2024-09-17T03:05:05.312 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T03:05:05.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T03:05:05.444 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: pdonnell@redhat.com first_in_suite: false job_id: '7908415' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIkyP6uczF9Odc6nUmWEDkznd4+3GSexQxw/po+/jO+QSbTqfrHBHCT5gqw8cdPWoaUn8vQUFmzBeworePvjNk= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqvrmwqsgWn70v84FkhDINnJA6jQh0nOktWk4hHdOD055amKJCCLamqdWjI7+blq+IlvOjgV2YDOjYivYebbp0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T03:05:05.444 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T03:05:05.445 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T03:05:05.445 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T03:05:05.447 INFO:teuthology.task.internal:Checking packages... 2024-09-17T03:05:05.468 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T03:05:05.469 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T03:05:05.469 INFO:teuthology.packaging:ref: None 2024-09-17T03:05:05.469 INFO:teuthology.packaging:tag: None 2024-09-17T03:05:05.469 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T03:05:05.469 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:05:05.469 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T03:05:05.617 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:05:05.619 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T03:05:05.627 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T03:05:05.627 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T03:05:05.635 INFO:teuthology.task.internal:Saving configuration 2024-09-17T03:05:05.653 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T03:05:05.663 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T03:05:05.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 02:59:18.049212', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIJ+GGvXHxS25sM+SLaPxJayE7e8NKXBCJVwXtcWdaXi'} 2024-09-17T03:05:05.709 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 02:59:18.051173', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBbANoVMo7Ek85hJpxCkEdOxSAcjX0NQ37x2NOSKHMmy'} 2024-09-17T03:05:05.709 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T03:05:05.716 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T03:05:05.716 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T03:05:05.716 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T03:05:05.782 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f63bcf0fac0>, signals=[15]) 2024-09-17T03:05:05.782 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T03:05:05.788 INFO:teuthology.task.internal:Opening connections... 2024-09-17T03:05:05.789 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:05:05.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:05:05.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:05:05.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:05:05.947 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T03:05:05.960 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-17T03:05:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-17T03:05:05.966 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-17T03:05:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T03:05:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T03:05:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy 2024-09-17T03:05:06.016 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-17T03:05:06.036 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-17T03:05:06.041 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-17T03:05:06.041 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Ubuntu" 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="22.04" 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_CODENAME=jammy 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:ID=ubuntu 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE=debian 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T03:05:06.088 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T03:05:06.089 INFO:teuthology.orchestra.run.smithi099.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T03:05:06.089 INFO:teuthology.orchestra.run.smithi099.stdout:UBUNTU_CODENAME=jammy 2024-09-17T03:05:06.089 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-17T03:05:06.110 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T03:05:06.119 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T03:05:06.133 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T03:05:06.133 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-17T03:05:06.135 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-17T03:05:06.140 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T03:05:06.150 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T03:05:06.150 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-17T03:05:06.184 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-17T03:05:06.190 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T03:05:06.241 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T03:05:06.257 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-17T03:05:06.257 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-17T03:05:06.257 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-17T03:05:06.257 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T03:05:06.258 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T03:05:06.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:05:06.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T03:05:06.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:05:06.266 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T03:05:06.426 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T03:05:06.426 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T03:05:06.427 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-17T03:05:06.432 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-56-generic 2024-09-17T03:05:06.433 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T03:05:06.433 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T03:05:06.433 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-17T03:05:06.433 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T03:05:06.447 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T03:05:06.447 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T03:05:06.448 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-17T03:05:06.453 INFO:teuthology.orchestra.run.smithi099.stdout:5.15.0-56-generic 2024-09-17T03:05:06.453 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T03:05:06.454 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T03:05:06.454 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-17T03:05:06.454 DEBUG:teuthology.orchestra.run.smithi099:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T03:05:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T03:05:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T03:05:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: git config --global init.defaultBranch 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-09-17T03:05:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:hint: git branch -m 2024-09-17T03:05:06.611 INFO:teuthology.orchestra.run.smithi046.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T03:05:06.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T03:05:06.679 INFO:teuthology.orchestra.run.smithi099.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: git config --global init.defaultBranch 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: 2024-09-17T03:05:06.680 INFO:teuthology.orchestra.run.smithi099.stderr:hint: git branch -m 2024-09-17T03:05:06.681 INFO:teuthology.orchestra.run.smithi099.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T03:05:06.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T03:05:06.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T03:05:06.740 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T03:05:06.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T03:05:06.805 DEBUG:teuthology.orchestra.run.smithi099:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T03:05:32.349 INFO:teuthology.orchestra.run.smithi099.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T03:05:32.349 INFO:teuthology.orchestra.run.smithi099.stderr: * [new branch] main -> origin/main 2024-09-17T03:05:32.377 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T03:05:32.377 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T03:05:32.377 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T03:05:32.377 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T03:05:32.377 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T03:05:32.377 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T03:05:32.378 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T03:05:32.378 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T03:05:32.378 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T03:05:32.378 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T03:05:32.378 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T03:05:32.378 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T03:05:32.382 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T03:05:32.382 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T03:05:32.382 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T03:05:32.383 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T03:05:32.384 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T03:05:32.385 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T03:05:32.386 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T03:05:32.386 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T03:05:32.386 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T03:05:32.386 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T03:05:32.386 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T03:05:32.387 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T03:05:32.387 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T03:05:32.387 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T03:05:32.387 INFO:teuthology.orchestra.run.smithi099.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T03:05:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T03:05:33.051 INFO:teuthology.orchestra.run.smithi046.stderr: * [new branch] main -> origin/main 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T03:05:33.082 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T03:05:33.084 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T03:05:33.084 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T03:05:33.084 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T03:05:33.085 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T03:05:33.088 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T03:05:33.089 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T03:05:33.090 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T03:05:33.091 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T03:05:36.971 INFO:teuthology.orchestra.run.smithi099.stderr:Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 59% (2077/3520) Updating files: 59% (2079/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2436/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2902/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T03:05:36.976 INFO:teuthology.orchestra.run.smithi099.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T03:05:36.998 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T03:05:36.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T03:05:37.153 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T03:05:37.153 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T03:05:37.156 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2024-09-17T03:05:37.940 INFO:teuthology.orchestra.run.smithi046.stderr:Updating files: 29% (1029/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2058/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 68% (2427/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 81% (2886/3520) Updating files: 82% (2887/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T03:05:37.945 INFO:teuthology.orchestra.run.smithi046.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T03:05:37.970 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T03:05:37.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T03:05:38.122 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T03:05:38.122 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T03:05:38.125 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-09-17T03:05:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17 03:05:37 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T03:05:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2024-09-17 03:05:37-- 2024-09-17T03:05:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 1.0s 2024-09-17T03:05:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 43M in 0.9s (50.7 MB/s) 2024-09-17T03:05:38.159 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T03:05:38.159 DEBUG:teuthology.orchestra.run.smithi099:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T03:05:38.413 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T03:05:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17 03:05:39 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T03:05:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-09-17 03:05:39-- 2024-09-17T03:05:39.104 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 1.0s 2024-09-17T03:05:39.105 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 43M in 0.8s (51.7 MB/s) 2024-09-17T03:05:39.108 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T03:05:39.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T03:05:39.349 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T03:05:40.391 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T03:05:40.395 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T03:05:40.496 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T03:05:41.317 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T03:05:41.322 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T03:05:41.431 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T03:05:44.528 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T03:05:44.639 INFO:teuthology.orchestra.run.smithi099.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:05:45.215 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T03:05:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:07.342 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub' 2024-09-17T03:06:07.356 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T03:06:07.377 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-17T03:06:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.003 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T03:06:08.016 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T03:06:08.079 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T03:06:08.085 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T03:06:08.230 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T03:06:08.230 INFO:teuthology.orchestra.run.smithi099.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T03:06:08.230 INFO:teuthology.orchestra.run.smithi099.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T03:06:08.266 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-17T03:06:08.436 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-09-17T03:06:08.449 DEBUG:teuthology.orchestra.run.smithi099:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T03:06:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T03:06:08.456 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T03:06:08.457 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T03:06:08.457 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.474 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.474 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T03:06:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T03:06:08.498 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T03:06:08.499 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T03:06:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T03:06:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T03:06:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T03:06:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T03:06:08.500 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T03:06:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T03:06:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T03:06:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T03:06:08.501 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T03:06:08.557 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T03:06:08.569 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T03:06:08.570 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T03:06:08.571 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T03:06:08.572 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T03:06:08.573 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T03:06:08.574 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T03:06:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T03:06:08.576 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T03:06:08.577 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T03:06:08.578 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T03:06:08.579 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T03:06:08.580 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T03:06:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T03:06:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T03:06:08.583 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T03:06:08.584 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T03:06:08.585 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T03:06:08.586 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T03:06:08.587 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T03:06:08.588 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T03:06:08.589 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T03:06:08.590 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T03:06:08.591 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T03:06:08.592 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T03:06:08.593 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T03:06:08.594 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T03:06:08.595 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T03:06:08.596 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T03:06:08.597 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T03:06:08.598 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T03:06:08.599 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T03:06:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T03:06:08.601 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T03:06:08.602 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T03:06:08.603 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T03:06:08.604 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T03:06:08.605 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T03:06:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T03:06:08.607 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T03:06:08.608 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T03:06:08.609 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T03:06:08.610 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T03:06:08.611 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T03:06:08.612 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T03:06:08.613 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T03:06:08.614 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T03:06:08.615 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T03:06:08.616 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T03:06:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T03:06:08.618 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T03:06:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T03:06:08.620 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T03:06:08.621 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T03:06:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T03:06:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T03:06:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T03:06:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T03:06:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T03:06:08.672 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T03:06:08.673 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T03:06:08.674 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T03:06:08.675 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T03:06:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T03:06:08.677 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T03:06:08.678 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T03:06:08.679 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T03:06:08.680 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T03:06:08.681 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T03:06:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T03:06:08.683 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T03:06:08.684 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T03:06:08.685 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T03:06:08.686 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T03:06:08.687 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T03:06:08.688 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T03:06:08.689 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T03:06:08.690 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T03:06:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T03:06:08.692 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T03:06:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T03:06:08.694 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T03:06:08.695 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T03:06:08.696 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T03:06:08.697 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T03:06:08.698 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T03:06:08.699 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T03:06:08.700 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T03:06:08.701 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T03:06:08.702 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T03:06:08.703 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T03:06:08.704 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T03:06:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T03:06:08.706 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T03:06:08.707 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T03:06:08.708 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T03:06:08.709 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T03:06:08.710 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T03:06:08.711 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T03:06:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T03:06:08.713 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T03:06:08.714 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T03:06:08.715 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T03:06:08.716 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T03:06:08.717 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T03:06:08.756 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T03:06:08.756 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T03:06:08.756 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T03:06:08.756 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T03:06:08.757 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T03:06:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T03:06:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T03:06:08.760 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T03:06:08.761 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T03:06:08.762 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T03:06:08.763 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T03:06:08.764 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T03:06:08.765 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T03:06:08.766 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T03:06:08.767 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T03:06:08.768 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T03:06:08.769 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T03:06:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T03:06:08.771 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T03:06:08.772 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T03:06:08.773 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T03:06:08.774 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T03:06:08.775 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T03:06:08.776 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T03:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T03:06:08.778 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T03:06:08.779 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T03:06:08.780 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T03:06:08.781 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T03:06:08.782 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T03:06:08.783 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T03:06:08.784 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T03:06:08.785 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T03:06:08.786 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T03:06:08.787 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T03:06:08.788 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T03:06:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T03:06:08.790 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T03:06:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T03:06:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T03:06:08.793 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T03:06:08.794 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T03:06:08.795 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T03:06:08.796 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T03:06:08.797 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T03:06:08.835 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T03:06:08.836 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T03:06:08.837 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T03:06:08.838 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T03:06:08.839 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T03:06:08.840 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T03:06:08.841 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T03:06:08.842 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T03:06:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T03:06:08.844 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T03:06:08.845 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T03:06:08.845 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T03:06:08.845 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T03:06:08.845 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T03:06:08.845 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T03:06:08.848 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T03:06:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T03:06:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T03:06:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T03:06:08.852 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T03:06:08.853 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T03:06:08.854 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T03:06:08.855 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T03:06:08.856 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T03:06:08.857 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T03:06:08.858 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T03:06:08.859 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T03:06:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T03:06:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T03:06:08.862 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T03:06:08.863 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T03:06:08.864 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T03:06:08.865 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T03:06:08.866 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T03:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T03:06:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T03:06:08.869 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T03:06:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T03:06:08.871 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T03:06:08.872 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T03:06:08.873 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T03:06:08.874 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T03:06:08.875 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T03:06:08.876 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T03:06:08.877 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T03:06:08.883 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.883 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.883 DEBUG:teuthology.orchestra.run.smithi099:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T03:06:08.890 INFO:teuthology.orchestra.run.smithi099.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T03:06:08.890 INFO:teuthology.orchestra.run.smithi099.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T03:06:08.890 INFO:teuthology.orchestra.run.smithi099.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T03:06:08.890 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T03:06:08.890 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.891 DEBUG:teuthology.orchestra.run.smithi099:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T03:06:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:08.933 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi099.front.sepia.ceph.com to complete... 2024-09-17T03:06:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T03:06:09.279 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T03:06:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub' 2024-09-17T03:06:09.326 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T03:06:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T03:06:09.345 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T03:06:09.348 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-17T03:06:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T03:06:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T03:06:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T03:06:09.545 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:09.551 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-17T03:06:09.713 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T03:06:09.722 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T03:06:09.722 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T03:06:09.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:09.742 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:09.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:09.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T03:06:09.766 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T03:06:09.767 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T03:06:09.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T03:06:09.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T03:06:09.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T03:06:09.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T03:06:09.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T03:06:09.771 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T03:06:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T03:06:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T03:06:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T03:06:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T03:06:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T03:06:09.772 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T03:06:09.786 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T03:06:09.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T03:06:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T03:06:09.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T03:06:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T03:06:09.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T03:06:09.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T03:06:09.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T03:06:09.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T03:06:09.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T03:06:09.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T03:06:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T03:06:09.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T03:06:09.799 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T03:06:09.800 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T03:06:09.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T03:06:09.836 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T03:06:09.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T03:06:09.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T03:06:09.836 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T03:06:09.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T03:06:09.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T03:06:09.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T03:06:09.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T03:06:09.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T03:06:09.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T03:06:09.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T03:06:09.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T03:06:09.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T03:06:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T03:06:09.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T03:06:09.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T03:06:09.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T03:06:09.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T03:06:09.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T03:06:09.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T03:06:09.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T03:06:09.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T03:06:09.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T03:06:09.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T03:06:09.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T03:06:09.853 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T03:06:09.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T03:06:09.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T03:06:09.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T03:06:09.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T03:06:09.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T03:06:09.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T03:06:09.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T03:06:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T03:06:09.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T03:06:09.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T03:06:09.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T03:06:09.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T03:06:09.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T03:06:09.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T03:06:09.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T03:06:09.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T03:06:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T03:06:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T03:06:09.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T03:06:09.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T03:06:09.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T03:06:09.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T03:06:09.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T03:06:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T03:06:09.873 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T03:06:09.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T03:06:09.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T03:06:09.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T03:06:09.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T03:06:09.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T03:06:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T03:06:09.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T03:06:09.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T03:06:09.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T03:06:09.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T03:06:09.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T03:06:09.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T03:06:09.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T03:06:09.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T03:06:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T03:06:09.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T03:06:09.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T03:06:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T03:06:09.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T03:06:09.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T03:06:09.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T03:06:09.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T03:06:09.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T03:06:09.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T03:06:09.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T03:06:09.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T03:06:09.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T03:06:09.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T03:06:09.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T03:06:09.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T03:06:09.930 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T03:06:09.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T03:06:09.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T03:06:09.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T03:06:09.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T03:06:09.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T03:06:09.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T03:06:09.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T03:06:09.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T03:06:09.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T03:06:09.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T03:06:09.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T03:06:09.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T03:06:09.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T03:06:09.961 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T03:06:09.962 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T03:06:09.963 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T03:06:09.964 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T03:06:09.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T03:06:09.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T03:06:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T03:06:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T03:06:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T03:06:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T03:06:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T03:06:09.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T03:06:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T03:06:09.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T03:06:09.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T03:06:09.976 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T03:06:09.977 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T03:06:09.978 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T03:06:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T03:06:09.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T03:06:09.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T03:06:09.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T03:06:09.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T03:06:09.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T03:06:09.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T03:06:09.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T03:06:09.987 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T03:06:09.987 INFO:teuthology.orchestra.run.smithi099.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T03:06:09.987 INFO:teuthology.orchestra.run.smithi099.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T03:06:10.013 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T03:06:10.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T03:06:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T03:06:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T03:06:10.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T03:06:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T03:06:10.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T03:06:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T03:06:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T03:06:10.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T03:06:10.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T03:06:10.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T03:06:10.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T03:06:10.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T03:06:10.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T03:06:10.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T03:06:10.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T03:06:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T03:06:10.043 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T03:06:10.044 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T03:06:10.045 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T03:06:10.046 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T03:06:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T03:06:10.048 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T03:06:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T03:06:10.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T03:06:10.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T03:06:10.052 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T03:06:10.053 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T03:06:10.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T03:06:10.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T03:06:10.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T03:06:10.057 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T03:06:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T03:06:10.059 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T03:06:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T03:06:10.074 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T03:06:10.075 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T03:06:10.076 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T03:06:10.077 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T03:06:10.078 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T03:06:10.078 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T03:06:10.080 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T03:06:10.080 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T03:06:10.080 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T03:06:10.080 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T03:06:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T03:06:10.082 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T03:06:10.083 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T03:06:10.084 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T03:06:10.085 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T03:06:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T03:06:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T03:06:10.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T03:06:10.089 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T03:06:10.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T03:06:10.126 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T03:06:10.126 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T03:06:10.127 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T03:06:10.128 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T03:06:10.129 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T03:06:10.130 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T03:06:10.131 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T03:06:10.132 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T03:06:10.133 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T03:06:10.134 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T03:06:10.135 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T03:06:10.136 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T03:06:10.137 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T03:06:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T03:06:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T03:06:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T03:06:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T03:06:10.142 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T03:06:10.143 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T03:06:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T03:06:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T03:06:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T03:06:10.147 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T03:06:10.148 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T03:06:10.149 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T03:06:10.150 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T03:06:10.151 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T03:06:10.152 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T03:06:10.153 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T03:06:10.154 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T03:06:10.155 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T03:06:10.156 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T03:06:10.157 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T03:06:10.162 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:10.162 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:10.162 DEBUG:teuthology.orchestra.run.smithi046:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T03:06:10.176 INFO:teuthology.orchestra.run.smithi046.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T03:06:10.176 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T03:06:10.176 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T03:06:10.177 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T03:06:10.177 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:10.177 DEBUG:teuthology.orchestra.run.smithi046:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T03:06:10.220 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-09-17T03:06:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-09-17T03:06:10.732 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T03:06:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-17T03:06:10.985 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:06:11.297 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T03:06:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T03:06:11.349 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T03:06:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T03:06:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T03:06:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T03:06:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T03:06:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-17T03:06:41.121 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-17T03:06:41.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:06:41.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:42.701 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T03:06:42.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:06:42.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:59.517 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-17T03:07:01.057 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-17T03:07:08.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:07:08.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:10.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:07:10.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:11.585 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-17T03:07:13.117 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-17T03:07:23.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:07:23.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:25.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:07:25.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:26.653 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-17T03:07:41.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:07:41.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:44.733 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-17T03:08:02.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:08:02.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:08:03.045 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-17T03:08:03.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:08:03.730 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T03:08:03.731 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-17T03:08:03.781 INFO:teuthology.orchestra.run.smithi099.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:08:03.781 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:08:03.781 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T03:08:03.781 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T03:08:04.782 DEBUG:teuthology.parallel:result is None 2024-09-17T03:08:25.122 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T03:08:40.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:08:40.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:08:40.416 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-17T03:08:41.247 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:08:41.247 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T03:08:41.247 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-17T03:08:41.299 INFO:teuthology.orchestra.run.smithi046.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:08:41.300 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T03:08:41.300 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T03:08:41.300 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T03:08:42.303 DEBUG:teuthology.parallel:result is None 2024-09-17T03:08:42.304 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T03:08:42.318 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T03:08:42.318 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T03:08:42.321 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T03:08:42.330 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T03:08:42.368 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T03:08:42.379 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T03:08:42.379 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T03:08:42.382 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T03:08:42.415 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T03:08:42.421 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T03:08:42.422 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T03:08:42.446 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T03:08:42.520 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:08:42.547 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:08:42.554 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:08:42.574 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:08:42.577 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T03:08:42.584 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T03:08:42.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T03:08:42.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T03:08:42.609 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T03:08:42.618 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T03:08:42.618 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T03:08:42.650 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T03:08:42.661 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T03:08:42.708 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T03:08:42.760 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:08:42.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T03:08:42.833 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T03:08:42.842 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T03:08:42.893 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:08:42.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T03:08:42.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-17T03:08:42.969 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-17T03:08:43.119 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T03:08:43.126 INFO:teuthology.task.internal:Starting timer... 2024-09-17T03:08:43.126 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T03:08:43.137 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T03:08:43.146 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2024-09-17T03:08:43.146 DEBUG:teuthology.task.selinux:Excluding smithi099: OS 'ubuntu' does not support SELinux 2024-09-17T03:08:43.146 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T03:08:43.146 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T03:08:43.147 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T03:08:43.147 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T03:08:43.155 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T03:08:43.168 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T03:08:43.169 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi099.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T03:17:05.377 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com')] 2024-09-17T03:17:05.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:17:05.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:17:05.463 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-17T03:17:05.576 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-17T03:17:05.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:17:05.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:17:05.657 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-17T03:17:05.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-17T03:17:05.768 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T03:17:05.779 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T03:17:05.779 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T03:17:05.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:17:05.783 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T03:17:05.783 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Command line: ntpd -gq 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: ---------------------------------------------------- 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: corporation. Support and training for ntp-4 are 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: available at https://www.nwtime.org/support 2024-09-17T03:17:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: ---------------------------------------------------- 2024-09-17T03:17:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: proto: precision = 0.042 usec (-24) 2024-09-17T03:17:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: basedate set to 2022-02-04 2024-09-17T03:17:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: gps base set to 2022-02-06 (week 2196) 2024-09-17T03:17:05.821 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T03:17:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T03:17:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T03:17:05.821 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T03:17:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T03:17:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T03:17:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T03:17:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listen normally on 3 ens1f0 172.21.15.46:123 2024-09-17T03:17:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listen normally on 4 lo [::1]:123 2024-09-17T03:17:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%3]:123 2024-09-17T03:17:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:05 ntpd[18413]: Listening on routing socket on fd #22 for interface updates 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Command line: ntpd -gq 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: ---------------------------------------------------- 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: corporation. Support and training for ntp-4 are 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: available at https://www.nwtime.org/support 2024-09-17T03:17:05.848 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: ---------------------------------------------------- 2024-09-17T03:17:05.849 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: proto: precision = 0.041 usec (-24) 2024-09-17T03:17:05.849 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: basedate set to 2022-02-04 2024-09-17T03:17:05.849 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: gps base set to 2022-02-06 (week 2196) 2024-09-17T03:17:05.849 INFO:teuthology.orchestra.run.smithi099.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T03:17:05.850 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T03:17:05.850 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T03:17:05.850 INFO:teuthology.orchestra.run.smithi099.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T03:17:05.851 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T03:17:05.851 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T03:17:05.851 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T03:17:05.851 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listen normally on 3 enp3s0f1 172.21.15.99:123 2024-09-17T03:17:05.851 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listen normally on 4 lo [::1]:123 2024-09-17T03:17:05.852 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d803%5]:123 2024-09-17T03:17:05.852 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:05 ntpd[18536]: Listening on routing socket on fd #22 for interface updates 2024-09-17T03:17:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:06 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:06.848 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:06 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:06.848 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:06 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:06.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:06 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:06.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:06 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:06.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:06 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:06.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:06 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:07.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:07 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:07.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:07 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:07.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:07 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:07.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:07 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:07.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:07 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:07.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:07 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:08 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:08 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:08 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:08 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:08 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:08 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:08 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:08 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:08 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:08 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:08 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:08 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:09.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:09 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:10 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:10 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:11.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:11 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:11.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:11 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:11.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:11 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:11.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:11 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:11.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:11 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:11.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:11 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:12.821 INFO:teuthology.orchestra.run.smithi046.stdout:17 Sep 03:17:12 ntpd[18413]: ntpd: time slew +0.003965 s 2024-09-17T03:17:12.821 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew +0.003965s 2024-09-17T03:17:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T03:17:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:17 Sep 03:17:12 ntpd[18413]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T03:17:12.849 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T03:17:12.850 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T03:17:12.851 INFO:teuthology.orchestra.run.smithi099.stdout:17 Sep 03:17:12 ntpd[18536]: ntpd: time slew -0.002632 s 2024-09-17T03:17:12.851 INFO:teuthology.orchestra.run.smithi099.stdout:ntpd: time slew -0.002632s 2024-09-17T03:17:12.851 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T03:17:12.851 INFO:teuthology.orchestra.run.smithi099.stderr:17 Sep 03:17:12 ntpd[18536]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T03:17:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T03:17:12.902 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-09-17T03:17:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.902 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.947 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T03:17:12.947 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2024-09-17T03:17:12.947 INFO:teuthology.orchestra.run.smithi099.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.947 INFO:teuthology.orchestra.run.smithi099.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.947 INFO:teuthology.orchestra.run.smithi099.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.948 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T03:17:12.948 INFO:teuthology.run_tasks:Running task install... 2024-09-17T03:17:12.956 DEBUG:teuthology.task.install:project ceph 2024-09-17T03:17:12.956 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T03:17:12.957 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T03:17:12.957 INFO:teuthology.task.install:Using flavor: default 2024-09-17T03:17:12.963 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T03:17:12.963 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T03:17:12.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2024-09-17T03:17:12.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-key list | grep Ceph 2024-09-17T03:17:13.068 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T03:17:13.092 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T03:17:13.119 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T03:17:13.120 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T03:17:13.121 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T03:17:13.121 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T03:17:13.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:17:13.143 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T03:17:13.143 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T03:17:13.144 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T03:17:13.145 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T03:17:13.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:17:13.264 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T03:17:13.264 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:17:13.281 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T03:17:13.281 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:17:13.375 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:17:13.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T03:17:13.392 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:17:13.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T03:17:13.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-09-17T03:17:13.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-17T03:17:13.581 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T03:17:13.581 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T03:17:13.587 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T03:17:13.598 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T03:17:13.604 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T03:17:13.631 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T03:17:13.648 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T03:17:13.653 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T03:17:13.669 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T03:17:13.675 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T03:17:13.691 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T03:17:13.698 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T03:17:13.714 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T03:17:13.727 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T03:17:13.730 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T03:17:13.748 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T03:17:13.796 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T03:17:13.878 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T03:17:14.202 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 29.2 kB in 1s (51.0 kB/s) 2024-09-17T03:17:14.204 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 29.2 kB in 1s (52.0 kB/s) 2024-09-17T03:17:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-17T03:17:15.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:17:15.276 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-17T03:17:15.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:17:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-17T03:17:15.369 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-17T03:17:15.537 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-17T03:17:15.537 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-17T03:17:15.555 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-17T03:17:15.555 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-17T03:17:15.724 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T03:17:15.725 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T03:17:15.725 INFO:teuthology.orchestra.run.smithi099.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T03:17:15.725 INFO:teuthology.orchestra.run.smithi099.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T03:17:15.725 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T03:17:15.725 INFO:teuthology.orchestra.run.smithi099.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T03:17:15.726 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T03:17:15.726 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2024-09-17T03:17:15.726 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T03:17:15.727 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T03:17:15.727 INFO:teuthology.orchestra.run.smithi099.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T03:17:15.727 INFO:teuthology.orchestra.run.smithi099.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T03:17:15.727 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T03:17:15.728 INFO:teuthology.orchestra.run.smithi099.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T03:17:15.728 INFO:teuthology.orchestra.run.smithi099.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T03:17:15.728 INFO:teuthology.orchestra.run.smithi099.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T03:17:15.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T03:17:15.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T03:17:15.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T03:17:15.729 INFO:teuthology.orchestra.run.smithi099.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T03:17:15.730 INFO:teuthology.orchestra.run.smithi099.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T03:17:15.731 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2024-09-17T03:17:15.731 INFO:teuthology.orchestra.run.smithi099.stdout: btrfs-tools 2024-09-17T03:17:15.750 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T03:17:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T03:17:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T03:17:15.750 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T03:17:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T03:17:15.751 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T03:17:15.751 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T03:17:15.752 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-09-17T03:17:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T03:17:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T03:17:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T03:17:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T03:17:15.752 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T03:17:15.754 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T03:17:15.755 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-09-17T03:17:15.756 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2024-09-17T03:17:15.886 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2024-09-17T03:17:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T03:17:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T03:17:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T03:17:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T03:17:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T03:17:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T03:17:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T03:17:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T03:17:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T03:17:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T03:17:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T03:17:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T03:17:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T03:17:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T03:17:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T03:17:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T03:17:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T03:17:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T03:17:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T03:17:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T03:17:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T03:17:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T03:17:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T03:17:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T03:17:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T03:17:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T03:17:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T03:17:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: socat unzip xmlstarlet zip 2024-09-17T03:17:15.920 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-09-17T03:17:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T03:17:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T03:17:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T03:17:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T03:17:15.921 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T03:17:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T03:17:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T03:17:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T03:17:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T03:17:15.922 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T03:17:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T03:17:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T03:17:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T03:17:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T03:17:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T03:17:15.923 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T03:17:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T03:17:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T03:17:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T03:17:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T03:17:15.924 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T03:17:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T03:17:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T03:17:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T03:17:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T03:17:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T03:17:15.925 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T03:17:15.926 INFO:teuthology.orchestra.run.smithi046.stdout: socat unzip xmlstarlet zip 2024-09-17T03:17:15.953 INFO:teuthology.orchestra.run.smithi099.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T03:17:15.953 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 239 MB of archives. 2024-09-17T03:17:15.953 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T03:17:15.954 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T03:17:15.971 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T03:17:15.979 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T03:17:15.989 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T03:17:15.989 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 239 MB of archives. 2024-09-17T03:17:15.989 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T03:17:15.989 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T03:17:16.007 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T03:17:16.017 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T03:17:16.020 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T03:17:16.036 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T03:17:16.041 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T03:17:16.044 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T03:17:16.056 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T03:17:16.058 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T03:17:16.079 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T03:17:16.082 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T03:17:16.086 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T03:17:16.093 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T03:17:16.093 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T03:17:16.095 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T03:17:16.102 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T03:17:16.105 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T03:17:16.106 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T03:17:16.106 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T03:17:16.112 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T03:17:16.114 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T03:17:16.115 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T03:17:16.117 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T03:17:16.129 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T03:17:16.131 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T03:17:16.131 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T03:17:16.132 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T03:17:16.134 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T03:17:16.134 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T03:17:16.135 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T03:17:16.142 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T03:17:16.145 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T03:17:16.149 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T03:17:16.150 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T03:17:16.150 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T03:17:16.150 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T03:17:16.150 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T03:17:16.150 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T03:17:16.150 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T03:17:16.153 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T03:17:16.155 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T03:17:16.156 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T03:17:16.157 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T03:17:16.163 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T03:17:16.163 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T03:17:16.163 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T03:17:16.165 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T03:17:16.166 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T03:17:16.168 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T03:17:16.170 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T03:17:16.172 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T03:17:16.173 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T03:17:16.174 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T03:17:16.181 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T03:17:16.181 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T03:17:16.181 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T03:17:16.182 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T03:17:16.182 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T03:17:16.185 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T03:17:16.186 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T03:17:16.186 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T03:17:16.187 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T03:17:16.187 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T03:17:16.196 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T03:17:16.197 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T03:17:16.198 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T03:17:16.202 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T03:17:16.202 INFO:teuthology.orchestra.run.smithi099.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T03:17:16.203 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T03:17:16.203 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T03:17:16.204 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T03:17:16.205 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T03:17:16.206 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T03:17:16.213 INFO:teuthology.orchestra.run.smithi099.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T03:17:16.213 INFO:teuthology.orchestra.run.smithi099.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T03:17:16.215 INFO:teuthology.orchestra.run.smithi099.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T03:17:16.216 INFO:teuthology.orchestra.run.smithi099.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T03:17:16.217 INFO:teuthology.orchestra.run.smithi099.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T03:17:16.219 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T03:17:16.220 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T03:17:16.220 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T03:17:16.221 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T03:17:16.221 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T03:17:16.230 INFO:teuthology.orchestra.run.smithi099.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T03:17:16.231 INFO:teuthology.orchestra.run.smithi099.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T03:17:16.233 INFO:teuthology.orchestra.run.smithi099.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T03:17:16.233 INFO:teuthology.orchestra.run.smithi099.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T03:17:16.234 INFO:teuthology.orchestra.run.smithi099.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T03:17:16.237 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T03:17:16.237 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T03:17:16.238 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T03:17:16.240 INFO:teuthology.orchestra.run.smithi099.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T03:17:16.242 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T03:17:16.243 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T03:17:16.247 INFO:teuthology.orchestra.run.smithi099.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T03:17:16.254 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T03:17:16.255 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T03:17:16.256 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T03:17:16.258 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T03:17:16.258 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T03:17:16.273 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T03:17:16.274 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T03:17:16.276 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T03:17:16.276 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T03:17:16.277 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T03:17:16.291 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T03:17:16.292 INFO:teuthology.orchestra.run.smithi099.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T03:17:16.293 INFO:teuthology.orchestra.run.smithi099.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T03:17:16.294 INFO:teuthology.orchestra.run.smithi099.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T03:17:16.294 INFO:teuthology.orchestra.run.smithi099.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T03:17:16.296 INFO:teuthology.orchestra.run.smithi099.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T03:17:16.307 INFO:teuthology.orchestra.run.smithi099.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T03:17:16.315 INFO:teuthology.orchestra.run.smithi099.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T03:17:16.318 INFO:teuthology.orchestra.run.smithi099.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T03:17:16.318 INFO:teuthology.orchestra.run.smithi099.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T03:17:16.335 INFO:teuthology.orchestra.run.smithi099.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T03:17:16.335 INFO:teuthology.orchestra.run.smithi099.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T03:17:16.335 INFO:teuthology.orchestra.run.smithi099.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T03:17:16.336 INFO:teuthology.orchestra.run.smithi099.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T03:17:16.337 INFO:teuthology.orchestra.run.smithi099.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T03:17:16.337 INFO:teuthology.orchestra.run.smithi099.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T03:17:16.340 INFO:teuthology.orchestra.run.smithi099.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T03:17:16.340 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T03:17:16.341 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T03:17:16.342 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T03:17:16.343 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T03:17:16.344 INFO:teuthology.orchestra.run.smithi099.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T03:17:16.345 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T03:17:16.345 INFO:teuthology.orchestra.run.smithi099.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T03:17:16.345 INFO:teuthology.orchestra.run.smithi099.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T03:17:16.349 INFO:teuthology.orchestra.run.smithi099.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T03:17:16.362 INFO:teuthology.orchestra.run.smithi099.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T03:17:16.363 INFO:teuthology.orchestra.run.smithi099.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T03:17:16.364 INFO:teuthology.orchestra.run.smithi099.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T03:17:16.365 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T03:17:16.367 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T03:17:16.367 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T03:17:16.367 INFO:teuthology.orchestra.run.smithi099.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T03:17:16.370 INFO:teuthology.orchestra.run.smithi099.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T03:17:16.370 INFO:teuthology.orchestra.run.smithi099.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T03:17:16.371 INFO:teuthology.orchestra.run.smithi099.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T03:17:16.377 INFO:teuthology.orchestra.run.smithi099.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T03:17:16.379 INFO:teuthology.orchestra.run.smithi099.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T03:17:16.383 INFO:teuthology.orchestra.run.smithi099.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T03:17:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T03:17:16.384 INFO:teuthology.orchestra.run.smithi099.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T03:17:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T03:17:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T03:17:16.385 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T03:17:16.386 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T03:17:16.387 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T03:17:16.388 INFO:teuthology.orchestra.run.smithi099.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T03:17:16.394 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T03:17:16.395 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T03:17:16.396 INFO:teuthology.orchestra.run.smithi099.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T03:17:16.396 INFO:teuthology.orchestra.run.smithi099.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T03:17:16.397 INFO:teuthology.orchestra.run.smithi099.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T03:17:16.397 INFO:teuthology.orchestra.run.smithi099.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T03:17:16.399 INFO:teuthology.orchestra.run.smithi099.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T03:17:16.399 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T03:17:16.400 INFO:teuthology.orchestra.run.smithi099.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T03:17:16.403 INFO:teuthology.orchestra.run.smithi099.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T03:17:16.406 INFO:teuthology.orchestra.run.smithi099.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T03:17:16.412 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T03:17:16.413 INFO:teuthology.orchestra.run.smithi099.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T03:17:16.413 INFO:teuthology.orchestra.run.smithi099.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T03:17:16.414 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T03:17:16.414 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T03:17:16.415 INFO:teuthology.orchestra.run.smithi099.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T03:17:16.418 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T03:17:16.420 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T03:17:16.421 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T03:17:16.422 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T03:17:16.429 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T03:17:16.430 INFO:teuthology.orchestra.run.smithi099.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T03:17:16.432 INFO:teuthology.orchestra.run.smithi099.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T03:17:16.434 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T03:17:16.434 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T03:17:16.446 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T03:17:16.446 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T03:17:16.448 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T03:17:16.448 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T03:17:16.450 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T03:17:16.451 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T03:17:16.457 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T03:17:16.463 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T03:17:16.464 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T03:17:16.465 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T03:17:16.466 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T03:17:16.480 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T03:17:16.587 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T03:17:16.598 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T03:17:16.603 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T03:17:16.604 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T03:17:16.607 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T03:17:16.608 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T03:17:16.614 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T03:17:16.681 INFO:teuthology.orchestra.run.smithi099.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T03:17:16.682 INFO:teuthology.orchestra.run.smithi099.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T03:17:16.688 INFO:teuthology.orchestra.run.smithi099.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T03:17:17.568 INFO:teuthology.orchestra.run.smithi099.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T03:17:17.703 INFO:teuthology.orchestra.run.smithi099.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T03:17:17.748 INFO:teuthology.orchestra.run.smithi099.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T03:17:17.793 INFO:teuthology.orchestra.run.smithi099.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T03:17:17.938 INFO:teuthology.orchestra.run.smithi099.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T03:17:18.040 INFO:teuthology.orchestra.run.smithi099.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T03:17:18.708 INFO:teuthology.orchestra.run.smithi099.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T03:17:18.709 INFO:teuthology.orchestra.run.smithi099.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T03:17:18.777 INFO:teuthology.orchestra.run.smithi099.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T03:17:18.818 INFO:teuthology.orchestra.run.smithi099.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T03:17:18.863 INFO:teuthology.orchestra.run.smithi099.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T03:17:18.880 INFO:teuthology.orchestra.run.smithi099.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T03:17:21.059 INFO:teuthology.orchestra.run.smithi099.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T03:17:21.217 INFO:teuthology.orchestra.run.smithi099.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T03:17:21.226 INFO:teuthology.orchestra.run.smithi099.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T03:17:22.517 INFO:teuthology.orchestra.run.smithi099.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T03:17:22.531 INFO:teuthology.orchestra.run.smithi099.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T03:17:22.567 INFO:teuthology.orchestra.run.smithi099.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T03:17:22.568 INFO:teuthology.orchestra.run.smithi099.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T03:17:22.570 INFO:teuthology.orchestra.run.smithi099.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T03:17:22.585 INFO:teuthology.orchestra.run.smithi099.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T03:17:23.238 INFO:teuthology.orchestra.run.smithi099.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T03:17:23.402 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T03:17:23.403 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T03:17:23.408 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T03:17:23.892 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 239 MB in 7s (32.6 MB/s) 2024-09-17T03:17:23.895 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T03:17:23.968 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T03:17:23.972 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T03:17:23.973 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T03:17:24.000 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T03:17:24.038 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T03:17:24.103 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T03:17:24.104 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T03:17:24.108 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T03:17:24.146 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T03:17:24.448 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T03:17:24.462 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T03:17:24.483 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T03:17:24.483 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T03:17:24.491 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T03:17:24.517 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T03:17:24.530 INFO:teuthology.orchestra.run.smithi046.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T03:17:24.543 INFO:teuthology.orchestra.run.smithi046.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T03:17:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T03:17:24.801 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T03:17:24.815 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T03:17:24.851 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T03:17:25.138 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T03:17:25.152 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T03:17:25.221 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T03:17:25.573 INFO:teuthology.orchestra.run.smithi046.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T03:17:25.583 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T03:17:25.598 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T03:17:25.642 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T03:17:25.697 INFO:teuthology.orchestra.run.smithi046.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T03:17:25.697 INFO:teuthology.orchestra.run.smithi046.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T03:17:25.978 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T03:17:25.993 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T03:17:26.036 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T03:17:26.380 INFO:teuthology.orchestra.run.smithi046.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T03:17:26.381 INFO:teuthology.orchestra.run.smithi046.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T03:17:26.383 INFO:teuthology.orchestra.run.smithi046.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T03:17:26.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T03:17:26.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T03:17:26.384 INFO:teuthology.orchestra.run.smithi046.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T03:17:26.398 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T03:17:26.413 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T03:17:26.465 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:17:26.667 INFO:teuthology.orchestra.run.smithi046.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T03:17:26.835 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T03:17:26.850 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T03:17:26.901 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:17:27.246 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T03:17:27.261 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T03:17:27.342 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:17:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 239 MB in 11s (22.3 MB/s) 2024-09-17T03:17:27.473 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T03:17:27.541 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T03:17:27.544 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T03:17:27.580 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T03:17:27.625 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T03:17:27.640 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T03:17:27.675 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T03:17:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T03:17:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T03:17:27.974 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T03:17:27.996 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librados2. 2024-09-17T03:17:28.010 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:28.053 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:28.286 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T03:17:28.300 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T03:17:28.344 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T03:17:28.689 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T03:17:28.704 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T03:17:28.705 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libnbd0. 2024-09-17T03:17:28.721 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T03:17:28.764 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T03:17:28.789 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T03:17:29.061 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librbd1. 2024-09-17T03:17:29.076 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:29.119 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:29.152 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T03:17:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T03:17:29.192 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T03:17:29.521 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T03:17:29.539 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T03:17:29.579 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T03:17:29.655 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T03:17:29.670 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:29.722 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:29.991 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rados. 2024-09-17T03:17:30.005 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:30.016 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T03:17:30.031 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T03:17:30.041 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:30.082 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:17:30.385 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T03:17:30.402 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:30.435 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T03:17:30.444 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:30.453 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T03:17:30.485 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:17:30.680 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T03:17:30.697 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:30.738 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:30.830 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T03:17:30.845 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T03:17:30.888 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:17:31.049 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T03:17:31.065 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:31.107 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:31.225 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T03:17:31.239 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T03:17:31.283 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T03:17:31.444 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T03:17:31.459 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T03:17:31.494 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T03:17:31.561 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librados2. 2024-09-17T03:17:31.579 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:31.627 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T03:17:31.770 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:31.813 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:32.080 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnbd0. 2024-09-17T03:17:32.095 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T03:17:32.139 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T03:17:32.183 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T03:17:32.198 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T03:17:32.241 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T03:17:32.450 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librbd1. 2024-09-17T03:17:32.465 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:32.494 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T03:17:32.508 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:32.509 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T03:17:32.544 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T03:17:32.822 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T03:17:32.837 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T03:17:32.880 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T03:17:32.919 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T03:17:32.934 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:32.969 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:33.225 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua5.1. 2024-09-17T03:17:33.230 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2024-09-17T03:17:33.240 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T03:17:33.245 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:33.284 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T03:17:33.288 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:33.583 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T03:17:33.596 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua-any. 2024-09-17T03:17:33.598 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:33.610 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T03:17:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:33.645 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T03:17:33.856 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package zip. 2024-09-17T03:17:33.871 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T03:17:33.915 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T03:17:34.119 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T03:17:34.135 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:34.169 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:34.251 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package unzip. 2024-09-17T03:17:34.267 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T03:17:34.310 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T03:17:34.447 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T03:17:34.462 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:34.505 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:34.697 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package luarocks. 2024-09-17T03:17:34.712 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T03:17:34.747 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T03:17:34.833 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T03:17:34.848 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T03:17:34.891 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T03:17:35.152 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T03:17:35.168 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:35.180 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librgw2. 2024-09-17T03:17:35.194 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:35.210 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:35.247 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:35.605 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T03:17:35.621 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T03:17:35.655 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T03:17:35.891 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T03:17:35.905 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:35.907 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T03:17:35.923 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T03:17:35.949 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:35.966 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T03:17:36.268 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T03:17:36.284 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T03:17:36.294 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T03:17:36.309 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T03:17:36.327 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T03:17:36.352 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T03:17:36.639 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T03:17:36.654 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua5.1. 2024-09-17T03:17:36.671 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T03:17:36.705 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:36.713 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T03:17:36.975 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-any. 2024-09-17T03:17:36.995 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T03:17:37.033 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T03:17:37.117 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-common. 2024-09-17T03:17:37.132 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:37.175 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:37.244 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package zip. 2024-09-17T03:17:37.259 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T03:17:37.294 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T03:17:37.572 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package unzip. 2024-09-17T03:17:37.587 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T03:17:37.630 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T03:17:38.034 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package luarocks. 2024-09-17T03:17:38.050 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T03:17:38.092 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T03:17:38.475 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2024-09-17T03:17:38.490 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:38.533 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:38.656 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-base. 2024-09-17T03:17:38.671 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:38.732 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:39.186 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T03:17:39.201 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:39.244 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:39.302 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T03:17:39.316 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T03:17:39.360 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T03:17:39.530 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T03:17:39.541 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T03:17:39.572 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T03:17:39.697 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T03:17:39.711 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T03:17:39.755 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T03:17:39.875 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T03:17:39.887 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:39.925 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:40.083 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T03:17:40.106 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T03:17:40.149 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T03:17:40.370 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2024-09-17T03:17:40.381 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:40.419 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T03:17:40.420 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:40.432 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T03:17:40.503 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T03:17:40.805 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T03:17:40.815 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T03:17:40.855 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T03:17:41.158 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T03:17:41.168 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T03:17:41.208 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T03:17:41.528 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-portend. 2024-09-17T03:17:41.537 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T03:17:41.578 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T03:17:41.708 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2024-09-17T03:17:41.719 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:41.767 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T03:17:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T03:17:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T03:17:42.225 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T03:17:42.235 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T03:17:42.275 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T03:17:42.312 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T03:17:42.326 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T03:17:42.362 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T03:17:42.632 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T03:17:42.644 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T03:17:42.673 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T03:17:42.719 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T03:17:42.724 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T03:17:42.771 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T03:17:42.977 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T03:17:42.993 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T03:17:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T03:17:43.149 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T03:17:43.158 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T03:17:43.208 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T03:17:43.329 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T03:17:43.342 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T03:17:43.379 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T03:17:43.603 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T03:17:43.613 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T03:17:43.660 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T03:17:43.690 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T03:17:43.703 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T03:17:43.740 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T03:17:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T03:17:44.023 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T03:17:44.052 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T03:17:44.263 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-mako. 2024-09-17T03:17:44.273 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T03:17:44.322 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T03:17:44.354 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-portend. 2024-09-17T03:17:44.367 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T03:17:44.404 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T03:17:44.708 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T03:17:44.722 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T03:17:44.725 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T03:17:44.735 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T03:17:44.762 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T03:17:44.792 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T03:17:45.069 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T03:17:45.081 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T03:17:45.118 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T03:17:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T03:17:45.138 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T03:17:45.179 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T03:17:45.472 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T03:17:45.472 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webob. 2024-09-17T03:17:45.482 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T03:17:45.484 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T03:17:45.523 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T03:17:45.530 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T03:17:45.842 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T03:17:45.854 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T03:17:45.876 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T03:17:45.885 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T03:17:45.891 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T03:17:45.928 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T03:17:46.195 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T03:17:46.207 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T03:17:46.244 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T03:17:46.271 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T03:17:46.280 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T03:17:46.321 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T03:17:46.506 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2024-09-17T03:17:46.518 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T03:17:46.547 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T03:17:46.557 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-paste. 2024-09-17T03:17:46.566 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T03:17:46.598 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T03:17:46.867 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T03:17:46.880 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T03:17:46.917 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T03:17:46.952 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T03:17:46.962 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T03:17:47.010 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T03:17:47.203 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T03:17:47.216 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T03:17:47.253 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T03:17:47.288 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T03:17:47.298 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T03:17:47.329 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T03:17:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2024-09-17T03:17:47.560 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T03:17:47.597 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T03:17:47.624 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T03:17:47.633 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T03:17:47.674 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T03:17:47.934 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T03:17:47.946 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T03:17:47.986 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T03:17:47.994 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T03:17:48.004 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T03:17:48.044 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T03:17:48.262 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T03:17:48.274 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T03:17:48.303 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T03:17:48.407 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T03:17:48.416 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T03:17:48.457 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T03:17:48.614 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2024-09-17T03:17:48.627 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T03:17:48.664 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T03:17:48.793 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-idna. 2024-09-17T03:17:48.802 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T03:17:49.577 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T03:17:49.922 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T03:17:49.932 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T03:17:49.972 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T03:17:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T03:17:50.087 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T03:17:50.126 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T03:17:50.292 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-requests. 2024-09-17T03:17:50.302 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T03:17:50.342 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T03:17:50.412 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T03:17:50.424 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T03:17:50.461 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T03:17:50.678 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T03:17:50.688 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T03:17:50.728 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T03:17:50.749 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T03:17:50.761 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T03:17:50.790 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T03:17:51.031 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T03:17:51.040 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:51.068 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T03:17:51.080 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T03:17:51.081 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:51.118 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T03:17:51.437 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T03:17:51.445 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:51.478 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:51.480 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T03:17:51.493 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T03:17:51.532 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T03:17:51.798 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T03:17:51.807 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T03:17:51.835 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-idna. 2024-09-17T03:17:51.847 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T03:17:51.848 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T03:17:51.885 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T03:17:52.130 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T03:17:52.142 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T03:17:52.150 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T03:17:52.160 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T03:17:52.200 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T03:17:52.204 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T03:17:52.507 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests. 2024-09-17T03:17:52.520 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T03:17:52.545 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T03:17:52.555 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T03:17:52.595 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:52.877 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T03:17:52.890 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T03:17:52.927 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T03:17:52.932 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T03:17:52.941 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:53.213 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T03:17:53.226 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:53.577 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T03:17:53.587 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:53.627 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:53.668 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T03:17:53.680 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:53.718 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:54.020 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T03:17:54.033 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T03:17:54.070 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T03:17:54.373 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T03:17:54.385 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T03:17:54.423 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T03:17:54.439 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph. 2024-09-17T03:17:54.448 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:54.489 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:54.776 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T03:17:54.789 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:54.791 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T03:17:54.801 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:54.826 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:54.841 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:55.203 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T03:17:55.212 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T03:17:55.213 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:55.225 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:55.262 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:55.269 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:55.631 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephadm. 2024-09-17T03:17:55.641 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:55.680 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:55.807 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T03:17:55.819 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:55.848 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:55.958 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T03:17:55.968 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T03:17:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T03:17:56.336 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T03:17:56.346 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T03:17:56.378 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T03:17:56.593 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2024-09-17T03:17:56.605 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:56.644 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:56.962 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T03:17:56.975 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:57.012 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:57.374 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-babel. 2024-09-17T03:17:57.383 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T03:17:57.415 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T03:17:57.424 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T03:17:57.428 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:57.473 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:57.719 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T03:17:57.728 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T03:17:57.769 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T03:17:57.810 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2024-09-17T03:17:57.822 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:17:57.859 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:58.122 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T03:17:58.131 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:58.164 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:58.170 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T03:17:58.183 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T03:17:58.228 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T03:17:58.509 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T03:17:58.518 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T03:17:58.559 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T03:17:58.572 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T03:17:58.585 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T03:17:58.622 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T03:17:58.795 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-routes. 2024-09-17T03:17:58.804 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T03:17:58.845 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T03:17:59.349 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T03:17:59.359 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:17:59.399 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:17:59.602 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-babel. 2024-09-17T03:17:59.614 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T03:17:59.677 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T03:17:59.955 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T03:17:59.967 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T03:18:00.005 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T03:18:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T03:18:00.328 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:00.366 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:00.711 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T03:18:00.724 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T03:18:00.769 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T03:18:01.055 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2024-09-17T03:18:01.068 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T03:18:01.105 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T03:18:01.408 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T03:18:01.421 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:01.458 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T03:18:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T03:18:01.687 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T03:18:02.241 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T03:18:02.250 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T03:18:02.368 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T03:18:02.772 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T03:18:02.781 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T03:18:02.831 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T03:18:03.150 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T03:18:03.160 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T03:18:03.201 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T03:18:03.688 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T03:18:03.701 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T03:18:03.738 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T03:18:03.935 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T03:18:03.945 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:03.985 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:04.240 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T03:18:04.253 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T03:18:04.368 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T03:18:04.755 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T03:18:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T03:18:04.804 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T03:18:04.929 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T03:18:04.940 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T03:18:05.014 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T03:18:05.324 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T03:18:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T03:18:05.374 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T03:18:05.383 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T03:18:05.396 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T03:18:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T03:18:05.745 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T03:18:05.755 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T03:18:05.794 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T03:18:06.106 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T03:18:06.116 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T03:18:06.124 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T03:18:06.136 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:06.155 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T03:18:06.174 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:06.520 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T03:18:06.531 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T03:18:06.603 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T03:18:06.940 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T03:18:06.950 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T03:18:06.990 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T03:18:07.236 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T03:18:07.249 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T03:18:07.293 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T03:18:07.343 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T03:18:07.353 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T03:18:07.401 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T03:18:07.571 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T03:18:07.585 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T03:18:07.629 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T03:18:07.763 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T03:18:07.773 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T03:18:07.813 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T03:18:07.925 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T03:18:07.938 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T03:18:07.974 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T03:18:08.236 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T03:18:08.249 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T03:18:08.286 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T03:18:08.622 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T03:18:08.636 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T03:18:08.643 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T03:18:08.652 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:08.680 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T03:18:08.693 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T03:18:09.014 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T03:18:09.021 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T03:18:09.030 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T03:18:09.058 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T03:18:09.071 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T03:18:09.370 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T03:18:09.383 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T03:18:09.408 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T03:18:09.417 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T03:18:09.428 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T03:18:09.457 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T03:18:09.710 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package jq. 2024-09-17T03:18:09.721 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T03:18:09.741 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T03:18:09.753 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T03:18:09.760 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T03:18:09.797 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T03:18:10.022 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package socat. 2024-09-17T03:18:10.035 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T03:18:10.080 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T03:18:10.400 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T03:18:10.411 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T03:18:10.441 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T03:18:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T03:18:10.457 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:10.495 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:10.780 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T03:18:10.793 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T03:18:10.829 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-test. 2024-09-17T03:18:10.830 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T03:18:10.839 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:10.879 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:11.208 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T03:18:11.221 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T03:18:11.258 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T03:18:11.519 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2024-09-17T03:18:11.533 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T03:18:11.577 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T03:18:11.797 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2024-09-17T03:18:11.810 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T03:18:11.846 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T03:18:12.108 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T03:18:12.122 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T03:18:12.181 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T03:18:12.554 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2024-09-17T03:18:12.566 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:12.603 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:12.816 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T03:18:12.827 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:12.866 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:13.237 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T03:18:13.248 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:13.287 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:13.590 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T03:18:13.601 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T03:18:13.640 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T03:18:13.934 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T03:18:13.945 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:13.983 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:14.253 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T03:18:14.265 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:14.303 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:14.499 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T03:18:14.513 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:14.565 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:14.739 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T03:18:14.750 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:14.814 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:14.945 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T03:18:14.959 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:15.003 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:15.133 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T03:18:15.146 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T03:18:15.183 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T03:18:15.297 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T03:18:15.311 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T03:18:15.355 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T03:18:15.513 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T03:18:15.524 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T03:18:15.563 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T03:18:15.616 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T03:18:15.629 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:15.666 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:15.919 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T03:18:15.932 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T03:18:15.933 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T03:18:15.946 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T03:18:15.975 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T03:18:15.977 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:16.246 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T03:18:16.260 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:16.305 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:16.355 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package pkg-config. 2024-09-17T03:18:16.367 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T03:18:16.405 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T03:18:16.574 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T03:18:16.587 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T03:18:16.623 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T03:18:16.675 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T03:18:16.687 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T03:18:16.725 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T03:18:16.944 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T03:18:16.961 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T03:18:17.002 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T03:18:17.144 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T03:18:17.157 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T03:18:17.203 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T03:18:17.382 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T03:18:17.397 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T03:18:17.440 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T03:18:17.505 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T03:18:17.518 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T03:18:17.572 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T03:18:17.844 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pkg-config. 2024-09-17T03:18:17.859 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T03:18:17.900 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T03:18:17.902 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T03:18:17.912 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T03:18:17.950 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T03:18:18.164 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T03:18:18.178 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T03:18:18.220 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T03:18:18.222 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T03:18:18.233 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T03:18:18.270 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T03:18:18.623 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-py. 2024-09-17T03:18:18.635 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T03:18:18.667 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T03:18:18.672 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T03:18:18.681 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T03:18:18.725 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T03:18:18.986 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T03:18:19.001 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T03:18:19.001 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T03:18:19.013 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T03:18:19.035 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T03:18:19.051 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T03:18:19.297 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T03:18:19.312 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T03:18:19.357 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T03:18:19.513 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T03:18:19.525 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T03:18:19.563 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T03:18:19.667 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T03:18:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T03:18:19.724 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T03:18:19.866 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-toml. 2024-09-17T03:18:19.877 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T03:18:19.907 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T03:18:20.070 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2024-09-17T03:18:20.084 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T03:18:20.127 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T03:18:20.186 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T03:18:20.197 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T03:18:20.235 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T03:18:20.456 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T03:18:20.471 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T03:18:20.564 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T03:18:20.576 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T03:18:20.605 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T03:18:20.729 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T03:18:20.850 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T03:18:20.863 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T03:18:20.899 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T03:18:21.201 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T03:18:21.215 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T03:18:21.259 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T03:18:21.545 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-toml. 2024-09-17T03:18:21.559 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T03:18:21.595 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T03:18:21.595 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package radosgw. 2024-09-17T03:18:21.607 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:21.645 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:21.840 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T03:18:21.854 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T03:18:21.897 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T03:18:22.226 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T03:18:22.240 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T03:18:22.284 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T03:18:22.540 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T03:18:22.553 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:22.587 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T03:18:22.598 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:22.602 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T03:18:22.637 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T03:18:22.898 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T03:18:23.171 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T03:18:23.322 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T03:18:23.349 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2024-09-17T03:18:23.363 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:23.407 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:23.580 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T03:18:23.990 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T03:18:24.268 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T03:18:24.283 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T03:18:24.326 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:24.606 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T03:18:24.625 INFO:teuthology.orchestra.run.smithi099.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T03:18:24.677 INFO:teuthology.orchestra.run.smithi099.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T03:18:24.714 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:24.881 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T03:18:25.032 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T03:18:25.267 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user cephadm....done 2024-09-17T03:18:25.298 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T03:18:25.404 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T03:18:25.664 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T03:18:25.715 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T03:18:25.923 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T03:18:26.075 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T03:18:26.343 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T03:18:26.370 INFO:teuthology.orchestra.run.smithi046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T03:18:26.384 INFO:teuthology.orchestra.run.smithi046.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T03:18:26.426 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:26.594 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T03:18:26.703 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T03:18:26.929 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2024-09-17T03:18:27.013 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T03:18:27.081 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T03:18:27.382 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T03:18:27.412 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T03:18:27.650 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T03:18:27.750 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T03:18:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T03:18:27.918 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T03:18:28.103 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T03:18:28.186 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T03:18:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T03:18:28.437 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T03:18:28.496 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T03:18:28.704 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T03:18:28.824 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T03:18:29.090 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T03:18:29.208 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T03:18:29.357 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:29.451 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T03:18:29.659 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T03:18:29.719 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T03:18:29.759 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T03:18:29.869 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T03:18:29.978 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T03:18:30.004 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T03:18:30.238 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T03:18:30.506 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T03:18:30.513 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T03:18:30.672 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T03:18:30.782 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T03:18:30.932 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T03:18:31.050 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:31.307 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T03:18:31.310 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T03:18:31.337 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T03:18:31.393 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T03:18:31.427 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T03:18:31.536 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T03:18:31.553 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T03:18:31.679 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T03:18:31.895 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T03:18:31.955 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T03:18:32.021 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T03:18:32.081 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T03:18:32.289 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T03:18:32.357 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T03:18:32.599 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T03:18:32.773 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T03:18:32.804 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T03:18:32.860 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T03:18:32.908 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T03:18:32.988 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T03:18:33.034 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T03:18:33.373 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T03:18:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T03:18:33.720 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T03:18:33.766 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T03:18:33.987 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T03:18:34.017 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T03:18:34.088 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T03:18:34.248 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T03:18:34.327 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T03:18:34.452 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T03:18:34.565 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T03:18:34.967 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T03:18:35.164 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T03:18:35.226 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T03:18:35.439 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T03:18:35.565 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T03:18:35.619 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T03:18:35.733 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T03:18:35.954 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T03:18:36.059 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T03:18:36.297 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T03:18:36.493 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T03:18:36.648 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T03:18:36.752 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T03:18:36.924 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T03:18:37.050 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:37.154 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T03:18:37.360 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T03:18:37.464 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T03:18:37.703 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T03:18:37.790 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T03:18:38.158 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T03:18:38.409 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T03:18:38.526 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:38.852 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T03:18:38.917 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T03:18:39.177 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:18:39.212 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T03:18:39.295 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T03:18:39.554 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T03:18:39.697 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T03:18:39.823 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T03:18:40.125 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T03:18:40.384 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T03:18:40.526 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T03:18:40.651 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:18:40.760 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:18:40.777 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T03:18:40.895 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T03:18:41.087 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T03:18:41.179 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T03:18:41.305 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T03:18:41.347 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T03:18:41.431 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T03:18:41.614 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T03:18:41.741 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T03:18:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T03:18:42.000 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T03:18:42.142 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T03:18:42.243 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:18:42.383 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T03:18:42.426 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T03:18:42.544 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T03:18:42.712 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T03:18:42.811 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T03:18:42.930 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T03:18:42.980 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T03:18:43.230 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T03:18:43.256 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T03:18:43.532 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T03:18:43.573 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T03:18:43.832 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T03:18:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T03:18:43.958 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T03:18:44.093 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T03:18:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T03:18:44.181 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T03:18:44.226 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T03:18:44.420 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T03:18:44.519 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T03:18:44.546 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T03:18:44.645 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T03:18:44.939 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T03:18:44.955 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T03:18:45.215 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T03:18:45.281 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T03:18:45.341 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T03:18:45.557 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T03:18:45.617 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:18:45.682 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T03:18:45.743 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T03:18:45.792 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T03:18:45.864 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T03:18:46.086 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T03:18:46.135 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T03:18:46.261 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T03:18:46.396 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T03:18:46.671 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T03:18:46.757 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T03:18:46.858 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T03:18:46.947 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T03:18:47.073 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T03:18:47.126 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T03:18:47.341 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T03:18:47.395 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T03:18:47.483 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T03:18:47.670 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T03:18:47.801 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T03:18:48.064 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T03:18:48.119 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T03:18:48.357 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T03:18:48.497 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T03:18:48.648 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T03:18:48.693 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T03:18:48.827 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:48.937 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:48.941 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T03:18:49.054 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T03:18:49.200 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T03:18:49.391 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:49.460 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T03:18:49.517 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:49.643 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T03:18:49.882 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T03:18:49.961 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:50.062 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:50.188 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:50.196 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T03:18:50.397 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T03:18:50.397 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T03:18:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T03:18:50.659 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:50.759 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:50.885 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T03:18:51.040 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:51.165 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:51.292 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:51.426 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:51.538 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:51.544 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T03:18:51.664 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:51.790 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T03:18:52.083 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:52.184 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:52.285 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:52.529 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T03:18:52.529 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T03:18:52.847 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:52.973 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.099 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.211 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.337 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.471 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.505 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.656 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:53.783 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T03:18:54.015 INFO:teuthology.orchestra.run.smithi099.stdout:Adding group ceph....done 2024-09-17T03:18:54.433 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user ceph....done 2024-09-17T03:18:54.602 INFO:teuthology.orchestra.run.smithi099.stdout:Setting system user ceph properties....done 2024-09-17T03:18:54.619 INFO:teuthology.orchestra.run.smithi099.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T03:18:54.761 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T03:18:55.129 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:55.162 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T03:18:55.255 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:55.397 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:55.523 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:55.633 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:55.733 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:56.278 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2024-09-17T03:18:56.316 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:56.528 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:18:56.528 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:18:57.041 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2024-09-17T03:18:57.201 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:57.209 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2024-09-17T03:18:57.228 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T03:18:57.335 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:57.375 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T03:18:57.774 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T03:18:57.874 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:18:57.875 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:18:58.471 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:58.479 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:58.695 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:18:58.695 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:18:58.833 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T03:18:59.423 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:59.530 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:59.549 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:18:59.746 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T03:18:59.746 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T03:19:00.118 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:19:00.118 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:19:00.349 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:00.562 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T03:19:00.562 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T03:19:00.816 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:01.226 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:01.314 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T03:19:01.529 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T03:19:01.529 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T03:19:02.012 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:02.244 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T03:19:02.244 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T03:19:02.252 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:02.378 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:02.522 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:02.738 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T03:19:02.739 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T03:19:02.905 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:03.127 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T03:19:03.127 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T03:19:03.434 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:03.677 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:03.765 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:03.913 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:04.071 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T03:19:04.072 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T03:19:04.133 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:04.570 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T03:19:04.679 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T03:19:04.725 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:04.823 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T03:19:04.867 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:05.019 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:05.235 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T03:19:05.236 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T03:19:05.598 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T03:19:05.895 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:06.037 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:06.189 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:06.288 INFO:teuthology.orchestra.run.smithi099.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T03:19:06.293 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T03:19:06.331 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T03:19:06.369 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-17T03:19:06.552 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-17T03:19:06.553 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-17T03:19:06.701 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T03:19:06.741 INFO:teuthology.orchestra.run.smithi099.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T03:19:06.742 INFO:teuthology.orchestra.run.smithi099.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T03:19:06.742 INFO:teuthology.orchestra.run.smithi099.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T03:19:06.742 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T03:19:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T03:19:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T03:19:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T03:19:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T03:19:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T03:19:06.743 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T03:19:06.743 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2024-09-17T03:19:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T03:19:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T03:19:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T03:19:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T03:19:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T03:19:06.745 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2024-09-17T03:19:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T03:19:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T03:19:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T03:19:06.801 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T03:19:06.802 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2024-09-17T03:19:06.802 INFO:teuthology.orchestra.run.smithi099.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T03:19:06.802 INFO:teuthology.orchestra.run.smithi099.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T03:19:06.802 INFO:teuthology.orchestra.run.smithi099.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T03:19:06.802 INFO:teuthology.orchestra.run.smithi099.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T03:19:06.802 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T03:19:06.803 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T03:19:06.803 INFO:teuthology.orchestra.run.smithi099.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T03:19:06.804 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2024-09-17T03:19:06.804 INFO:teuthology.orchestra.run.smithi099.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T03:19:06.953 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T03:19:06.990 INFO:teuthology.orchestra.run.smithi099.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T03:19:06.990 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 55.8 MB of archives. 2024-09-17T03:19:06.990 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T03:19:06.990 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T03:19:07.612 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T03:19:07.628 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T03:19:07.694 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T03:19:07.752 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T03:19:07.753 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T03:19:07.754 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T03:19:07.782 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T03:19:07.784 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T03:19:07.787 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T03:19:07.789 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T03:19:07.790 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T03:19:07.792 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T03:19:07.856 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T03:19:07.856 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T03:19:07.857 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T03:19:07.937 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T03:19:08.532 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T03:19:08.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T03:19:08.614 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-09-17T03:19:08.617 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T03:19:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T03:19:08.619 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T03:19:08.622 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T03:19:08.623 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T03:19:08.627 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T03:19:08.629 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T03:19:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T03:19:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T03:19:08.712 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T03:19:08.759 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T03:19:08.770 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T03:19:08.801 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-09-17T03:19:08.802 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-09-17T03:19:08.846 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T03:19:08.849 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T03:19:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T03:19:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T03:19:08.851 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T03:19:08.868 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T03:19:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T03:19:08.999 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T03:19:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T03:19:09.000 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T03:19:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T03:19:09.000 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-09-17T03:19:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T03:19:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T03:19:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T03:19:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T03:19:09.001 INFO:teuthology.orchestra.run.smithi046.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T03:19:09.002 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-09-17T03:19:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T03:19:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T03:19:09.003 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T03:19:09.059 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-09-17T03:19:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T03:19:09.060 INFO:teuthology.orchestra.run.smithi046.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T03:19:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T03:19:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T03:19:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T03:19:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T03:19:09.061 INFO:teuthology.orchestra.run.smithi046.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T03:19:09.062 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2024-09-17T03:19:09.063 INFO:teuthology.orchestra.run.smithi046.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T03:19:09.249 INFO:teuthology.orchestra.run.smithi046.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T03:19:09.250 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 55.8 MB of archives. 2024-09-17T03:19:09.250 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T03:19:09.250 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T03:19:09.354 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T03:19:09.354 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T03:19:09.355 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T03:19:09.355 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T03:19:09.355 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T03:19:09.355 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T03:19:09.368 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T03:19:09.368 INFO:teuthology.orchestra.run.smithi099.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T03:19:09.753 INFO:teuthology.orchestra.run.smithi099.stdout:Preconfiguring packages ... 2024-09-17T03:19:09.873 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T03:19:09.889 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T03:19:09.920 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 55.8 MB in 3s (21.8 MB/s) 2024-09-17T03:19:10.013 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T03:19:10.014 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T03:19:10.015 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T03:19:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T03:19:10.049 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T03:19:10.058 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T03:19:10.060 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T03:19:10.061 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T03:19:10.063 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T03:19:10.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T03:19:10.118 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T03:19:10.118 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T03:19:10.190 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T03:19:10.194 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T03:19:10.199 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T03:19:10.328 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T03:19:10.884 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T03:19:10.887 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T03:19:10.888 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T03:19:10.889 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T03:19:10.890 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T03:19:10.894 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T03:19:10.896 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T03:19:10.942 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T03:19:10.943 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T03:19:10.980 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T03:19:10.993 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T03:19:11.026 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T03:19:11.036 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T03:19:11.112 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T03:19:11.116 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T03:19:11.116 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T03:19:11.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T03:19:11.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T03:19:11.137 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T03:19:11.138 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T03:19:11.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T03:19:11.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T03:19:11.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T03:19:11.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T03:19:11.621 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T03:19:11.621 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T03:19:11.684 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T03:19:11.684 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T03:19:12.086 INFO:teuthology.orchestra.run.smithi046.stdout:Preconfiguring packages ... 2024-09-17T03:19:12.098 INFO:teuthology.orchestra.run.smithi099.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T03:19:12.236 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 55.8 MB in 3s (21.3 MB/s) 2024-09-17T03:19:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package flex. 2024-09-17T03:19:12.438 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T03:19:12.442 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T03:19:12.480 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T03:19:12.507 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T03:19:12.510 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T03:19:12.633 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T03:19:13.000 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T03:19:13.178 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T03:19:13.284 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T03:19:13.591 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T03:19:14.114 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T03:19:14.117 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T03:19:14.243 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T03:19:14.387 INFO:teuthology.orchestra.run.smithi046.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T03:19:14.646 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package flex. 2024-09-17T03:19:14.672 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T03:19:14.719 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T03:19:14.722 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T03:19:14.765 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T03:19:14.782 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T03:19:15.071 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T03:19:15.298 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T03:19:15.466 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package bison. 2024-09-17T03:19:15.466 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T03:19:15.536 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T03:19:15.540 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T03:19:15.585 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T03:19:15.842 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T03:19:15.977 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T03:19:15.993 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T03:19:16.036 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T03:19:16.330 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package dump. 2024-09-17T03:19:16.345 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T03:19:16.380 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T03:19:16.430 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T03:19:16.434 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T03:19:16.557 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T03:19:16.650 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package indent. 2024-09-17T03:19:16.665 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:16.717 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T03:19:16.919 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T03:19:16.970 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T03:19:16.984 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T03:19:17.028 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T03:19:17.055 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T03:19:17.348 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T03:19:17.363 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T03:19:17.369 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T03:19:17.398 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T03:19:17.643 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T03:19:17.658 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T03:19:17.701 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T03:19:17.763 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package bison. 2024-09-17T03:19:17.837 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T03:19:17.841 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T03:19:17.883 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T03:19:18.038 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T03:19:18.053 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T03:19:18.096 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T03:19:18.298 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T03:19:18.341 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T03:19:18.382 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T03:19:18.398 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T03:19:18.440 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:18.685 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T03:19:18.686 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package dump. 2024-09-17T03:19:18.700 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T03:19:18.701 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T03:19:18.734 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T03:19:18.744 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T03:19:19.004 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T03:19:19.019 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T03:19:19.062 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T03:19:19.065 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package indent. 2024-09-17T03:19:19.081 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:19.123 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T03:19:19.359 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T03:19:19.376 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T03:19:19.417 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T03:19:19.746 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T03:19:19.763 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T03:19:19.804 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T03:19:20.099 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T03:19:20.116 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T03:19:20.157 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T03:19:20.448 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T03:19:20.452 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T03:19:20.465 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T03:19:20.468 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T03:19:20.506 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:20.510 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:20.743 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T03:19:20.759 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T03:19:20.762 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T03:19:20.779 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T03:19:20.792 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:20.820 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:21.082 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T03:19:21.098 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T03:19:21.129 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T03:19:21.145 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T03:19:21.165 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T03:19:21.187 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:21.468 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T03:19:21.485 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T03:19:21.530 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T03:19:21.540 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T03:19:21.557 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T03:19:21.600 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T03:19:22.101 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T03:19:22.117 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T03:19:22.159 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T03:19:22.520 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T03:19:22.537 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:22.579 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:22.948 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T03:19:22.964 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:23.006 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:23.121 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T03:19:23.137 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T03:19:23.179 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:23.293 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T03:19:23.309 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:23.342 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:23.467 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T03:19:23.484 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T03:19:23.525 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:23.545 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T03:19:23.560 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T03:19:23.595 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T03:19:23.828 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T03:19:23.844 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T03:19:23.878 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:24.115 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T03:19:24.130 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T03:19:24.173 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T03:19:24.239 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T03:19:24.256 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T03:19:24.297 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T03:19:24.485 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package modemmanager. 2024-09-17T03:19:24.501 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T03:19:24.534 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:24.642 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T03:19:24.658 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T03:19:24.700 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T03:19:25.053 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T03:19:25.070 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:25.099 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package network-manager. 2024-09-17T03:19:25.111 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:25.114 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T03:19:25.157 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T03:19:25.439 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T03:19:25.455 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:25.497 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:25.775 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T03:19:25.789 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ppp. 2024-09-17T03:19:25.792 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:25.805 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T03:19:25.825 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:26.019 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T03:19:26.036 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T03:19:26.069 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T03:19:26.162 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T03:19:26.453 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T03:19:26.467 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T03:19:26.553 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T03:19:26.614 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T03:19:26.631 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T03:19:26.664 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T03:19:26.780 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T03:19:26.795 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T03:19:26.830 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T03:19:26.992 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package modemmanager. 2024-09-17T03:19:27.009 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T03:19:27.050 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:27.066 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T03:19:27.081 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T03:19:27.116 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T03:19:27.615 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager. 2024-09-17T03:19:27.632 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T03:19:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T03:19:27.812 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T03:19:27.828 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:27.872 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:28.331 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ppp. 2024-09-17T03:19:28.348 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T03:19:28.433 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T03:19:28.448 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T03:19:28.535 INFO:teuthology.orchestra.run.smithi099.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T03:19:28.565 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql-common (238) ... 2024-09-17T03:19:28.715 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T03:19:29.023 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T03:19:29.027 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T03:19:29.038 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:29.042 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T03:19:29.084 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:29.085 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T03:19:29.321 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T03:19:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T03:19:29.379 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T03:19:29.682 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T03:19:29.699 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T03:19:29.740 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T03:19:30.019 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T03:19:30.036 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:30.069 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:30.084 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql. 2024-09-17T03:19:30.099 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T03:19:30.151 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql (14+238) ... 2024-09-17T03:19:30.394 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T03:19:30.409 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T03:19:30.453 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T03:19:30.649 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T03:19:30.665 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T03:19:30.696 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T03:19:30.711 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T03:19:30.726 INFO:teuthology.orchestra.run.smithi046.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T03:19:30.754 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T03:19:30.755 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-common (238) ... 2024-09-17T03:19:31.074 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T03:19:31.089 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T03:19:31.132 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T03:19:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T03:19:31.214 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T03:19:31.258 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:31.443 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package tcl. 2024-09-17T03:19:31.459 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T03:19:31.531 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T03:19:31.853 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T03:19:31.862 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T03:19:31.904 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T03:19:32.283 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T03:19:32.299 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T03:19:32.341 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T03:19:32.383 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql. 2024-09-17T03:19:32.399 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T03:19:32.441 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql (14+238) ... 2024-09-17T03:19:32.693 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T03:19:32.710 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T03:19:32.719 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T03:19:32.735 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T03:19:32.752 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T03:19:32.785 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T03:19:33.004 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T03:19:33.021 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T03:19:33.062 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T03:19:33.063 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T03:19:33.308 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T03:19:33.381 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T03:19:33.398 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T03:19:33.434 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T03:19:33.439 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T03:19:33.535 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:33.645 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T03:19:33.717 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl. 2024-09-17T03:19:33.734 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T03:19:33.770 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T03:19:33.789 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T03:19:33.896 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:34.086 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T03:19:34.103 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T03:19:34.136 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T03:19:34.482 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T03:19:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T03:19:34.540 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T03:19:34.851 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T03:19:34.927 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T03:19:34.944 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T03:19:34.977 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T03:19:34.985 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T03:19:35.272 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T03:19:35.337 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T03:19:35.337 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T03:19:35.524 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T03:19:35.650 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T03:19:35.785 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:35.902 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T03:19:36.003 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T03:19:36.046 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T03:19:36.129 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:36.155 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T03:19:36.256 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T03:19:36.399 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T03:19:36.524 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T03:19:36.659 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:36.802 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T03:19:36.928 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T03:19:37.107 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T03:19:37.201 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T03:19:37.541 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T03:19:37.588 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T03:19:37.588 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T03:19:37.688 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T03:19:37.789 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T03:19:37.853 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T03:19:38.338 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T03:19:38.413 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T03:19:38.464 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T03:19:38.514 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T03:19:38.588 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T03:19:38.648 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T03:19:38.707 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T03:19:38.774 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T03:19:38.797 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T03:19:38.867 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T03:19:38.942 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:38.993 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T03:19:39.068 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T03:19:39.178 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T03:19:39.194 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T03:19:39.304 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:39.430 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T03:19:39.614 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T03:19:39.715 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T03:19:39.857 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T03:19:39.983 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T03:19:40.087 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T03:19:40.185 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:40.311 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T03:19:40.315 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:19:40.315 INFO:teuthology.orchestra.run.smithi099.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T03:19:40.315 INFO:teuthology.orchestra.run.smithi099.stdout:which means they are currently configured by ifupdown: 2024-09-17T03:19:40.315 INFO:teuthology.orchestra.run.smithi099.stdout:- enp3s0f1 2024-09-17T03:19:40.315 INFO:teuthology.orchestra.run.smithi099.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T03:19:40.316 INFO:teuthology.orchestra.run.smithi099.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T03:19:40.316 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:19:40.437 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T03:19:40.447 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T03:19:40.546 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T03:19:40.655 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T03:19:40.729 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T03:19:40.799 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T03:19:40.868 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T03:19:40.924 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T03:19:41.109 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T03:19:41.235 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:41.277 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T03:19:41.361 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T03:19:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T03:19:41.647 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T03:19:41.992 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:42.152 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:42.208 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:19:42.208 INFO:teuthology.orchestra.run.smithi046.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T03:19:42.208 INFO:teuthology.orchestra.run.smithi046.stdout:which means they are currently configured by ifupdown: 2024-09-17T03:19:42.213 INFO:teuthology.orchestra.run.smithi046.stdout:- ens1f0 2024-09-17T03:19:42.213 INFO:teuthology.orchestra.run.smithi046.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T03:19:42.213 INFO:teuthology.orchestra.run.smithi046.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T03:19:42.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:19:42.312 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T03:19:42.346 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T03:19:42.437 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql-common (238) ... 2024-09-17T03:19:42.748 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T03:19:43.160 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T03:19:43.410 INFO:teuthology.orchestra.run.smithi099.stdout:Adding user postgres to group ssl-cert 2024-09-17T03:19:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:19:43.658 INFO:teuthology.orchestra.run.smithi099.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T03:19:43.911 INFO:teuthology.orchestra.run.smithi099.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T03:19:43.911 INFO:teuthology.orchestra.run.smithi099.stdout:Removing obsolete dictionary files: 2024-09-17T03:19:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T03:19:44.084 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T03:19:44.210 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T03:19:44.336 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-common (238) ... 2024-09-17T03:19:44.643 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T03:19:45.121 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:45.271 INFO:teuthology.orchestra.run.smithi046.stdout:Adding user postgres to group ssl-cert 2024-09-17T03:19:45.452 INFO:teuthology.orchestra.run.smithi099.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T03:19:45.481 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T03:19:45.501 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:19:45.501 INFO:teuthology.orchestra.run.smithi046.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T03:19:45.529 INFO:teuthology.orchestra.run.smithi099.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T03:19:45.529 INFO:teuthology.orchestra.run.smithi099.stdout:This user must also own the server process. 2024-09-17T03:19:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:19:45.530 INFO:teuthology.orchestra.run.smithi099.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T03:19:45.530 INFO:teuthology.orchestra.run.smithi099.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T03:19:45.530 INFO:teuthology.orchestra.run.smithi099.stdout:The default text search configuration will be set to "english". 2024-09-17T03:19:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:19:45.530 INFO:teuthology.orchestra.run.smithi099.stdout:Data page checksums are disabled. 2024-09-17T03:19:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:19:45.531 INFO:teuthology.orchestra.run.smithi099.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T03:19:45.532 INFO:teuthology.orchestra.run.smithi099.stdout:creating subdirectories ... ok 2024-09-17T03:19:45.532 INFO:teuthology.orchestra.run.smithi099.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T03:19:45.554 INFO:teuthology.orchestra.run.smithi099.stdout:selecting default max_connections ... 100 2024-09-17T03:19:45.587 INFO:teuthology.orchestra.run.smithi099.stdout:selecting default shared_buffers ... 128MB 2024-09-17T03:19:45.601 INFO:teuthology.orchestra.run.smithi099.stdout:selecting default time zone ... Etc/UTC 2024-09-17T03:19:45.602 INFO:teuthology.orchestra.run.smithi099.stdout:creating configuration files ... ok 2024-09-17T03:19:45.762 INFO:teuthology.orchestra.run.smithi099.stdout:running bootstrap script ... ok 2024-09-17T03:19:45.773 INFO:teuthology.orchestra.run.smithi046.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T03:19:45.773 INFO:teuthology.orchestra.run.smithi046.stdout:Removing obsolete dictionary files: 2024-09-17T03:19:46.423 INFO:teuthology.orchestra.run.smithi099.stdout:performing post-bootstrap initialization ... ok 2024-09-17T03:19:46.535 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T03:19:47.011 INFO:teuthology.orchestra.run.smithi099.stdout:syncing data to disk ... ok 2024-09-17T03:19:47.020 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T03:19:47.467 INFO:teuthology.orchestra.run.smithi046.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T03:19:47.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T03:19:47.519 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout:This user must also own the server process. 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout:The default text search configuration will be set to "english". 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout:Data page checksums are disabled. 2024-09-17T03:19:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:19:47.546 INFO:teuthology.orchestra.run.smithi046.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T03:19:47.546 INFO:teuthology.orchestra.run.smithi046.stdout:creating subdirectories ... ok 2024-09-17T03:19:47.546 INFO:teuthology.orchestra.run.smithi046.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T03:19:47.571 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default max_connections ... 100 2024-09-17T03:19:47.605 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default shared_buffers ... 128MB 2024-09-17T03:19:47.619 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default time zone ... Etc/UTC 2024-09-17T03:19:47.620 INFO:teuthology.orchestra.run.smithi046.stdout:creating configuration files ... ok 2024-09-17T03:19:47.790 INFO:teuthology.orchestra.run.smithi046.stdout:running bootstrap script ... ok 2024-09-17T03:19:48.406 INFO:teuthology.orchestra.run.smithi046.stdout:performing post-bootstrap initialization ... ok 2024-09-17T03:19:48.947 INFO:teuthology.orchestra.run.smithi046.stdout:syncing data to disk ... ok 2024-09-17T03:19:49.484 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T03:19:50.589 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:50.704 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T03:19:50.805 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up postgresql (14+238) ... 2024-09-17T03:19:50.931 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:51.057 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:51.316 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T03:19:51.317 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T03:19:52.042 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T03:19:52.151 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T03:19:52.504 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T03:19:52.548 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:52.656 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T03:19:52.817 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T03:19:52.818 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql (14+238) ... 2024-09-17T03:19:52.913 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T03:19:53.039 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T03:19:53.299 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T03:19:53.299 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T03:19:54.091 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T03:19:54.115 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T03:19:54.192 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T03:19:54.215 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T03:19:54.744 INFO:teuthology.orchestra.run.smithi099.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T03:19:54.750 DEBUG:teuthology.parallel:result is None 2024-09-17T03:19:54.845 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T03:19:54.996 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T03:19:56.457 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T03:19:56.549 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T03:19:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T03:19:57.110 DEBUG:teuthology.parallel:result is None 2024-09-17T03:19:57.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:19:57.262 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2024-09-17T03:19:57.285 INFO:teuthology.orchestra.run.smithi046.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:19:57.285 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:19:57.286 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T03:19:57.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:19:57.426 DEBUG:teuthology.orchestra.run.smithi099:> dpkg-query -W -f '${Version}' ceph 2024-09-17T03:19:57.450 INFO:teuthology.orchestra.run.smithi099.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:19:57.450 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T03:19:57.450 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T03:19:57.452 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T03:19:57.453 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:19:57.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T03:19:57.473 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:19:57.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T03:19:57.514 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T03:19:57.514 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:19:57.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T03:19:57.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T03:19:57.603 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:19:57.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T03:19:57.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T03:19:57.685 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T03:19:57.685 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:19:57.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T03:19:57.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T03:19:57.772 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:19:57.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T03:19:57.793 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T03:19:57.853 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T03:19:57.853 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:19:57.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T03:19:57.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T03:19:57.935 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:19:57.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T03:19:57.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T03:19:58.021 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T03:19:58.099 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T03:19:58.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-09-17T03:19:58.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-09-17T03:19:58.123 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T03:19:58.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T03:19:58.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T03:19:58.185 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T03:19:58.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T03:19:58.223 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T03:19:58.260 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T03:19:58.260 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T03:19:58.260 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T03:19:58.260 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:19:58.260 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T03:19:58.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T03:19:58.391 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:19:58.392 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T03:19:58.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T03:19:58.495 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T03:19:58.496 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T03:19:58.496 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908415', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIkyP6uczF9Odc6nUmWEDkznd4+3GSexQxw/po+/jO+QSbTqfrHBHCT5gqw8cdPWoaUn8vQUFmzBeworePvjNk=', 'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqvrmwqsgWn70v84FkhDINnJA6jQh0nOktWk4hHdOD055amKJCCLamqdWjI7+blq+IlvOjgV2YDOjYivYebbp0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T03:19:58.497 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T03:19:58.500 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T03:19:58.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T03:19:58.556 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T03:19:58.576 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:19:58.576 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T03:19:58.629 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T03:19:58.629 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-17T03:19:58.681 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T03:19:58.681 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:58.682 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 906 Links: 1 2024-09-17T03:19:58.682 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:58.682 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-17 03:13:16.736675330 +0000 2024-09-17T03:19:58.682 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-17 03:13:16.460677796 +0000 2024-09-17T03:19:58.682 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-17 03:13:16.460677796 +0000 2024-09-17T03:19:58.682 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-17 03:13:16.460677796 +0000 2024-09-17T03:19:58.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T03:19:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-17T03:19:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-17T03:19:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000253373 s, 2.0 MB/s 2024-09-17T03:19:58.744 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T03:19:58.795 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-17T03:19:58.845 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 924 Links: 1 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-17 03:13:17.264670615 +0000 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-17 03:13:17.008672901 +0000 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-17 03:13:17.008672901 +0000 2024-09-17T03:19:58.846 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-17 03:13:17.008672901 +0000 2024-09-17T03:19:58.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T03:19:58.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-17T03:19:58.911 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-17T03:19:58.911 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000245969 s, 2.1 MB/s 2024-09-17T03:19:58.913 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T03:19:58.967 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-17T03:19:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T03:19:59.017 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:59.017 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 938 Links: 1 2024-09-17T03:19:59.017 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:59.017 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-17 03:13:17.804665792 +0000 2024-09-17T03:19:59.017 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-17 03:13:17.524668293 +0000 2024-09-17T03:19:59.018 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-17 03:13:17.524668293 +0000 2024-09-17T03:19:59.018 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-17 03:13:17.524668293 +0000 2024-09-17T03:19:59.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T03:19:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-17T03:19:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-17T03:19:59.079 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000252444 s, 2.0 MB/s 2024-09-17T03:19:59.081 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T03:19:59.135 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-17T03:19:59.189 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 958 Links: 1 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-17 03:13:18.336661039 +0000 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-17 03:13:18.060663504 +0000 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-17 03:13:18.060663504 +0000 2024-09-17T03:19:59.190 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-17 03:13:18.060663504 +0000 2024-09-17T03:19:59.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T03:19:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-17T03:19:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-17T03:19:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000262034 s, 2.0 MB/s 2024-09-17T03:19:59.254 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T03:19:59.307 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:19:59.308 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:19:59.308 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T03:19:59.316 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T03:19:59.316 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-17T03:19:59.366 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T03:19:59.366 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:59.366 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 892 Links: 1 2024-09-17T03:19:59.366 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:59.366 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-17 03:13:12.178854135 +0000 2024-09-17T03:19:59.367 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-17 03:13:11.918856458 +0000 2024-09-17T03:19:59.367 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-17 03:13:11.918856458 +0000 2024-09-17T03:19:59.367 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-17 03:13:11.918856458 +0000 2024-09-17T03:19:59.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T03:19:59.428 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-17T03:19:59.428 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-17T03:19:59.428 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000424596 s, 1.2 MB/s 2024-09-17T03:19:59.430 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T03:19:59.483 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-17T03:19:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 908 Links: 1 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-17 03:13:12.766848883 +0000 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-17 03:13:12.502851241 +0000 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-17 03:13:12.502851241 +0000 2024-09-17T03:19:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-17 03:13:12.502851241 +0000 2024-09-17T03:19:59.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T03:19:59.595 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-17T03:19:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-17T03:19:59.596 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000376388 s, 1.4 MB/s 2024-09-17T03:19:59.597 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T03:19:59.652 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 926 Links: 1 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-17 03:13:13.374843452 +0000 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-17 03:13:13.114845774 +0000 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-17 03:13:13.114845774 +0000 2024-09-17T03:19:59.706 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-17 03:13:13.114845774 +0000 2024-09-17T03:19:59.707 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T03:19:59.768 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-17T03:19:59.768 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-17T03:19:59.768 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000379374 s, 1.3 MB/s 2024-09-17T03:19:59.770 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T03:19:59.823 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 945 Links: 1 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-17 03:13:13.998837879 +0000 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-17 03:13:13.706840486 +0000 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-17 03:13:13.706840486 +0000 2024-09-17T03:19:59.874 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-17 03:13:13.706840486 +0000 2024-09-17T03:19:59.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T03:19:59.940 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-17T03:19:59.940 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-17T03:19:59.940 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000395514 s, 1.3 MB/s 2024-09-17T03:19:59.942 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T03:19:59.996 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:19:59.996 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T03:19:59.996 INFO:tasks.ceph:Generating config... 2024-09-17T03:20:00.000 INFO:tasks.ceph:[client] pid file = 2024-09-17T03:20:00.001 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-17T03:20:00.001 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-17T03:20:00.001 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-17T03:20:00.001 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T03:20:00.001 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T03:20:00.002 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T03:20:00.002 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T03:20:00.002 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T03:20:00.002 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T03:20:00.002 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T03:20:00.002 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T03:20:00.003 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T03:20:00.003 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T03:20:00.003 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T03:20:00.003 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T03:20:00.003 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T03:20:00.003 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T03:20:00.004 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T03:20:00.004 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T03:20:00.004 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T03:20:00.004 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T03:20:00.004 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T03:20:00.004 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T03:20:00.005 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T03:20:00.005 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T03:20:00.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T03:20:00.068 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T03:20:00.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T03:20:00.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T03:20:00.252 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.99'), ('mon.c', '[v2:172.21.15.99:3301,v1:172.21.15.99:6790]')] 2024-09-17T03:20:00.252 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T03:20:00.254 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:00.254 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T03:20:00.305 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.99 --addv c '[v2:172.21.15.99:3301,v1:172.21.15.99:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:00.409 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:00.409 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 6c25af6b-3689-4426-acb2-6ebe6de6efd5 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 6c25af6b-3689-4426-acb2-6ebe6de6efd5 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2024-09-17T03:20:00.412822+0000 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-17T03:20:00.412822+0000 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 17 (quincy) 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] mon.c 2024-09-17T03:20:00.410 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T03:20:00.412 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T03:20:00.419 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6c25af6b-3689-4426-acb2-6ebe6de6efd5... 2024-09-17T03:20:00.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T03:20:00.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T03:20:00.503 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-17T03:20:00.503 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-09-17T03:20:00.504 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-09-17T03:20:00.505 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-09-17T03:20:00.506 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-09-17T03:20:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790]" 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 6c25af6b-3689-4426-acb2-6ebe6de6efd5 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-09-17T03:20:00.508 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-09-17T03:20:00.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-09-17T03:20:00.510 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-17T03:20:00.511 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug finisher = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-09-17T03:20:00.512 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T03:20:00.513 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: mon down mkfs grace = 300 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-09-17T03:20:00.514 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-09-17T03:20:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2024-09-17T03:20:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T03:20:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2024-09-17T03:20:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2024-09-17T03:20:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.46,172.21.15.99,[v2:172.21.15.99:3301,v1:172.21.15.99:6790]" 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on skipped message = False 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 6c25af6b-3689-4426-acb2-6ebe6de6efd5 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2024-09-17T03:20:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug trim objects = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2024-09-17T03:20:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = bitmap 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-17T03:20:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-17T03:20:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug finisher = 20 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2024-09-17T03:20:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2024-09-17T03:20:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: mon down mkfs grace = 300 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = "" 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2024-09-17T03:20:00.526 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2024-09-17T03:20:00.527 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2024-09-17T03:20:00.527 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2024-09-17T03:20:00.527 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2024-09-17T03:20:00.527 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2024-09-17T03:20:00.528 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T03:20:00.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T03:20:00.637 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T03:20:00.637 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:00.637 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T03:20:00.646 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:00.646 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T03:20:00.701 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:20:00.701 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:00.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T03:20:00.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T03:20:00.777 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:00.777 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:00.833 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:20:00.834 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:00.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T03:20:00.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T03:20:00.870 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:00.870 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:00.921 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T03:20:00.921 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T03:20:00.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T03:20:00.988 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T03:20:00.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T03:20:01.064 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T03:20:01.083 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T03:20:01.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T03:20:01.149 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T03:20:01.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T03:20:01.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T03:20:01.282 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T03:20:01.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T03:20:01.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T03:20:01.374 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T03:20:01.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T03:20:01.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T03:20:01.504 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T03:20:01.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T03:20:01.529 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T03:20:01.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T03:20:01.583 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T03:20:01.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T03:20:01.653 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T03:20:01.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T03:20:01.722 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T03:20:01.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T03:20:01.790 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T03:20:01.807 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T03:20:01.807 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T03:20:01.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T03:20:01.827 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:01.827 INFO:tasks.ceph:role: osd.0 2024-09-17T03:20:01.827 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:20:01.827 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:01.888 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:01.889 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:01.889 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:01.913 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-17T03:20:02.037 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-17T03:20:02.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T03:20:02.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T03:20:02.165 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:02.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:02.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T03:20:02.228 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:02.228 INFO:tasks.ceph:role: osd.1 2024-09-17T03:20:02.228 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:20:02.228 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:02.288 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:02.312 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-17T03:20:02.315 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-17T03:20:02.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T03:20:02.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T03:20:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:02.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:02.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T03:20:02.504 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:02.504 INFO:tasks.ceph:role: osd.2 2024-09-17T03:20:02.504 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:20:02.504 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:02.564 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:02.565 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:02.565 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:02.589 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-17T03:20:02.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-17T03:20:02.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T03:20:02.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T03:20:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:02.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:02.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T03:20:02.779 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:02.780 INFO:tasks.ceph:role: osd.3 2024-09-17T03:20:02.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:20:02.780 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T03:20:02.839 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:02.839 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:02.839 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:02.840 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:02.865 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-09-17T03:20:02.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-09-17T03:20:02.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T03:20:02.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T03:20:03.015 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:03.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:03.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:03.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:03.162+0000 7fd31042e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T03:20:03.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:03.162+0000 7fd31042e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T03:20:03.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:03.162+0000 7fd31042e7c0 -1 bdev(0x55c10c578a00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:03.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:03.162+0000 7fd31042e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T03:20:06.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T03:20:06.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:06.538+0000 7facfeb517c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T03:20:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:06.538+0000 7facfeb517c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T03:20:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:06.538+0000 7facfeb517c0 -1 bdev(0x563642ee2a00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:06.538+0000 7facfeb517c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T03:20:09.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T03:20:09.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:09.918+0000 7f9fa249e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T03:20:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:09.918+0000 7f9fa249e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T03:20:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:09.918+0000 7f9fa249e7c0 -1 bdev(0x55a146f9ea00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:09.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:09.918+0000 7f9fa249e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T03:20:13.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T03:20:13.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:13.210+0000 7f8a2b82b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T03:20:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:13.210+0000 7f8a2b82b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T03:20:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:13.214+0000 7f8a2b82b7c0 -1 bdev(0x55d94a9aaa00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:13.214+0000 7f8a2b82b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T03:20:16.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T03:20:16.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T03:20:16.765 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:16.765 INFO:tasks.ceph:role: osd.4 2024-09-17T03:20:16.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:20:16.765 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:16.837 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:17.246 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-17T03:20:17.250 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-17T03:20:17.250 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T03:20:17.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T03:20:17.403 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:17.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:17.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T03:20:17.468 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:17.469 INFO:tasks.ceph:role: osd.5 2024-09-17T03:20:17.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:20:17.469 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T03:20:17.536 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:17.536 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:17.536 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:17.537 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:17.898 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-17T03:20:17.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-17T03:20:17.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T03:20:17.971 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T03:20:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:18.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:18.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T03:20:18.093 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:18.093 INFO:tasks.ceph:role: osd.6 2024-09-17T03:20:18.093 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:20:18.093 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T03:20:18.164 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:18.165 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:18.557 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-17T03:20:18.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-17T03:20:18.560 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T03:20:18.659 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T03:20:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:18.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:18.676 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T03:20:18.737 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:20:18.737 INFO:tasks.ceph:role: osd.7 2024-09-17T03:20:18.737 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:20:18.737 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T03:20:18.809 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:20:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:20:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:20:18.810 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:20:19.200 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-17T03:20:19.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-17T03:20:19.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T03:20:19.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T03:20:19.363 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T03:20:19.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:20:19.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:19.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:19.504+0000 7fc6bee2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T03:20:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:19.508+0000 7fc6bee2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T03:20:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:19.508+0000 7fc6bee2b7c0 -1 bdev(0x55947d716a00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:19.508+0000 7fc6bee2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T03:20:22.751 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T03:20:22.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:22.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:22.880+0000 7f9f3e3037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T03:20:22.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:22.880+0000 7f9f3e3037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T03:20:22.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:22.880+0000 7f9f3e3037c0 -1 bdev(0x5643da496a00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:22.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:22.880+0000 7f9f3e3037c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T03:20:26.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T03:20:26.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:26.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:26.276+0000 7f73f282f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T03:20:26.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:26.276+0000 7f73f282f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T03:20:26.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:26.276+0000 7f73f282f7c0 -1 bdev(0x55f5c7b94a00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:26.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:26.276+0000 7f73f282f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T03:20:28.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:20:28.501 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:20:29.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T03:20:29.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:29.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:29.672+0000 7fee14d637c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T03:20:29.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:29.672+0000 7fee14d637c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T03:20:29.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:29.672+0000 7fee14d637c0 -1 bdev(0x558ea2e2ca00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T03:20:29.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:29.672+0000 7fee14d637c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T03:20:32.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T03:20:32.915 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T03:20:32.915 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:32.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T03:20:32.938 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:32.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T03:20:33.006 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T03:20:33.074 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T03:20:33.138 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T03:20:33.206 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.206 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T03:20:33.270 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T03:20:33.334 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T03:20:33.355 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T03:20:33.416 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T03:20:33.476 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.476 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T03:20:33.536 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T03:20:33.596 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T03:20:33.656 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T03:20:33.716 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.716 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T03:20:33.725 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:33.725 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T03:20:33.778 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.778 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T03:20:33.786 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:33.786 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T03:20:33.836 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T03:20:33.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T03:20:33.839 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw== 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w== 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g== 2024-09-17T03:20:33.880 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA== 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q== 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA== 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA== 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw== 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g== 2024-09-17T03:20:33.881 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg== 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ== 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA== 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g== 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A== 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:20:33.882 INFO:teuthology.orchestra.run.smithi046.stdout:[client.2] 2024-09-17T03:20:33.883 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg== 2024-09-17T03:20:33.883 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-09-17T03:20:33.883 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA== 2024-09-17T03:20:33.883 INFO:teuthology.orchestra.run.smithi046.stdout:[client.3] 2024-09-17T03:20:33.883 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ== 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw== 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w== 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g== 2024-09-17T03:20:33.893 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg== 2024-09-17T03:20:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[client.2] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA== 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:[client.3] 2024-09-17T03:20:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ== 2024-09-17T03:20:33.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:20:33.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:20:34.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.151 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:20:34.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:20:34.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:20:34.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:20:34.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:34.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:34.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:34.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:35.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:35.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:35.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:35.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:20:35.182 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T03:20:35.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T03:20:35.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T03:20:35.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T03:20:35.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T03:20:35.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T03:20:35.492 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T03:20:35.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T03:20:35.575 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T03:20:35.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T03:20:35.718 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:35.722 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:20:35.772 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T03:20:35.773 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T03:20:35.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T03:20:35.776 INFO:tasks.ceph.mon.a:Started 2024-09-17T03:20:35.776 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T03:20:35.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T03:20:35.815 INFO:tasks.ceph.mon.b:Started 2024-09-17T03:20:35.815 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T03:20:35.816 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T03:20:35.818 INFO:tasks.ceph.mon.c:Started 2024-09-17T03:20:35.818 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T03:20:35.819 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T03:20:35.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T03:20:35.822 INFO:tasks.ceph.mgr.y:Started 2024-09-17T03:20:35.822 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T03:20:35.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T03:20:35.827 INFO:tasks.ceph.mgr.x:Started 2024-09-17T03:20:35.827 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T03:20:35.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T03:20:35.830 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T03:20:35.830 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T03:20:35.875 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T03:20:35.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T03:20:35.924 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T03:20:35.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T03:20:35.967 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T03:20:35.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T03:20:36.016 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T03:20:36.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T03:20:36.065 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T03:20:36.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T03:20:36.116 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T03:20:36.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T03:20:36.164 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T03:20:36.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T03:20:36.213 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T03:20:36.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T03:20:36.263 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T03:20:36.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T03:20:36.308 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T03:20:36.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T03:20:36.357 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T03:20:36.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T03:20:36.403 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T03:20:36.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T03:20:36.447 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T03:20:36.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T03:20:36.493 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T03:20:36.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T03:20:36.541 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T03:20:36.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T03:20:36.588 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T03:20:36.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T03:20:36.631 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T03:20:36.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T03:20:36.675 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T03:20:36.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T03:20:36.719 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T03:20:36.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T03:20:36.763 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T03:20:36.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T03:20:36.808 DEBUG:tasks.ceph:set 22 configs 2024-09-17T03:20:36.808 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.628+0000 7f33ffe00640 1 -- 172.21.15.46:0/1570359392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f33d4005320 con 0x7f33e4003a60 2024-09-17T03:20:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 -- 172.21.15.46:0/3180470424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc004fa0 msgr2=0x7f6ddc15f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 --2- 172.21.15.46:0/3180470424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc004fa0 0x7f6ddc15f010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000b080 tx=0x7f6dc8013f10 comp rx=0 tx=0).stop 2024-09-17T03:20:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 -- 172.21.15.46:0/3180470424 shutdown_connections 2024-09-17T03:20:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 --2- 172.21.15.46:0/3180470424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ddc0d1190 0x7f6ddc15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 --2- 172.21.15.46:0/3180470424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc004fa0 0x7f6ddc15f010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 --2- 172.21.15.46:0/3180470424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6dbc0041d0 0x7f6ddc1f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 -- 172.21.15.46:0/3180470424 >> 172.21.15.46:0/3180470424 conn(0x7f6ddc0b41f0 msgr2=0x7f6ddc06f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 -- 172.21.15.46:0/3180470424 shutdown_connections 2024-09-17T03:20:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.632+0000 7f6ddbe00640 1 -- 172.21.15.46:0/3180470424 wait complete. 2024-09-17T03:20:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a52a00640 1 -- 172.21.15.46:0/2282690081 <== mon.1 v2:172.21.15.99:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v18)=0 v18) ==== 108+0+0 (secure 0 0 0) 0x7f2a4c0036d0 con 0x7f2a3c003680 2024-09-17T03:20:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 -- 172.21.15.46:0/2282690081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3c003680 msgr2=0x7f2a5c140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 --2- 172.21.15.46:0/2282690081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3c003680 0x7f2a5c140760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c011b40 tx=0x7f2a4c012f10 comp rx=0 tx=0).stop 2024-09-17T03:20:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 -- 172.21.15.46:0/2282690081 shutdown_connections 2024-09-17T03:20:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 --2- 172.21.15.46:0/2282690081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a40004f50 0x7f2a5c143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 --2- 172.21.15.46:0/2282690081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a400040c0 0x7f2a5c142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 --2- 172.21.15.46:0/2282690081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a3c003680 0x7f2a5c140760 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c011b40 tx=0x7f2a4c012f10 comp rx=0 tx=0).stop 2024-09-17T03:20:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 -- 172.21.15.46:0/2282690081 >> 172.21.15.46:0/2282690081 conn(0x7f2a5c0b35f0 msgr2=0x7f2a5c0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 -- 172.21.15.46:0/2282690081 shutdown_connections 2024-09-17T03:20:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.636+0000 7f2a61800640 1 -- 172.21.15.46:0/2282690081 wait complete. 2024-09-17T03:20:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.640+0000 7f33f6a00640 1 -- 172.21.15.46:0/1570359392 <== mon.1 v2:172.21.15.99:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33f0003670 con 0x7f33e4003a60 2024-09-17T03:20:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.640+0000 7f33f6a00640 1 -- 172.21.15.46:0/1570359392 <== mon.1 v2:172.21.15.99:3300/0 12 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v19)=0 v19) ==== 123+0+0 (secure 0 0 0) 0x7f33f0003a50 con 0x7f33e4003a60 2024-09-17T03:20:41.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 -- 172.21.15.46:0/1570359392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33e4003a60 msgr2=0x7f34001ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1570359392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33e4003a60 0x7f34001ffee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005fd0 tx=0x7f33f0013e90 comp rx=0 tx=0).stop 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 -- 172.21.15.46:0/1570359392 shutdown_connections 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1570359392 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34000746e0 0x7f34000b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1570359392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e4004810 0x7f34000b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 --2- 172.21.15.46:0/1570359392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33e4003a60 0x7f34001ffee0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 -- 172.21.15.46:0/1570359392 >> 172.21.15.46:0/1570359392 conn(0x7f34000b41f0 msgr2=0x7f34000cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 -- 172.21.15.46:0/1570359392 shutdown_connections 2024-09-17T03:20:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:41.644+0000 7f33ffe00640 1 -- 172.21.15.46:0/1570359392 wait complete. 2024-09-17T03:20:41.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.671 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi046') 2024-09-17T03:20:41.671 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 -- 172.21.15.46:0/3548415480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 msgr2=0x7fc7f0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 --2- 172.21.15.46:0/3548415480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc7f0004c20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005f20 tx=0x7fc7f401e830 comp rx=0 tx=0).stop 2024-09-17T03:20:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 -- 172.21.15.46:0/3548415480 shutdown_connections 2024-09-17T03:20:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 --2- 172.21.15.46:0/3548415480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc7f0004c20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 -- 172.21.15.46:0/3548415480 >> 172.21.15.46:0/3548415480 conn(0x7fc8080708c0 msgr2=0x7fc808070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 -- 172.21.15.46:0/3548415480 shutdown_connections 2024-09-17T03:20:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.388+0000 7fc80fa00640 1 -- 172.21.15.46:0/3548415480 wait complete. 2024-09-17T03:20:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 Processor -- start 2024-09-17T03:20:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 -- start start 2024-09-17T03:20:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc80813efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8081455c0 0x7fc8081fc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081fcf00 0x7fc8081fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc80815d920 con 0x7fc7f0004810 2024-09-17T03:20:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc80815d7a0 con 0x7fc8081fcf00 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.392+0000 7fc80fa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc808071c00 con 0x7fc8081455c0 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc80813efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8081455c0 0x7fc8081fc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc80813efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41254/0 (socket says 172.21.15.46:41254) 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80d000640 1 -- 172.21.15.46:0/3823438979 learned_addr learned my addr 172.21.15.46:0/3823438979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80da00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081fcf00 0x7fc8081fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 -- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081fcf00 msgr2=0x7fc8081fd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081fcf00 0x7fc8081fd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 -- 172.21.15.46:0/3823438979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 msgr2=0x7fc80813efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc80813efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 -- 172.21.15.46:0/3823438979 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc808154ed0 con 0x7fc8081455c0 2024-09-17T03:20:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80da00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081fcf00 0x7fc8081fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc807e00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8081455c0 0x7fc8081fc9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4004000 tx=0x7fc7f400f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40af050 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f4005290 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40046d0 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80813fa60 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc808140020 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7f40bb020 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc7f4003900 con 0x7fc8081455c0 2024-09-17T03:20:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.396+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc808072160 con 0x7fc8081455c0 2024-09-17T03:20:42.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.400+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc7f4016370 con 0x7fc8081455c0 2024-09-17T03:20:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.460+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fc8080d2260 con 0x7fc8081455c0 2024-09-17T03:20:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc804c00640 1 -- 172.21.15.46:0/3823438979 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v20)=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fc7f4003b00 con 0x7fc8081455c0 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8081455c0 msgr2=0x7fc8081fc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8081455c0 0x7fc8081fc9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4004000 tx=0x7fc7f400f440 comp rx=0 tx=0).stop 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 shutdown_connections 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081fcf00 0x7fc8081fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8081455c0 0x7fc8081fc9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 --2- 172.21.15.46:0/3823438979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7f0004810 0x7fc80813efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 >> 172.21.15.46:0/3823438979 conn(0x7fc8080708c0 msgr2=0x7fc80815db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 shutdown_connections 2024-09-17T03:20:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.468+0000 7fc80fa00640 1 -- 172.21.15.46:0/3823438979 wait complete. 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 -- 172.21.15.46:0/3083892533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 msgr2=0x7f85b8004460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 --2- 172.21.15.46:0/3083892533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85b8004460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85b8004ad0 tx=0x7f85c401d730 comp rx=0 tx=0).stop 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 -- 172.21.15.46:0/3083892533 shutdown_connections 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 --2- 172.21.15.46:0/3083892533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85b8004460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 --2- 172.21.15.46:0/3083892533 >> v2:172.21.15.99:3300/0 conn(0x7f85b8004b00 0x7f85b8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 --2- 172.21.15.46:0/3083892533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b80066c0 0x7f85b8003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 -- 172.21.15.46:0/3083892533 >> 172.21.15.46:0/3083892533 conn(0x7f85d80b41f0 msgr2=0x7f85d80b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 -- 172.21.15.46:0/3083892533 shutdown_connections 2024-09-17T03:20:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 -- 172.21.15.46:0/3083892533 wait complete. 2024-09-17T03:20:42.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 Processor -- start 2024-09-17T03:20:42.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.472+0000 7f85ddc00640 1 -- start start 2024-09-17T03:20:42.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85d80d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8004b00 0x7f85d81fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b80066c0 0x7f85d81fc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85d80d1710 con 0x7f85b8004050 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85d80d1590 con 0x7f85b80066c0 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f85d80d1890 con 0x7f85b8004b00 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85d80d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85d80d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41276/0 (socket says 172.21.15.46:41276) 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 -- 172.21.15.46:0/2125795899 learned_addr learned my addr 172.21.15.46:0/2125795899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7400640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8004b00 0x7f85d81fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85dd200640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b80066c0 0x7f85d81fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 -- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8004b00 msgr2=0x7f85d81fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8004b00 0x7f85d81fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 -- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b80066c0 msgr2=0x7f85d81fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b80066c0 0x7f85d81fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 -- 172.21.15.46:0/2125795899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d80c9040 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85d7e00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85d80d0a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85c8003ed0 tx=0x7f85c8013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80126b0 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85c8012850 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d815e910 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8012c40 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d815e0a0 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85c801e430 con 0x7f85b8004050 2024-09-17T03:20:42.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f85c80279b0 con 0x7f85b8004050 2024-09-17T03:20:42.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.476+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d80d1df0 con 0x7f85b8004050 2024-09-17T03:20:42.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.480+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f85c8012de0 con 0x7f85b8004050 2024-09-17T03:20:42.489 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.490 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 -- 172.21.15.46:0/3666721109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 msgr2=0x7f73180066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 --2- 172.21.15.46:0/3666721109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73180066b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f732c005fd0 tx=0x7f732c01e4b0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 -- 172.21.15.46:0/3666721109 shutdown_connections 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 --2- 172.21.15.46:0/3666721109 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 0x7f731800d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 --2- 172.21.15.46:0/3666721109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73180066b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 -- 172.21.15.46:0/3666721109 >> 172.21.15.46:0/3666721109 conn(0x7f73380b41f0 msgr2=0x7f73380b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 -- 172.21.15.46:0/3666721109 shutdown_connections 2024-09-17T03:20:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.520+0000 7f733e800640 1 -- 172.21.15.46:0/3666721109 wait complete. 2024-09-17T03:20:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.524+0000 7f733e800640 1 Processor -- start 2024-09-17T03:20:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.524+0000 7f733e800640 1 -- start start 2024-09-17T03:20:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73380ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7318006bf0 0x7f7338159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 0x7f73380cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73380cea30 con 0x7f7318006bf0 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73380ce8b0 con 0x7f7318003bb0 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f73380d1190 con 0x7f731800b290 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7318006bf0 0x7f7338159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73380ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73380ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:33856/0 (socket says 172.21.15.46:33856) 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 0x7f73380cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 -- 172.21.15.46:0/3502963982 learned_addr learned my addr 172.21.15.46:0/3502963982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7318006bf0 0x7f7338159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41288/0 (socket says 172.21.15.46:41288) 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 -- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 msgr2=0x7f73380cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 0x7f73380cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 -- 172.21.15.46:0/3502963982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7318006bf0 msgr2=0x7f7338159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7318006bf0 0x7f7338159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 -- 172.21.15.46:0/3502963982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380b0330 con 0x7f7318003bb0 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733de00640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 0x7f73380cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733d400640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73380ca5b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c10 tx=0x7f732c0056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0aa050 con 0x7f7318003bb0 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f732c0046d0 con 0x7f7318003bb0 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73380c9800 con 0x7f7318003bb0 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.528+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c004b20 con 0x7f7318003bb0 2024-09-17T03:20:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.532+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73380c9ad0 con 0x7f7318003bb0 2024-09-17T03:20:42.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.532+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f732c0bf020 con 0x7f7318003bb0 2024-09-17T03:20:42.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.532+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f732c004cc0 con 0x7f7318003bb0 2024-09-17T03:20:42.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.532+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73380d1660 con 0x7f7318003bb0 2024-09-17T03:20:42.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.532+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f732c0c6020 con 0x7f7318003bb0 2024-09-17T03:20:42.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.548+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f85d80b4f30 con 0x7f85b8004050 2024-09-17T03:20:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.548+0000 7f85cfe00640 1 -- 172.21.15.46:0/2125795899 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f85c8027bb0 con 0x7f85b8004050 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 msgr2=0x7f85d80d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85d80d0a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85c8003ed0 tx=0x7f85c8013820 comp rx=0 tx=0).stop 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 shutdown_connections 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b80066c0 0x7f85d81fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8004b00 0x7f85d81fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 --2- 172.21.15.46:0/2125795899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8004050 0x7f85d80d0a30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 >> 172.21.15.46:0/2125795899 conn(0x7f85d80b41f0 msgr2=0x7f85d80cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 shutdown_connections 2024-09-17T03:20:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.552+0000 7f85ddc00640 1 -- 172.21.15.46:0/2125795899 wait complete. 2024-09-17T03:20:42.570 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.596+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f73380b4f30 con 0x7f7318003bb0 2024-09-17T03:20:42.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f7335600640 1 -- 172.21.15.46:0/3502963982 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f732c015080 con 0x7f7318003bb0 2024-09-17T03:20:42.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 msgr2=0x7f73380ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73380ca5b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c10 tx=0x7f732c0056a0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 shutdown_connections 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f731800b290 0x7f73380cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7318006bf0 0x7f7338159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 --2- 172.21.15.46:0/3502963982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318003bb0 0x7f73380ca5b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 >> 172.21.15.46:0/3502963982 conn(0x7f73380b41f0 msgr2=0x7f73381580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 shutdown_connections 2024-09-17T03:20:42.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.604+0000 7f733e800640 1 -- 172.21.15.46:0/3502963982 wait complete. 2024-09-17T03:20:42.623 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.624 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.624 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.624 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 -- 172.21.15.46:0/3185715225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398008790 msgr2=0x7fe39800d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3185715225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398008790 0x7fe39800d370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8073cf0 tx=0x7fe3b001c590 comp rx=0 tx=0).stop 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 -- 172.21.15.46:0/3185715225 shutdown_connections 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3185715225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398008790 0x7fe39800d370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3185715225 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe398007d10 0x7fe398008120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 -- 172.21.15.46:0/3185715225 >> 172.21.15.46:0/3185715225 conn(0x7fe3b80b4560 msgr2=0x7fe3b80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 -- 172.21.15.46:0/3185715225 shutdown_connections 2024-09-17T03:20:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.704+0000 7fe3bda00640 1 -- 172.21.15.46:0/3185715225 wait complete. 2024-09-17T03:20:42.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 Processor -- start 2024-09-17T03:20:42.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 -- start start 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 0x7fe3b81fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe398007d10 0x7fe3b80cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe398008790 0x7fe3b80d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80b5a50 con 0x7fe398003ae0 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80b58d0 con 0x7fe398008790 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b80b5bd0 con 0x7fe398007d10 2024-09-17T03:20:42.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 0x7fe3b81fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 0x7fe3b81fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41310/0 (socket says 172.21.15.46:41310) 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 -- 172.21.15.46:0/3642323893 learned_addr learned my addr 172.21.15.46:0/3642323893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7400640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe398007d10 0x7fe3b80cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bd000640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe398008790 0x7fe3b80d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 -- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe398007d10 msgr2=0x7fe3b80cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe398007d10 0x7fe3b80cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 -- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe398008790 msgr2=0x7fe3b80d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe398008790 0x7fe3b80d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 -- 172.21.15.46:0/3642323893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b80cdc00 con 0x7fe398003ae0 2024-09-17T03:20:42.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3b7e00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 0x7fe3b81fc9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80117b0 tx=0x7fe3a8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8010bb0 con 0x7fe398003ae0 2024-09-17T03:20:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3a8022070 con 0x7fe398003ae0 2024-09-17T03:20:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b81f8de0 con 0x7fe398003ae0 2024-09-17T03:20:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a801d670 con 0x7fe398003ae0 2024-09-17T03:20:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b81f93a0 con 0x7fe398003ae0 2024-09-17T03:20:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3a801d810 con 0x7fe398003ae0 2024-09-17T03:20:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.708+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe3a8026cb0 con 0x7fe398003ae0 2024-09-17T03:20:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.712+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b81f9ae0 con 0x7fe398003ae0 2024-09-17T03:20:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.712+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe3a801dce0 con 0x7fe398003ae0 2024-09-17T03:20:42.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.776+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fe3b80719d0 con 0x7fe398003ae0 2024-09-17T03:20:42.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.776+0000 7fe3afe00640 1 -- 172.21.15.46:0/3642323893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v23)=0 v23) ==== 119+0+0 (secure 0 0 0) 0x7fe3a8026410 con 0x7fe398003ae0 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 msgr2=0x7fe3b81fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 0x7fe3b81fc9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80117b0 tx=0x7fe3a8011bb0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 shutdown_connections 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe398008790 0x7fe3b80d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe398007d10 0x7fe3b80cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 --2- 172.21.15.46:0/3642323893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe398003ae0 0x7fe3b81fc9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 >> 172.21.15.46:0/3642323893 conn(0x7fe3b80b4560 msgr2=0x7fe3b815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 shutdown_connections 2024-09-17T03:20:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.780+0000 7fe3bda00640 1 -- 172.21.15.46:0/3642323893 wait complete. 2024-09-17T03:20:42.800 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.800 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.800 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.800 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.800 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.800 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi046') 2024-09-17T03:20:42.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config dump 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 -- 172.21.15.46:0/3944446100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f0160460 msgr2=0x7fb9f0160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 --2- 172.21.15.46:0/3944446100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f0160460 0x7fb9f0160870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b77a0 tx=0x7fb9e001c890 comp rx=0 tx=0).stop 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 -- 172.21.15.46:0/3944446100 shutdown_connections 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 --2- 172.21.15.46:0/3944446100 >> v2:172.21.15.99:3300/0 conn(0x7fb9f015d960 0x7fb9f015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 --2- 172.21.15.46:0/3944446100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f0160460 0x7fb9f0160870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 -- 172.21.15.46:0/3944446100 >> 172.21.15.46:0/3944446100 conn(0x7fb9f00b7a20 msgr2=0x7fb9f00b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 -- 172.21.15.46:0/3944446100 shutdown_connections 2024-09-17T03:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 -- 172.21.15.46:0/3944446100 wait complete. 2024-09-17T03:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 Processor -- start 2024-09-17T03:20:42.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 -- start start 2024-09-17T03:20:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.956+0000 7fb9f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 0x7fb9f01f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f0160460 0x7fb9f006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9f0160db0 0x7fb9f006de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f0169bf0 con 0x7fb9f015d960 2024-09-17T03:20:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f0169a70 con 0x7fb9f0160460 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f0169d70 con 0x7fb9f0160db0 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 0x7fb9f01f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 0x7fb9f01f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41336/0 (socket says 172.21.15.46:41336) 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 -- 172.21.15.46:0/2109269172 learned_addr learned my addr 172.21.15.46:0/2109269172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9e7e00640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f0160460 0x7fb9f006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f5200640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9f0160db0 0x7fb9f006de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 -- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9f0160db0 msgr2=0x7fb9f006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9f0160db0 0x7fb9f006de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 -- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f0160460 msgr2=0x7fb9f006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f0160460 0x7fb9f006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 -- 172.21.15.46:0/2109269172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f015b1c0 con 0x7fb9f015d960 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9e7e00640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f0160460 0x7fb9f006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9efe00640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 0x7fb9f01f8440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00b0040 tx=0x7fb9e0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e00b2040 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e0005310 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f015b4a0 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e00046d0 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f015ba60 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9e0004c00 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9e00ad780 con 0x7fb9f015d960 2024-09-17T03:20:42.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.960+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f014efb0 con 0x7fb9f015d960 2024-09-17T03:20:42.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:42.964+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb9e00164b0 con 0x7fb9f015d960 2024-09-17T03:20:43.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.024+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb9f014f1b0 con 0x7fb9f015d960 2024-09-17T03:20:43.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.024+0000 7fb9ecc00640 1 -- 172.21.15.46:0/2109269172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7fb9e00ad980 con 0x7fb9f015d960 2024-09-17T03:20:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:20:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:20:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:20:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:20:43.026 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:20:43.027 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:20:43.028 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:20:43.028 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:20:43.028 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:20:43.028 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:20:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 msgr2=0x7fb9f01f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 0x7fb9f01f8440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00b0040 tx=0x7fb9e0005d60 comp rx=0 tx=0).stop 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 shutdown_connections 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9f0160db0 0x7fb9f006de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f0160460 0x7fb9f006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 --2- 172.21.15.46:0/2109269172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f015d960 0x7fb9f01f8440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 >> 172.21.15.46:0/2109269172 conn(0x7fb9f00b7a20 msgr2=0x7fb9f0154390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 shutdown_connections 2024-09-17T03:20:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.028+0000 7fb9f7200640 1 -- 172.21.15.46:0/2109269172 wait complete. 2024-09-17T03:20:43.047 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T03:20:43.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T03:20:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 -- 172.21.15.46:0/1828920291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 msgr2=0x7fd8480b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 --2- 172.21.15.46:0/1828920291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd8480b5a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd8480b1740 tx=0x7fd83c01c8c0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 -- 172.21.15.46:0/1828920291 shutdown_connections 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 --2- 172.21.15.46:0/1828920291 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd848161160 0x7fd848074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 --2- 172.21.15.46:0/1828920291 >> v2:172.21.15.99:3300/0 conn(0x7fd848160d50 0x7fd848074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 --2- 172.21.15.46:0/1828920291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd8480b5a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 -- 172.21.15.46:0/1828920291 >> 172.21.15.46:0/1828920291 conn(0x7fd8480708c0 msgr2=0x7fd848070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 -- 172.21.15.46:0/1828920291 shutdown_connections 2024-09-17T03:20:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.208+0000 7fd850000640 1 -- 172.21.15.46:0/1828920291 wait complete. 2024-09-17T03:20:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 Processor -- start 2024-09-17T03:20:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 -- start start 2024-09-17T03:20:43.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd84814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd848160d50 0x7fd84813bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd848161160 0x7fd84815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd848072230 con 0x7fd848157030 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480720b0 con 0x7fd848160d50 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd8480723b0 con 0x7fd848161160 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd84814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd84814c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41360/0 (socket says 172.21.15.46:41360) 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 -- 172.21.15.46:0/3089005743 learned_addr learned my addr 172.21.15.46:0/3089005743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84e000640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd848161160 0x7fd84815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84cc00640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd848160d50 0x7fd84813bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:43.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 -- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd848161160 msgr2=0x7fd84815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd848161160 0x7fd84815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 -- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd848160d50 msgr2=0x7fd84813bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd848160d50 0x7fd84813bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 -- 172.21.15.46:0/3089005743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84814d400 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd84d600640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd84814c160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01af70 tx=0x7fd83c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0b2040 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd83c004320 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84814d6e0 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0052d0 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84814bc40 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd83c004a60 con 0x7fd848157030 2024-09-17T03:20:43.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.212+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd83c003d00 con 0x7fd848157030 2024-09-17T03:20:43.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.216+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8480728a0 con 0x7fd848157030 2024-09-17T03:20:43.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.216+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd83c0bd060 con 0x7fd848157030 2024-09-17T03:20:43.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.280+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd8480d2260 con 0x7fd848157030 2024-09-17T03:20:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd845600640 1 -- 172.21.15.46:0/3089005743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd83c0164b0 con 0x7fd848157030 2024-09-17T03:20:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-09-17T03:20:43.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 msgr2=0x7fd84814c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd84814c160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01af70 tx=0x7fd83c005d60 comp rx=0 tx=0).stop 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 shutdown_connections 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd848161160 0x7fd84815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd848160d50 0x7fd84813bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 --2- 172.21.15.46:0/3089005743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848157030 0x7fd84814c160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 >> 172.21.15.46:0/3089005743 conn(0x7fd8480708c0 msgr2=0x7fd8480b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 shutdown_connections 2024-09-17T03:20:43.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:43.284+0000 7fd850000640 1 -- 172.21.15.46:0/3089005743 wait complete. 2024-09-17T03:20:43.305 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T03:20:43.305 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T03:20:43.305 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:43.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T03:20:43.325 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:43.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T03:20:43.385 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:43.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T03:20:43.445 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:20:43.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T03:20:43.505 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:43.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T03:20:43.526 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:43.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T03:20:43.588 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:43.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T03:20:43.648 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:20:43.648 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T03:20:43.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 5991f809-e043-4b36-aa08-043a41133140 0 2024-09-17T03:20:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 -- 172.21.15.99:0/1733477849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180074b20 msgr2=0x7f318015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 --2- 172.21.15.99:0/1733477849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180074b20 0x7f318015cf90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3178009bc0 tx=0x7f317801cb10 comp rx=0 tx=0).stop 2024-09-17T03:20:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 -- 172.21.15.99:0/1733477849 shutdown_connections 2024-09-17T03:20:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 --2- 172.21.15.99:0/1733477849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180074b20 0x7f318015cf90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 -- 172.21.15.99:0/1733477849 >> 172.21.15.99:0/1733477849 conn(0x7f31800708c0 msgr2=0x7f3180070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 -- 172.21.15.99:0/1733477849 shutdown_connections 2024-09-17T03:20:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 -- 172.21.15.99:0/1733477849 wait complete. 2024-09-17T03:20:43.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 Processor -- start 2024-09-17T03:20:43.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.900+0000 7f3188200640 1 -- start start 2024-09-17T03:20:43.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 0x7f31801556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180157030 0x7f318013fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31800b3320 0x7f31800b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f318016b4e0 con 0x7f31800b3320 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f318016b360 con 0x7f3180157030 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f318016b660 con 0x7f31800b5a80 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 0x7f31801556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 0x7f31801556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:58452/0 (socket says 172.21.15.99:58452) 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 -- 172.21.15.99:0/2041872669 learned_addr learned my addr 172.21.15.99:0/2041872669 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3186200640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31800b3320 0x7f31800b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3184e00640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180157030 0x7f318013fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 -- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180157030 msgr2=0x7f318013fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180157030 0x7f318013fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 -- 172.21.15.99:0/2041872669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31800b3320 msgr2=0x7f31800b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31800b3320 0x7f31800b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 -- 172.21.15.99:0/2041872669 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31800b13e0 con 0x7f31800b5a80 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3186200640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31800b3320 0x7f31800b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3185800640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 0x7f31801556c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f317400c430 tx=0x7f317400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174011790 con 0x7f31800b5a80 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3174017070 con 0x7f31800b5a80 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174011ce0 con 0x7f31800b5a80 2024-09-17T03:20:43.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31800b3c70 con 0x7f31800b5a80 2024-09-17T03:20:43.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f318006d600 con 0x7f31800b5a80 2024-09-17T03:20:43.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f317401c460 con 0x7f31800b5a80 2024-09-17T03:20:43.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.904+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f3174012920 con 0x7f31800b5a80 2024-09-17T03:20:43.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.908+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318015cf90 con 0x7f31800b5a80 2024-09-17T03:20:43.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.908+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f31800b3c70 con 0x7f31800b5a80 2024-09-17T03:20:43.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.976+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5991f809-e043-4b36-aa08-043a41133140", "id": 0} v 0) -- 0x7f31800d2260 con 0x7f31800b5a80 2024-09-17T03:20:43.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f317d600640 1 -- 172.21.15.99:0/2041872669 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5991f809-e043-4b36-aa08-043a41133140", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f3174012420 con 0x7f31800b5a80 2024-09-17T03:20:43.985 INFO:teuthology.orchestra.run.smithi099.stdout:0 2024-09-17T03:20:43.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 msgr2=0x7f31801556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:43.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 0x7f31801556c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f317400c430 tx=0x7f317400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 shutdown_connections 2024-09-17T03:20:43.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31800b3320 0x7f31800b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3180157030 0x7f318013fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 --2- 172.21.15.99:0/2041872669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31800b5a80 0x7f31801556c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:43.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 >> 172.21.15.99:0/2041872669 conn(0x7f31800708c0 msgr2=0x7f318015e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:43.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 shutdown_connections 2024-09-17T03:20:43.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:43.984+0000 7f3188200640 1 -- 172.21.15.99:0/2041872669 wait complete. 2024-09-17T03:20:44.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 332bd3c4-c4cd-4788-9a2d-ae1d68f4686f 1 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 -- 172.21.15.99:0/948151200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa440b5a80 msgr2=0x7faa44074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 --2- 172.21.15.99:0/948151200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa440b5a80 0x7faa44074410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009840 tx=0x7faa2c01c620 comp rx=0 tx=0).stop 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 -- 172.21.15.99:0/948151200 shutdown_connections 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 --2- 172.21.15.99:0/948151200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa440b5a80 0x7faa44074410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 -- 172.21.15.99:0/948151200 >> 172.21.15.99:0/948151200 conn(0x7faa440708c0 msgr2=0x7faa44070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 -- 172.21.15.99:0/948151200 shutdown_connections 2024-09-17T03:20:44.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 -- 172.21.15.99:0/948151200 wait complete. 2024-09-17T03:20:44.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 Processor -- start 2024-09-17T03:20:44.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.156+0000 7faa4ac00640 1 -- start start 2024-09-17T03:20:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa44074950 0x7faa4414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 0x7faa441f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa44157030 0x7faa4414f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa4416b640 con 0x7faa44074950 2024-09-17T03:20:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa4416b4c0 con 0x7faa44157030 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faa4416b7c0 con 0x7faa440b5a80 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 0x7faa441f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa48c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa44157030 0x7faa4414f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 0x7faa441f8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:58466/0 (socket says 172.21.15.99:58466) 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 -- 172.21.15.99:0/2978845183 learned_addr learned my addr 172.21.15.99:0/2978845183 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa43e00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa44074950 0x7faa4414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 -- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa44157030 msgr2=0x7faa4414f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa44157030 0x7faa4414f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 -- 172.21.15.99:0/2978845183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa44074950 msgr2=0x7faa4414c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa44074950 0x7faa4414c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 -- 172.21.15.99:0/2978845183 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa44138ab0 con 0x7faa440b5a80 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa48c00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa44157030 0x7faa4414f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa3be00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 0x7faa441f8fb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa2c01a250 tx=0x7faa2c0aa5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0aab20 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa2c0aadb0 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa44138d90 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c005290 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa44139350 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa2c004a60 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.160+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7faa2c003c20 con 0x7faa440b5a80 2024-09-17T03:20:44.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.164+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa44074410 con 0x7faa440b5a80 2024-09-17T03:20:44.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.164+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faa2c0a7070 con 0x7faa440b5a80 2024-09-17T03:20:44.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.236+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd new", "uuid": "332bd3c4-c4cd-4788-9a2d-ae1d68f4686f", "id": 1} v 0) -- 0x7faa440d2260 con 0x7faa440b5a80 2024-09-17T03:20:44.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa40c00640 1 -- 172.21.15.99:0/2978845183 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "332bd3c4-c4cd-4788-9a2d-ae1d68f4686f", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7faa2c016330 con 0x7faa440b5a80 2024-09-17T03:20:44.242 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-17T03:20:44.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 msgr2=0x7faa441f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 0x7faa441f8fb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa2c01a250 tx=0x7faa2c0aa5f0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 shutdown_connections 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa44157030 0x7faa4414f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa440b5a80 0x7faa441f8fb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 --2- 172.21.15.99:0/2978845183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa44074950 0x7faa4414c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 >> 172.21.15.99:0/2978845183 conn(0x7faa440708c0 msgr2=0x7faa441504d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 shutdown_connections 2024-09-17T03:20:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.240+0000 7faa4ac00640 1 -- 172.21.15.99:0/2978845183 wait complete. 2024-09-17T03:20:44.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new a6821825-458d-44c4-b5e2-a72d7f7cb840 2 2024-09-17T03:20:44.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 -- 172.21.15.99:0/2813007086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb01527b0 msgr2=0x7f4bb015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2813007086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb01527b0 0x7f4bb015bb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0070710 tx=0x7f4b9c01c740 comp rx=0 tx=0).stop 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 -- 172.21.15.99:0/2813007086 shutdown_connections 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2813007086 >> v2:172.21.15.46:3300/0 conn(0x7f4bb00b4de0 0x7f4bb00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2813007086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb01527b0 0x7f4bb015bb60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 -- 172.21.15.99:0/2813007086 >> 172.21.15.99:0/2813007086 conn(0x7f4bb00b2040 msgr2=0x7f4bb00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 -- 172.21.15.99:0/2813007086 shutdown_connections 2024-09-17T03:20:44.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 -- 172.21.15.99:0/2813007086 wait complete. 2024-09-17T03:20:44.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 Processor -- start 2024-09-17T03:20:44.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.412+0000 7f4bb5400640 1 -- start start 2024-09-17T03:20:44.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb00b4de0 0x7f4bb0154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb00b5c70 0x7f4bb0158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bb01527b0 0x7f4bb01588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0168950 con 0x7f4bb00b5c70 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01687d0 con 0x7f4bb00b4de0 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb0168ad0 con 0x7f4bb01527b0 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bae800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb00b4de0 0x7f4bb0154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bae800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb00b4de0 0x7f4bb0154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41454/0 (socket says 172.21.15.99:41454) 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bae800640 1 -- 172.21.15.99:0/2191098206 learned_addr learned my addr 172.21.15.99:0/2191098206 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bb01527b0 0x7f4bb01588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bade00640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb00b5c70 0x7f4bb0158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 -- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb00b4de0 msgr2=0x7f4bb0154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb00b4de0 0x7f4bb0154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 -- 172.21.15.99:0/2191098206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb00b5c70 msgr2=0x7f4bb0158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb00b5c70 0x7f4bb0158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 -- 172.21.15.99:0/2191098206 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb00cfa70 con 0x7f4bb01527b0 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bade00640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb00b5c70 0x7f4bb0158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4baf200640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bb01527b0 0x7f4bb01588d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800e990 tx=0x7f4ba800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8004d30 con 0x7f4bb01527b0 2024-09-17T03:20:44.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ba800cb60 con 0x7f4bb01527b0 2024-09-17T03:20:44.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8005020 con 0x7f4bb01527b0 2024-09-17T03:20:44.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb00cfd50 con 0x7f4bb01527b0 2024-09-17T03:20:44.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb00cd430 con 0x7f4bb01527b0 2024-09-17T03:20:44.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ba8005550 con 0x7f4bb01527b0 2024-09-17T03:20:44.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f4ba8018dd0 con 0x7f4bb01527b0 2024-09-17T03:20:44.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.416+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb00b51f0 con 0x7f4bb01527b0 2024-09-17T03:20:44.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.420+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4ba80058a0 con 0x7f4bb01527b0 2024-09-17T03:20:44.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.488+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a6821825-458d-44c4-b5e2-a72d7f7cb840", "id": 2} v 0) -- 0x7f4bb00b15c0 con 0x7f4bb01527b0 2024-09-17T03:20:44.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4ba6a00640 1 -- 172.21.15.99:0/2191098206 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a6821825-458d-44c4-b5e2-a72d7f7cb840", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f4ba8003980 con 0x7f4bb01527b0 2024-09-17T03:20:44.498 INFO:teuthology.orchestra.run.smithi099.stdout:2 2024-09-17T03:20:44.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bb01527b0 msgr2=0x7f4bb01588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bb01527b0 0x7f4bb01588d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800e990 tx=0x7f4ba800ee50 comp rx=0 tx=0).stop 2024-09-17T03:20:44.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 shutdown_connections 2024-09-17T03:20:44.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bb01527b0 0x7f4bb01588d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb00b5c70 0x7f4bb0158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 --2- 172.21.15.99:0/2191098206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bb00b4de0 0x7f4bb0154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 >> 172.21.15.99:0/2191098206 conn(0x7f4bb00b2040 msgr2=0x7f4bb00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 shutdown_connections 2024-09-17T03:20:44.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.496+0000 7f4bb5400640 1 -- 172.21.15.99:0/2191098206 wait complete. 2024-09-17T03:20:44.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new d9adfef4-8f8c-4aac-876e-4014eafeac6d 3 2024-09-17T03:20:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 -- 172.21.15.99:0/3109068323 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c0744d0 msgr2=0x7ff85c06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 --2- 172.21.15.99:0/3109068323 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c0744d0 0x7ff85c06ea60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff848009840 tx=0x7ff84801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 -- 172.21.15.99:0/3109068323 shutdown_connections 2024-09-17T03:20:44.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 --2- 172.21.15.99:0/3109068323 >> v2:172.21.15.99:3300/0 conn(0x7ff85c06efa0 0x7ff85c06f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 --2- 172.21.15.99:0/3109068323 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c0744d0 0x7ff85c06ea60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 -- 172.21.15.99:0/3109068323 >> 172.21.15.99:0/3109068323 conn(0x7ff85c0b2c40 msgr2=0x7ff85c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 -- 172.21.15.99:0/3109068323 shutdown_connections 2024-09-17T03:20:44.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.668+0000 7ff861200640 1 -- 172.21.15.99:0/3109068323 wait complete. 2024-09-17T03:20:44.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 Processor -- start 2024-09-17T03:20:44.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 -- start start 2024-09-17T03:20:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 0x7ff85c14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85c15c5d0 0x7ff85c0aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c1fcb00 0x7ff85c073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c168ea0 con 0x7ff85c1fcb00 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c168d20 con 0x7ff85c15c5d0 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c169020 con 0x7ff85c06efa0 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 0x7ff85c14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85c15c5d0 0x7ff85c0aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 0x7ff85c14f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:58502/0 (socket says 172.21.15.99:58502) 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859e00640 1 -- 172.21.15.99:0/3564759866 learned_addr learned my addr 172.21.15.99:0/3564759866 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff85a800640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c1fcb00 0x7ff85c073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 -- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 msgr2=0x7ff85c14f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 0x7ff85c14f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 -- 172.21.15.99:0/3564759866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c1fcb00 msgr2=0x7ff85c073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c1fcb00 0x7ff85c073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 -- 172.21.15.99:0/3564759866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c0ab780 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859e00640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 0x7ff85c14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff859400640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85c15c5d0 0x7ff85c0aa560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8480ac000 tx=0x7ff8480a57c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8480b2040 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff848005020 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c1fc020 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848005470 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c0ab2a0 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff848004a60 con 0x7ff85c15c5d0 2024-09-17T03:20:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.672+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff848003da0 con 0x7ff85c15c5d0 2024-09-17T03:20:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.676+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c0ab4d0 con 0x7ff85c15c5d0 2024-09-17T03:20:44.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.676+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff8480bc060 con 0x7ff85c15c5d0 2024-09-17T03:20:44.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.756+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9adfef4-8f8c-4aac-876e-4014eafeac6d", "id": 3} v 0) -- 0x7ff85c06f3b0 con 0x7ff85c15c5d0 2024-09-17T03:20:44.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.760+0000 7ff852000640 1 -- 172.21.15.99:0/3564759866 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9adfef4-8f8c-4aac-876e-4014eafeac6d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff848016410 con 0x7ff85c15c5d0 2024-09-17T03:20:44.766 INFO:teuthology.orchestra.run.smithi099.stdout:3 2024-09-17T03:20:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85c15c5d0 msgr2=0x7ff85c0aa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85c15c5d0 0x7ff85c0aa560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8480ac000 tx=0x7ff8480a57c0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 shutdown_connections 2024-09-17T03:20:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c1fcb00 0x7ff85c073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85c15c5d0 0x7ff85c0aa560 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 --2- 172.21.15.99:0/3564759866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff85c06efa0 0x7ff85c14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 >> 172.21.15.99:0/3564759866 conn(0x7ff85c0b2c40 msgr2=0x7ff85c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 shutdown_connections 2024-09-17T03:20:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.764+0000 7ff861200640 1 -- 172.21.15.99:0/3564759866 wait complete. 2024-09-17T03:20:44.774 INFO:tasks.ceph.mgr.y.smithi046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T03:20:44.774 INFO:tasks.ceph.mgr.y.smithi046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T03:20:44.774 INFO:tasks.ceph.mgr.y.smithi046.stderr: from numpy import show_config as show_numpy_config 2024-09-17T03:20:44.799 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 72138866-38d1-45f3-a209-6f9a4523c997 4 2024-09-17T03:20:44.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 -- 172.21.15.99:0/2835865915 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e700d11b0 msgr2=0x7f7e7015ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 --2- 172.21.15.99:0/2835865915 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e700d11b0 0x7f7e7015ea90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009900 tx=0x7f7e5801c850 comp rx=0 tx=0).stop 2024-09-17T03:20:44.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 -- 172.21.15.99:0/2835865915 shutdown_connections 2024-09-17T03:20:44.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 --2- 172.21.15.99:0/2835865915 >> v2:172.21.15.99:3300/0 conn(0x7f7e7006f1e0 0x7f7e7006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 --2- 172.21.15.99:0/2835865915 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e700d11b0 0x7f7e7015ea90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 -- 172.21.15.99:0/2835865915 >> 172.21.15.99:0/2835865915 conn(0x7f7e7006d1f0 msgr2=0x7f7e7006d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:44.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 -- 172.21.15.99:0/2835865915 shutdown_connections 2024-09-17T03:20:44.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 -- 172.21.15.99:0/2835865915 wait complete. 2024-09-17T03:20:44.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 Processor -- start 2024-09-17T03:20:44.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.964+0000 7f7e75400640 1 -- start start 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7006f1e0 0x7f7e700c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 0x7f7e701f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 0x7f7e7015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70168b50 con 0x7f7e7006f1e0 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701689d0 con 0x7f7e70072550 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7e70168cd0 con 0x7f7e701f8f30 2024-09-17T03:20:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e74a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 0x7f7e7015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 0x7f7e701f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e74a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 0x7f7e7015a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:58526/0 (socket says 172.21.15.99:58526) 2024-09-17T03:20:44.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e74a00640 1 -- 172.21.15.99:0/2348980260 learned_addr learned my addr 172.21.15.99:0/2348980260 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:44.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 0x7f7e701f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41498/0 (socket says 172.21.15.99:41498) 2024-09-17T03:20:44.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e6fe00640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7006f1e0 0x7f7e700c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 -- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 msgr2=0x7f7e7015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 0x7f7e7015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 -- 172.21.15.99:0/2348980260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7006f1e0 msgr2=0x7f7e700c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7006f1e0 0x7f7e700c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 -- 172.21.15.99:0/2348980260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e700cfe60 con 0x7f7e70072550 2024-09-17T03:20:44.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e74a00640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 0x7f7e7015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e67400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 0x7f7e701f89f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e700cf9a0 tx=0x7f7e580a5600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e580b2040 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e580042e0 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e700b02a0 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58003860 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e700b0860 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7e58003a00 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7e580b6420 con 0x7f7e70072550 2024-09-17T03:20:44.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.968+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e701f94d0 con 0x7f7e70072550 2024-09-17T03:20:44.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:44.972+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7e580164d0 con 0x7f7e70072550 2024-09-17T03:20:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.040+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72138866-38d1-45f3-a209-6f9a4523c997", "id": 4} v 0) -- 0x7f7e700c6490 con 0x7f7e70072550 2024-09-17T03:20:45.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e6cc00640 1 -- 172.21.15.99:0/2348980260 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72138866-38d1-45f3-a209-6f9a4523c997", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f7e58003d90 con 0x7f7e70072550 2024-09-17T03:20:45.048 INFO:teuthology.orchestra.run.smithi099.stdout:4 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 msgr2=0x7f7e701f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 0x7f7e701f89f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e700cf9a0 tx=0x7f7e580a5600 comp rx=0 tx=0).stop 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 shutdown_connections 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e701f8f30 0x7f7e7015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70072550 0x7f7e701f89f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 --2- 172.21.15.99:0/2348980260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e7006f1e0 0x7f7e700c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.044+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 >> 172.21.15.99:0/2348980260 conn(0x7f7e7006d1f0 msgr2=0x7f7e700b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.048+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 shutdown_connections 2024-09-17T03:20:45.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.048+0000 7f7e75400640 1 -- 172.21.15.99:0/2348980260 wait complete. 2024-09-17T03:20:45.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 77e04047-9271-4fdf-aac9-29db70985c96 5 2024-09-17T03:20:45.157 INFO:tasks.ceph.mgr.x.smithi099.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T03:20:45.157 INFO:tasks.ceph.mgr.x.smithi099.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T03:20:45.157 INFO:tasks.ceph.mgr.x.smithi099.stderr: from numpy import show_config as show_numpy_config 2024-09-17T03:20:45.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.216+0000 7f0b46800640 1 -- 172.21.15.99:0/1605484808 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b400b5a80 msgr2=0x7f0b40074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.216+0000 7f0b46800640 1 --2- 172.21.15.99:0/1605484808 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b400b5a80 0x7f0b40074410 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009900 tx=0x7f0b2c01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- 172.21.15.99:0/1605484808 shutdown_connections 2024-09-17T03:20:45.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 --2- 172.21.15.99:0/1605484808 >> v2:172.21.15.99:3300/0 conn(0x7f0b40074950 0x7f0b4015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 --2- 172.21.15.99:0/1605484808 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b400b5a80 0x7f0b40074410 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- 172.21.15.99:0/1605484808 >> 172.21.15.99:0/1605484808 conn(0x7f0b400708c0 msgr2=0x7f0b40070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- 172.21.15.99:0/1605484808 shutdown_connections 2024-09-17T03:20:45.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- 172.21.15.99:0/1605484808 wait complete. 2024-09-17T03:20:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 Processor -- start 2024-09-17T03:20:45.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- start start 2024-09-17T03:20:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b40074950 0x7f0b4015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 0x7f0b401f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b40157030 0x7f0b40149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4016a360 con 0x7f0b40074950 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4016a1e0 con 0x7f0b400b5a80 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.220+0000 7f0b46800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4016a4e0 con 0x7f0b40157030 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 0x7f0b401f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 0x7f0b401f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41516/0 (socket says 172.21.15.99:41516) 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 -- 172.21.15.99:0/2875123715 learned_addr learned my addr 172.21.15.99:0/2875123715 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b40157030 0x7f0b40149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3f400640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b40074950 0x7f0b4015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 -- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b40157030 msgr2=0x7f0b40149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b40157030 0x7f0b40149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 -- 172.21.15.99:0/2875123715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b40074950 msgr2=0x7f0b4015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b40074950 0x7f0b4015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 -- 172.21.15.99:0/2875123715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4014a4a0 con 0x7f0b400b5a80 2024-09-17T03:20:45.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b3ea00640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 0x7f0b401f92a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009850 tx=0x7f0b2c0a5710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0b2040 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b2c0043b0 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4015f4f0 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c005290 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b400b44b0 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b2c004a60 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0b2c003ab0 con 0x7f0b400b5a80 2024-09-17T03:20:45.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.224+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4015cf90 con 0x7f0b400b5a80 2024-09-17T03:20:45.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.228+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0b2c0bc060 con 0x7f0b400b5a80 2024-09-17T03:20:45.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.296+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "77e04047-9271-4fdf-aac9-29db70985c96", "id": 5} v 0) -- 0x7f0b400d2260 con 0x7f0b400b5a80 2024-09-17T03:20:45.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.300+0000 7f0b37400640 1 -- 172.21.15.99:0/2875123715 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "77e04047-9271-4fdf-aac9-29db70985c96", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0b2c0164d0 con 0x7f0b400b5a80 2024-09-17T03:20:45.304 INFO:teuthology.orchestra.run.smithi099.stdout:5 2024-09-17T03:20:45.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 msgr2=0x7f0b401f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 0x7f0b401f92a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009850 tx=0x7f0b2c0a5710 comp rx=0 tx=0).stop 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 shutdown_connections 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b40157030 0x7f0b40149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b400b5a80 0x7f0b401f92a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 --2- 172.21.15.99:0/2875123715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b40074950 0x7f0b4015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 >> 172.21.15.99:0/2875123715 conn(0x7f0b400708c0 msgr2=0x7f0b4015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 shutdown_connections 2024-09-17T03:20:45.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.304+0000 7f0b46800640 1 -- 172.21.15.99:0/2875123715 wait complete. 2024-09-17T03:20:45.324 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a 6 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 -- 172.21.15.99:0/872319263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8161160 msgr2=0x7f44f8074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 --2- 172.21.15.99:0/872319263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8161160 0x7f44f8074950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f44f80713e0 tx=0x7f44f001cd80 comp rx=0 tx=0).stop 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 -- 172.21.15.99:0/872319263 shutdown_connections 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 --2- 172.21.15.99:0/872319263 >> v2:172.21.15.46:3300/0 conn(0x7f44f8160d50 0x7f44f8074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 --2- 172.21.15.99:0/872319263 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8161160 0x7f44f8074950 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 --2- 172.21.15.99:0/872319263 >> v2:172.21.15.99:3300/0 conn(0x7f44f8157030 0x7f44f80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 -- 172.21.15.99:0/872319263 >> 172.21.15.99:0/872319263 conn(0x7f44f80708c0 msgr2=0x7f44f8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 -- 172.21.15.99:0/872319263 shutdown_connections 2024-09-17T03:20:45.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.472+0000 7f44ff000640 1 -- 172.21.15.99:0/872319263 wait complete. 2024-09-17T03:20:45.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 Processor -- start 2024-09-17T03:20:45.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 -- start start 2024-09-17T03:20:45.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8157030 0x7f44f8137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44f8160d50 0x7f44f814c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f8161160 0x7f44f815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8072290 con 0x7f44f8160d50 2024-09-17T03:20:45.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8072110 con 0x7f44f8161160 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44ff000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f44f8072410 con 0x7f44f8157030 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44f7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8157030 0x7f44f8137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44f7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8157030 0x7f44f8137980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.99:58550/0 (socket says 172.21.15.99:58550) 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.476+0000 7f44f7e00640 1 -- 172.21.15.99:0/4290720265 learned_addr learned my addr 172.21.15.99:0/4290720265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44f7400640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44f8160d50 0x7f44f814c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f8161160 0x7f44f815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 -- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8157030 msgr2=0x7f44f8137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8157030 0x7f44f8137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 -- 172.21.15.99:0/4290720265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44f8160d50 msgr2=0x7f44f814c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44f8160d50 0x7f44f814c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 -- 172.21.15.99:0/4290720265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f8137ec0 con 0x7f44f8161160 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44f7400640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44f8160d50 0x7f44f814c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44fd000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f8161160 0x7f44f815bac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44f00ac000 tx=0x7f44f0005aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00b2040 con 0x7f44f8161160 2024-09-17T03:20:45.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44f0004b40 con 0x7f44f8161160 2024-09-17T03:20:45.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0003fc0 con 0x7f44f8161160 2024-09-17T03:20:45.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f814ac60 con 0x7f44f8161160 2024-09-17T03:20:45.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f813a6d0 con 0x7f44f8161160 2024-09-17T03:20:45.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44f00be060 con 0x7f44f8161160 2024-09-17T03:20:45.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f44f0005880 con 0x7f44f8161160 2024-09-17T03:20:45.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.480+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f814a9d0 con 0x7f44f8161160 2024-09-17T03:20:45.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.484+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f44f00d7860 con 0x7f44f8161160 2024-09-17T03:20:45.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.552+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a", "id": 6} v 0) -- 0x7f44f80728c0 con 0x7f44f8161160 2024-09-17T03:20:45.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44efe00640 1 -- 172.21.15.99:0/4290720265 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f44f00c5070 con 0x7f44f8161160 2024-09-17T03:20:45.559 INFO:teuthology.orchestra.run.smithi099.stdout:6 2024-09-17T03:20:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f8161160 msgr2=0x7f44f815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f8161160 0x7f44f815bac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44f00ac000 tx=0x7f44f0005aa0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 shutdown_connections 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f8161160 0x7f44f815bac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44f8160d50 0x7f44f814c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 --2- 172.21.15.99:0/4290720265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44f8157030 0x7f44f8137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 >> 172.21.15.99:0/4290720265 conn(0x7f44f80708c0 msgr2=0x7f44f81502c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 shutdown_connections 2024-09-17T03:20:45.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.556+0000 7f44ff000640 1 -- 172.21.15.99:0/4290720265 wait complete. 2024-09-17T03:20:45.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6 7 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 -- 172.21.15.99:0/2829139152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc160d50 msgr2=0x7fd8cc074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/2829139152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc160d50 0x7fd8cc074410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009840 tx=0x7fd8b801c820 comp rx=0 tx=0).stop 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 -- 172.21.15.99:0/2829139152 shutdown_connections 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/2829139152 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc160d50 0x7fd8cc074410 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/2829139152 >> v2:172.21.15.99:3300/0 conn(0x7fd8cc157030 0x7fd8cc0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 -- 172.21.15.99:0/2829139152 >> 172.21.15.99:0/2829139152 conn(0x7fd8cc0708c0 msgr2=0x7fd8cc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 -- 172.21.15.99:0/2829139152 shutdown_connections 2024-09-17T03:20:45.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 -- 172.21.15.99:0/2829139152 wait complete. 2024-09-17T03:20:45.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 Processor -- start 2024-09-17T03:20:45.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.732+0000 7fd8d3e00640 1 -- start start 2024-09-17T03:20:45.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc157030 0x7fd8cc0b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 0x7fd8cc1f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8cc161160 0x7fd8cc155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc16a800 con 0x7fd8cc161160 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc16a680 con 0x7fd8cc160d50 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc16a980 con 0x7fd8cc157030 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 0x7fd8cc1f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 0x7fd8cc1f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41554/0 (socket says 172.21.15.99:41554) 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d0a00640 1 -- 172.21.15.99:0/841508498 learned_addr learned my addr 172.21.15.99:0/841508498 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc157030 0x7fd8cc0b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1e00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8cc161160 0x7fd8cc155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 -- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 msgr2=0x7fd8cc1f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 0x7fd8cc1f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 -- 172.21.15.99:0/841508498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8cc161160 msgr2=0x7fd8cc155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8cc161160 0x7fd8cc155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 -- 172.21.15.99:0/841508498 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc0b3660 con 0x7fd8cc157030 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d0a00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 0x7fd8cc1f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1e00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8cc161160 0x7fd8cc155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d1400640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc157030 0x7fd8cc0b2d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000ea20 tx=0x7fd8c000eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:45.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000ce40 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8c0004020 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc0b3940 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0004470 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc153ce0 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8c0003990 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd8c0009440 con 0x7fd8cc157030 2024-09-17T03:20:45.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.736+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0d2260 con 0x7fd8cc157030 2024-09-17T03:20:45.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.740+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd8c0003ce0 con 0x7fd8cc157030 2024-09-17T03:20:45.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.808+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6", "id": 7} v 0) -- 0x7fd8cc0d1e00 con 0x7fd8cc157030 2024-09-17T03:20:45.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8c9600640 1 -- 172.21.15.99:0/841508498 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd8c0021c90 con 0x7fd8cc157030 2024-09-17T03:20:45.817 INFO:teuthology.orchestra.run.smithi099.stdout:7 2024-09-17T03:20:45.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc157030 msgr2=0x7fd8cc0b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc157030 0x7fd8cc0b2d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000ea20 tx=0x7fd8c000eee0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 shutdown_connections 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8cc161160 0x7fd8cc155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8cc160d50 0x7fd8cc1f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 --2- 172.21.15.99:0/841508498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8cc157030 0x7fd8cc0b2d80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 >> 172.21.15.99:0/841508498 conn(0x7fd8cc0708c0 msgr2=0x7fd8cc072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 shutdown_connections 2024-09-17T03:20:45.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:20:45.816+0000 7fd8d3e00640 1 -- 172.21.15.99:0/841508498 wait complete. 2024-09-17T03:20:45.838 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T03:20:45.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T03:20:45.842 INFO:tasks.ceph.osd.0:Started 2024-09-17T03:20:45.842 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T03:20:45.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T03:20:45.846 INFO:tasks.ceph.osd.1:Started 2024-09-17T03:20:45.846 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T03:20:45.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T03:20:45.850 INFO:tasks.ceph.osd.2:Started 2024-09-17T03:20:45.850 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T03:20:45.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T03:20:45.855 INFO:tasks.ceph.osd.3:Started 2024-09-17T03:20:45.855 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T03:20:45.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T03:20:45.858 INFO:tasks.ceph.osd.4:Started 2024-09-17T03:20:45.858 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T03:20:45.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T03:20:45.861 INFO:tasks.ceph.osd.5:Started 2024-09-17T03:20:45.861 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T03:20:45.862 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T03:20:45.864 INFO:tasks.ceph.osd.6:Started 2024-09-17T03:20:45.865 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T03:20:45.865 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T03:20:45.868 INFO:tasks.ceph.osd.7:Started 2024-09-17T03:20:45.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:20:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 -- 172.21.15.46:0/435321504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82680b5c70 msgr2=0x7f826815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 --2- 172.21.15.46:0/435321504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82680b5c70 0x7f826815bb60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8268070710 tx=0x7f825801c800 comp rx=0 tx=0).stop 2024-09-17T03:20:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 -- 172.21.15.46:0/435321504 shutdown_connections 2024-09-17T03:20:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 --2- 172.21.15.46:0/435321504 >> v2:172.21.15.99:3300/0 conn(0x7f82681527b0 0x7f8268074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 --2- 172.21.15.46:0/435321504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82680b5c70 0x7f826815bb60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 -- 172.21.15.46:0/435321504 >> 172.21.15.46:0/435321504 conn(0x7f82680b2040 msgr2=0x7f82680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 -- 172.21.15.46:0/435321504 shutdown_connections 2024-09-17T03:20:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.076+0000 7f826de00640 1 -- 172.21.15.46:0/435321504 wait complete. 2024-09-17T03:20:46.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 Processor -- start 2024-09-17T03:20:46.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 -- start start 2024-09-17T03:20:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 0x7f8268149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82680b5c70 0x7f826815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82681527b0 0x7f826815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8268168b20 con 0x7f82681527b0 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82681689a0 con 0x7f82680b4de0 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f826de00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8268168ca0 con 0x7f82680b5c70 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f8267400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82681527b0 0x7f826815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f8266a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 0x7f8268149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f8266a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 0x7f8268149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:33950/0 (socket says 172.21.15.46:33950) 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f8266a00640 1 -- 172.21.15.46:0/3291536512 learned_addr learned my addr 172.21.15.46:0/3291536512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82680b5c70 0x7f826815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 -- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 msgr2=0x7f8268149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 0x7f8268149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 -- 172.21.15.46:0/3291536512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82681527b0 msgr2=0x7f826815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82681527b0 0x7f826815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 -- 172.21.15.46:0/3291536512 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82680ce0a0 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f8267400640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82681527b0 0x7f826815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f8266a00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 0x7f8268149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.080+0000 7f825fe00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82680b5c70 0x7f826815a770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f825000d680 tx=0x7f825000db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250017070 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8250004310 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82680ce5a0 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250005280 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f826814e110 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8250009820 con 0x7f82680b5c70 2024-09-17T03:20:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8250004da0 con 0x7f82680b5c70 2024-09-17T03:20:46.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82680ce310 con 0x7f82680b5c70 2024-09-17T03:20:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.084+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8250020020 con 0x7f82680b5c70 2024-09-17T03:20:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.148+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f826815bb60 con 0x7f82680b5c70 2024-09-17T03:20:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.148+0000 7f825f400640 1 -- 172.21.15.46:0/3291536512 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f82500047b0 con 0x7f82680b5c70 2024-09-17T03:20:46.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:20:46.150 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:20:45.817503+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82680b5c70 msgr2=0x7f826815a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82680b5c70 0x7f826815a770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f825000d680 tx=0x7f825000db40 comp rx=0 tx=0).stop 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 shutdown_connections 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82681527b0 0x7f826815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82680b5c70 0x7f826815a770 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 --2- 172.21.15.46:0/3291536512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82680b4de0 0x7f8268149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 >> 172.21.15.46:0/3291536512 conn(0x7f82680b2040 msgr2=0x7f82680b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 shutdown_connections 2024-09-17T03:20:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:20:46.152+0000 7f826de00640 1 -- 172.21.15.46:0/3291536512 wait complete. 2024-09-17T03:20:46.171 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T03:20:46.172 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T03:20:46.977 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 -- 172.21.15.46:0/2860669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654074b20 msgr2=0x7f065415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 --2- 172.21.15.46:0/2860669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654074b20 0x7f065415cf90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06540713e0 tx=0x7f064c01c770 comp rx=0 tx=0).stop 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 -- 172.21.15.46:0/2860669917 shutdown_connections 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 --2- 172.21.15.46:0/2860669917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654074b20 0x7f065415cf90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 --2- 172.21.15.46:0/2860669917 >> v2:172.21.15.99:3300/0 conn(0x7f0654157030 0x7f06540b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 -- 172.21.15.46:0/2860669917 >> 172.21.15.46:0/2860669917 conn(0x7f06540708c0 msgr2=0x7f0654070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 -- 172.21.15.46:0/2860669917 shutdown_connections 2024-09-17T03:20:47.129 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.128+0000 7f0658a00640 1 -- 172.21.15.46:0/2860669917 wait complete. 2024-09-17T03:20:47.131 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 Processor -- start 2024-09-17T03:20:47.131 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 -- start start 2024-09-17T03:20:47.135 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0654157030 0x7f0654153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:47.135 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 0x7f06540b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0654140dd0 0x7f06541544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f065416a790 con 0x7f0654160d50 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f065416a610 con 0x7f0654157030 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f065416a910 con 0x7f0654140dd0 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0649400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 0x7f06540b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0649400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 0x7f06540b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41410/0 (socket says 172.21.15.46:41410) 2024-09-17T03:20:47.136 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0649400640 1 -- 172.21.15.46:0/1610504189 learned_addr learned my addr 172.21.15.46:0/1610504189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0654157030 0x7f0654153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0652800640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0654140dd0 0x7f06541544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 -- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0654140dd0 msgr2=0x7f06541544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0654140dd0 0x7f06541544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 -- 172.21.15.46:0/1610504189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 msgr2=0x7f06540b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 0x7f06540b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.137 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 -- 172.21.15.46:0/1610504189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06541532e0 con 0x7f0654157030 2024-09-17T03:20:47.138 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0649400640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 0x7f06540b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:20:47.138 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0651e00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0654157030 0x7f0654153da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f064400ea40 tx=0x7f064400ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:47.138 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064400ce40 con 0x7f0654157030 2024-09-17T03:20:47.138 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0644004c20 con 0x7f0654157030 2024-09-17T03:20:47.138 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06540b4c10 con 0x7f0654157030 2024-09-17T03:20:47.138 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644005020 con 0x7f0654157030 2024-09-17T03:20:47.139 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541535a0 con 0x7f0654157030 2024-09-17T03:20:47.139 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0644005550 con 0x7f0654157030 2024-09-17T03:20:47.139 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.132+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0644005840 con 0x7f0654157030 2024-09-17T03:20:47.139 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.136+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06540d2260 con 0x7f0654157030 2024-09-17T03:20:47.140 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.136+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0644018470 con 0x7f0654157030 2024-09-17T03:20:47.186 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.184+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 2) ==== 48655+0+0 (secure 0 0 0) 0x7f0644018650 con 0x7f0654157030 2024-09-17T03:20:47.208 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0654156740 con 0x7f0654157030 2024-09-17T03:20:47.208 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f064aa00640 1 -- 172.21.15.46:0/1610504189 <== mon.1 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0644012760 con 0x7f0654157030 2024-09-17T03:20:47.208 INFO:teuthology.misc.health.smithi046.stdout: 2024-09-17T03:20:47.209 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:20:45.817503+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0654157030 msgr2=0x7f0654153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0654157030 0x7f0654153da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f064400ea40 tx=0x7f064400ef00 comp rx=0 tx=0).stop 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 shutdown_connections 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0654140dd0 0x7f06541544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0654160d50 0x7f06540b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 --2- 172.21.15.46:0/1610504189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0654157030 0x7f0654153da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 >> 172.21.15.46:0/1610504189 conn(0x7f06540708c0 msgr2=0x7f065415e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 shutdown_connections 2024-09-17T03:20:47.210 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:47.208+0000 7f0658a00640 1 -- 172.21.15.46:0/1610504189 wait complete. 2024-09-17T03:20:47.232 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T03:20:47.498 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-17T03:20:47.496+0000 7f19c16577c0 -1 Falling back to public interface 2024-09-17T03:20:47.532 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-17T03:20:47.532+0000 7fc981d0b7c0 -1 Falling back to public interface 2024-09-17T03:20:47.556 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-17T03:20:47.556+0000 7fb3b553c7c0 -1 Falling back to public interface 2024-09-17T03:20:47.592 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-17T03:20:47.592+0000 7fae877297c0 -1 Falling back to public interface 2024-09-17T03:20:47.594 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-17T03:20:47.592+0000 7fcad18ab7c0 -1 Falling back to public interface 2024-09-17T03:20:47.614 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-17T03:20:47.612+0000 7f46cadcf7c0 -1 Falling back to public interface 2024-09-17T03:20:47.636 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-17T03:20:47.636+0000 7fc1f1f437c0 -1 Falling back to public interface 2024-09-17T03:20:47.774 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-17T03:20:47.772+0000 7f68956e67c0 -1 Falling back to public interface 2024-09-17T03:20:48.197 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:20:48.195+0000 7fbe78400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T03:20:48.199 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:20:48.199+0000 7fbe78400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T03:20:51.718 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-17T03:20:51.716+0000 7f19c16577c0 -1 osd.5 0 log_to_monitors true 2024-09-17T03:20:51.719 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-17T03:20:51.715+0000 7fc981d0b7c0 -1 osd.1 0 log_to_monitors true 2024-09-17T03:20:51.793 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-17T03:20:51.791+0000 7fb3b553c7c0 -1 osd.0 0 log_to_monitors true 2024-09-17T03:20:51.873 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-17T03:20:51.872+0000 7f46cadcf7c0 -1 osd.6 0 log_to_monitors true 2024-09-17T03:20:51.890 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-17T03:20:51.888+0000 7fcad18ab7c0 -1 osd.4 0 log_to_monitors true 2024-09-17T03:20:51.906 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-17T03:20:51.903+0000 7fc1f1f437c0 -1 osd.2 0 log_to_monitors true 2024-09-17T03:20:51.912 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-17T03:20:51.911+0000 7fae877297c0 -1 osd.3 0 log_to_monitors true 2024-09-17T03:20:52.502 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-17T03:20:52.500+0000 7f68956e67c0 -1 osd.7 0 log_to_monitors true 2024-09-17T03:20:53.912 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-17T03:20:53.911+0000 7fc97cc00640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T03:20:53.944 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-09-17T03:20:53.939+0000 7fc972c00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:53.955 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-17T03:20:53.952+0000 7f19bc600640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T03:20:54.002 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-17T03:20:54.000+0000 7f19b2600640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:54.040 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.251+0000 7f4666400640 1 -- 172.21.15.46:0/1280929403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600cda80 msgr2=0x7f46600cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.251+0000 7f4666400640 1 --2- 172.21.15.46:0/1280929403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600cda80 0x7f46600cde90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4658009bc0 tx=0x7f465801cab0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.251+0000 7f4666400640 1 -- 172.21.15.46:0/1280929403 shutdown_connections 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.251+0000 7f4666400640 1 --2- 172.21.15.46:0/1280929403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600cda80 0x7f46600cde90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.251+0000 7f4666400640 1 --2- 172.21.15.46:0/1280929403 >> v2:172.21.15.46:3300/0 conn(0x7f46600746e0 0x7f466015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.251+0000 7f4666400640 1 -- 172.21.15.46:0/1280929403 >> 172.21.15.46:0/1280929403 conn(0x7f46600b41f0 msgr2=0x7f46600b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.255+0000 7f4666400640 1 -- 172.21.15.46:0/1280929403 shutdown_connections 2024-09-17T03:20:54.257 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.255+0000 7f4666400640 1 -- 172.21.15.46:0/1280929403 wait complete. 2024-09-17T03:20:54.259 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 Processor -- start 2024-09-17T03:20:54.259 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 -- start start 2024-09-17T03:20:54.262 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 0x7f466014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:54.262 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4660074c30 0x7f46600cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:54.262 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46600cda80 0x7f46600d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:54.262 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4660169f30 con 0x7f46600cda80 2024-09-17T03:20:54.262 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4660169db0 con 0x7f4660074c30 2024-09-17T03:20:54.262 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4666400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f466016a0b0 con 0x7f46600746e0 2024-09-17T03:20:54.263 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4665000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 0x7f466014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:54.263 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4665000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 0x7f466014f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:48052/0 (socket says 172.21.15.46:48052) 2024-09-17T03:20:54.263 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f4665000640 1 -- 172.21.15.46:0/3617290289 learned_addr learned my addr 172.21.15.46:0/3617290289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:20:54.263 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.259+0000 7f465fe00640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4660074c30 0x7f46600cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4665000640 1 -- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4660074c30 msgr2=0x7f46600cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4665000640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4660074c30 0x7f46600cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4665000640 1 -- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46600cda80 msgr2=0x7f46600d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4665000640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46600cda80 0x7f46600d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4665000640 1 -- 172.21.15.46:0/3617290289 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466014ece0 con 0x7f46600746e0 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4665000640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 0x7f466014f960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f465400b440 tx=0x7f465400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:54.264 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654015020 con 0x7f46600746e0 2024-09-17T03:20:54.265 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46540119a0 con 0x7f46600746e0 2024-09-17T03:20:54.265 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654011df0 con 0x7f46600746e0 2024-09-17T03:20:54.265 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601fcce0 con 0x7f46600746e0 2024-09-17T03:20:54.265 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4660154cf0 con 0x7f46600746e0 2024-09-17T03:20:54.265 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.263+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466015c920 con 0x7f46600746e0 2024-09-17T03:20:54.267 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.267+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f4654018020 con 0x7f46600746e0 2024-09-17T03:20:54.268 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.267+0000 7f465cc00640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46400731a0 0x7f46400756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:20:54.268 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.267+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f46540935a0 con 0x7f46600746e0 2024-09-17T03:20:54.269 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.267+0000 7f465fe00640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46400731a0 0x7f46400756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:20:54.270 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.267+0000 7f465fe00640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46400731a0 0x7f46400756b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f46600b3970 tx=0x7f464c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:20:54.276 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.275+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f465401f020 con 0x7f46600746e0 2024-09-17T03:20:54.585 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.583+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46600b4f30 con 0x7f46600746e0 2024-09-17T03:20:54.585 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.583+0000 7f465cc00640 1 -- 172.21.15.46:0/3617290289 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f465400d040 con 0x7f46600746e0 2024-09-17T03:20:54.586 INFO:teuthology.misc.health.smithi046.stdout: 2024-09-17T03:20:54.586 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":13,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:20:53.781148+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:20:54.589 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46400731a0 msgr2=0x7f46400756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:54.589 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46400731a0 0x7f46400756b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f46600b3970 tx=0x7f464c009040 comp rx=0 tx=0).stop 2024-09-17T03:20:54.589 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 msgr2=0x7f466014f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 0x7f466014f960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f465400b440 tx=0x7f465400b900 comp rx=0 tx=0).stop 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 shutdown_connections 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46400731a0 0x7f46400756b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46600cda80 0x7f46600d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4660074c30 0x7f46600cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 --2- 172.21.15.46:0/3617290289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46600746e0 0x7f466014f960 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 >> 172.21.15.46:0/3617290289 conn(0x7f46600b41f0 msgr2=0x7f4660075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:20:54.590 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 shutdown_connections 2024-09-17T03:20:54.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:20:54.587+0000 7f4666400640 1 -- 172.21.15.46:0/3617290289 wait complete. 2024-09-17T03:20:54.621 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T03:20:55.356 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-17T03:20:55.355+0000 7fb3b0e00640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T03:20:55.359 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-17T03:20:55.359+0000 7fae82600640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T03:20:55.380 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-09-17T03:20:55.379+0000 7fb3a6400640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:55.396 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-09-17T03:20:55.395+0000 7fae78600640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:55.412 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-17T03:20:55.411+0000 7fc1ece00640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T03:20:55.432 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-09-17T03:20:55.431+0000 7fc1e2e00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:55.624 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-17T03:20:55.620+0000 7f46c6600640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T03:20:55.650 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-17T03:20:55.648+0000 7f6891000640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T03:20:55.658 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-17T03:20:55.656+0000 7f46bbc00640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:55.674 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-17T03:20:55.672+0000 7f6886600640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:55.726 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-17T03:20:55.724+0000 7fcacc800640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T03:20:55.754 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-17T03:20:55.752+0000 7fcac2800640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:20:58.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:20:58.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:21:01.428 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 -- 172.21.15.46:0/55770118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 msgr2=0x7f136c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 --2- 172.21.15.46:0/55770118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c0b5a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f135c009840 tx=0x7f135c01c820 comp rx=0 tx=0).stop 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 -- 172.21.15.46:0/55770118 shutdown_connections 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 --2- 172.21.15.46:0/55770118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 0x7f136c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 --2- 172.21.15.46:0/55770118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c0b5a80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 -- 172.21.15.46:0/55770118 >> 172.21.15.46:0/55770118 conn(0x7f136c0708c0 msgr2=0x7f136c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 -- 172.21.15.46:0/55770118 shutdown_connections 2024-09-17T03:21:01.591 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.587+0000 7f1373a00640 1 -- 172.21.15.46:0/55770118 wait complete. 2024-09-17T03:21:01.593 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 Processor -- start 2024-09-17T03:21:01.593 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 -- start start 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 0x7f136c14ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f136c161160 0x7f136c0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f136c16a790 con 0x7f136c157030 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f136c16a610 con 0x7f136c161160 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1373a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f136c16a910 con 0x7f136c160d50 2024-09-17T03:21:01.596 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42954/0 (socket says 172.21.15.46:42954) 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 -- 172.21.15.46:0/3010279265 learned_addr learned my addr 172.21.15.46:0/3010279265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1363e00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 0x7f136c14ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f136c161160 0x7f136c0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 -- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 msgr2=0x7f136c14ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 0x7f136c14ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 -- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f136c161160 msgr2=0x7f136c0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f136c161160 0x7f136c0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371000640 1 -- 172.21.15.46:0/3010279265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c1fa750 con 0x7f136c157030 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1363e00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 0x7f136c14ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:01.597 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.591+0000 7f1371a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f136c161160 0x7f136c0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:01.598 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1371000640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c152130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f135c009790 tx=0x7f135c0a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:01.598 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0043c0 con 0x7f136c157030 2024-09-17T03:21:01.598 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f135c0be070 con 0x7f136c157030 2024-09-17T03:21:01.598 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c003860 con 0x7f136c157030 2024-09-17T03:21:01.598 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c1faa30 con 0x7f136c157030 2024-09-17T03:21:01.599 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c1faff0 con 0x7f136c157030 2024-09-17T03:21:01.599 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136c0d2260 con 0x7f136c157030 2024-09-17T03:21:01.599 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f135c003a00 con 0x7f136c157030 2024-09-17T03:21:01.599 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1369600640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1348073200 0x7f1348075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:01.600 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f135c12f120 con 0x7f136c157030 2024-09-17T03:21:01.600 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.595+0000 7f1363e00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1348073200 0x7f1348075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:01.600 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.599+0000 7f1363e00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1348073200 0x7f1348075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f136c0b5f70 tx=0x7f1354005d50 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:01.613 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.611+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f135c0c8020 con 0x7f136c157030 2024-09-17T03:21:01.789 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.787+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f136c06b250 con 0x7f136c157030 2024-09-17T03:21:01.790 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.787+0000 7f1369600640 1 -- 172.21.15.46:0/3010279265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12860 (secure 0 0 0) 0x7f135c016410 con 0x7f136c157030 2024-09-17T03:21:01.790 INFO:teuthology.misc.health.smithi046.stdout: 2024-09-17T03:21:01.791 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":17,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:20:57.960808+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:01.792 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1348073200 msgr2=0x7f1348075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1348073200 0x7f1348075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f136c0b5f70 tx=0x7f1354005d50 comp rx=0 tx=0).stop 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 msgr2=0x7f136c152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c152130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f135c009790 tx=0x7f135c0a67b0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 shutdown_connections 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1348073200 0x7f1348075710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f136c161160 0x7f136c0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f136c160d50 0x7f136c14ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 --2- 172.21.15.46:0/3010279265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f136c157030 0x7f136c152130 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 >> 172.21.15.46:0/3010279265 conn(0x7f136c0708c0 msgr2=0x7f136c15f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 shutdown_connections 2024-09-17T03:21:01.793 INFO:teuthology.misc.health.smithi046.stderr:2024-09-17T03:21:01.791+0000 7f1373a00640 1 -- 172.21.15.46:0/3010279265 wait complete. 2024-09-17T03:21:01.811 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T03:21:01.811 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T03:21:01.811 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:21:01.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:21:01.815 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:21:01.815 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:21:01.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:21:01.818 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:21:01.818 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:21:01.818 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:21:01.821 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:21:01.822 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:21:01.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:21:01.825 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:21:01.825 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T03:21:01.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:01.922 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:21:01.926 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:21:01.953 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:21:01.953 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:21:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 -- 172.21.15.46:0/1610735615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c074b20 msgr2=0x7f6d3c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 --2- 172.21.15.46:0/1610735615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c074b20 0x7f6d3c15cf90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009bc0 tx=0x7f6d3401cb10 comp rx=0 tx=0).stop 2024-09-17T03:21:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 -- 172.21.15.46:0/1610735615 shutdown_connections 2024-09-17T03:21:01.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 --2- 172.21.15.46:0/1610735615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c074b20 0x7f6d3c15cf90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 -- 172.21.15.46:0/1610735615 >> 172.21.15.46:0/1610735615 conn(0x7f6d3c0708c0 msgr2=0x7f6d3c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 -- 172.21.15.46:0/1610735615 shutdown_connections 2024-09-17T03:21:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.991+0000 7f6d41e00640 1 -- 172.21.15.46:0/1610735615 wait complete. 2024-09-17T03:21:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 Processor -- start 2024-09-17T03:21:01.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 -- start start 2024-09-17T03:21:01.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 0x7f6d3c144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d3c157030 0x7f6d3c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c14afc0 0x7f6d3c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c16b590 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c16b410 con 0x7f6d3c14afc0 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d41e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c16b710 con 0x7f6d3c157030 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 0x7f6d3c144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 0x7f6d3c144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42982/0 (socket says 172.21.15.46:42982) 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d3aa00640 1 -- 172.21.15.46:0/163797239 learned_addr learned my addr 172.21.15.46:0/163797239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d33e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d3c157030 0x7f6d3c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.995+0000 7f6d3b400640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c14afc0 0x7f6d3c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:02.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d3aa00640 1 -- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d3c157030 msgr2=0x7f6d3c14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d3aa00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d3c157030 0x7f6d3c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d3aa00640 1 -- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c14afc0 msgr2=0x7f6d3c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d3aa00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c14afc0 0x7f6d3c14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d3aa00640 1 -- 172.21.15.46:0/163797239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c14c770 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d3aa00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 0x7f6d3c144000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c3f0 tx=0x7f6d2c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c011790 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d2c017070 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c011ce0 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c14ca50 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c14d010 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c15cf90 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d2c01c460 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:01.999+0000 7f6d33400640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d18073200 0x7f6d18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.003+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6d2c093bb0 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.003+0000 7f6d33e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d18073200 0x7f6d18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:02.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.003+0000 7f6d33e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d18073200 0x7f6d18075710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0b7600 tx=0x7f6d24009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.007+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d2c05e960 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.211+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d3c0d2260 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.211+0000 7f6d33400640 1 -- 172.21.15.46:0/163797239 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1433 (secure 0 0 0) 0x7f6d2c063390 con 0x7f6d3c0b5a80 2024-09-17T03:21:02.215 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:02.216 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"btime":"2024-09-17T03:21:01:932253+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T03:21:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-09-17T03:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d18073200 msgr2=0x7f6d18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d18073200 0x7f6d18075710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0b7600 tx=0x7f6d24009040 comp rx=0 tx=0).stop 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 msgr2=0x7f6d3c144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 0x7f6d3c144000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c3f0 tx=0x7f6d2c00c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 shutdown_connections 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d18073200 0x7f6d18075710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d3c14afc0 0x7f6d3c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d3c157030 0x7f6d3c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 --2- 172.21.15.46:0/163797239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d3c0b5a80 0x7f6d3c144000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 >> 172.21.15.46:0/163797239 conn(0x7f6d3c0708c0 msgr2=0x7f6d3c15e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 shutdown_connections 2024-09-17T03:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:02.215+0000 7f6d41e00640 1 -- 172.21.15.46:0/163797239 wait complete. 2024-09-17T03:21:04.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 -- 172.21.15.46:0/1637217925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94801536f0 msgr2=0x7f948015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 --2- 172.21.15.46:0/1637217925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94801536f0 0x7f948015d480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9474009840 tx=0x7f947401c790 comp rx=0 tx=0).stop 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 -- 172.21.15.46:0/1637217925 shutdown_connections 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 --2- 172.21.15.46:0/1637217925 >> v2:172.21.15.99:3300/0 conn(0x7f9480160060 0x7f9480074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 --2- 172.21.15.46:0/1637217925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94801536f0 0x7f948015d480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 -- 172.21.15.46:0/1637217925 >> 172.21.15.46:0/1637217925 conn(0x7f94800b4560 msgr2=0x7f94800d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:04.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 -- 172.21.15.46:0/1637217925 shutdown_connections 2024-09-17T03:21:04.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.406+0000 7f9487000640 1 -- 172.21.15.46:0/1637217925 wait complete. 2024-09-17T03:21:04.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 Processor -- start 2024-09-17T03:21:04.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 -- start start 2024-09-17T03:21:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94801536f0 0x7f94801527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 0x7f948006e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9480160470 0x7f948015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f948016a440 con 0x7f9480160060 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f948016a2c0 con 0x7f94801536f0 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9487000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f948016a5c0 con 0x7f9480160470 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9485200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 0x7f948006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9485200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 0x7f948006e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42368/0 (socket says 172.21.15.46:42368) 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9485200640 1 -- 172.21.15.46:0/2326786614 learned_addr learned my addr 172.21.15.46:0/2326786614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9485c00640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94801536f0 0x7f94801527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.410+0000 7f9486600640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9480160470 0x7f948015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9485200640 1 -- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9480160470 msgr2=0x7f948015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9485200640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9480160470 0x7f948015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9485200640 1 -- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94801536f0 msgr2=0x7f94801527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9485200640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94801536f0 0x7f94801527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9485200640 1 -- 172.21.15.46:0/2326786614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948014ee00 con 0x7f9480160060 2024-09-17T03:21:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9485200640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 0x7f948006e960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e9f0 tx=0x7f946c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00cd80 con 0x7f9480160060 2024-09-17T03:21:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f946c004020 con 0x7f9480160060 2024-09-17T03:21:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c004470 con 0x7f9480160060 2024-09-17T03:21:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948014f0e0 con 0x7f9480160060 2024-09-17T03:21:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948006ce80 con 0x7f9480160060 2024-09-17T03:21:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9480152d90 con 0x7f9480160060 2024-09-17T03:21:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.414+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f946c003600 con 0x7f9480160060 2024-09-17T03:21:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.418+0000 7f947d600640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9460073200 0x7f9460075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.418+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f946c094690 con 0x7f9480160060 2024-09-17T03:21:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.418+0000 7f9485c00640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9460073200 0x7f9460075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.418+0000 7f9485c00640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9460073200 0x7f9460075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94740ae000 tx=0x7f947401afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.422+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f946c05f420 con 0x7f9480160060 2024-09-17T03:21:04.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.630+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f948015d480 con 0x7f9480160060 2024-09-17T03:21:04.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.630+0000 7f947d600640 1 -- 172.21.15.46:0/2326786614 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3911 (secure 0 0 0) 0x7f946c063e50 con 0x7f9480160060 2024-09-17T03:21:04.635 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:04.635 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"btime":"2024-09-17T03:21:02:933777+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T03:21:04.635 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9460073200 msgr2=0x7f9460075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9460073200 0x7f9460075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94740ae000 tx=0x7f947401afd0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 msgr2=0x7f948006e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 0x7f948006e960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e9f0 tx=0x7f946c00edf0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 shutdown_connections 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9460073200 0x7f9460075710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9480160470 0x7f948015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9480160060 0x7f948006e960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 --2- 172.21.15.46:0/2326786614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94801536f0 0x7f94801527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 >> 172.21.15.46:0/2326786614 conn(0x7f94800b4560 msgr2=0x7f94800715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 shutdown_connections 2024-09-17T03:21:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.634+0000 7f9487000640 1 -- 172.21.15.46:0/2326786614 wait complete. 2024-09-17T03:21:04.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:21:04.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 -- 172.21.15.46:0/1346972928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40b5a80 msgr2=0x7f30d4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 --2- 172.21.15.46:0/1346972928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40b5a80 0x7f30d4074410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009900 tx=0x7f30bc01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 -- 172.21.15.46:0/1346972928 shutdown_connections 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 --2- 172.21.15.46:0/1346972928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d4074950 0x7f30d415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 --2- 172.21.15.46:0/1346972928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40b5a80 0x7f30d4074410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 -- 172.21.15.46:0/1346972928 >> 172.21.15.46:0/1346972928 conn(0x7f30d40708c0 msgr2=0x7f30d4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 -- 172.21.15.46:0/1346972928 shutdown_connections 2024-09-17T03:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.810+0000 7f30da000640 1 -- 172.21.15.46:0/1346972928 wait complete. 2024-09-17T03:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 Processor -- start 2024-09-17T03:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 -- start start 2024-09-17T03:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 0x7f30d41492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d40b5a80 0x7f30d414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d4157030 0x7f30d4150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30d416a310 con 0x7f30d4074950 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30d416a190 con 0x7f30d40b5a80 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f30d416a490 con 0x7f30d4157030 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 0x7f30d41492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 0x7f30d41492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42382/0 (socket says 172.21.15.46:42382) 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 -- 172.21.15.46:0/3636493141 learned_addr learned my addr 172.21.15.46:0/3636493141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30cbe00640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d40b5a80 0x7f30d414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3e00640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d4157030 0x7f30d4150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 -- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d4157030 msgr2=0x7f30d4150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d4157030 0x7f30d4150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 -- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d40b5a80 msgr2=0x7f30d414e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d40b5a80 0x7f30d414e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 -- 172.21.15.46:0/3636493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d41fb610 con 0x7f30d4074950 2024-09-17T03:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30d3400640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 0x7f30d41492e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f30c400c3c0 tx=0x7f30c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4017070 con 0x7f30d4074950 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30c40053c0 con 0x7f30d4074950 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4003860 con 0x7f30d4074950 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d414df30 con 0x7f30d4074950 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.814+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41fad70 con 0x7f30d4074950 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.818+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d41f70c0 con 0x7f30d4074950 2024-09-17T03:21:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.818+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30c401a070 con 0x7f30d4074950 2024-09-17T03:21:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.818+0000 7f30cb400640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b0073230 0x7f30b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.818+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f30c4093290 con 0x7f30d4074950 2024-09-17T03:21:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.822+0000 7f30cbe00640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b0073230 0x7f30b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.822+0000 7f30cbe00640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b0073230 0x7f30b0075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0af000 tx=0x7f30bc01b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.826+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30d4074d60 con 0x7f30d4074950 2024-09-17T03:21:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:04.998+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f30d415cf90 con 0x7f30d4074950 2024-09-17T03:21:05.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30cb400640 1 -- 172.21.15.46:0/3636493141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f30c405df20 con 0x7f30d4074950 2024-09-17T03:21:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:05.005 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:21:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:21:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:21:05.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:05.005 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:21:05.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b0073230 msgr2=0x7f30b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:05.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b0073230 0x7f30b0075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0af000 tx=0x7f30bc01b120 comp rx=0 tx=0).stop 2024-09-17T03:21:05.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 msgr2=0x7f30d41492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 0x7f30d41492e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f30c400c3c0 tx=0x7f30c400c880 comp rx=0 tx=0).stop 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 shutdown_connections 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b0073230 0x7f30b0075740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d4157030 0x7f30d4150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d40b5a80 0x7f30d414e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 --2- 172.21.15.46:0/3636493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d4074950 0x7f30d41492e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 >> 172.21.15.46:0/3636493141 conn(0x7f30d40708c0 msgr2=0x7f30d40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 shutdown_connections 2024-09-17T03:21:05.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.002+0000 7f30da000640 1 -- 172.21.15.46:0/3636493141 wait complete. 2024-09-17T03:21:05.026 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:21:05.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 -- 172.21.15.46:0/2822310735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4157030 msgr2=0x7f3ee40b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 --2- 172.21.15.46:0/2822310735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4157030 0x7f3ee40b5a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009900 tx=0x7f3ed801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 -- 172.21.15.46:0/2822310735 shutdown_connections 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 --2- 172.21.15.46:0/2822310735 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ee4160d50 0x7f3ee4074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 --2- 172.21.15.46:0/2822310735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4157030 0x7f3ee40b5a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 -- 172.21.15.46:0/2822310735 >> 172.21.15.46:0/2822310735 conn(0x7f3ee40708c0 msgr2=0x7f3ee4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 -- 172.21.15.46:0/2822310735 shutdown_connections 2024-09-17T03:21:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 -- 172.21.15.46:0/2822310735 wait complete. 2024-09-17T03:21:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 Processor -- start 2024-09-17T03:21:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.186+0000 7f3eebc00640 1 -- start start 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4157030 0x7f3ee4145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 0x7f3ee41f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ee4161160 0x7f3ee415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee416a820 con 0x7f3ee4160d50 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee416a6a0 con 0x7f3ee4157030 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee416a9a0 con 0x7f3ee4161160 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 0x7f3ee41f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 0x7f3ee41f9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42398/0 (socket says 172.21.15.46:42398) 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3646691000 learned_addr learned my addr 172.21.15.46:0/3646691000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee9200640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4157030 0x7f3ee4145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee9c00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ee4161160 0x7f3ee415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ee4161160 msgr2=0x7f3ee415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ee4161160 0x7f3ee415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4157030 msgr2=0x7f3ee4145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4157030 0x7f3ee4145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 -- 172.21.15.46:0/3646691000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee414fc20 con 0x7f3ee4160d50 2024-09-17T03:21:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee3e00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 0x7f3ee41f9f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000d6a0 tx=0x7f3ed000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0017070 con 0x7f3ee4160d50 2024-09-17T03:21:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed0004310 con 0x7f3ee4160d50 2024-09-17T03:21:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0005280 con 0x7f3ee4160d50 2024-09-17T03:21:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee414ff00 con 0x7f3ee4160d50 2024-09-17T03:21:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee41504c0 con 0x7f3ee4160d50 2024-09-17T03:21:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.190+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee40d2260 con 0x7f3ee4160d50 2024-09-17T03:21:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.194+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ed0005420 con 0x7f3ee4160d50 2024-09-17T03:21:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.194+0000 7f3ee0c00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ec4073200 0x7f3ec4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.194+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3ed0094180 con 0x7f3ee4160d50 2024-09-17T03:21:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.194+0000 7f3ee9200640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ec4073200 0x7f3ec4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.194+0000 7f3ee9200640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ec4073200 0x7f3ec4075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed801b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:05.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.198+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ed0020020 con 0x7f3ee4160d50 2024-09-17T03:21:05.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.374+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3ee40b1a90 con 0x7f3ee4160d50 2024-09-17T03:21:06.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:05.998+0000 7f3ee0c00640 1 -- 172.21.15.46:0/3646691000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f3ed005ee90 con 0x7f3ee4160d50 2024-09-17T03:21:06.003 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:21:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.002+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ec4073200 msgr2=0x7f3ec4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.002+0000 7f3eebc00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ec4073200 0x7f3ec4075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed801b120 comp rx=0 tx=0).stop 2024-09-17T03:21:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.002+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 msgr2=0x7f3ee41f9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.002+0000 7f3eebc00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 0x7f3ee41f9f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000d6a0 tx=0x7f3ed000db60 comp rx=0 tx=0).stop 2024-09-17T03:21:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 shutdown_connections 2024-09-17T03:21:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ec4073200 0x7f3ec4075710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ee4161160 0x7f3ee415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ee4160d50 0x7f3ee41f9f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 --2- 172.21.15.46:0/3646691000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee4157030 0x7f3ee4145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 >> 172.21.15.46:0/3646691000 conn(0x7f3ee40708c0 msgr2=0x7f3ee415e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 shutdown_connections 2024-09-17T03:21:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.006+0000 7f3eebc00640 1 -- 172.21.15.46:0/3646691000 wait complete. 2024-09-17T03:21:06.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.202+0000 7eff58c00640 1 -- 172.21.15.46:0/4019950927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 msgr2=0x7eff540b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.202+0000 7eff58c00640 1 --2- 172.21.15.46:0/4019950927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff540b5a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff44009900 tx=0x7eff4401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 -- 172.21.15.46:0/4019950927 shutdown_connections 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 --2- 172.21.15.46:0/4019950927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 0x7eff54074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 --2- 172.21.15.46:0/4019950927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff540b5a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 -- 172.21.15.46:0/4019950927 >> 172.21.15.46:0/4019950927 conn(0x7eff540708c0 msgr2=0x7eff54070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 -- 172.21.15.46:0/4019950927 shutdown_connections 2024-09-17T03:21:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 -- 172.21.15.46:0/4019950927 wait complete. 2024-09-17T03:21:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 Processor -- start 2024-09-17T03:21:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 -- start start 2024-09-17T03:21:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff541439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 0x7eff54148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.206+0000 7eff58c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff54161160 0x7eff5414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff58c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff5416a820 con 0x7eff54157030 2024-09-17T03:21:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff58c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff5416a6a0 con 0x7eff54161160 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff58c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7eff5416a9a0 con 0x7eff54160d50 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff541439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff541439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42424/0 (socket says 172.21.15.46:42424) 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 -- 172.21.15.46:0/2394058251 learned_addr learned my addr 172.21.15.46:0/2394058251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff52800640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff54161160 0x7eff5414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff4be00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 0x7eff54148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 -- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 msgr2=0x7eff54148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 0x7eff54148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 -- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff54161160 msgr2=0x7eff5414ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff54161160 0x7eff5414ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 -- 172.21.15.46:0/2394058251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5414f810 con 0x7eff54157030 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff4be00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 0x7eff54148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff51e00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff541439b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff44009850 tx=0x7eff440a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440b2040 con 0x7eff54157030 2024-09-17T03:21:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff440043b0 con 0x7eff54157030 2024-09-17T03:21:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44005290 con 0x7eff54157030 2024-09-17T03:21:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff54143ef0 con 0x7eff54157030 2024-09-17T03:21:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff541512b0 con 0x7eff54157030 2024-09-17T03:21:06.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff540d2260 con 0x7eff54157030 2024-09-17T03:21:06.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.210+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7eff440bd060 con 0x7eff54157030 2024-09-17T03:21:06.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.214+0000 7eff4a000640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30073230 0x7eff30075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:06.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.214+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7eff4412dc40 con 0x7eff54157030 2024-09-17T03:21:06.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.214+0000 7eff4be00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30073230 0x7eff30075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:06.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.214+0000 7eff4be00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30073230 0x7eff30075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005fd0 tx=0x7eff3c005de0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:06.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.218+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7eff440164d0 con 0x7eff54157030 2024-09-17T03:21:06.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:06.394+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7eff540b1a90 con 0x7eff54157030 2024-09-17T03:21:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.002+0000 7eff4a000640 1 -- 172.21.15.46:0/2394058251 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7eff440f87e0 con 0x7eff54157030 2024-09-17T03:21:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:21:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30073230 msgr2=0x7eff30075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30073230 0x7eff30075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005fd0 tx=0x7eff3c005de0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 msgr2=0x7eff541439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff541439b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff44009850 tx=0x7eff440a5620 comp rx=0 tx=0).stop 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 shutdown_connections 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30073230 0x7eff30075740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff54161160 0x7eff5414ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff54160d50 0x7eff54148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 --2- 172.21.15.46:0/2394058251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff54157030 0x7eff541439b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 >> 172.21.15.46:0/2394058251 conn(0x7eff540708c0 msgr2=0x7eff5415e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 shutdown_connections 2024-09-17T03:21:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.006+0000 7eff58c00640 1 -- 172.21.15.46:0/2394058251 wait complete. 2024-09-17T03:21:07.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 -- 172.21.15.46:0/897709217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a80cd2e0 msgr2=0x7f06a8070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 --2- 172.21.15.46:0/897709217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a80cd2e0 0x7f06a8070e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0698009900 tx=0x7f069801c880 comp rx=0 tx=0).stop 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 -- 172.21.15.46:0/897709217 shutdown_connections 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 --2- 172.21.15.46:0/897709217 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06a81602a0 0x7f06a81606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 --2- 172.21.15.46:0/897709217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a80cd2e0 0x7f06a8070e80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 -- 172.21.15.46:0/897709217 >> 172.21.15.46:0/897709217 conn(0x7f06a80b2c40 msgr2=0x7f06a80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 -- 172.21.15.46:0/897709217 shutdown_connections 2024-09-17T03:21:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.206+0000 7f06ad800640 1 -- 172.21.15.46:0/897709217 wait complete. 2024-09-17T03:21:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 Processor -- start 2024-09-17T03:21:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 -- start start 2024-09-17T03:21:07.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a80cd2e0 0x7f06a813b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:07.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06a815cd20 0x7f06a81fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:07.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 0x7f06a81fd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:07.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06a816a3e0 con 0x7f06a81602a0 2024-09-17T03:21:07.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06a816a260 con 0x7f06a80cd2e0 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06ad800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f06a816a560 con 0x7f06a815cd20 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.210+0000 7f06a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 0x7f06a81fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a6a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a80cd2e0 0x7f06a813b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 0x7f06a81fd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42440/0 (socket says 172.21.15.46:42440) 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 -- 172.21.15.46:0/1797610157 learned_addr learned my addr 172.21.15.46:0/1797610157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f069fe00640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06a815cd20 0x7f06a81fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 -- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06a815cd20 msgr2=0x7f06a81fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06a815cd20 0x7f06a81fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 -- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a80cd2e0 msgr2=0x7f06a813b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a80cd2e0 0x7f06a813b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 -- 172.21.15.46:0/1797610157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a81fcc50 con 0x7f06a81602a0 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a6a00640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a80cd2e0 0x7f06a813b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:07.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06a7400640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 0x7f06a81fd670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c730 tx=0x7f06a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0003b80 con 0x7f06a81602a0 2024-09-17T03:21:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06a0003d20 con 0x7f06a81602a0 2024-09-17T03:21:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a8145a80 con 0x7f06a81602a0 2024-09-17T03:21:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0005330 con 0x7f06a81602a0 2024-09-17T03:21:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81fa4e0 con 0x7f06a81602a0 2024-09-17T03:21:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a8070e80 con 0x7f06a81602a0 2024-09-17T03:21:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.214+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f06a00056e0 con 0x7f06a81602a0 2024-09-17T03:21:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.218+0000 7f069ea00640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0684073230 0x7f0684075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.218+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f06a0093fe0 con 0x7f06a81602a0 2024-09-17T03:21:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.218+0000 7f06a6a00640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0684073230 0x7f0684075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:07.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.218+0000 7f06a6a00640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0684073230 0x7f0684075740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06980af000 tx=0x7f069801b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.222+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f06a005ea60 con 0x7f06a81602a0 2024-09-17T03:21:07.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:07.430+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f06a8074ba0 con 0x7f06a81602a0 2024-09-17T03:21:08.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.038+0000 7f069ea00640 1 -- 172.21.15.46:0/1797610157 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f06a0063490 con 0x7f06a81602a0 2024-09-17T03:21:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T03:21:08.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0684073230 msgr2=0x7f0684075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0684073230 0x7f0684075740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06980af000 tx=0x7f069801b120 comp rx=0 tx=0).stop 2024-09-17T03:21:08.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 msgr2=0x7f06a81fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 0x7f06a81fd670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c730 tx=0x7f06a000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 shutdown_connections 2024-09-17T03:21:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0684073230 0x7f0684075740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06a81602a0 0x7f06a81fd670 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06a815cd20 0x7f06a81fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 --2- 172.21.15.46:0/1797610157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06a80cd2e0 0x7f06a813b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.046+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 >> 172.21.15.46:0/1797610157 conn(0x7f06a80b2c40 msgr2=0x7f06a80701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:08.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.050+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 shutdown_connections 2024-09-17T03:21:08.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.050+0000 7f06ad800640 1 -- 172.21.15.46:0/1797610157 wait complete. 2024-09-17T03:21:08.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 -- 172.21.15.46:0/881131443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 msgr2=0x7f1f8c068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 --2- 172.21.15.46:0/881131443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c068df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0ce8f0 tx=0x7f1f8001c890 comp rx=0 tx=0).stop 2024-09-17T03:21:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 -- 172.21.15.46:0/881131443 shutdown_connections 2024-09-17T03:21:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 --2- 172.21.15.46:0/881131443 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f8c160920 0x7f1f8c070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 --2- 172.21.15.46:0/881131443 >> v2:172.21.15.99:3300/0 conn(0x7f1f8c069330 0x7f1f8c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 --2- 172.21.15.46:0/881131443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c068df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 -- 172.21.15.46:0/881131443 >> 172.21.15.46:0/881131443 conn(0x7f1f8c0b35f0 msgr2=0x7f1f8c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 -- 172.21.15.46:0/881131443 shutdown_connections 2024-09-17T03:21:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.246+0000 7f1f93600640 1 -- 172.21.15.46:0/881131443 wait complete. 2024-09-17T03:21:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 Processor -- start 2024-09-17T03:21:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 -- start start 2024-09-17T03:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f8c069330 0x7f1f8c1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c160920 0x7f1f8c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d450 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d2d0 con 0x7f1f8c160920 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f93600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d5d0 con 0x7f1f8c069330 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42458/0 (socket says 172.21.15.46:42458) 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 -- 172.21.15.46:0/2900300413 learned_addr learned my addr 172.21.15.46:0/2900300413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f90c00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f8c069330 0x7f1f8c1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f91600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c160920 0x7f1f8c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 -- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f8c069330 msgr2=0x7f1f8c1467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f8c069330 0x7f1f8c1467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 -- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c160920 msgr2=0x7f1f8c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c160920 0x7f1f8c15afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.250+0000 7f1f8be00640 1 -- 172.21.15.46:0/2900300413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c159f00 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f8be00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c146d00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c940 tx=0x7f1f7800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007bb0 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c15a1e0 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f78007e40 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800f690 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c15a7a0 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c06db10 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f7800f830 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.254+0000 7f1f88c00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f6c073200 0x7f1f6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.258+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f78094920 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.258+0000 7f1f90c00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f6c073200 0x7f1f6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.258+0000 7f1f90c00640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f6c073200 0x7f1f6c075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f800af000 tx=0x7f1f8001b0d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:08.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.262+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f78014030 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.434+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f8c160010 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.434+0000 7f1f88c00640 1 -- 172.21.15.46:0/2900300413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16642 (secure 0 0 0) 0x7f1f78064e20 con 0x7f1f8c15c7b0 2024-09-17T03:21:08.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:08.439 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:07.435983+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:08.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f6c073200 msgr2=0x7f1f6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f6c073200 0x7f1f6c075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f800af000 tx=0x7f1f8001b0d0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 msgr2=0x7f1f8c146d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c146d00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c940 tx=0x7f1f7800ce00 comp rx=0 tx=0).stop 2024-09-17T03:21:08.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 shutdown_connections 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f6c073200 0x7f1f6c075710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f8c160920 0x7f1f8c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f8c15c7b0 0x7f1f8c146d00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 --2- 172.21.15.46:0/2900300413 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f8c069330 0x7f1f8c1467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 >> 172.21.15.46:0/2900300413 conn(0x7f1f8c0b35f0 msgr2=0x7f1f8c0b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 shutdown_connections 2024-09-17T03:21:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.438+0000 7f1f93600640 1 -- 172.21.15.46:0/2900300413 wait complete. 2024-09-17T03:21:08.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 -- 172.21.15.46:0/1128038874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0749a0 msgr2=0x7f33ac074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 --2- 172.21.15.46:0/1128038874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0749a0 0x7f33ac074db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0cfc90 tx=0x7f339c01c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 -- 172.21.15.46:0/1128038874 shutdown_connections 2024-09-17T03:21:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 --2- 172.21.15.46:0/1128038874 >> v2:172.21.15.99:3300/0 conn(0x7f33ac0b3340 0x7f33ac0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 --2- 172.21.15.46:0/1128038874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0749a0 0x7f33ac074db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 -- 172.21.15.46:0/1128038874 >> 172.21.15.46:0/1128038874 conn(0x7f33ac0b2040 msgr2=0x7f33ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 -- 172.21.15.46:0/1128038874 shutdown_connections 2024-09-17T03:21:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 -- 172.21.15.46:0/1128038874 wait complete. 2024-09-17T03:21:08.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 Processor -- start 2024-09-17T03:21:08.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.618+0000 7f33b2000640 1 -- start start 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ac0749a0 0x7f33ac145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 0x7f33ac13d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33ac0cefa0 0x7f33ac13ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac169d20 con 0x7f33ac0b3340 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac169ba0 con 0x7f33ac0749a0 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac169ea0 con 0x7f33ac0cefa0 2024-09-17T03:21:08.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 0x7f33ac13d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 0x7f33ac13d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42490/0 (socket says 172.21.15.46:42490) 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 -- 172.21.15.46:0/700255437 learned_addr learned my addr 172.21.15.46:0/700255437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33abe00640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33ac0cefa0 0x7f33ac13ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33ab400640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ac0749a0 0x7f33ac145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 -- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33ac0cefa0 msgr2=0x7f33ac13ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33ac0cefa0 0x7f33ac13ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 -- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ac0749a0 msgr2=0x7f33ac145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ac0749a0 0x7f33ac145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 -- 172.21.15.46:0/700255437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac145ed0 con 0x7f33ac0b3340 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33aaa00640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 0x7f33ac13d8d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f339800b3e0 tx=0x7f339800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398015020 con 0x7f33ac0b3340 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3398011920 con 0x7f33ac0b3340 2024-09-17T03:21:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398011d70 con 0x7f33ac0b3340 2024-09-17T03:21:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac13de10 con 0x7f33ac0b3340 2024-09-17T03:21:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac1f9290 con 0x7f33ac0b3340 2024-09-17T03:21:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac1460a0 con 0x7f33ac0b3340 2024-09-17T03:21:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.622+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3398018020 con 0x7f33ac0b3340 2024-09-17T03:21:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.626+0000 7f33a3400640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f338c073200 0x7f338c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.626+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3398093e10 con 0x7f33ac0b3340 2024-09-17T03:21:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.626+0000 7f33ab400640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f338c073200 0x7f338c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.626+0000 7f33ab400640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f338c073200 0x7f338c075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f339c0af000 tx=0x7f339c0a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.630+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3398097030 con 0x7f33ac0b3340 2024-09-17T03:21:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33ac074db0 con 0x7f33ac0b3340 2024-09-17T03:21:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33a3400640 1 -- 172.21.15.46:0/700255437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16642 (secure 0 0 0) 0x7f339801f030 con 0x7f33ac0b3340 2024-09-17T03:21:08.810 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:08.810 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:07.435983+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f338c073200 msgr2=0x7f338c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f338c073200 0x7f338c075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f339c0af000 tx=0x7f339c0a7040 comp rx=0 tx=0).stop 2024-09-17T03:21:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 msgr2=0x7f33ac13d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 0x7f33ac13d8d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f339800b3e0 tx=0x7f339800b8a0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 shutdown_connections 2024-09-17T03:21:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f338c073200 0x7f338c075710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33ac0cefa0 0x7f33ac13ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33ac0b3340 0x7f33ac13d8d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 --2- 172.21.15.46:0/700255437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ac0749a0 0x7f33ac145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 >> 172.21.15.46:0/700255437 conn(0x7f33ac0b2040 msgr2=0x7f33ac0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 shutdown_connections 2024-09-17T03:21:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.806+0000 7f33b2000640 1 -- 172.21.15.46:0/700255437 wait complete. 2024-09-17T03:21:08.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 -- 172.21.15.46:0/1557292995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03280b4de0 msgr2=0x7f03280b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 --2- 172.21.15.46:0/1557292995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03280b4de0 0x7f03280b51f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0320009c80 tx=0x7f032001cce0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 -- 172.21.15.46:0/1557292995 shutdown_connections 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 --2- 172.21.15.46:0/1557292995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03280b4de0 0x7f03280b51f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 --2- 172.21.15.46:0/1557292995 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03281527b0 0x7f032815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 -- 172.21.15.46:0/1557292995 >> 172.21.15.46:0/1557292995 conn(0x7f03280b2040 msgr2=0x7f03280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 -- 172.21.15.46:0/1557292995 shutdown_connections 2024-09-17T03:21:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 -- 172.21.15.46:0/1557292995 wait complete. 2024-09-17T03:21:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.990+0000 7f032d400640 1 Processor -- start 2024-09-17T03:21:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 -- start start 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03280b4de0 0x7f03281519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03280b5c70 0x7f03280ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 0x7f03280aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0328168ab0 con 0x7f03281527b0 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0328168930 con 0x7f03280b5c70 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0328168c30 con 0x7f03280b4de0 2024-09-17T03:21:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 0x7f03280aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 0x7f03280aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42512/0 (socket says 172.21.15.46:42512) 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 -- 172.21.15.46:0/1514859309 learned_addr learned my addr 172.21.15.46:0/1514859309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f031de00640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03280b5c70 0x7f03280ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0326800640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03280b4de0 0x7f03281519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 -- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03280b4de0 msgr2=0x7f03281519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03280b4de0 0x7f03281519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 -- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03280b5c70 msgr2=0x7f03280ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03280b5c70 0x7f03280ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 -- 172.21.15.46:0/1514859309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0328155a20 con 0x7f03281527b0 2024-09-17T03:21:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f0327200640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 0x7f03280aee20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f032001b390 tx=0x7f0320005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200bb070 con 0x7f03281527b0 2024-09-17T03:21:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0320004b40 con 0x7f03281527b0 2024-09-17T03:21:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0328154be0 con 0x7f03281527b0 2024-09-17T03:21:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200b2040 con 0x7f03281527b0 2024-09-17T03:21:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.994+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0328154900 con 0x7f03281527b0 2024-09-17T03:21:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.998+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03280b51f0 con 0x7f03281527b0 2024-09-17T03:21:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.998+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f03200be070 con 0x7f03281527b0 2024-09-17T03:21:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.998+0000 7f031f400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0304073200 0x7f0304075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.998+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f032012dff0 con 0x7f03281527b0 2024-09-17T03:21:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.998+0000 7f0326800640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0304073200 0x7f0304075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:08.998+0000 7f0326800640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0304073200 0x7f0304075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0318007fd0 tx=0x7f0318009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:09.002+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0320016990 con 0x7f03281527b0 2024-09-17T03:21:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:09.230+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0328159970 con 0x7f03281527b0 2024-09-17T03:21:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.046+0000 7f031f400640 1 -- 172.21.15.46:0/1514859309 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f03200f89f0 con 0x7f03281527b0 2024-09-17T03:21:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0304073200 msgr2=0x7f0304075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0304073200 0x7f0304075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0318007fd0 tx=0x7f0318009040 comp rx=0 tx=0).stop 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 msgr2=0x7f03280aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 0x7f03280aee20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f032001b390 tx=0x7f0320005b00 comp rx=0 tx=0).stop 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 shutdown_connections 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0304073200 0x7f0304075710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03281527b0 0x7f03280aee20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03280b5c70 0x7f03280ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 --2- 172.21.15.46:0/1514859309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03280b4de0 0x7f03281519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 >> 172.21.15.46:0/1514859309 conn(0x7f03280b2040 msgr2=0x7f03280b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 shutdown_connections 2024-09-17T03:21:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.050+0000 7f032d400640 1 -- 172.21.15.46:0/1514859309 wait complete. 2024-09-17T03:21:10.076 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:21:10.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 -- 172.21.15.46:0/526084278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 msgr2=0x7fcfc4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/526084278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfc4003e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009b40 tx=0x7fcfd001d270 comp rx=0 tx=0).stop 2024-09-17T03:21:10.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe5c00640 1 -- 172.21.15.46:0/526084278 reap_dead start 2024-09-17T03:21:10.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 -- 172.21.15.46:0/526084278 shutdown_connections 2024-09-17T03:21:10.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 -- 172.21.15.46:0/526084278 >> 172.21.15.46:0/526084278 conn(0x7fcfe00b2040 msgr2=0x7fcfe00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:10.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 -- 172.21.15.46:0/526084278 shutdown_connections 2024-09-17T03:21:10.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 -- 172.21.15.46:0/526084278 wait complete. 2024-09-17T03:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 Processor -- start 2024-09-17T03:21:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.242+0000 7fcfe7c00640 1 -- start start 2024-09-17T03:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfe0156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcfe014dc40 0x7fcfe01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfe014ad10 0x7fcfe014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe0166950 con 0x7fcfc4003a10 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe01667d0 con 0x7fcfe014ad10 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe0166ad0 con 0x7fcfe014dc40 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfe0156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfe0156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42526/0 (socket says 172.21.15.46:42526) 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 -- 172.21.15.46:0/1769264990 learned_addr learned my addr 172.21.15.46:0/1769264990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 -- 172.21.15.46:0/1769264990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcfe014dc40 msgr2=0x7fcfe01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcfe014dc40 0x7fcfe01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 -- 172.21.15.46:0/1769264990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfe014ad10 msgr2=0x7fcfe014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfe014ad10 0x7fcfe014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 -- 172.21.15.46:0/1769264990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe0155910 con 0x7fcfc4003a10 2024-09-17T03:21:10.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe5200640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfe0156990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfd001cce0 tx=0x7fcfd0005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:10.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00bb030 con 0x7fcfc4003a10 2024-09-17T03:21:10.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfd00046d0 con 0x7fcfc4003a10 2024-09-17T03:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe0155bd0 con 0x7fcfc4003a10 2024-09-17T03:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0004b20 con 0x7fcfc4003a10 2024-09-17T03:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe0156190 con 0x7fcfc4003a10 2024-09-17T03:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe00cd670 con 0x7fcfc4003a10 2024-09-17T03:21:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.246+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcfd00be030 con 0x7fcfc4003a10 2024-09-17T03:21:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.250+0000 7fcfdd600640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfbc073200 0x7fcfbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.250+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcfd012f620 con 0x7fcfc4003a10 2024-09-17T03:21:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.250+0000 7fcfd7e00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfbc073200 0x7fcfbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.250+0000 7fcfd7e00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfbc073200 0x7fcfbc075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfe014e3b0 tx=0x7fcfc8000f20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.254+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcfd0016f80 con 0x7fcfc4003a10 2024-09-17T03:21:10.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:10.466+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcfe0158000 con 0x7fcfc4003a10 2024-09-17T03:21:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.046+0000 7fcfdd600640 1 -- 172.21.15.46:0/1769264990 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fcfd00fa5a0 con 0x7fcfc4003a10 2024-09-17T03:21:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfbc073200 msgr2=0x7fcfbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfbc073200 0x7fcfbc075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfe014e3b0 tx=0x7fcfc8000f20 comp rx=0 tx=0).stop 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 msgr2=0x7fcfe0156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfe0156990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcfd001cce0 tx=0x7fcfd0005500 comp rx=0 tx=0).stop 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 shutdown_connections 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfbc073200 0x7fcfbc075710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfe014ad10 0x7fcfe014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcfe014dc40 0x7fcfe01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 --2- 172.21.15.46:0/1769264990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfc4003a10 0x7fcfe0156990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 >> 172.21.15.46:0/1769264990 conn(0x7fcfe00b2040 msgr2=0x7fcfe0157350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 shutdown_connections 2024-09-17T03:21:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.050+0000 7fcfe7c00640 1 -- 172.21.15.46:0/1769264990 wait complete. 2024-09-17T03:21:11.079 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:21:11.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 -- 172.21.15.46:0/4103972829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00cefa0 msgr2=0x7fd6a00cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/4103972829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00cefa0 0x7fd6a00cf3b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00cfc90 tx=0x7fd68801cd80 comp rx=0 tx=0).stop 2024-09-17T03:21:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 -- 172.21.15.46:0/4103972829 shutdown_connections 2024-09-17T03:21:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/4103972829 >> v2:172.21.15.99:3300/0 conn(0x7fd6a00b3340 0x7fd6a00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/4103972829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00cefa0 0x7fd6a00cf3b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 -- 172.21.15.46:0/4103972829 >> 172.21.15.46:0/4103972829 conn(0x7fd6a00b2040 msgr2=0x7fd6a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 -- 172.21.15.46:0/4103972829 shutdown_connections 2024-09-17T03:21:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.238+0000 7fd6a5e00640 1 -- 172.21.15.46:0/4103972829 wait complete. 2024-09-17T03:21:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 Processor -- start 2024-09-17T03:21:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 -- start start 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 0x7fd6a015f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a00b3340 0x7fd6a013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6a00cefa0 0x7fd6a013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0169cd0 con 0x7fd6a00749a0 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0169b50 con 0x7fd6a00b3340 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a0169e50 con 0x7fd6a00cefa0 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 0x7fd6a015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 0x7fd6a015f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42542/0 (socket says 172.21.15.46:42542) 2024-09-17T03:21:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 -- 172.21.15.46:0/1484693813 learned_addr learned my addr 172.21.15.46:0/1484693813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd696000640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a00b3340 0x7fd6a013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69f400640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6a00cefa0 0x7fd6a013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 -- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6a00cefa0 msgr2=0x7fd6a013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6a00cefa0 0x7fd6a013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 -- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a00b3340 msgr2=0x7fd6a013e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a00b3340 0x7fd6a013e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 -- 172.21.15.46:0/1484693813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a015e5c0 con 0x7fd6a00749a0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd69ea00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 0x7fd6a015f980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd69000b440 tx=0x7fd69000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd690015020 con 0x7fd6a00749a0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd690011920 con 0x7fd6a00749a0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd690011d70 con 0x7fd6a00749a0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a015e8a0 con 0x7fd6a00749a0 2024-09-17T03:21:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.242+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a015ee60 con 0x7fd6a00749a0 2024-09-17T03:21:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.246+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a00b3750 con 0x7fd6a00749a0 2024-09-17T03:21:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.246+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd690018020 con 0x7fd6a00749a0 2024-09-17T03:21:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.246+0000 7fd697e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd67c073230 0x7fd67c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.246+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd690093d90 con 0x7fd6a00749a0 2024-09-17T03:21:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.246+0000 7fd696000640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd67c073230 0x7fd67c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.246+0000 7fd696000640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd67c073230 0x7fd67c075740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6880af000 tx=0x7fd68801b5c0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:11.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.250+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd69001f030 con 0x7fd6a00749a0 2024-09-17T03:21:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.462+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a015a6e0 con 0x7fd6a00749a0 2024-09-17T03:21:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.462+0000 7fd697e00640 1 -- 172.21.15.46:0/1484693813 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7fd69005e6c0 con 0x7fd6a00749a0 2024-09-17T03:21:11.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:11.466 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd67c073230 msgr2=0x7fd67c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd67c073230 0x7fd67c075740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd6880af000 tx=0x7fd68801b5c0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 msgr2=0x7fd6a015f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 0x7fd6a015f980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd69000b440 tx=0x7fd69000b900 comp rx=0 tx=0).stop 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 shutdown_connections 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd67c073230 0x7fd67c075740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6a00cefa0 0x7fd6a013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a00b3340 0x7fd6a013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 --2- 172.21.15.46:0/1484693813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a00749a0 0x7fd6a015f980 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 >> 172.21.15.46:0/1484693813 conn(0x7fd6a00b2040 msgr2=0x7fd6a015ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 shutdown_connections 2024-09-17T03:21:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.466+0000 7fd6a5e00640 1 -- 172.21.15.46:0/1484693813 wait complete. 2024-09-17T03:21:11.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 -- 172.21.15.46:0/2807955903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7748160060 msgr2=0x7f7748074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 --2- 172.21.15.46:0/2807955903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7748160060 0x7f7748074240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7730009900 tx=0x7f773001c850 comp rx=0 tx=0).stop 2024-09-17T03:21:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 -- 172.21.15.46:0/2807955903 shutdown_connections 2024-09-17T03:21:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 --2- 172.21.15.46:0/2807955903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7748160060 0x7f7748074240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 --2- 172.21.15.46:0/2807955903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77481536f0 0x7f774815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 -- 172.21.15.46:0/2807955903 >> 172.21.15.46:0/2807955903 conn(0x7f77480b4560 msgr2=0x7f77480d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 -- 172.21.15.46:0/2807955903 shutdown_connections 2024-09-17T03:21:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.642+0000 7f7747200640 1 -- 172.21.15.46:0/2807955903 wait complete. 2024-09-17T03:21:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 Processor -- start 2024-09-17T03:21:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 -- start start 2024-09-17T03:21:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 0x7f7748166960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7748160060 0x7f77481f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7748160470 0x7f7748151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f774816a3f0 con 0x7f77481536f0 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f774816a270 con 0x7f7748160470 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7747200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f774816a570 con 0x7f7748160060 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 0x7f7748166960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 0x7f7748166960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42548/0 (socket says 172.21.15.46:42548) 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 -- 172.21.15.46:0/2662251048 learned_addr learned my addr 172.21.15.46:0/2662251048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f773fe00640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7748160060 0x7f77481f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7746800640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7748160470 0x7f7748151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 -- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7748160060 msgr2=0x7f77481f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7748160060 0x7f77481f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 -- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7748160470 msgr2=0x7f7748151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7748160470 0x7f7748151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.646+0000 7f7745e00640 1 -- 172.21.15.46:0/2662251048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77481f9340 con 0x7f77481536f0 2024-09-17T03:21:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f7745e00640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 0x7f7748166960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f773800e7f0 tx=0x7f773800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773800ccc0 con 0x7f77481536f0 2024-09-17T03:21:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77481fca70 con 0x7f77481536f0 2024-09-17T03:21:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f773800ce80 con 0x7f77481536f0 2024-09-17T03:21:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7738005280 con 0x7f77481536f0 2024-09-17T03:21:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77481fd030 con 0x7f77481536f0 2024-09-17T03:21:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774815d480 con 0x7f77481536f0 2024-09-17T03:21:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.650+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7738005420 con 0x7f77481536f0 2024-09-17T03:21:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.654+0000 7f773e000640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7724073200 0x7f7724075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.654+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7738094cf0 con 0x7f77481536f0 2024-09-17T03:21:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.654+0000 7f773fe00640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7724073200 0x7f7724075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.654+0000 7f773fe00640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7724073200 0x7f7724075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77300af000 tx=0x7f773001b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.658+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f773805fbf0 con 0x7f77481536f0 2024-09-17T03:21:11.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.830+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f774815c4c0 con 0x7f77481536f0 2024-09-17T03:21:11.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f773e000640 1 -- 172.21.15.46:0/2662251048 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f773805f4f0 con 0x7f77481536f0 2024-09-17T03:21:11.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:11.837 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7724073200 msgr2=0x7f7724075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7724073200 0x7f7724075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77300af000 tx=0x7f773001b090 comp rx=0 tx=0).stop 2024-09-17T03:21:11.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 msgr2=0x7f7748166960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 0x7f7748166960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f773800e7f0 tx=0x7f773800ecb0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 shutdown_connections 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7724073200 0x7f7724075710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7748160470 0x7f7748151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7748160060 0x7f77481f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 --2- 172.21.15.46:0/2662251048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77481536f0 0x7f7748166960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 >> 172.21.15.46:0/2662251048 conn(0x7f77480b4560 msgr2=0x7f7748071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 shutdown_connections 2024-09-17T03:21:11.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:11.834+0000 7f7747200640 1 -- 172.21.15.46:0/2662251048 wait complete. 2024-09-17T03:21:11.861 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:21:11.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 -- 172.21.15.46:0/3094194033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58074950 msgr2=0x7f1d5815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 --2- 172.21.15.46:0/3094194033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58074950 0x7f1d5815cf90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d580b1740 tx=0x7f1d5001ccf0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 -- 172.21.15.46:0/3094194033 shutdown_connections 2024-09-17T03:21:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 --2- 172.21.15.46:0/3094194033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58074950 0x7f1d5815cf90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 --2- 172.21.15.46:0/3094194033 >> v2:172.21.15.99:3300/0 conn(0x7f1d580b5a80 0x7f1d58074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 -- 172.21.15.46:0/3094194033 >> 172.21.15.46:0/3094194033 conn(0x7f1d580708c0 msgr2=0x7f1d58070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 -- 172.21.15.46:0/3094194033 shutdown_connections 2024-09-17T03:21:12.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.014+0000 7f1d5da00640 1 -- 172.21.15.46:0/3094194033 wait complete. 2024-09-17T03:21:12.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 Processor -- start 2024-09-17T03:21:12.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 -- start start 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58074950 0x7f1d5814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 0x7f1d58149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d58157030 0x7f1d581516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5816a310 con 0x7f1d580b5a80 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5816a190 con 0x7f1d58074950 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d5da00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5816a490 con 0x7f1d58157030 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d4e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 0x7f1d58149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d4e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 0x7f1d58149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42576/0 (socket says 172.21.15.46:42576) 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.018+0000 7f1d4e000640 1 -- 172.21.15.46:0/1466432943 learned_addr learned my addr 172.21.15.46:0/1466432943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4e000640 1 -- 172.21.15.46:0/1466432943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d58157030 msgr2=0x7f1d581516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4e000640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d58157030 0x7f1d581516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4e000640 1 -- 172.21.15.46:0/1466432943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58074950 msgr2=0x7f1d5814a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4e000640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58074950 0x7f1d5814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4e000640 1 -- 172.21.15.46:0/1466432943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d58156460 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4e000640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 0x7f1d58149aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000d720 tx=0x7f1d4000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40017070 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d40009d20 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d580b2670 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40005280 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d580b2c30 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d581561d0 con 0x7f1d580b5a80 2024-09-17T03:21:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.022+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1d40005420 con 0x7f1d580b5a80 2024-09-17T03:21:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.026+0000 7f1d4f400640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d34073200 0x7f1d34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.026+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1d400942d0 con 0x7f1d580b5a80 2024-09-17T03:21:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.026+0000 7f1d56a00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d34073200 0x7f1d34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.026+0000 7f1d56a00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d34073200 0x7f1d34075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007fd0 tx=0x7f1d48007940 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:12.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.030+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1d4005f370 con 0x7f1d580b5a80 2024-09-17T03:21:12.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.202+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d5806b540 con 0x7f1d34073200 2024-09-17T03:21:12.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.206+0000 7f1d4f400640 1 -- 172.21.15.46:0/1466432943 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243502 (secure 0 0 0) 0x7f1d480050d0 con 0x7f1d34073200 2024-09-17T03:21:12.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:12.217 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d34073200 msgr2=0x7f1d34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d34073200 0x7f1d34075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007fd0 tx=0x7f1d48007940 comp rx=0 tx=0).stop 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 msgr2=0x7f1d58149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 0x7f1d58149aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000d720 tx=0x7f1d4000dbe0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 shutdown_connections 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d34073200 0x7f1d34075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d58157030 0x7f1d581516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d580b5a80 0x7f1d58149aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 --2- 172.21.15.46:0/1466432943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58074950 0x7f1d5814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 >> 172.21.15.46:0/1466432943 conn(0x7f1d580708c0 msgr2=0x7f1d5814f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 shutdown_connections 2024-09-17T03:21:12.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.210+0000 7f1d5da00640 1 -- 172.21.15.46:0/1466432943 wait complete. 2024-09-17T03:21:12.234 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T03:21:11.196102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2564,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2625536,"data_stored":1630748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190285"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.009001+0000","last_change":"2024-09-17T03:21:06.009001+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.009001+0000","last_undegraded":"2024-09-17T03:21:06.009001+0000","last_fullsized":"2024-09-17T03:21:06.009001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014686+0000","last_change":"2024-09-17T03:21:06.014686+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014686+0000","last_undegraded":"2024-09-17T03:21:06.014686+0000","last_fullsized":"2024-09-17T03:21:06.014686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.008995+0000","last_change":"2024-09-17T03:21:06.008995+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.008995+0000","last_undegraded":"2024-09-17T03:21:06.008995+0000","last_fullsized":"2024-09-17T03:21:06.008995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.015234+0000","last_change":"2024-09-17T03:21:06.015234+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.015234+0000","last_undegraded":"2024-09-17T03:21:06.015234+0000","last_fullsized":"2024-09-17T03:21:06.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.017150+0000","last_change":"2024-09-17T03:21:06.017150+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.017150+0000","last_undegraded":"2024-09-17T03:21:06.017150+0000","last_fullsized":"2024-09-17T03:21:06.017150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.013362+0000","last_change":"2024-09-17T03:21:06.013362+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.013362+0000","last_undegraded":"2024-09-17T03:21:06.013362+0000","last_fullsized":"2024-09-17T03:21:06.013362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014718+0000","last_change":"2024-09-17T03:21:06.014718+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014718+0000","last_undegraded":"2024-09-17T03:21:06.014718+0000","last_fullsized":"2024-09-17T03:21:06.014718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020665+0000","last_change":"2024-09-17T03:21:06.020665+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020665+0000","last_undegraded":"2024-09-17T03:21:06.020665+0000","last_fullsized":"2024-09-17T03:21:06.020665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.015331+0000","last_change":"2024-09-17T03:21:06.015331+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.015331+0000","last_undegraded":"2024-09-17T03:21:06.015331+0000","last_fullsized":"2024-09-17T03:21:06.015331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020787+0000","last_change":"2024-09-17T03:21:06.020787+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020787+0000","last_undegraded":"2024-09-17T03:21:06.020787+0000","last_fullsized":"2024-09-17T03:21:06.020787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020803+0000","last_change":"2024-09-17T03:21:06.020803+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020803+0000","last_undegraded":"2024-09-17T03:21:06.020803+0000","last_fullsized":"2024-09-17T03:21:06.020803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T03:21:06.012586+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:06.012586+0000","last_peered":"2024-09-17T03:21:06.012586+0000","last_clean":"2024-09-17T03:21:06.012586+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:06.012586+0000","last_undegraded":"2024-09-17T03:21:06.012586+0000","last_fullsized":"2024-09-17T03:21:06.012586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010108+0000","last_change":"2024-09-17T03:21:06.010108+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010108+0000","last_undegraded":"2024-09-17T03:21:06.010108+0000","last_fullsized":"2024-09-17T03:21:06.010108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020721+0000","last_change":"2024-09-17T03:21:06.020721+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020721+0000","last_undegraded":"2024-09-17T03:21:06.020721+0000","last_fullsized":"2024-09-17T03:21:06.020721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010242+0000","last_change":"2024-09-17T03:21:06.010242+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010242+0000","last_undegraded":"2024-09-17T03:21:06.010242+0000","last_fullsized":"2024-09-17T03:21:06.010242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014135+0000","last_change":"2024-09-17T03:21:06.014135+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014135+0000","last_undegraded":"2024-09-17T03:21:06.014135+0000","last_fullsized":"2024-09-17T03:21:06.014135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.011067+0000","last_change":"2024-09-17T03:21:06.011067+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.011067+0000","last_undegraded":"2024-09-17T03:21:06.011067+0000","last_fullsized":"2024-09-17T03:21:06.011067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010198+0000","last_change":"2024-09-17T03:21:06.010198+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010198+0000","last_undegraded":"2024-09-17T03:21:06.010198+0000","last_fullsized":"2024-09-17T03:21:06.010198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010118+0000","last_change":"2024-09-17T03:21:06.010118+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010118+0000","last_undegraded":"2024-09-17T03:21:06.010118+0000","last_fullsized":"2024-09-17T03:21:06.010118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.011132+0000","last_change":"2024-09-17T03:21:06.011132+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.011132+0000","last_undegraded":"2024-09-17T03:21:06.011132+0000","last_fullsized":"2024-09-17T03:21:06.011132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.008976+0000","last_change":"2024-09-17T03:21:06.008976+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.008976+0000","last_undegraded":"2024-09-17T03:21:06.008976+0000","last_fullsized":"2024-09-17T03:21:06.008976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014197+0000","last_change":"2024-09-17T03:21:06.014197+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014197+0000","last_undegraded":"2024-09-17T03:21:06.014197+0000","last_fullsized":"2024-09-17T03:21:06.014197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":643205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":47020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":159744,"data_stored":47020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":765952,"data_stored":643205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:12.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 -- 172.21.15.46:0/2035287688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b4de0 msgr2=0x7f50b80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 --2- 172.21.15.46:0/2035287688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b4de0 0x7f50b80b51f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009c80 tx=0x7f50b001cce0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 -- 172.21.15.46:0/2035287688 shutdown_connections 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 --2- 172.21.15.46:0/2035287688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b4de0 0x7f50b80b51f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 --2- 172.21.15.46:0/2035287688 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50b80b5c70 0x7f50b815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 -- 172.21.15.46:0/2035287688 >> 172.21.15.46:0/2035287688 conn(0x7f50b80b2040 msgr2=0x7f50b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 -- 172.21.15.46:0/2035287688 shutdown_connections 2024-09-17T03:21:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.434+0000 7f50bd000640 1 -- 172.21.15.46:0/2035287688 wait complete. 2024-09-17T03:21:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 Processor -- start 2024-09-17T03:21:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 -- start start 2024-09-17T03:21:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50b80b4de0 0x7f50b8158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b5c70 0x7f50b80cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 0x7f50b80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8168ab0 con 0x7f50b81527b0 2024-09-17T03:21:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8168930 con 0x7f50b80b5c70 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f50b8168c30 con 0x7f50b80b4de0 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 0x7f50b80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 0x7f50b80ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38260/0 (socket says 172.21.15.46:38260) 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 -- 172.21.15.46:0/3591343746 learned_addr learned my addr 172.21.15.46:0/3591343746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b5e00640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50b80b4de0 0x7f50b8158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50afe00640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b5c70 0x7f50b80cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 -- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50b80b4de0 msgr2=0x7f50b8158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50b80b4de0 0x7f50b8158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 -- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b5c70 msgr2=0x7f50b80cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b5c70 0x7f50b80cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 -- 172.21.15.46:0/3591343746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b80cf350 con 0x7f50b81527b0 2024-09-17T03:21:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50b6800640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 0x7f50b80ce390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50b001b390 tx=0x7f50b0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00bb070 con 0x7f50b81527b0 2024-09-17T03:21:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50b0004b40 con 0x7f50b81527b0 2024-09-17T03:21:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00b2040 con 0x7f50b81527b0 2024-09-17T03:21:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b80cf630 con 0x7f50b81527b0 2024-09-17T03:21:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.438+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b8156870 con 0x7f50b81527b0 2024-09-17T03:21:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.442+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b80cf0c0 con 0x7f50b81527b0 2024-09-17T03:21:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.442+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50b0004000 con 0x7f50b81527b0 2024-09-17T03:21:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.442+0000 7f50ae000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5094073230 0x7f5094075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.442+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f50b012e230 con 0x7f50b81527b0 2024-09-17T03:21:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.442+0000 7f50b5e00640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5094073230 0x7f5094075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.442+0000 7f50b5e00640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5094073230 0x7f5094075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009ad0 tx=0x7f50a80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.446+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f50b0016990 con 0x7f50b81527b0 2024-09-17T03:21:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.614+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50b806b540 con 0x7f5094073230 2024-09-17T03:21:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50ae000640 1 -- 172.21.15.46:0/3591343746 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243502 (secure 0 0 0) 0x7f50a8011b20 con 0x7f5094073230 2024-09-17T03:21:12.626 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5094073230 msgr2=0x7f5094075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5094073230 0x7f5094075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009ad0 tx=0x7f50a80091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 msgr2=0x7f50b80ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 0x7f50b80ce390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f50b001b390 tx=0x7f50b0005b00 comp rx=0 tx=0).stop 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 shutdown_connections 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5094073230 0x7f5094075740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50b81527b0 0x7f50b80ce390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50b80b5c70 0x7f50b80cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 --2- 172.21.15.46:0/3591343746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50b80b4de0 0x7f50b8158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 >> 172.21.15.46:0/3591343746 conn(0x7f50b80b2040 msgr2=0x7f50b815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 shutdown_connections 2024-09-17T03:21:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.622+0000 7f50bd000640 1 -- 172.21.15.46:0/3591343746 wait complete. 2024-09-17T03:21:12.647 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T03:21:11.196102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2564,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2625536,"data_stored":1630748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190285"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.009001+0000","last_change":"2024-09-17T03:21:06.009001+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.009001+0000","last_undegraded":"2024-09-17T03:21:06.009001+0000","last_fullsized":"2024-09-17T03:21:06.009001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014686+0000","last_change":"2024-09-17T03:21:06.014686+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014686+0000","last_undegraded":"2024-09-17T03:21:06.014686+0000","last_fullsized":"2024-09-17T03:21:06.014686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.008995+0000","last_change":"2024-09-17T03:21:06.008995+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.008995+0000","last_undegraded":"2024-09-17T03:21:06.008995+0000","last_fullsized":"2024-09-17T03:21:06.008995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.015234+0000","last_change":"2024-09-17T03:21:06.015234+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.015234+0000","last_undegraded":"2024-09-17T03:21:06.015234+0000","last_fullsized":"2024-09-17T03:21:06.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.017150+0000","last_change":"2024-09-17T03:21:06.017150+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.017150+0000","last_undegraded":"2024-09-17T03:21:06.017150+0000","last_fullsized":"2024-09-17T03:21:06.017150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.013362+0000","last_change":"2024-09-17T03:21:06.013362+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.013362+0000","last_undegraded":"2024-09-17T03:21:06.013362+0000","last_fullsized":"2024-09-17T03:21:06.013362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014718+0000","last_change":"2024-09-17T03:21:06.014718+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014718+0000","last_undegraded":"2024-09-17T03:21:06.014718+0000","last_fullsized":"2024-09-17T03:21:06.014718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020665+0000","last_change":"2024-09-17T03:21:06.020665+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020665+0000","last_undegraded":"2024-09-17T03:21:06.020665+0000","last_fullsized":"2024-09-17T03:21:06.020665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.015331+0000","last_change":"2024-09-17T03:21:06.015331+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.015331+0000","last_undegraded":"2024-09-17T03:21:06.015331+0000","last_fullsized":"2024-09-17T03:21:06.015331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020787+0000","last_change":"2024-09-17T03:21:06.020787+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020787+0000","last_undegraded":"2024-09-17T03:21:06.020787+0000","last_fullsized":"2024-09-17T03:21:06.020787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020803+0000","last_change":"2024-09-17T03:21:06.020803+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020803+0000","last_undegraded":"2024-09-17T03:21:06.020803+0000","last_fullsized":"2024-09-17T03:21:06.020803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T03:21:06.012586+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:06.012586+0000","last_peered":"2024-09-17T03:21:06.012586+0000","last_clean":"2024-09-17T03:21:06.012586+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:06.012586+0000","last_undegraded":"2024-09-17T03:21:06.012586+0000","last_fullsized":"2024-09-17T03:21:06.012586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010108+0000","last_change":"2024-09-17T03:21:06.010108+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010108+0000","last_undegraded":"2024-09-17T03:21:06.010108+0000","last_fullsized":"2024-09-17T03:21:06.010108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020721+0000","last_change":"2024-09-17T03:21:06.020721+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020721+0000","last_undegraded":"2024-09-17T03:21:06.020721+0000","last_fullsized":"2024-09-17T03:21:06.020721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010242+0000","last_change":"2024-09-17T03:21:06.010242+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010242+0000","last_undegraded":"2024-09-17T03:21:06.010242+0000","last_fullsized":"2024-09-17T03:21:06.010242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014135+0000","last_change":"2024-09-17T03:21:06.014135+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014135+0000","last_undegraded":"2024-09-17T03:21:06.014135+0000","last_fullsized":"2024-09-17T03:21:06.014135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.011067+0000","last_change":"2024-09-17T03:21:06.011067+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.011067+0000","last_undegraded":"2024-09-17T03:21:06.011067+0000","last_fullsized":"2024-09-17T03:21:06.011067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010198+0000","last_change":"2024-09-17T03:21:06.010198+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010198+0000","last_undegraded":"2024-09-17T03:21:06.010198+0000","last_fullsized":"2024-09-17T03:21:06.010198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010118+0000","last_change":"2024-09-17T03:21:06.010118+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010118+0000","last_undegraded":"2024-09-17T03:21:06.010118+0000","last_fullsized":"2024-09-17T03:21:06.010118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.011132+0000","last_change":"2024-09-17T03:21:06.011132+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.011132+0000","last_undegraded":"2024-09-17T03:21:06.011132+0000","last_fullsized":"2024-09-17T03:21:06.011132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.008976+0000","last_change":"2024-09-17T03:21:06.008976+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.008976+0000","last_undegraded":"2024-09-17T03:21:06.008976+0000","last_fullsized":"2024-09-17T03:21:06.008976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014197+0000","last_change":"2024-09-17T03:21:06.014197+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014197+0000","last_undegraded":"2024-09-17T03:21:06.014197+0000","last_fullsized":"2024-09-17T03:21:06.014197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":643205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":47020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":159744,"data_stored":47020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":765952,"data_stored":643205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:12.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 -- 172.21.15.46:0/1464309157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 msgr2=0x7f0c98074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/1464309157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c98074410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0c980713e0 tx=0x7f0c8001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 -- 172.21.15.46:0/1464309157 shutdown_connections 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/1464309157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c98074410 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/1464309157 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98161160 0x7f0c98074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/1464309157 >> v2:172.21.15.99:3300/0 conn(0x7f0c98157030 0x7f0c980b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 -- 172.21.15.46:0/1464309157 >> 172.21.15.46:0/1464309157 conn(0x7f0c980708c0 msgr2=0x7f0c98070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 -- 172.21.15.46:0/1464309157 shutdown_connections 2024-09-17T03:21:12.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.846+0000 7f0c9ea00640 1 -- 172.21.15.46:0/1464309157 wait complete. 2024-09-17T03:21:12.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 Processor -- start 2024-09-17T03:21:12.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 -- start start 2024-09-17T03:21:12.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98157030 0x7f0c98150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c981fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c98161160 0x7f0c981fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98072230 con 0x7f0c98160d50 2024-09-17T03:21:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980720b0 con 0x7f0c98161160 2024-09-17T03:21:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0c980723b0 con 0x7f0c98157030 2024-09-17T03:21:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c981fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c981fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38276/0 (socket says 172.21.15.46:38276) 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 -- 172.21.15.46:0/73106072 learned_addr learned my addr 172.21.15.46:0/73106072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ca00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c98161160 0x7f0c981fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c97e00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98157030 0x7f0c98150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 -- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98157030 msgr2=0x7f0c98150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98157030 0x7f0c98150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 -- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c98161160 msgr2=0x7f0c981fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c98161160 0x7f0c981fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 -- 172.21.15.46:0/73106072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980b40b0 con 0x7f0c98160d50 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c97e00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98157030 0x7f0c98150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c8fe00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c981fc530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c800b0040 tx=0x7f0c800a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800b2040 con 0x7f0c98160d50 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c800043b0 con 0x7f0c98160d50 2024-09-17T03:21:12.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80003860 con 0x7f0c98160d50 2024-09-17T03:21:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c98161890 con 0x7f0c98160d50 2024-09-17T03:21:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.850+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c98164810 con 0x7f0c98160d50 2024-09-17T03:21:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.854+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c98161680 con 0x7f0c98160d50 2024-09-17T03:21:12.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.854+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0c80003a00 con 0x7f0c98160d50 2024-09-17T03:21:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.854+0000 7f0c94c00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c74073230 0x7f0c74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.854+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c8012df20 con 0x7f0c98160d50 2024-09-17T03:21:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.854+0000 7f0c97e00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c74073230 0x7f0c74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.854+0000 7f0c97e00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c74073230 0x7f0c74075740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005fd0 tx=0x7f0c88005e40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:12.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:12.858+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c800164b0 con 0x7f0c98160d50 2024-09-17T03:21:12.942 INFO:tasks.ceph.mds.b.smithi099.stderr:2024-09-17T03:21:12.944+0000 7fcb4fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:21:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.030+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c9806b540 con 0x7f0c74073230 2024-09-17T03:21:13.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.034+0000 7f0c94c00640 1 -- 172.21.15.46:0/73106072 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243502 (secure 0 0 0) 0x7f0c8800eac0 con 0x7f0c74073230 2024-09-17T03:21:13.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:13.041 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c74073230 msgr2=0x7f0c74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c74073230 0x7f0c74075740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005fd0 tx=0x7f0c88005e40 comp rx=0 tx=0).stop 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 msgr2=0x7f0c981fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c981fc530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c800b0040 tx=0x7f0c800a55c0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 shutdown_connections 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c74073230 0x7f0c74075740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c98161160 0x7f0c981fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c98160d50 0x7f0c981fc530 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 --2- 172.21.15.46:0/73106072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c98157030 0x7f0c98150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 >> 172.21.15.46:0/73106072 conn(0x7f0c980708c0 msgr2=0x7f0c9806e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 shutdown_connections 2024-09-17T03:21:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.038+0000 7f0c9ea00640 1 -- 172.21.15.46:0/73106072 wait complete. 2024-09-17T03:21:13.061 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T03:21:11.196102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2564,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2625536,"data_stored":1630748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190285"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.009001+0000","last_change":"2024-09-17T03:21:06.009001+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.009001+0000","last_undegraded":"2024-09-17T03:21:06.009001+0000","last_fullsized":"2024-09-17T03:21:06.009001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014686+0000","last_change":"2024-09-17T03:21:06.014686+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014686+0000","last_undegraded":"2024-09-17T03:21:06.014686+0000","last_fullsized":"2024-09-17T03:21:06.014686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.008995+0000","last_change":"2024-09-17T03:21:06.008995+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.008995+0000","last_undegraded":"2024-09-17T03:21:06.008995+0000","last_fullsized":"2024-09-17T03:21:06.008995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.015234+0000","last_change":"2024-09-17T03:21:06.015234+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.015234+0000","last_undegraded":"2024-09-17T03:21:06.015234+0000","last_fullsized":"2024-09-17T03:21:06.015234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.017150+0000","last_change":"2024-09-17T03:21:06.017150+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.017150+0000","last_undegraded":"2024-09-17T03:21:06.017150+0000","last_fullsized":"2024-09-17T03:21:06.017150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.013362+0000","last_change":"2024-09-17T03:21:06.013362+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.013362+0000","last_undegraded":"2024-09-17T03:21:06.013362+0000","last_fullsized":"2024-09-17T03:21:06.013362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014718+0000","last_change":"2024-09-17T03:21:06.014718+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014718+0000","last_undegraded":"2024-09-17T03:21:06.014718+0000","last_fullsized":"2024-09-17T03:21:06.014718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020665+0000","last_change":"2024-09-17T03:21:06.020665+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020665+0000","last_undegraded":"2024-09-17T03:21:06.020665+0000","last_fullsized":"2024-09-17T03:21:06.020665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.015331+0000","last_change":"2024-09-17T03:21:06.015331+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.015331+0000","last_undegraded":"2024-09-17T03:21:06.015331+0000","last_fullsized":"2024-09-17T03:21:06.015331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020787+0000","last_change":"2024-09-17T03:21:06.020787+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020787+0000","last_undegraded":"2024-09-17T03:21:06.020787+0000","last_fullsized":"2024-09-17T03:21:06.020787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020803+0000","last_change":"2024-09-17T03:21:06.020803+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020803+0000","last_undegraded":"2024-09-17T03:21:06.020803+0000","last_fullsized":"2024-09-17T03:21:06.020803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T03:21:06.012586+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:06.012586+0000","last_peered":"2024-09-17T03:21:06.012586+0000","last_clean":"2024-09-17T03:21:06.012586+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:06.012586+0000","last_undegraded":"2024-09-17T03:21:06.012586+0000","last_fullsized":"2024-09-17T03:21:06.012586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010108+0000","last_change":"2024-09-17T03:21:06.010108+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010108+0000","last_undegraded":"2024-09-17T03:21:06.010108+0000","last_fullsized":"2024-09-17T03:21:06.010108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.020721+0000","last_change":"2024-09-17T03:21:06.020721+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.020721+0000","last_undegraded":"2024-09-17T03:21:06.020721+0000","last_fullsized":"2024-09-17T03:21:06.020721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010242+0000","last_change":"2024-09-17T03:21:06.010242+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010242+0000","last_undegraded":"2024-09-17T03:21:06.010242+0000","last_fullsized":"2024-09-17T03:21:06.010242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014135+0000","last_change":"2024-09-17T03:21:06.014135+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014135+0000","last_undegraded":"2024-09-17T03:21:06.014135+0000","last_fullsized":"2024-09-17T03:21:06.014135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.011067+0000","last_change":"2024-09-17T03:21:06.011067+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.011067+0000","last_undegraded":"2024-09-17T03:21:06.011067+0000","last_fullsized":"2024-09-17T03:21:06.011067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010198+0000","last_change":"2024-09-17T03:21:06.010198+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010198+0000","last_undegraded":"2024-09-17T03:21:06.010198+0000","last_fullsized":"2024-09-17T03:21:06.010198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.010118+0000","last_change":"2024-09-17T03:21:06.010118+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.010118+0000","last_undegraded":"2024-09-17T03:21:06.010118+0000","last_fullsized":"2024-09-17T03:21:06.010118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.011132+0000","last_change":"2024-09-17T03:21:06.011132+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.011132+0000","last_undegraded":"2024-09-17T03:21:06.011132+0000","last_fullsized":"2024-09-17T03:21:06.011132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.008976+0000","last_change":"2024-09-17T03:21:06.008976+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.008976+0000","last_undegraded":"2024-09-17T03:21:06.008976+0000","last_fullsized":"2024-09-17T03:21:06.008976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:21:07.005025+0000","last_change":"2024-09-17T03:21:07.005025+0000","last_active":"2024-09-17T03:21:07.005025+0000","last_peered":"2024-09-17T03:21:07.005025+0000","last_clean":"2024-09-17T03:21:07.005025+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:07.005025+0000","last_undegraded":"2024-09-17T03:21:07.005025+0000","last_fullsized":"2024-09-17T03:21:07.005025+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-17T03:21:06.014197+0000","last_change":"2024-09-17T03:21:06.014197+0000","last_active":"2024-09-17T03:21:05.998750+0000","last_peered":"2024-09-17T03:21:05.998750+0000","last_clean":"2024-09-17T03:21:05.998750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:21:06.014197+0000","last_undegraded":"2024-09-17T03:21:06.014197+0000","last_fullsized":"2024-09-17T03:21:06.014197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":643205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":47020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":159744,"data_stored":47020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":765952,"data_stored":643205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:13.065 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-17T03:21:13.065 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.065 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-17T03:21:13.065 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.065 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-17T03:21:13.065 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.066 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-17T03:21:13.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.068 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.009001+0000', 'last_change': '2024-09-17T03:21:06.009001+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.009001+0000', 'last_undegraded': '2024-09-17T03:21:06.009001+0000', 'last_fullsized': '2024-09-17T03:21:06.009001+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-17T03:21:13.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.014686+0000', 'last_change': '2024-09-17T03:21:06.014686+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.014686+0000', 'last_undegraded': '2024-09-17T03:21:06.014686+0000', 'last_fullsized': '2024-09-17T03:21:06.014686+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.008995+0000', 'last_change': '2024-09-17T03:21:06.008995+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.008995+0000', 'last_undegraded': '2024-09-17T03:21:06.008995+0000', 'last_fullsized': '2024-09-17T03:21:06.008995+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.015234+0000', 'last_change': '2024-09-17T03:21:06.015234+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.015234+0000', 'last_undegraded': '2024-09-17T03:21:06.015234+0000', 'last_fullsized': '2024-09-17T03:21:06.015234+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.070 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.017150+0000', 'last_change': '2024-09-17T03:21:06.017150+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.017150+0000', 'last_undegraded': '2024-09-17T03:21:06.017150+0000', 'last_fullsized': '2024-09-17T03:21:06.017150+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.013362+0000', 'last_change': '2024-09-17T03:21:06.013362+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.013362+0000', 'last_undegraded': '2024-09-17T03:21:06.013362+0000', 'last_fullsized': '2024-09-17T03:21:06.013362+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-17T03:21:13.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.014718+0000', 'last_change': '2024-09-17T03:21:06.014718+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.014718+0000', 'last_undegraded': '2024-09-17T03:21:06.014718+0000', 'last_fullsized': '2024-09-17T03:21:06.014718+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.020665+0000', 'last_change': '2024-09-17T03:21:06.020665+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.020665+0000', 'last_undegraded': '2024-09-17T03:21:06.020665+0000', 'last_fullsized': '2024-09-17T03:21:06.020665+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.015331+0000', 'last_change': '2024-09-17T03:21:06.015331+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.015331+0000', 'last_undegraded': '2024-09-17T03:21:06.015331+0000', 'last_fullsized': '2024-09-17T03:21:06.015331+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:21:13.072 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.020787+0000', 'last_change': '2024-09-17T03:21:06.020787+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.020787+0000', 'last_undegraded': '2024-09-17T03:21:06.020787+0000', 'last_fullsized': '2024-09-17T03:21:06.020787+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.020803+0000', 'last_change': '2024-09-17T03:21:06.020803+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.020803+0000', 'last_undegraded': '2024-09-17T03:21:06.020803+0000', 'last_fullsized': '2024-09-17T03:21:06.020803+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-17T03:21:13.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.010108+0000', 'last_change': '2024-09-17T03:21:06.010108+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.010108+0000', 'last_undegraded': '2024-09-17T03:21:06.010108+0000', 'last_fullsized': '2024-09-17T03:21:06.010108+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.020721+0000', 'last_change': '2024-09-17T03:21:06.020721+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.020721+0000', 'last_undegraded': '2024-09-17T03:21:06.020721+0000', 'last_fullsized': '2024-09-17T03:21:06.020721+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.010242+0000', 'last_change': '2024-09-17T03:21:06.010242+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.010242+0000', 'last_undegraded': '2024-09-17T03:21:06.010242+0000', 'last_fullsized': '2024-09-17T03:21:06.010242+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:21:13.074 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.014135+0000', 'last_change': '2024-09-17T03:21:06.014135+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.014135+0000', 'last_undegraded': '2024-09-17T03:21:06.014135+0000', 'last_fullsized': '2024-09-17T03:21:06.014135+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.011067+0000', 'last_change': '2024-09-17T03:21:06.011067+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.011067+0000', 'last_undegraded': '2024-09-17T03:21:06.011067+0000', 'last_fullsized': '2024-09-17T03:21:06.011067+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-17T03:21:13.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.010198+0000', 'last_change': '2024-09-17T03:21:06.010198+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.010198+0000', 'last_undegraded': '2024-09-17T03:21:06.010198+0000', 'last_fullsized': '2024-09-17T03:21:06.010198+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.010118+0000', 'last_change': '2024-09-17T03:21:06.010118+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.010118+0000', 'last_undegraded': '2024-09-17T03:21:06.010118+0000', 'last_fullsized': '2024-09-17T03:21:06.010118+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.011132+0000', 'last_change': '2024-09-17T03:21:06.011132+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.011132+0000', 'last_undegraded': '2024-09-17T03:21:06.011132+0000', 'last_fullsized': '2024-09-17T03:21:06.011132+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:21:13.076 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.008976+0000', 'last_change': '2024-09-17T03:21:06.008976+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.008976+0000', 'last_undegraded': '2024-09-17T03:21:06.008976+0000', 'last_fullsized': '2024-09-17T03:21:06.008976+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:21:07.005025+0000', 'last_change': '2024-09-17T03:21:07.005025+0000', 'last_active': '2024-09-17T03:21:07.005025+0000', 'last_peered': '2024-09-17T03:21:07.005025+0000', 'last_clean': '2024-09-17T03:21:07.005025+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:07.005025+0000', 'last_undegraded': '2024-09-17T03:21:07.005025+0000', 'last_fullsized': '2024-09-17T03:21:07.005025+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:07.005025+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-17T03:21:13.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:21:06.014197+0000', 'last_change': '2024-09-17T03:21:06.014197+0000', 'last_active': '2024-09-17T03:21:05.998750+0000', 'last_peered': '2024-09-17T03:21:05.998750+0000', 'last_clean': '2024-09-17T03:21:05.998750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:21:06.014197+0000', 'last_undegraded': '2024-09-17T03:21:06.014197+0000', 'last_fullsized': '2024-09-17T03:21:06.014197+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'last_clean_scrub_stamp': '2024-09-17T03:21:05.998750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:21:13.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 -- 172.21.15.46:0/3535073167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8160d50 msgr2=0x7f77f8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 --2- 172.21.15.46:0/3535073167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8160d50 0x7f77f8074410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009900 tx=0x7f77e001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 -- 172.21.15.46:0/3535073167 shutdown_connections 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 --2- 172.21.15.46:0/3535073167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8160d50 0x7f77f8074410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 --2- 172.21.15.46:0/3535073167 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77f8157030 0x7f77f80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 -- 172.21.15.46:0/3535073167 >> 172.21.15.46:0/3535073167 conn(0x7f77f80708c0 msgr2=0x7f77f8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 -- 172.21.15.46:0/3535073167 shutdown_connections 2024-09-17T03:21:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.258+0000 7f77fe400640 1 -- 172.21.15.46:0/3535073167 wait complete. 2024-09-17T03:21:13.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 Processor -- start 2024-09-17T03:21:13.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 -- start start 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f8157030 0x7f77f814b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77f8160d50 0x7f77f8138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 0x7f77f81fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77f816a7d0 con 0x7f77f8161160 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77f816a650 con 0x7f77f8157030 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77fe400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f77f816a950 con 0x7f77f8160d50 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 0x7f77f81fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 0x7f77f81fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38294/0 (socket says 172.21.15.46:38294) 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 -- 172.21.15.46:0/524772743 learned_addr learned my addr 172.21.15.46:0/524772743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77efe00640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77f8160d50 0x7f77f8138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f8157030 0x7f77f814b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 -- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77f8160d50 msgr2=0x7f77f8138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77f8160d50 0x7f77f8138c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 -- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f8157030 msgr2=0x7f77f814b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f8157030 0x7f77f814b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 -- 172.21.15.46:0/524772743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f814c2f0 con 0x7f77f8161160 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77f7e00640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 0x7f77f81fcc40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c730 tx=0x7f77f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.262+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003b80 con 0x7f77f8161160 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f0003d40 con 0x7f77f8161160 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f814c5d0 con 0x7f77f8161160 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f00052d0 con 0x7f77f8161160 2024-09-17T03:21:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f8153c30 con 0x7f77f8161160 2024-09-17T03:21:13.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f814c060 con 0x7f77f8161160 2024-09-17T03:21:13.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77f0005470 con 0x7f77f8161160 2024-09-17T03:21:13.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77ef400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f77d4073230 0x7f77d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f77f0094060 con 0x7f77f8161160 2024-09-17T03:21:13.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.266+0000 7f77f7400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f77d4073230 0x7f77d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.270+0000 7f77f7400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f77d4073230 0x7f77d4075740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009ad0 tx=0x7f77e80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:13.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.270+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77f005f070 con 0x7f77f8161160 2024-09-17T03:21:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.506+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f77f80d2260 con 0x7f77f8161160 2024-09-17T03:21:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.506+0000 7f77ef400640 1 -- 172.21.15.46:0/524772743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1055 (secure 0 0 0) 0x7f77f005e970 con 0x7f77f8161160 2024-09-17T03:21:13.512 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:13.512 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726543255,"num_in_osds":8,"osd_in_since":1726543245,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":241414144,"bytes_avail":772852699136,"bytes_total":773094113280,"write_bytes_sec":2317,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":244315}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T03:20:41.127725+0000","services":{}},"progress_events":{}} 2024-09-17T03:21:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f77d4073230 msgr2=0x7f77d4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f77d4073230 0x7f77d4075740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009ad0 tx=0x7f77e80091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 msgr2=0x7f77f81fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 0x7f77f81fcc40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c730 tx=0x7f77f000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 shutdown_connections 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f77d4073230 0x7f77d4075740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77f8161160 0x7f77f81fcc40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f77f8160d50 0x7f77f8138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 --2- 172.21.15.46:0/524772743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f8157030 0x7f77f814b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 >> 172.21.15.46:0/524772743 conn(0x7f77f80708c0 msgr2=0x7f77f806e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 shutdown_connections 2024-09-17T03:21:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.510+0000 7f77fe400640 1 -- 172.21.15.46:0/524772743 wait complete. 2024-09-17T03:21:13.534 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:21:13.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 -- 172.21.15.46:0/2523636456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 msgr2=0x7f89340b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 --2- 172.21.15.46:0/2523636456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f89340b51f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f892c009bc0 tx=0x7f892c01cae0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 -- 172.21.15.46:0/2523636456 shutdown_connections 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 --2- 172.21.15.46:0/2523636456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f89340b51f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 --2- 172.21.15.46:0/2523636456 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89341527b0 0x7f893415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 -- 172.21.15.46:0/2523636456 >> 172.21.15.46:0/2523636456 conn(0x7f89340b2040 msgr2=0x7f89340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 -- 172.21.15.46:0/2523636456 shutdown_connections 2024-09-17T03:21:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 -- 172.21.15.46:0/2523636456 wait complete. 2024-09-17T03:21:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 Processor -- start 2024-09-17T03:21:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.690+0000 7f8938e00640 1 -- start start 2024-09-17T03:21:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f8934150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89340b5c70 0x7f89340b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89341527b0 0x7f89340ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8934168ab0 con 0x7f89340b4de0 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8934168930 con 0x7f89341527b0 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8934168c30 con 0x7f89340b5c70 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f8934150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f8934150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38312/0 (socket says 172.21.15.46:38312) 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 -- 172.21.15.46:0/1896139104 learned_addr learned my addr 172.21.15.46:0/1896139104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931400640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89340b5c70 0x7f89340b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8932800640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89341527b0 0x7f89340ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 -- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89340b5c70 msgr2=0x7f89340b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89340b5c70 0x7f89340b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 -- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89341527b0 msgr2=0x7f89340ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89341527b0 0x7f89340ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 -- 172.21.15.46:0/1896139104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89341fc2b0 con 0x7f89340b4de0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931400640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89340b5c70 0x7f89340b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8932800640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89341527b0 0x7f89340ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8931e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f8934150be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8924001ec0 tx=0x7f892400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924017070 con 0x7f89340b4de0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8924003600 con 0x7f89340b4de0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89341fc590 con 0x7f89340b4de0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924003a50 con 0x7f89340b4de0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89341fcb50 con 0x7f89340b4de0 2024-09-17T03:21:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.694+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89340b51f0 con 0x7f89340b4de0 2024-09-17T03:21:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.698+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f892401a070 con 0x7f89340b4de0 2024-09-17T03:21:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.698+0000 7f892a000640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8914073200 0x7f8914075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.698+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89240935e0 con 0x7f89340b4de0 2024-09-17T03:21:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.698+0000 7f8931400640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8914073200 0x7f8914075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.698+0000 7f8931400640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8914073200 0x7f8914075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89340c63c0 tx=0x7f89200091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.702+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f89340b5730 con 0x7f89340b4de0 2024-09-17T03:21:13.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.874+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8934151120 con 0x7f8914073200 2024-09-17T03:21:13.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.878+0000 7f892a000640 1 -- 172.21.15.46:0/1896139104 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f8934151120 con 0x7f8914073200 2024-09-17T03:21:13.885 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:13.886 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8914073200 msgr2=0x7f8914075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8914073200 0x7f8914075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89340c63c0 tx=0x7f89200091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 msgr2=0x7f8934150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f8934150be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8924001ec0 tx=0x7f892400c890 comp rx=0 tx=0).stop 2024-09-17T03:21:13.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 shutdown_connections 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8914073200 0x7f8914075710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89341527b0 0x7f89340ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89340b5c70 0x7f89340b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 --2- 172.21.15.46:0/1896139104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b4de0 0x7f8934150be0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 >> 172.21.15.46:0/1896139104 conn(0x7f89340b2040 msgr2=0x7f89341581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 shutdown_connections 2024-09-17T03:21:13.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:13.882+0000 7f8938e00640 1 -- 172.21.15.46:0/1896139104 wait complete. 2024-09-17T03:21:13.906 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T03:21:13.196557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235756,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738964,"statfs":{"total":773094113280,"available":772852699136,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.187264"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:08.073787+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:08.073460+0000","last_became_peered":"2024-09-17T03:21:08.073460+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:56.409644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034270+0000","last_change":"2024-09-17T03:21:08.046022+0000","last_active":"2024-09-17T03:21:09.034270+0000","last_peered":"2024-09-17T03:21:09.034270+0000","last_clean":"2024-09-17T03:21:09.034270+0000","last_became_active":"2024-09-17T03:21:08.045462+0000","last_became_peered":"2024-09-17T03:21:08.045462+0000","last_unstale":"2024-09-17T03:21:09.034270+0000","last_undegraded":"2024-09-17T03:21:09.034270+0000","last_fullsized":"2024-09-17T03:21:09.034270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405803+0000","last_change":"2024-09-17T03:21:08.057050+0000","last_active":"2024-09-17T03:21:10.405803+0000","last_peered":"2024-09-17T03:21:10.405803+0000","last_clean":"2024-09-17T03:21:10.405803+0000","last_became_active":"2024-09-17T03:21:08.056744+0000","last_became_peered":"2024-09-17T03:21:08.056744+0000","last_unstale":"2024-09-17T03:21:10.405803+0000","last_undegraded":"2024-09-17T03:21:10.405803+0000","last_fullsized":"2024-09-17T03:21:10.405803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:52.233846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405049+0000","last_change":"2024-09-17T03:21:08.072759+0000","last_active":"2024-09-17T03:21:10.405049+0000","last_peered":"2024-09-17T03:21:10.405049+0000","last_clean":"2024-09-17T03:21:10.405049+0000","last_became_active":"2024-09-17T03:21:08.072444+0000","last_became_peered":"2024-09-17T03:21:08.072444+0000","last_unstale":"2024-09-17T03:21:10.405049+0000","last_undegraded":"2024-09-17T03:21:10.405049+0000","last_fullsized":"2024-09-17T03:21:10.405049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:09.811529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036226+0000","last_change":"2024-09-17T03:21:08.045612+0000","last_active":"2024-09-17T03:21:09.036226+0000","last_peered":"2024-09-17T03:21:09.036226+0000","last_clean":"2024-09-17T03:21:09.036226+0000","last_became_active":"2024-09-17T03:21:08.044072+0000","last_became_peered":"2024-09-17T03:21:08.044072+0000","last_unstale":"2024-09-17T03:21:09.036226+0000","last_undegraded":"2024-09-17T03:21:09.036226+0000","last_fullsized":"2024-09-17T03:21:09.036226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406443+0000","last_change":"2024-09-17T03:21:08.060425+0000","last_active":"2024-09-17T03:21:10.406443+0000","last_peered":"2024-09-17T03:21:10.406443+0000","last_clean":"2024-09-17T03:21:10.406443+0000","last_became_active":"2024-09-17T03:21:08.060122+0000","last_became_peered":"2024-09-17T03:21:08.060122+0000","last_unstale":"2024-09-17T03:21:10.406443+0000","last_undegraded":"2024-09-17T03:21:10.406443+0000","last_fullsized":"2024-09-17T03:21:10.406443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:49.768654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075521+0000","last_change":"2024-09-17T03:21:08.067905+0000","last_active":"2024-09-17T03:21:09.075521+0000","last_peered":"2024-09-17T03:21:09.075521+0000","last_clean":"2024-09-17T03:21:09.075521+0000","last_became_active":"2024-09-17T03:21:08.066280+0000","last_became_peered":"2024-09-17T03:21:08.066280+0000","last_unstale":"2024-09-17T03:21:09.075521+0000","last_undegraded":"2024-09-17T03:21:09.075521+0000","last_fullsized":"2024-09-17T03:21:09.075521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075384+0000","last_change":"2024-09-17T03:21:08.065069+0000","last_active":"2024-09-17T03:21:09.075384+0000","last_peered":"2024-09-17T03:21:09.075384+0000","last_clean":"2024-09-17T03:21:09.075384+0000","last_became_active":"2024-09-17T03:21:08.064712+0000","last_became_peered":"2024-09-17T03:21:08.064712+0000","last_unstale":"2024-09-17T03:21:09.075384+0000","last_undegraded":"2024-09-17T03:21:09.075384+0000","last_fullsized":"2024-09-17T03:21:09.075384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037730+0000","last_change":"2024-09-17T03:21:08.071508+0000","last_active":"2024-09-17T03:21:09.037730+0000","last_peered":"2024-09-17T03:21:09.037730+0000","last_clean":"2024-09-17T03:21:09.037730+0000","last_became_active":"2024-09-17T03:21:08.071167+0000","last_became_peered":"2024-09-17T03:21:08.071167+0000","last_unstale":"2024-09-17T03:21:09.037730+0000","last_undegraded":"2024-09-17T03:21:09.037730+0000","last_fullsized":"2024-09-17T03:21:09.037730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:05.628131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036578+0000","last_change":"2024-09-17T03:21:08.066995+0000","last_active":"2024-09-17T03:21:09.036578+0000","last_peered":"2024-09-17T03:21:09.036578+0000","last_clean":"2024-09-17T03:21:09.036578+0000","last_became_active":"2024-09-17T03:21:08.066620+0000","last_became_peered":"2024-09-17T03:21:08.066620+0000","last_unstale":"2024-09-17T03:21:09.036578+0000","last_undegraded":"2024-09-17T03:21:09.036578+0000","last_fullsized":"2024-09-17T03:21:09.036578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:53.398923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034599+0000","last_change":"2024-09-17T03:21:08.060591+0000","last_active":"2024-09-17T03:21:09.034599+0000","last_peered":"2024-09-17T03:21:09.034599+0000","last_clean":"2024-09-17T03:21:09.034599+0000","last_became_active":"2024-09-17T03:21:08.056048+0000","last_became_peered":"2024-09-17T03:21:08.056048+0000","last_unstale":"2024-09-17T03:21:09.034599+0000","last_undegraded":"2024-09-17T03:21:09.034599+0000","last_fullsized":"2024-09-17T03:21:09.034599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402958+0000","last_change":"2024-09-17T03:21:08.068237+0000","last_active":"2024-09-17T03:21:10.402958+0000","last_peered":"2024-09-17T03:21:10.402958+0000","last_clean":"2024-09-17T03:21:10.402958+0000","last_became_active":"2024-09-17T03:21:08.067863+0000","last_became_peered":"2024-09-17T03:21:08.067863+0000","last_unstale":"2024-09-17T03:21:10.402958+0000","last_undegraded":"2024-09-17T03:21:10.402958+0000","last_fullsized":"2024-09-17T03:21:10.402958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:18.563998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406412+0000","last_change":"2024-09-17T03:21:08.060741+0000","last_active":"2024-09-17T03:21:10.406412+0000","last_peered":"2024-09-17T03:21:10.406412+0000","last_clean":"2024-09-17T03:21:10.406412+0000","last_became_active":"2024-09-17T03:21:08.060429+0000","last_became_peered":"2024-09-17T03:21:08.060429+0000","last_unstale":"2024-09-17T03:21:10.406412+0000","last_undegraded":"2024-09-17T03:21:10.406412+0000","last_fullsized":"2024-09-17T03:21:10.406412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:20.270467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075420+0000","last_change":"2024-09-17T03:21:08.072066+0000","last_active":"2024-09-17T03:21:09.075420+0000","last_peered":"2024-09-17T03:21:09.075420+0000","last_clean":"2024-09-17T03:21:09.075420+0000","last_became_active":"2024-09-17T03:21:08.070966+0000","last_became_peered":"2024-09-17T03:21:08.070966+0000","last_unstale":"2024-09-17T03:21:09.075420+0000","last_undegraded":"2024-09-17T03:21:09.075420+0000","last_fullsized":"2024-09-17T03:21:09.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034202+0000","last_change":"2024-09-17T03:21:08.049953+0000","last_active":"2024-09-17T03:21:09.034202+0000","last_peered":"2024-09-17T03:21:09.034202+0000","last_clean":"2024-09-17T03:21:09.034202+0000","last_became_active":"2024-09-17T03:21:08.049526+0000","last_became_peered":"2024-09-17T03:21:08.049526+0000","last_unstale":"2024-09-17T03:21:09.034202+0000","last_undegraded":"2024-09-17T03:21:09.034202+0000","last_fullsized":"2024-09-17T03:21:09.034202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:59.117173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036398+0000","last_change":"2024-09-17T03:21:08.051667+0000","last_active":"2024-09-17T03:21:09.036398+0000","last_peered":"2024-09-17T03:21:09.036398+0000","last_clean":"2024-09-17T03:21:09.036398+0000","last_became_active":"2024-09-17T03:21:08.051099+0000","last_became_peered":"2024-09-17T03:21:08.051099+0000","last_unstale":"2024-09-17T03:21:09.036398+0000","last_undegraded":"2024-09-17T03:21:09.036398+0000","last_fullsized":"2024-09-17T03:21:09.036398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:26.245059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037066+0000","last_change":"2024-09-17T03:21:08.052647+0000","last_active":"2024-09-17T03:21:09.037066+0000","last_peered":"2024-09-17T03:21:09.037066+0000","last_clean":"2024-09-17T03:21:09.037066+0000","last_became_active":"2024-09-17T03:21:08.052328+0000","last_became_peered":"2024-09-17T03:21:08.052328+0000","last_unstale":"2024-09-17T03:21:09.037066+0000","last_undegraded":"2024-09-17T03:21:09.037066+0000","last_fullsized":"2024-09-17T03:21:09.037066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:48.836360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075599+0000","last_change":"2024-09-17T03:21:08.068699+0000","last_active":"2024-09-17T03:21:09.075599+0000","last_peered":"2024-09-17T03:21:09.075599+0000","last_clean":"2024-09-17T03:21:09.075599+0000","last_became_active":"2024-09-17T03:21:08.068338+0000","last_became_peered":"2024-09-17T03:21:08.068338+0000","last_unstale":"2024-09-17T03:21:09.075599+0000","last_undegraded":"2024-09-17T03:21:09.075599+0000","last_fullsized":"2024-09-17T03:21:09.075599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036872+0000","last_change":"2024-09-17T03:21:08.059892+0000","last_active":"2024-09-17T03:21:09.036872+0000","last_peered":"2024-09-17T03:21:09.036872+0000","last_clean":"2024-09-17T03:21:09.036872+0000","last_became_active":"2024-09-17T03:21:08.059568+0000","last_became_peered":"2024-09-17T03:21:08.059568+0000","last_unstale":"2024-09-17T03:21:09.036872+0000","last_undegraded":"2024-09-17T03:21:09.036872+0000","last_fullsized":"2024-09-17T03:21:09.036872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:03.752767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034157+0000","last_change":"2024-09-17T03:21:08.049305+0000","last_active":"2024-09-17T03:21:09.034157+0000","last_peered":"2024-09-17T03:21:09.034157+0000","last_clean":"2024-09-17T03:21:09.034157+0000","last_became_active":"2024-09-17T03:21:08.048911+0000","last_became_peered":"2024-09-17T03:21:08.048911+0000","last_unstale":"2024-09-17T03:21:09.034157+0000","last_undegraded":"2024-09-17T03:21:09.034157+0000","last_fullsized":"2024-09-17T03:21:09.034157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402687+0000","last_change":"2024-09-17T03:21:08.060288+0000","last_active":"2024-09-17T03:21:10.402687+0000","last_peered":"2024-09-17T03:21:10.402687+0000","last_clean":"2024-09-17T03:21:10.402687+0000","last_became_active":"2024-09-17T03:21:08.059954+0000","last_became_peered":"2024-09-17T03:21:08.059954+0000","last_unstale":"2024-09-17T03:21:10.402687+0000","last_undegraded":"2024-09-17T03:21:10.402687+0000","last_fullsized":"2024-09-17T03:21:10.402687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075416+0000","last_change":"2024-09-17T03:21:08.067449+0000","last_active":"2024-09-17T03:21:09.075416+0000","last_peered":"2024-09-17T03:21:09.075416+0000","last_clean":"2024-09-17T03:21:09.075416+0000","last_became_active":"2024-09-17T03:21:08.066649+0000","last_became_peered":"2024-09-17T03:21:08.066649+0000","last_unstale":"2024-09-17T03:21:09.075416+0000","last_undegraded":"2024-09-17T03:21:09.075416+0000","last_fullsized":"2024-09-17T03:21:09.075416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037818+0000","last_change":"2024-09-17T03:21:08.055200+0000","last_active":"2024-09-17T03:21:09.037818+0000","last_peered":"2024-09-17T03:21:09.037818+0000","last_clean":"2024-09-17T03:21:09.037818+0000","last_became_active":"2024-09-17T03:21:08.054451+0000","last_became_peered":"2024-09-17T03:21:08.054451+0000","last_unstale":"2024-09-17T03:21:09.037818+0000","last_undegraded":"2024-09-17T03:21:09.037818+0000","last_fullsized":"2024-09-17T03:21:09.037818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:41.720407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036639+0000","last_change":"2024-09-17T03:21:07.042857+0000","last_active":"2024-09-17T03:21:09.036639+0000","last_peered":"2024-09-17T03:21:09.036639+0000","last_clean":"2024-09-17T03:21:09.036639+0000","last_became_active":"2024-09-17T03:21:07.042659+0000","last_became_peered":"2024-09-17T03:21:07.042659+0000","last_unstale":"2024-09-17T03:21:09.036639+0000","last_undegraded":"2024-09-17T03:21:09.036639+0000","last_fullsized":"2024-09-17T03:21:09.036639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034320+0000","last_change":"2024-09-17T03:21:08.054380+0000","last_active":"2024-09-17T03:21:09.034320+0000","last_peered":"2024-09-17T03:21:09.034320+0000","last_clean":"2024-09-17T03:21:09.034320+0000","last_became_active":"2024-09-17T03:21:08.053967+0000","last_became_peered":"2024-09-17T03:21:08.053967+0000","last_unstale":"2024-09-17T03:21:09.034320+0000","last_undegraded":"2024-09-17T03:21:09.034320+0000","last_fullsized":"2024-09-17T03:21:09.034320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074976+0000","last_change":"2024-09-17T03:21:07.037433+0000","last_active":"2024-09-17T03:21:09.074976+0000","last_peered":"2024-09-17T03:21:09.074976+0000","last_clean":"2024-09-17T03:21:09.074976+0000","last_became_active":"2024-09-17T03:21:07.035663+0000","last_became_peered":"2024-09-17T03:21:07.035663+0000","last_unstale":"2024-09-17T03:21:09.074976+0000","last_undegraded":"2024-09-17T03:21:09.074976+0000","last_fullsized":"2024-09-17T03:21:09.074976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036177+0000","last_change":"2024-09-17T03:21:07.033856+0000","last_active":"2024-09-17T03:21:09.036177+0000","last_peered":"2024-09-17T03:21:09.036177+0000","last_clean":"2024-09-17T03:21:09.036177+0000","last_became_active":"2024-09-17T03:21:07.032761+0000","last_became_peered":"2024-09-17T03:21:07.032761+0000","last_unstale":"2024-09-17T03:21:09.036177+0000","last_undegraded":"2024-09-17T03:21:09.036177+0000","last_fullsized":"2024-09-17T03:21:09.036177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036275+0000","last_change":"2024-09-17T03:21:08.046160+0000","last_active":"2024-09-17T03:21:09.036275+0000","last_peered":"2024-09-17T03:21:09.036275+0000","last_clean":"2024-09-17T03:21:09.036275+0000","last_became_active":"2024-09-17T03:21:08.045937+0000","last_became_peered":"2024-09-17T03:21:08.045937+0000","last_unstale":"2024-09-17T03:21:09.036275+0000","last_undegraded":"2024-09-17T03:21:09.036275+0000","last_fullsized":"2024-09-17T03:21:09.036275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:13.549239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075195+0000","last_change":"2024-09-17T03:21:07.042752+0000","last_active":"2024-09-17T03:21:09.075195+0000","last_peered":"2024-09-17T03:21:09.075195+0000","last_clean":"2024-09-17T03:21:09.075195+0000","last_became_active":"2024-09-17T03:21:07.042359+0000","last_became_peered":"2024-09-17T03:21:07.042359+0000","last_unstale":"2024-09-17T03:21:09.075195+0000","last_undegraded":"2024-09-17T03:21:09.075195+0000","last_fullsized":"2024-09-17T03:21:09.075195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034735+0000","last_change":"2024-09-17T03:21:08.060578+0000","last_active":"2024-09-17T03:21:09.034735+0000","last_peered":"2024-09-17T03:21:09.034735+0000","last_clean":"2024-09-17T03:21:09.034735+0000","last_became_active":"2024-09-17T03:21:08.059037+0000","last_became_peered":"2024-09-17T03:21:08.059037+0000","last_unstale":"2024-09-17T03:21:09.034735+0000","last_undegraded":"2024-09-17T03:21:09.034735+0000","last_fullsized":"2024-09-17T03:21:09.034735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037963+0000","last_change":"2024-09-17T03:21:08.052908+0000","last_active":"2024-09-17T03:21:09.037963+0000","last_peered":"2024-09-17T03:21:09.037963+0000","last_clean":"2024-09-17T03:21:09.037963+0000","last_became_active":"2024-09-17T03:21:08.052578+0000","last_became_peered":"2024-09-17T03:21:08.052578+0000","last_unstale":"2024-09-17T03:21:09.037963+0000","last_undegraded":"2024-09-17T03:21:09.037963+0000","last_fullsized":"2024-09-17T03:21:09.037963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:06.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075516+0000","last_change":"2024-09-17T03:21:07.042836+0000","last_active":"2024-09-17T03:21:09.075516+0000","last_peered":"2024-09-17T03:21:09.075516+0000","last_clean":"2024-09-17T03:21:09.075516+0000","last_became_active":"2024-09-17T03:21:07.042380+0000","last_became_peered":"2024-09-17T03:21:07.042380+0000","last_unstale":"2024-09-17T03:21:09.075516+0000","last_undegraded":"2024-09-17T03:21:09.075516+0000","last_fullsized":"2024-09-17T03:21:09.075516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405970+0000","last_change":"2024-09-17T03:21:08.066084+0000","last_active":"2024-09-17T03:21:10.405970+0000","last_peered":"2024-09-17T03:21:10.405970+0000","last_clean":"2024-09-17T03:21:10.405970+0000","last_became_active":"2024-09-17T03:21:08.065807+0000","last_became_peered":"2024-09-17T03:21:08.065807+0000","last_unstale":"2024-09-17T03:21:10.405970+0000","last_undegraded":"2024-09-17T03:21:10.405970+0000","last_fullsized":"2024-09-17T03:21:10.405970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:53.782411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034281+0000","last_change":"2024-09-17T03:21:07.033053+0000","last_active":"2024-09-17T03:21:09.034281+0000","last_peered":"2024-09-17T03:21:09.034281+0000","last_clean":"2024-09-17T03:21:09.034281+0000","last_became_active":"2024-09-17T03:21:07.030140+0000","last_became_peered":"2024-09-17T03:21:07.030140+0000","last_unstale":"2024-09-17T03:21:09.034281+0000","last_undegraded":"2024-09-17T03:21:09.034281+0000","last_fullsized":"2024-09-17T03:21:09.034281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033897+0000","last_change":"2024-09-17T03:21:08.057090+0000","last_active":"2024-09-17T03:21:09.033897+0000","last_peered":"2024-09-17T03:21:09.033897+0000","last_clean":"2024-09-17T03:21:09.033897+0000","last_became_active":"2024-09-17T03:21:08.055419+0000","last_became_peered":"2024-09-17T03:21:08.055419+0000","last_unstale":"2024-09-17T03:21:09.033897+0000","last_undegraded":"2024-09-17T03:21:09.033897+0000","last_fullsized":"2024-09-17T03:21:09.033897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:54.579062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406013+0000","last_change":"2024-09-17T03:21:08.059703+0000","last_active":"2024-09-17T03:21:10.406013+0000","last_peered":"2024-09-17T03:21:10.406013+0000","last_clean":"2024-09-17T03:21:10.406013+0000","last_became_active":"2024-09-17T03:21:08.059411+0000","last_became_peered":"2024-09-17T03:21:08.059411+0000","last_unstale":"2024-09-17T03:21:10.406013+0000","last_undegraded":"2024-09-17T03:21:10.406013+0000","last_fullsized":"2024-09-17T03:21:10.406013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:00.131726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074938+0000","last_change":"2024-09-17T03:21:07.043190+0000","last_active":"2024-09-17T03:21:09.074938+0000","last_peered":"2024-09-17T03:21:09.074938+0000","last_clean":"2024-09-17T03:21:09.074938+0000","last_became_active":"2024-09-17T03:21:07.042868+0000","last_became_peered":"2024-09-17T03:21:07.042868+0000","last_unstale":"2024-09-17T03:21:09.074938+0000","last_undegraded":"2024-09-17T03:21:09.074938+0000","last_fullsized":"2024-09-17T03:21:09.074938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075453+0000","last_change":"2024-09-17T03:21:08.070393+0000","last_active":"2024-09-17T03:21:09.075453+0000","last_peered":"2024-09-17T03:21:09.075453+0000","last_clean":"2024-09-17T03:21:09.075453+0000","last_became_active":"2024-09-17T03:21:08.069695+0000","last_became_peered":"2024-09-17T03:21:08.069695+0000","last_unstale":"2024-09-17T03:21:09.075453+0000","last_undegraded":"2024-09-17T03:21:09.075453+0000","last_fullsized":"2024-09-17T03:21:09.075453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402466+0000","last_change":"2024-09-17T03:21:07.037447+0000","last_active":"2024-09-17T03:21:10.402466+0000","last_peered":"2024-09-17T03:21:10.402466+0000","last_clean":"2024-09-17T03:21:10.402466+0000","last_became_active":"2024-09-17T03:21:07.037133+0000","last_became_peered":"2024-09-17T03:21:07.037133+0000","last_unstale":"2024-09-17T03:21:10.402466+0000","last_undegraded":"2024-09-17T03:21:10.402466+0000","last_fullsized":"2024-09-17T03:21:10.402466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405541+0000","last_change":"2024-09-17T03:21:08.064992+0000","last_active":"2024-09-17T03:21:10.405541+0000","last_peered":"2024-09-17T03:21:10.405541+0000","last_clean":"2024-09-17T03:21:10.405541+0000","last_became_active":"2024-09-17T03:21:08.064705+0000","last_became_peered":"2024-09-17T03:21:08.064705+0000","last_unstale":"2024-09-17T03:21:10.405541+0000","last_undegraded":"2024-09-17T03:21:10.405541+0000","last_fullsized":"2024-09-17T03:21:10.405541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.038518+0000","last_change":"2024-09-17T03:21:07.031511+0000","last_active":"2024-09-17T03:21:09.038518+0000","last_peered":"2024-09-17T03:21:09.038518+0000","last_clean":"2024-09-17T03:21:09.038518+0000","last_became_active":"2024-09-17T03:21:07.031175+0000","last_became_peered":"2024-09-17T03:21:07.031175+0000","last_unstale":"2024-09-17T03:21:09.038518+0000","last_undegraded":"2024-09-17T03:21:09.038518+0000","last_fullsized":"2024-09-17T03:21:09.038518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:23:29.396492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406155+0000","last_change":"2024-09-17T03:21:08.067321+0000","last_active":"2024-09-17T03:21:10.406155+0000","last_peered":"2024-09-17T03:21:10.406155+0000","last_clean":"2024-09-17T03:21:10.406155+0000","last_became_active":"2024-09-17T03:21:08.066953+0000","last_became_peered":"2024-09-17T03:21:08.066953+0000","last_unstale":"2024-09-17T03:21:10.406155+0000","last_undegraded":"2024-09-17T03:21:10.406155+0000","last_fullsized":"2024-09-17T03:21:10.406155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:38.976170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402786+0000","last_change":"2024-09-17T03:21:07.042316+0000","last_active":"2024-09-17T03:21:10.402786+0000","last_peered":"2024-09-17T03:21:10.402786+0000","last_clean":"2024-09-17T03:21:10.402786+0000","last_became_active":"2024-09-17T03:21:07.039133+0000","last_became_peered":"2024-09-17T03:21:07.039133+0000","last_unstale":"2024-09-17T03:21:10.402786+0000","last_undegraded":"2024-09-17T03:21:10.402786+0000","last_fullsized":"2024-09-17T03:21:10.402786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406653+0000","last_change":"2024-09-17T03:21:08.057737+0000","last_active":"2024-09-17T03:21:10.406653+0000","last_peered":"2024-09-17T03:21:10.406653+0000","last_clean":"2024-09-17T03:21:10.406653+0000","last_became_active":"2024-09-17T03:21:08.057560+0000","last_became_peered":"2024-09-17T03:21:08.057560+0000","last_unstale":"2024-09-17T03:21:10.406653+0000","last_undegraded":"2024-09-17T03:21:10.406653+0000","last_fullsized":"2024-09-17T03:21:10.406653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:07.761806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402646+0000","last_change":"2024-09-17T03:21:07.042337+0000","last_active":"2024-09-17T03:21:10.402646+0000","last_peered":"2024-09-17T03:21:10.402646+0000","last_clean":"2024-09-17T03:21:10.402646+0000","last_became_active":"2024-09-17T03:21:07.040846+0000","last_became_peered":"2024-09-17T03:21:07.040846+0000","last_unstale":"2024-09-17T03:21:10.402646+0000","last_undegraded":"2024-09-17T03:21:10.402646+0000","last_fullsized":"2024-09-17T03:21:10.402646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406114+0000","last_change":"2024-09-17T03:21:08.074560+0000","last_active":"2024-09-17T03:21:10.406114+0000","last_peered":"2024-09-17T03:21:10.406114+0000","last_clean":"2024-09-17T03:21:10.406114+0000","last_became_active":"2024-09-17T03:21:08.074267+0000","last_became_peered":"2024-09-17T03:21:08.074267+0000","last_unstale":"2024-09-17T03:21:10.406114+0000","last_undegraded":"2024-09-17T03:21:10.406114+0000","last_fullsized":"2024-09-17T03:21:10.406114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:09.894552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404671+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:10.404671+0000","last_peered":"2024-09-17T03:21:10.404671+0000","last_clean":"2024-09-17T03:21:10.404671+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:10.404671+0000","last_undegraded":"2024-09-17T03:21:10.404671+0000","last_fullsized":"2024-09-17T03:21:10.404671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036442+0000","last_change":"2024-09-17T03:21:08.046702+0000","last_active":"2024-09-17T03:21:09.036442+0000","last_peered":"2024-09-17T03:21:09.036442+0000","last_clean":"2024-09-17T03:21:09.036442+0000","last_became_active":"2024-09-17T03:21:08.046321+0000","last_became_peered":"2024-09-17T03:21:08.046321+0000","last_unstale":"2024-09-17T03:21:09.036442+0000","last_undegraded":"2024-09-17T03:21:09.036442+0000","last_fullsized":"2024-09-17T03:21:09.036442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404543+0000","last_change":"2024-09-17T03:21:07.045789+0000","last_active":"2024-09-17T03:21:10.404543+0000","last_peered":"2024-09-17T03:21:10.404543+0000","last_clean":"2024-09-17T03:21:10.404543+0000","last_became_active":"2024-09-17T03:21:07.045443+0000","last_became_peered":"2024-09-17T03:21:07.045443+0000","last_unstale":"2024-09-17T03:21:10.404543+0000","last_undegraded":"2024-09-17T03:21:10.404543+0000","last_fullsized":"2024-09-17T03:21:10.404543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:00.822415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033526+0000","last_change":"2024-09-17T03:21:08.040522+0000","last_active":"2024-09-17T03:21:09.033526+0000","last_peered":"2024-09-17T03:21:09.033526+0000","last_clean":"2024-09-17T03:21:09.033526+0000","last_became_active":"2024-09-17T03:21:08.037656+0000","last_became_peered":"2024-09-17T03:21:08.037656+0000","last_unstale":"2024-09-17T03:21:09.033526+0000","last_undegraded":"2024-09-17T03:21:09.033526+0000","last_fullsized":"2024-09-17T03:21:09.033526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:58.143540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402609+0000","last_change":"2024-09-17T03:21:07.035825+0000","last_active":"2024-09-17T03:21:10.402609+0000","last_peered":"2024-09-17T03:21:10.402609+0000","last_clean":"2024-09-17T03:21:10.402609+0000","last_became_active":"2024-09-17T03:21:07.035573+0000","last_became_peered":"2024-09-17T03:21:07.035573+0000","last_unstale":"2024-09-17T03:21:10.402609+0000","last_undegraded":"2024-09-17T03:21:10.402609+0000","last_fullsized":"2024-09-17T03:21:10.402609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037938+0000","last_change":"2024-09-17T03:21:08.051692+0000","last_active":"2024-09-17T03:21:09.037938+0000","last_peered":"2024-09-17T03:21:09.037938+0000","last_clean":"2024-09-17T03:21:09.037938+0000","last_became_active":"2024-09-17T03:21:08.050563+0000","last_became_peered":"2024-09-17T03:21:08.050563+0000","last_unstale":"2024-09-17T03:21:09.037938+0000","last_undegraded":"2024-09-17T03:21:09.037938+0000","last_fullsized":"2024-09-17T03:21:09.037938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:07.035552+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:07.035214+0000","last_became_peered":"2024-09-17T03:21:07.035214+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.557724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037050+0000","last_change":"2024-09-17T03:21:08.059028+0000","last_active":"2024-09-17T03:21:09.037050+0000","last_peered":"2024-09-17T03:21:09.037050+0000","last_clean":"2024-09-17T03:21:09.037050+0000","last_became_active":"2024-09-17T03:21:08.058709+0000","last_became_peered":"2024-09-17T03:21:08.058709+0000","last_unstale":"2024-09-17T03:21:09.037050+0000","last_undegraded":"2024-09-17T03:21:09.037050+0000","last_fullsized":"2024-09-17T03:21:09.037050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:21.370502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075234+0000","last_change":"2024-09-17T03:21:07.033103+0000","last_active":"2024-09-17T03:21:09.075234+0000","last_peered":"2024-09-17T03:21:09.075234+0000","last_clean":"2024-09-17T03:21:09.075234+0000","last_became_active":"2024-09-17T03:21:07.032795+0000","last_became_peered":"2024-09-17T03:21:07.032795+0000","last_unstale":"2024-09-17T03:21:09.075234+0000","last_undegraded":"2024-09-17T03:21:09.075234+0000","last_fullsized":"2024-09-17T03:21:09.075234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033821+0000","last_change":"2024-09-17T03:21:08.065685+0000","last_active":"2024-09-17T03:21:09.033821+0000","last_peered":"2024-09-17T03:21:09.033821+0000","last_clean":"2024-09-17T03:21:09.033821+0000","last_became_active":"2024-09-17T03:21:08.065243+0000","last_became_peered":"2024-09-17T03:21:08.065243+0000","last_unstale":"2024-09-17T03:21:09.033821+0000","last_undegraded":"2024-09-17T03:21:09.033821+0000","last_fullsized":"2024-09-17T03:21:09.033821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:23.445102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404390+0000","last_change":"2024-09-17T03:21:07.034832+0000","last_active":"2024-09-17T03:21:10.404390+0000","last_peered":"2024-09-17T03:21:10.404390+0000","last_clean":"2024-09-17T03:21:10.404390+0000","last_became_active":"2024-09-17T03:21:07.034455+0000","last_became_peered":"2024-09-17T03:21:07.034455+0000","last_unstale":"2024-09-17T03:21:10.404390+0000","last_undegraded":"2024-09-17T03:21:10.404390+0000","last_fullsized":"2024-09-17T03:21:10.404390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037637+0000","last_change":"2024-09-17T03:21:08.046797+0000","last_active":"2024-09-17T03:21:09.037637+0000","last_peered":"2024-09-17T03:21:09.037637+0000","last_clean":"2024-09-17T03:21:09.037637+0000","last_became_active":"2024-09-17T03:21:08.046355+0000","last_became_peered":"2024-09-17T03:21:08.046355+0000","last_unstale":"2024-09-17T03:21:09.037637+0000","last_undegraded":"2024-09-17T03:21:09.037637+0000","last_fullsized":"2024-09-17T03:21:09.037637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:41.458145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036511+0000","last_change":"2024-09-17T03:21:08.060646+0000","last_active":"2024-09-17T03:21:09.036511+0000","last_peered":"2024-09-17T03:21:09.036511+0000","last_clean":"2024-09-17T03:21:09.036511+0000","last_became_active":"2024-09-17T03:21:08.060341+0000","last_became_peered":"2024-09-17T03:21:08.060341+0000","last_unstale":"2024-09-17T03:21:09.036511+0000","last_undegraded":"2024-09-17T03:21:09.036511+0000","last_fullsized":"2024-09-17T03:21:09.036511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:06.388228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036654+0000","last_change":"2024-09-17T03:21:08.045211+0000","last_active":"2024-09-17T03:21:09.036654+0000","last_peered":"2024-09-17T03:21:09.036654+0000","last_clean":"2024-09-17T03:21:09.036654+0000","last_became_active":"2024-09-17T03:21:08.043718+0000","last_became_peered":"2024-09-17T03:21:08.043718+0000","last_unstale":"2024-09-17T03:21:09.036654+0000","last_undegraded":"2024-09-17T03:21:09.036654+0000","last_fullsized":"2024-09-17T03:21:09.036654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:41.789200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406079+0000","last_change":"2024-09-17T03:21:07.038659+0000","last_active":"2024-09-17T03:21:10.406079+0000","last_peered":"2024-09-17T03:21:10.406079+0000","last_clean":"2024-09-17T03:21:10.406079+0000","last_became_active":"2024-09-17T03:21:07.038247+0000","last_became_peered":"2024-09-17T03:21:07.038247+0000","last_unstale":"2024-09-17T03:21:10.406079+0000","last_undegraded":"2024-09-17T03:21:10.406079+0000","last_fullsized":"2024-09-17T03:21:10.406079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075179+0000","last_change":"2024-09-17T03:21:08.065090+0000","last_active":"2024-09-17T03:21:09.075179+0000","last_peered":"2024-09-17T03:21:09.075179+0000","last_clean":"2024-09-17T03:21:09.075179+0000","last_became_active":"2024-09-17T03:21:08.064729+0000","last_became_peered":"2024-09-17T03:21:08.064729+0000","last_unstale":"2024-09-17T03:21:09.075179+0000","last_undegraded":"2024-09-17T03:21:09.075179+0000","last_fullsized":"2024-09-17T03:21:09.075179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405784+0000","last_change":"2024-09-17T03:21:07.049981+0000","last_active":"2024-09-17T03:21:10.405784+0000","last_peered":"2024-09-17T03:21:10.405784+0000","last_clean":"2024-09-17T03:21:10.405784+0000","last_became_active":"2024-09-17T03:21:07.049635+0000","last_became_peered":"2024-09-17T03:21:07.049635+0000","last_unstale":"2024-09-17T03:21:10.405784+0000","last_undegraded":"2024-09-17T03:21:10.405784+0000","last_fullsized":"2024-09-17T03:21:10.405784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:11.341893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406212+0000","last_change":"2024-09-17T03:21:07.038577+0000","last_active":"2024-09-17T03:21:10.406212+0000","last_peered":"2024-09-17T03:21:10.406212+0000","last_clean":"2024-09-17T03:21:10.406212+0000","last_became_active":"2024-09-17T03:21:07.038261+0000","last_became_peered":"2024-09-17T03:21:07.038261+0000","last_unstale":"2024-09-17T03:21:10.406212+0000","last_undegraded":"2024-09-17T03:21:10.406212+0000","last_fullsized":"2024-09-17T03:21:10.406212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:02.813101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034518+0000","last_change":"2024-09-17T03:21:08.061875+0000","last_active":"2024-09-17T03:21:09.034518+0000","last_peered":"2024-09-17T03:21:09.034518+0000","last_clean":"2024-09-17T03:21:09.034518+0000","last_became_active":"2024-09-17T03:21:08.057042+0000","last_became_peered":"2024-09-17T03:21:08.057042+0000","last_unstale":"2024-09-17T03:21:09.034518+0000","last_undegraded":"2024-09-17T03:21:09.034518+0000","last_fullsized":"2024-09-17T03:21:09.034518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:18.945802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036926+0000","last_change":"2024-09-17T03:21:07.037271+0000","last_active":"2024-09-17T03:21:09.036926+0000","last_peered":"2024-09-17T03:21:09.036926+0000","last_clean":"2024-09-17T03:21:09.036926+0000","last_became_active":"2024-09-17T03:21:07.036859+0000","last_became_peered":"2024-09-17T03:21:07.036859+0000","last_unstale":"2024-09-17T03:21:09.036926+0000","last_undegraded":"2024-09-17T03:21:09.036926+0000","last_fullsized":"2024-09-17T03:21:09.036926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037073+0000","last_change":"2024-09-17T03:21:08.068273+0000","last_active":"2024-09-17T03:21:09.037073+0000","last_peered":"2024-09-17T03:21:09.037073+0000","last_clean":"2024-09-17T03:21:09.037073+0000","last_became_active":"2024-09-17T03:21:08.066628+0000","last_became_peered":"2024-09-17T03:21:08.066628+0000","last_unstale":"2024-09-17T03:21:09.037073+0000","last_undegraded":"2024-09-17T03:21:09.037073+0000","last_fullsized":"2024-09-17T03:21:09.037073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034788+0000","last_change":"2024-09-17T03:21:07.032795+0000","last_active":"2024-09-17T03:21:09.034788+0000","last_peered":"2024-09-17T03:21:09.034788+0000","last_clean":"2024-09-17T03:21:09.034788+0000","last_became_active":"2024-09-17T03:21:07.032288+0000","last_became_peered":"2024-09-17T03:21:07.032288+0000","last_unstale":"2024-09-17T03:21:09.034788+0000","last_undegraded":"2024-09-17T03:21:09.034788+0000","last_fullsized":"2024-09-17T03:21:09.034788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:41.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:16.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 -- 172.21.15.46:0/3400390192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec160a20 msgr2=0x7f1aec162e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 --2- 172.21.15.46:0/3400390192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec160a20 0x7f1aec162e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009840 tx=0x7f1ad401c820 comp rx=0 tx=0).stop 2024-09-17T03:21:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 -- 172.21.15.46:0/3400390192 shutdown_connections 2024-09-17T03:21:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 --2- 172.21.15.46:0/3400390192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec160a20 0x7f1aec162e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 -- 172.21.15.46:0/3400390192 >> 172.21.15.46:0/3400390192 conn(0x7f1aec0b2040 msgr2=0x7f1aec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 -- 172.21.15.46:0/3400390192 shutdown_connections 2024-09-17T03:21:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.074+0000 7f1af1400640 1 -- 172.21.15.46:0/3400390192 wait complete. 2024-09-17T03:21:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 Processor -- start 2024-09-17T03:21:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 -- start start 2024-09-17T03:21:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1aec15e8c0 0x7f1aec162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1aec160a20 0x7f1aec1664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 0x7f1aec074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec169e80 con 0x7f1aec163380 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1aec169d00 con 0x7f1aec15e8c0 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1aec16a000 con 0x7f1aec160a20 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 0x7f1aec074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 0x7f1aec074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38330/0 (socket says 172.21.15.46:38330) 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 -- 172.21.15.46:0/832371769 learned_addr learned my addr 172.21.15.46:0/832371769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1ae1e00640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1aec160a20 0x7f1aec1664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aea800640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1aec15e8c0 0x7f1aec162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 -- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1aec160a20 msgr2=0x7f1aec1664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1aec160a20 0x7f1aec1664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 -- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1aec15e8c0 msgr2=0x7f1aec162ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1aec15e8c0 0x7f1aec162ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 -- 172.21.15.46:0/832371769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aec166a00 con 0x7f1aec163380 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1aeb200640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 0x7f1aec074420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b9a0 tx=0x7f1ae400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4019070 con 0x7f1aec163380 2024-09-17T03:21:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ae400cc90 con 0x7f1aec163380 2024-09-17T03:21:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aec161a20 con 0x7f1aec163380 2024-09-17T03:21:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40146a0 con 0x7f1aec163380 2024-09-17T03:21:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.078+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aec161fe0 con 0x7f1aec163380 2024-09-17T03:21:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.082+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aec166c80 con 0x7f1aec163380 2024-09-17T03:21:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.082+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1ae401c070 con 0x7f1aec163380 2024-09-17T03:21:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.082+0000 7f1ae3400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ac8073230 0x7f1ac8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.082+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1ae4093e30 con 0x7f1aec163380 2024-09-17T03:21:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.082+0000 7f1aea800640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ac8073230 0x7f1ac8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.082+0000 7f1aea800640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ac8073230 0x7f1ac8075740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1adc007fd0 tx=0x7f1adc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.086+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1ae405e760 con 0x7f1aec163380 2024-09-17T03:21:17.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.258+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1aec15ecd0 con 0x7f1ac8073230 2024-09-17T03:21:17.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.266+0000 7f1ae3400640 1 -- 172.21.15.46:0/832371769 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f1aec15ecd0 con 0x7f1ac8073230 2024-09-17T03:21:17.271 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:17.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.266+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ac8073230 msgr2=0x7f1ac8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.266+0000 7f1af1400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ac8073230 0x7f1ac8075740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1adc007fd0 tx=0x7f1adc009040 comp rx=0 tx=0).stop 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.266+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 msgr2=0x7f1aec074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.266+0000 7f1af1400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 0x7f1aec074420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b9a0 tx=0x7f1ae400be60 comp rx=0 tx=0).stop 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 shutdown_connections 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ac8073230 0x7f1ac8075740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aec163380 0x7f1aec074420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1aec160a20 0x7f1aec1664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 --2- 172.21.15.46:0/832371769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1aec15e8c0 0x7f1aec162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 >> 172.21.15.46:0/832371769 conn(0x7f1aec0b2040 msgr2=0x7f1aec15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 shutdown_connections 2024-09-17T03:21:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.270+0000 7f1af1400640 1 -- 172.21.15.46:0/832371769 wait complete. 2024-09-17T03:21:17.293 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T03:21:17.197496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235756,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738964,"statfs":{"total":773094113280,"available":772852699136,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.167731"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:08.073787+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:08.073460+0000","last_became_peered":"2024-09-17T03:21:08.073460+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:56.409644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034270+0000","last_change":"2024-09-17T03:21:08.046022+0000","last_active":"2024-09-17T03:21:09.034270+0000","last_peered":"2024-09-17T03:21:09.034270+0000","last_clean":"2024-09-17T03:21:09.034270+0000","last_became_active":"2024-09-17T03:21:08.045462+0000","last_became_peered":"2024-09-17T03:21:08.045462+0000","last_unstale":"2024-09-17T03:21:09.034270+0000","last_undegraded":"2024-09-17T03:21:09.034270+0000","last_fullsized":"2024-09-17T03:21:09.034270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405803+0000","last_change":"2024-09-17T03:21:08.057050+0000","last_active":"2024-09-17T03:21:10.405803+0000","last_peered":"2024-09-17T03:21:10.405803+0000","last_clean":"2024-09-17T03:21:10.405803+0000","last_became_active":"2024-09-17T03:21:08.056744+0000","last_became_peered":"2024-09-17T03:21:08.056744+0000","last_unstale":"2024-09-17T03:21:10.405803+0000","last_undegraded":"2024-09-17T03:21:10.405803+0000","last_fullsized":"2024-09-17T03:21:10.405803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:52.233846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405049+0000","last_change":"2024-09-17T03:21:08.072759+0000","last_active":"2024-09-17T03:21:10.405049+0000","last_peered":"2024-09-17T03:21:10.405049+0000","last_clean":"2024-09-17T03:21:10.405049+0000","last_became_active":"2024-09-17T03:21:08.072444+0000","last_became_peered":"2024-09-17T03:21:08.072444+0000","last_unstale":"2024-09-17T03:21:10.405049+0000","last_undegraded":"2024-09-17T03:21:10.405049+0000","last_fullsized":"2024-09-17T03:21:10.405049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:09.811529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036226+0000","last_change":"2024-09-17T03:21:08.045612+0000","last_active":"2024-09-17T03:21:09.036226+0000","last_peered":"2024-09-17T03:21:09.036226+0000","last_clean":"2024-09-17T03:21:09.036226+0000","last_became_active":"2024-09-17T03:21:08.044072+0000","last_became_peered":"2024-09-17T03:21:08.044072+0000","last_unstale":"2024-09-17T03:21:09.036226+0000","last_undegraded":"2024-09-17T03:21:09.036226+0000","last_fullsized":"2024-09-17T03:21:09.036226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406443+0000","last_change":"2024-09-17T03:21:08.060425+0000","last_active":"2024-09-17T03:21:10.406443+0000","last_peered":"2024-09-17T03:21:10.406443+0000","last_clean":"2024-09-17T03:21:10.406443+0000","last_became_active":"2024-09-17T03:21:08.060122+0000","last_became_peered":"2024-09-17T03:21:08.060122+0000","last_unstale":"2024-09-17T03:21:10.406443+0000","last_undegraded":"2024-09-17T03:21:10.406443+0000","last_fullsized":"2024-09-17T03:21:10.406443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:49.768654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075521+0000","last_change":"2024-09-17T03:21:08.067905+0000","last_active":"2024-09-17T03:21:09.075521+0000","last_peered":"2024-09-17T03:21:09.075521+0000","last_clean":"2024-09-17T03:21:09.075521+0000","last_became_active":"2024-09-17T03:21:08.066280+0000","last_became_peered":"2024-09-17T03:21:08.066280+0000","last_unstale":"2024-09-17T03:21:09.075521+0000","last_undegraded":"2024-09-17T03:21:09.075521+0000","last_fullsized":"2024-09-17T03:21:09.075521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075384+0000","last_change":"2024-09-17T03:21:08.065069+0000","last_active":"2024-09-17T03:21:09.075384+0000","last_peered":"2024-09-17T03:21:09.075384+0000","last_clean":"2024-09-17T03:21:09.075384+0000","last_became_active":"2024-09-17T03:21:08.064712+0000","last_became_peered":"2024-09-17T03:21:08.064712+0000","last_unstale":"2024-09-17T03:21:09.075384+0000","last_undegraded":"2024-09-17T03:21:09.075384+0000","last_fullsized":"2024-09-17T03:21:09.075384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037730+0000","last_change":"2024-09-17T03:21:08.071508+0000","last_active":"2024-09-17T03:21:09.037730+0000","last_peered":"2024-09-17T03:21:09.037730+0000","last_clean":"2024-09-17T03:21:09.037730+0000","last_became_active":"2024-09-17T03:21:08.071167+0000","last_became_peered":"2024-09-17T03:21:08.071167+0000","last_unstale":"2024-09-17T03:21:09.037730+0000","last_undegraded":"2024-09-17T03:21:09.037730+0000","last_fullsized":"2024-09-17T03:21:09.037730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:05.628131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036578+0000","last_change":"2024-09-17T03:21:08.066995+0000","last_active":"2024-09-17T03:21:09.036578+0000","last_peered":"2024-09-17T03:21:09.036578+0000","last_clean":"2024-09-17T03:21:09.036578+0000","last_became_active":"2024-09-17T03:21:08.066620+0000","last_became_peered":"2024-09-17T03:21:08.066620+0000","last_unstale":"2024-09-17T03:21:09.036578+0000","last_undegraded":"2024-09-17T03:21:09.036578+0000","last_fullsized":"2024-09-17T03:21:09.036578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:53.398923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034599+0000","last_change":"2024-09-17T03:21:08.060591+0000","last_active":"2024-09-17T03:21:09.034599+0000","last_peered":"2024-09-17T03:21:09.034599+0000","last_clean":"2024-09-17T03:21:09.034599+0000","last_became_active":"2024-09-17T03:21:08.056048+0000","last_became_peered":"2024-09-17T03:21:08.056048+0000","last_unstale":"2024-09-17T03:21:09.034599+0000","last_undegraded":"2024-09-17T03:21:09.034599+0000","last_fullsized":"2024-09-17T03:21:09.034599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402958+0000","last_change":"2024-09-17T03:21:08.068237+0000","last_active":"2024-09-17T03:21:10.402958+0000","last_peered":"2024-09-17T03:21:10.402958+0000","last_clean":"2024-09-17T03:21:10.402958+0000","last_became_active":"2024-09-17T03:21:08.067863+0000","last_became_peered":"2024-09-17T03:21:08.067863+0000","last_unstale":"2024-09-17T03:21:10.402958+0000","last_undegraded":"2024-09-17T03:21:10.402958+0000","last_fullsized":"2024-09-17T03:21:10.402958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:18.563998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406412+0000","last_change":"2024-09-17T03:21:08.060741+0000","last_active":"2024-09-17T03:21:10.406412+0000","last_peered":"2024-09-17T03:21:10.406412+0000","last_clean":"2024-09-17T03:21:10.406412+0000","last_became_active":"2024-09-17T03:21:08.060429+0000","last_became_peered":"2024-09-17T03:21:08.060429+0000","last_unstale":"2024-09-17T03:21:10.406412+0000","last_undegraded":"2024-09-17T03:21:10.406412+0000","last_fullsized":"2024-09-17T03:21:10.406412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:20.270467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075420+0000","last_change":"2024-09-17T03:21:08.072066+0000","last_active":"2024-09-17T03:21:09.075420+0000","last_peered":"2024-09-17T03:21:09.075420+0000","last_clean":"2024-09-17T03:21:09.075420+0000","last_became_active":"2024-09-17T03:21:08.070966+0000","last_became_peered":"2024-09-17T03:21:08.070966+0000","last_unstale":"2024-09-17T03:21:09.075420+0000","last_undegraded":"2024-09-17T03:21:09.075420+0000","last_fullsized":"2024-09-17T03:21:09.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034202+0000","last_change":"2024-09-17T03:21:08.049953+0000","last_active":"2024-09-17T03:21:09.034202+0000","last_peered":"2024-09-17T03:21:09.034202+0000","last_clean":"2024-09-17T03:21:09.034202+0000","last_became_active":"2024-09-17T03:21:08.049526+0000","last_became_peered":"2024-09-17T03:21:08.049526+0000","last_unstale":"2024-09-17T03:21:09.034202+0000","last_undegraded":"2024-09-17T03:21:09.034202+0000","last_fullsized":"2024-09-17T03:21:09.034202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:59.117173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036398+0000","last_change":"2024-09-17T03:21:08.051667+0000","last_active":"2024-09-17T03:21:09.036398+0000","last_peered":"2024-09-17T03:21:09.036398+0000","last_clean":"2024-09-17T03:21:09.036398+0000","last_became_active":"2024-09-17T03:21:08.051099+0000","last_became_peered":"2024-09-17T03:21:08.051099+0000","last_unstale":"2024-09-17T03:21:09.036398+0000","last_undegraded":"2024-09-17T03:21:09.036398+0000","last_fullsized":"2024-09-17T03:21:09.036398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:26.245059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037066+0000","last_change":"2024-09-17T03:21:08.052647+0000","last_active":"2024-09-17T03:21:09.037066+0000","last_peered":"2024-09-17T03:21:09.037066+0000","last_clean":"2024-09-17T03:21:09.037066+0000","last_became_active":"2024-09-17T03:21:08.052328+0000","last_became_peered":"2024-09-17T03:21:08.052328+0000","last_unstale":"2024-09-17T03:21:09.037066+0000","last_undegraded":"2024-09-17T03:21:09.037066+0000","last_fullsized":"2024-09-17T03:21:09.037066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:48.836360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075599+0000","last_change":"2024-09-17T03:21:08.068699+0000","last_active":"2024-09-17T03:21:09.075599+0000","last_peered":"2024-09-17T03:21:09.075599+0000","last_clean":"2024-09-17T03:21:09.075599+0000","last_became_active":"2024-09-17T03:21:08.068338+0000","last_became_peered":"2024-09-17T03:21:08.068338+0000","last_unstale":"2024-09-17T03:21:09.075599+0000","last_undegraded":"2024-09-17T03:21:09.075599+0000","last_fullsized":"2024-09-17T03:21:09.075599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036872+0000","last_change":"2024-09-17T03:21:08.059892+0000","last_active":"2024-09-17T03:21:09.036872+0000","last_peered":"2024-09-17T03:21:09.036872+0000","last_clean":"2024-09-17T03:21:09.036872+0000","last_became_active":"2024-09-17T03:21:08.059568+0000","last_became_peered":"2024-09-17T03:21:08.059568+0000","last_unstale":"2024-09-17T03:21:09.036872+0000","last_undegraded":"2024-09-17T03:21:09.036872+0000","last_fullsized":"2024-09-17T03:21:09.036872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:03.752767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034157+0000","last_change":"2024-09-17T03:21:08.049305+0000","last_active":"2024-09-17T03:21:09.034157+0000","last_peered":"2024-09-17T03:21:09.034157+0000","last_clean":"2024-09-17T03:21:09.034157+0000","last_became_active":"2024-09-17T03:21:08.048911+0000","last_became_peered":"2024-09-17T03:21:08.048911+0000","last_unstale":"2024-09-17T03:21:09.034157+0000","last_undegraded":"2024-09-17T03:21:09.034157+0000","last_fullsized":"2024-09-17T03:21:09.034157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402687+0000","last_change":"2024-09-17T03:21:08.060288+0000","last_active":"2024-09-17T03:21:10.402687+0000","last_peered":"2024-09-17T03:21:10.402687+0000","last_clean":"2024-09-17T03:21:10.402687+0000","last_became_active":"2024-09-17T03:21:08.059954+0000","last_became_peered":"2024-09-17T03:21:08.059954+0000","last_unstale":"2024-09-17T03:21:10.402687+0000","last_undegraded":"2024-09-17T03:21:10.402687+0000","last_fullsized":"2024-09-17T03:21:10.402687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075416+0000","last_change":"2024-09-17T03:21:08.067449+0000","last_active":"2024-09-17T03:21:09.075416+0000","last_peered":"2024-09-17T03:21:09.075416+0000","last_clean":"2024-09-17T03:21:09.075416+0000","last_became_active":"2024-09-17T03:21:08.066649+0000","last_became_peered":"2024-09-17T03:21:08.066649+0000","last_unstale":"2024-09-17T03:21:09.075416+0000","last_undegraded":"2024-09-17T03:21:09.075416+0000","last_fullsized":"2024-09-17T03:21:09.075416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037818+0000","last_change":"2024-09-17T03:21:08.055200+0000","last_active":"2024-09-17T03:21:09.037818+0000","last_peered":"2024-09-17T03:21:09.037818+0000","last_clean":"2024-09-17T03:21:09.037818+0000","last_became_active":"2024-09-17T03:21:08.054451+0000","last_became_peered":"2024-09-17T03:21:08.054451+0000","last_unstale":"2024-09-17T03:21:09.037818+0000","last_undegraded":"2024-09-17T03:21:09.037818+0000","last_fullsized":"2024-09-17T03:21:09.037818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:41.720407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036639+0000","last_change":"2024-09-17T03:21:07.042857+0000","last_active":"2024-09-17T03:21:09.036639+0000","last_peered":"2024-09-17T03:21:09.036639+0000","last_clean":"2024-09-17T03:21:09.036639+0000","last_became_active":"2024-09-17T03:21:07.042659+0000","last_became_peered":"2024-09-17T03:21:07.042659+0000","last_unstale":"2024-09-17T03:21:09.036639+0000","last_undegraded":"2024-09-17T03:21:09.036639+0000","last_fullsized":"2024-09-17T03:21:09.036639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034320+0000","last_change":"2024-09-17T03:21:08.054380+0000","last_active":"2024-09-17T03:21:09.034320+0000","last_peered":"2024-09-17T03:21:09.034320+0000","last_clean":"2024-09-17T03:21:09.034320+0000","last_became_active":"2024-09-17T03:21:08.053967+0000","last_became_peered":"2024-09-17T03:21:08.053967+0000","last_unstale":"2024-09-17T03:21:09.034320+0000","last_undegraded":"2024-09-17T03:21:09.034320+0000","last_fullsized":"2024-09-17T03:21:09.034320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074976+0000","last_change":"2024-09-17T03:21:07.037433+0000","last_active":"2024-09-17T03:21:09.074976+0000","last_peered":"2024-09-17T03:21:09.074976+0000","last_clean":"2024-09-17T03:21:09.074976+0000","last_became_active":"2024-09-17T03:21:07.035663+0000","last_became_peered":"2024-09-17T03:21:07.035663+0000","last_unstale":"2024-09-17T03:21:09.074976+0000","last_undegraded":"2024-09-17T03:21:09.074976+0000","last_fullsized":"2024-09-17T03:21:09.074976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036177+0000","last_change":"2024-09-17T03:21:07.033856+0000","last_active":"2024-09-17T03:21:09.036177+0000","last_peered":"2024-09-17T03:21:09.036177+0000","last_clean":"2024-09-17T03:21:09.036177+0000","last_became_active":"2024-09-17T03:21:07.032761+0000","last_became_peered":"2024-09-17T03:21:07.032761+0000","last_unstale":"2024-09-17T03:21:09.036177+0000","last_undegraded":"2024-09-17T03:21:09.036177+0000","last_fullsized":"2024-09-17T03:21:09.036177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036275+0000","last_change":"2024-09-17T03:21:08.046160+0000","last_active":"2024-09-17T03:21:09.036275+0000","last_peered":"2024-09-17T03:21:09.036275+0000","last_clean":"2024-09-17T03:21:09.036275+0000","last_became_active":"2024-09-17T03:21:08.045937+0000","last_became_peered":"2024-09-17T03:21:08.045937+0000","last_unstale":"2024-09-17T03:21:09.036275+0000","last_undegraded":"2024-09-17T03:21:09.036275+0000","last_fullsized":"2024-09-17T03:21:09.036275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:13.549239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075195+0000","last_change":"2024-09-17T03:21:07.042752+0000","last_active":"2024-09-17T03:21:09.075195+0000","last_peered":"2024-09-17T03:21:09.075195+0000","last_clean":"2024-09-17T03:21:09.075195+0000","last_became_active":"2024-09-17T03:21:07.042359+0000","last_became_peered":"2024-09-17T03:21:07.042359+0000","last_unstale":"2024-09-17T03:21:09.075195+0000","last_undegraded":"2024-09-17T03:21:09.075195+0000","last_fullsized":"2024-09-17T03:21:09.075195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034735+0000","last_change":"2024-09-17T03:21:08.060578+0000","last_active":"2024-09-17T03:21:09.034735+0000","last_peered":"2024-09-17T03:21:09.034735+0000","last_clean":"2024-09-17T03:21:09.034735+0000","last_became_active":"2024-09-17T03:21:08.059037+0000","last_became_peered":"2024-09-17T03:21:08.059037+0000","last_unstale":"2024-09-17T03:21:09.034735+0000","last_undegraded":"2024-09-17T03:21:09.034735+0000","last_fullsized":"2024-09-17T03:21:09.034735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037963+0000","last_change":"2024-09-17T03:21:08.052908+0000","last_active":"2024-09-17T03:21:09.037963+0000","last_peered":"2024-09-17T03:21:09.037963+0000","last_clean":"2024-09-17T03:21:09.037963+0000","last_became_active":"2024-09-17T03:21:08.052578+0000","last_became_peered":"2024-09-17T03:21:08.052578+0000","last_unstale":"2024-09-17T03:21:09.037963+0000","last_undegraded":"2024-09-17T03:21:09.037963+0000","last_fullsized":"2024-09-17T03:21:09.037963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:06.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075516+0000","last_change":"2024-09-17T03:21:07.042836+0000","last_active":"2024-09-17T03:21:09.075516+0000","last_peered":"2024-09-17T03:21:09.075516+0000","last_clean":"2024-09-17T03:21:09.075516+0000","last_became_active":"2024-09-17T03:21:07.042380+0000","last_became_peered":"2024-09-17T03:21:07.042380+0000","last_unstale":"2024-09-17T03:21:09.075516+0000","last_undegraded":"2024-09-17T03:21:09.075516+0000","last_fullsized":"2024-09-17T03:21:09.075516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405970+0000","last_change":"2024-09-17T03:21:08.066084+0000","last_active":"2024-09-17T03:21:10.405970+0000","last_peered":"2024-09-17T03:21:10.405970+0000","last_clean":"2024-09-17T03:21:10.405970+0000","last_became_active":"2024-09-17T03:21:08.065807+0000","last_became_peered":"2024-09-17T03:21:08.065807+0000","last_unstale":"2024-09-17T03:21:10.405970+0000","last_undegraded":"2024-09-17T03:21:10.405970+0000","last_fullsized":"2024-09-17T03:21:10.405970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:53.782411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034281+0000","last_change":"2024-09-17T03:21:07.033053+0000","last_active":"2024-09-17T03:21:09.034281+0000","last_peered":"2024-09-17T03:21:09.034281+0000","last_clean":"2024-09-17T03:21:09.034281+0000","last_became_active":"2024-09-17T03:21:07.030140+0000","last_became_peered":"2024-09-17T03:21:07.030140+0000","last_unstale":"2024-09-17T03:21:09.034281+0000","last_undegraded":"2024-09-17T03:21:09.034281+0000","last_fullsized":"2024-09-17T03:21:09.034281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033897+0000","last_change":"2024-09-17T03:21:08.057090+0000","last_active":"2024-09-17T03:21:09.033897+0000","last_peered":"2024-09-17T03:21:09.033897+0000","last_clean":"2024-09-17T03:21:09.033897+0000","last_became_active":"2024-09-17T03:21:08.055419+0000","last_became_peered":"2024-09-17T03:21:08.055419+0000","last_unstale":"2024-09-17T03:21:09.033897+0000","last_undegraded":"2024-09-17T03:21:09.033897+0000","last_fullsized":"2024-09-17T03:21:09.033897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:54.579062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406013+0000","last_change":"2024-09-17T03:21:08.059703+0000","last_active":"2024-09-17T03:21:10.406013+0000","last_peered":"2024-09-17T03:21:10.406013+0000","last_clean":"2024-09-17T03:21:10.406013+0000","last_became_active":"2024-09-17T03:21:08.059411+0000","last_became_peered":"2024-09-17T03:21:08.059411+0000","last_unstale":"2024-09-17T03:21:10.406013+0000","last_undegraded":"2024-09-17T03:21:10.406013+0000","last_fullsized":"2024-09-17T03:21:10.406013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:00.131726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074938+0000","last_change":"2024-09-17T03:21:07.043190+0000","last_active":"2024-09-17T03:21:09.074938+0000","last_peered":"2024-09-17T03:21:09.074938+0000","last_clean":"2024-09-17T03:21:09.074938+0000","last_became_active":"2024-09-17T03:21:07.042868+0000","last_became_peered":"2024-09-17T03:21:07.042868+0000","last_unstale":"2024-09-17T03:21:09.074938+0000","last_undegraded":"2024-09-17T03:21:09.074938+0000","last_fullsized":"2024-09-17T03:21:09.074938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075453+0000","last_change":"2024-09-17T03:21:08.070393+0000","last_active":"2024-09-17T03:21:09.075453+0000","last_peered":"2024-09-17T03:21:09.075453+0000","last_clean":"2024-09-17T03:21:09.075453+0000","last_became_active":"2024-09-17T03:21:08.069695+0000","last_became_peered":"2024-09-17T03:21:08.069695+0000","last_unstale":"2024-09-17T03:21:09.075453+0000","last_undegraded":"2024-09-17T03:21:09.075453+0000","last_fullsized":"2024-09-17T03:21:09.075453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402466+0000","last_change":"2024-09-17T03:21:07.037447+0000","last_active":"2024-09-17T03:21:10.402466+0000","last_peered":"2024-09-17T03:21:10.402466+0000","last_clean":"2024-09-17T03:21:10.402466+0000","last_became_active":"2024-09-17T03:21:07.037133+0000","last_became_peered":"2024-09-17T03:21:07.037133+0000","last_unstale":"2024-09-17T03:21:10.402466+0000","last_undegraded":"2024-09-17T03:21:10.402466+0000","last_fullsized":"2024-09-17T03:21:10.402466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405541+0000","last_change":"2024-09-17T03:21:08.064992+0000","last_active":"2024-09-17T03:21:10.405541+0000","last_peered":"2024-09-17T03:21:10.405541+0000","last_clean":"2024-09-17T03:21:10.405541+0000","last_became_active":"2024-09-17T03:21:08.064705+0000","last_became_peered":"2024-09-17T03:21:08.064705+0000","last_unstale":"2024-09-17T03:21:10.405541+0000","last_undegraded":"2024-09-17T03:21:10.405541+0000","last_fullsized":"2024-09-17T03:21:10.405541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.038518+0000","last_change":"2024-09-17T03:21:07.031511+0000","last_active":"2024-09-17T03:21:09.038518+0000","last_peered":"2024-09-17T03:21:09.038518+0000","last_clean":"2024-09-17T03:21:09.038518+0000","last_became_active":"2024-09-17T03:21:07.031175+0000","last_became_peered":"2024-09-17T03:21:07.031175+0000","last_unstale":"2024-09-17T03:21:09.038518+0000","last_undegraded":"2024-09-17T03:21:09.038518+0000","last_fullsized":"2024-09-17T03:21:09.038518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:23:29.396492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406155+0000","last_change":"2024-09-17T03:21:08.067321+0000","last_active":"2024-09-17T03:21:10.406155+0000","last_peered":"2024-09-17T03:21:10.406155+0000","last_clean":"2024-09-17T03:21:10.406155+0000","last_became_active":"2024-09-17T03:21:08.066953+0000","last_became_peered":"2024-09-17T03:21:08.066953+0000","last_unstale":"2024-09-17T03:21:10.406155+0000","last_undegraded":"2024-09-17T03:21:10.406155+0000","last_fullsized":"2024-09-17T03:21:10.406155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:38.976170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402786+0000","last_change":"2024-09-17T03:21:07.042316+0000","last_active":"2024-09-17T03:21:10.402786+0000","last_peered":"2024-09-17T03:21:10.402786+0000","last_clean":"2024-09-17T03:21:10.402786+0000","last_became_active":"2024-09-17T03:21:07.039133+0000","last_became_peered":"2024-09-17T03:21:07.039133+0000","last_unstale":"2024-09-17T03:21:10.402786+0000","last_undegraded":"2024-09-17T03:21:10.402786+0000","last_fullsized":"2024-09-17T03:21:10.402786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406653+0000","last_change":"2024-09-17T03:21:08.057737+0000","last_active":"2024-09-17T03:21:10.406653+0000","last_peered":"2024-09-17T03:21:10.406653+0000","last_clean":"2024-09-17T03:21:10.406653+0000","last_became_active":"2024-09-17T03:21:08.057560+0000","last_became_peered":"2024-09-17T03:21:08.057560+0000","last_unstale":"2024-09-17T03:21:10.406653+0000","last_undegraded":"2024-09-17T03:21:10.406653+0000","last_fullsized":"2024-09-17T03:21:10.406653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:07.761806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402646+0000","last_change":"2024-09-17T03:21:07.042337+0000","last_active":"2024-09-17T03:21:10.402646+0000","last_peered":"2024-09-17T03:21:10.402646+0000","last_clean":"2024-09-17T03:21:10.402646+0000","last_became_active":"2024-09-17T03:21:07.040846+0000","last_became_peered":"2024-09-17T03:21:07.040846+0000","last_unstale":"2024-09-17T03:21:10.402646+0000","last_undegraded":"2024-09-17T03:21:10.402646+0000","last_fullsized":"2024-09-17T03:21:10.402646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406114+0000","last_change":"2024-09-17T03:21:08.074560+0000","last_active":"2024-09-17T03:21:10.406114+0000","last_peered":"2024-09-17T03:21:10.406114+0000","last_clean":"2024-09-17T03:21:10.406114+0000","last_became_active":"2024-09-17T03:21:08.074267+0000","last_became_peered":"2024-09-17T03:21:08.074267+0000","last_unstale":"2024-09-17T03:21:10.406114+0000","last_undegraded":"2024-09-17T03:21:10.406114+0000","last_fullsized":"2024-09-17T03:21:10.406114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:09.894552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404671+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:10.404671+0000","last_peered":"2024-09-17T03:21:10.404671+0000","last_clean":"2024-09-17T03:21:10.404671+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:10.404671+0000","last_undegraded":"2024-09-17T03:21:10.404671+0000","last_fullsized":"2024-09-17T03:21:10.404671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036442+0000","last_change":"2024-09-17T03:21:08.046702+0000","last_active":"2024-09-17T03:21:09.036442+0000","last_peered":"2024-09-17T03:21:09.036442+0000","last_clean":"2024-09-17T03:21:09.036442+0000","last_became_active":"2024-09-17T03:21:08.046321+0000","last_became_peered":"2024-09-17T03:21:08.046321+0000","last_unstale":"2024-09-17T03:21:09.036442+0000","last_undegraded":"2024-09-17T03:21:09.036442+0000","last_fullsized":"2024-09-17T03:21:09.036442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404543+0000","last_change":"2024-09-17T03:21:07.045789+0000","last_active":"2024-09-17T03:21:10.404543+0000","last_peered":"2024-09-17T03:21:10.404543+0000","last_clean":"2024-09-17T03:21:10.404543+0000","last_became_active":"2024-09-17T03:21:07.045443+0000","last_became_peered":"2024-09-17T03:21:07.045443+0000","last_unstale":"2024-09-17T03:21:10.404543+0000","last_undegraded":"2024-09-17T03:21:10.404543+0000","last_fullsized":"2024-09-17T03:21:10.404543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:00.822415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033526+0000","last_change":"2024-09-17T03:21:08.040522+0000","last_active":"2024-09-17T03:21:09.033526+0000","last_peered":"2024-09-17T03:21:09.033526+0000","last_clean":"2024-09-17T03:21:09.033526+0000","last_became_active":"2024-09-17T03:21:08.037656+0000","last_became_peered":"2024-09-17T03:21:08.037656+0000","last_unstale":"2024-09-17T03:21:09.033526+0000","last_undegraded":"2024-09-17T03:21:09.033526+0000","last_fullsized":"2024-09-17T03:21:09.033526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:58.143540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402609+0000","last_change":"2024-09-17T03:21:07.035825+0000","last_active":"2024-09-17T03:21:10.402609+0000","last_peered":"2024-09-17T03:21:10.402609+0000","last_clean":"2024-09-17T03:21:10.402609+0000","last_became_active":"2024-09-17T03:21:07.035573+0000","last_became_peered":"2024-09-17T03:21:07.035573+0000","last_unstale":"2024-09-17T03:21:10.402609+0000","last_undegraded":"2024-09-17T03:21:10.402609+0000","last_fullsized":"2024-09-17T03:21:10.402609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037938+0000","last_change":"2024-09-17T03:21:08.051692+0000","last_active":"2024-09-17T03:21:09.037938+0000","last_peered":"2024-09-17T03:21:09.037938+0000","last_clean":"2024-09-17T03:21:09.037938+0000","last_became_active":"2024-09-17T03:21:08.050563+0000","last_became_peered":"2024-09-17T03:21:08.050563+0000","last_unstale":"2024-09-17T03:21:09.037938+0000","last_undegraded":"2024-09-17T03:21:09.037938+0000","last_fullsized":"2024-09-17T03:21:09.037938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:07.035552+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:07.035214+0000","last_became_peered":"2024-09-17T03:21:07.035214+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.557724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037050+0000","last_change":"2024-09-17T03:21:08.059028+0000","last_active":"2024-09-17T03:21:09.037050+0000","last_peered":"2024-09-17T03:21:09.037050+0000","last_clean":"2024-09-17T03:21:09.037050+0000","last_became_active":"2024-09-17T03:21:08.058709+0000","last_became_peered":"2024-09-17T03:21:08.058709+0000","last_unstale":"2024-09-17T03:21:09.037050+0000","last_undegraded":"2024-09-17T03:21:09.037050+0000","last_fullsized":"2024-09-17T03:21:09.037050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:21.370502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075234+0000","last_change":"2024-09-17T03:21:07.033103+0000","last_active":"2024-09-17T03:21:09.075234+0000","last_peered":"2024-09-17T03:21:09.075234+0000","last_clean":"2024-09-17T03:21:09.075234+0000","last_became_active":"2024-09-17T03:21:07.032795+0000","last_became_peered":"2024-09-17T03:21:07.032795+0000","last_unstale":"2024-09-17T03:21:09.075234+0000","last_undegraded":"2024-09-17T03:21:09.075234+0000","last_fullsized":"2024-09-17T03:21:09.075234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033821+0000","last_change":"2024-09-17T03:21:08.065685+0000","last_active":"2024-09-17T03:21:09.033821+0000","last_peered":"2024-09-17T03:21:09.033821+0000","last_clean":"2024-09-17T03:21:09.033821+0000","last_became_active":"2024-09-17T03:21:08.065243+0000","last_became_peered":"2024-09-17T03:21:08.065243+0000","last_unstale":"2024-09-17T03:21:09.033821+0000","last_undegraded":"2024-09-17T03:21:09.033821+0000","last_fullsized":"2024-09-17T03:21:09.033821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:23.445102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404390+0000","last_change":"2024-09-17T03:21:07.034832+0000","last_active":"2024-09-17T03:21:10.404390+0000","last_peered":"2024-09-17T03:21:10.404390+0000","last_clean":"2024-09-17T03:21:10.404390+0000","last_became_active":"2024-09-17T03:21:07.034455+0000","last_became_peered":"2024-09-17T03:21:07.034455+0000","last_unstale":"2024-09-17T03:21:10.404390+0000","last_undegraded":"2024-09-17T03:21:10.404390+0000","last_fullsized":"2024-09-17T03:21:10.404390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037637+0000","last_change":"2024-09-17T03:21:08.046797+0000","last_active":"2024-09-17T03:21:09.037637+0000","last_peered":"2024-09-17T03:21:09.037637+0000","last_clean":"2024-09-17T03:21:09.037637+0000","last_became_active":"2024-09-17T03:21:08.046355+0000","last_became_peered":"2024-09-17T03:21:08.046355+0000","last_unstale":"2024-09-17T03:21:09.037637+0000","last_undegraded":"2024-09-17T03:21:09.037637+0000","last_fullsized":"2024-09-17T03:21:09.037637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:41.458145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036511+0000","last_change":"2024-09-17T03:21:08.060646+0000","last_active":"2024-09-17T03:21:09.036511+0000","last_peered":"2024-09-17T03:21:09.036511+0000","last_clean":"2024-09-17T03:21:09.036511+0000","last_became_active":"2024-09-17T03:21:08.060341+0000","last_became_peered":"2024-09-17T03:21:08.060341+0000","last_unstale":"2024-09-17T03:21:09.036511+0000","last_undegraded":"2024-09-17T03:21:09.036511+0000","last_fullsized":"2024-09-17T03:21:09.036511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:06.388228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036654+0000","last_change":"2024-09-17T03:21:08.045211+0000","last_active":"2024-09-17T03:21:09.036654+0000","last_peered":"2024-09-17T03:21:09.036654+0000","last_clean":"2024-09-17T03:21:09.036654+0000","last_became_active":"2024-09-17T03:21:08.043718+0000","last_became_peered":"2024-09-17T03:21:08.043718+0000","last_unstale":"2024-09-17T03:21:09.036654+0000","last_undegraded":"2024-09-17T03:21:09.036654+0000","last_fullsized":"2024-09-17T03:21:09.036654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:41.789200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406079+0000","last_change":"2024-09-17T03:21:07.038659+0000","last_active":"2024-09-17T03:21:10.406079+0000","last_peered":"2024-09-17T03:21:10.406079+0000","last_clean":"2024-09-17T03:21:10.406079+0000","last_became_active":"2024-09-17T03:21:07.038247+0000","last_became_peered":"2024-09-17T03:21:07.038247+0000","last_unstale":"2024-09-17T03:21:10.406079+0000","last_undegraded":"2024-09-17T03:21:10.406079+0000","last_fullsized":"2024-09-17T03:21:10.406079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075179+0000","last_change":"2024-09-17T03:21:08.065090+0000","last_active":"2024-09-17T03:21:09.075179+0000","last_peered":"2024-09-17T03:21:09.075179+0000","last_clean":"2024-09-17T03:21:09.075179+0000","last_became_active":"2024-09-17T03:21:08.064729+0000","last_became_peered":"2024-09-17T03:21:08.064729+0000","last_unstale":"2024-09-17T03:21:09.075179+0000","last_undegraded":"2024-09-17T03:21:09.075179+0000","last_fullsized":"2024-09-17T03:21:09.075179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405784+0000","last_change":"2024-09-17T03:21:07.049981+0000","last_active":"2024-09-17T03:21:10.405784+0000","last_peered":"2024-09-17T03:21:10.405784+0000","last_clean":"2024-09-17T03:21:10.405784+0000","last_became_active":"2024-09-17T03:21:07.049635+0000","last_became_peered":"2024-09-17T03:21:07.049635+0000","last_unstale":"2024-09-17T03:21:10.405784+0000","last_undegraded":"2024-09-17T03:21:10.405784+0000","last_fullsized":"2024-09-17T03:21:10.405784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:11.341893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406212+0000","last_change":"2024-09-17T03:21:07.038577+0000","last_active":"2024-09-17T03:21:10.406212+0000","last_peered":"2024-09-17T03:21:10.406212+0000","last_clean":"2024-09-17T03:21:10.406212+0000","last_became_active":"2024-09-17T03:21:07.038261+0000","last_became_peered":"2024-09-17T03:21:07.038261+0000","last_unstale":"2024-09-17T03:21:10.406212+0000","last_undegraded":"2024-09-17T03:21:10.406212+0000","last_fullsized":"2024-09-17T03:21:10.406212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:02.813101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034518+0000","last_change":"2024-09-17T03:21:08.061875+0000","last_active":"2024-09-17T03:21:09.034518+0000","last_peered":"2024-09-17T03:21:09.034518+0000","last_clean":"2024-09-17T03:21:09.034518+0000","last_became_active":"2024-09-17T03:21:08.057042+0000","last_became_peered":"2024-09-17T03:21:08.057042+0000","last_unstale":"2024-09-17T03:21:09.034518+0000","last_undegraded":"2024-09-17T03:21:09.034518+0000","last_fullsized":"2024-09-17T03:21:09.034518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:18.945802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036926+0000","last_change":"2024-09-17T03:21:07.037271+0000","last_active":"2024-09-17T03:21:09.036926+0000","last_peered":"2024-09-17T03:21:09.036926+0000","last_clean":"2024-09-17T03:21:09.036926+0000","last_became_active":"2024-09-17T03:21:07.036859+0000","last_became_peered":"2024-09-17T03:21:07.036859+0000","last_unstale":"2024-09-17T03:21:09.036926+0000","last_undegraded":"2024-09-17T03:21:09.036926+0000","last_fullsized":"2024-09-17T03:21:09.036926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037073+0000","last_change":"2024-09-17T03:21:08.068273+0000","last_active":"2024-09-17T03:21:09.037073+0000","last_peered":"2024-09-17T03:21:09.037073+0000","last_clean":"2024-09-17T03:21:09.037073+0000","last_became_active":"2024-09-17T03:21:08.066628+0000","last_became_peered":"2024-09-17T03:21:08.066628+0000","last_unstale":"2024-09-17T03:21:09.037073+0000","last_undegraded":"2024-09-17T03:21:09.037073+0000","last_fullsized":"2024-09-17T03:21:09.037073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034788+0000","last_change":"2024-09-17T03:21:07.032795+0000","last_active":"2024-09-17T03:21:09.034788+0000","last_peered":"2024-09-17T03:21:09.034788+0000","last_clean":"2024-09-17T03:21:09.034788+0000","last_became_active":"2024-09-17T03:21:07.032288+0000","last_became_peered":"2024-09-17T03:21:07.032288+0000","last_unstale":"2024-09-17T03:21:09.034788+0000","last_undegraded":"2024-09-17T03:21:09.034788+0000","last_fullsized":"2024-09-17T03:21:09.034788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:41.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:17.296 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:21:17.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T03:21:17.298 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T03:21:17.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 -- 172.21.15.46:0/4260944927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3900b54f0 msgr2=0x7fd39015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 --2- 172.21.15.46:0/4260944927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3900b54f0 0x7fd39015bb60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd388009bc0 tx=0x7fd38801cae0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 -- 172.21.15.46:0/4260944927 shutdown_connections 2024-09-17T03:21:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 --2- 172.21.15.46:0/4260944927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3900b54f0 0x7fd39015bb60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 -- 172.21.15.46:0/4260944927 >> 172.21.15.46:0/4260944927 conn(0x7fd3900b2040 msgr2=0x7fd3900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 -- 172.21.15.46:0/4260944927 shutdown_connections 2024-09-17T03:21:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.490+0000 7fd397e00640 1 -- 172.21.15.46:0/4260944927 wait complete. 2024-09-17T03:21:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 Processor -- start 2024-09-17T03:21:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 -- start start 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3900b5c70 0x7fd39013b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 0x7fd39013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901fc800 0x7fd390159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900b4540 con 0x7fd3901527b0 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900b43c0 con 0x7fd3901fc800 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd3900b46c0 con 0x7fd3900b5c70 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 0x7fd39013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 0x7fd39013e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38356/0 (socket says 172.21.15.46:38356) 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 -- 172.21.15.46:0/3852191371 learned_addr learned my addr 172.21.15.46:0/3852191371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd395400640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3900b5c70 0x7fd39013b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 -- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3900b5c70 msgr2=0x7fd39013b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3900b5c70 0x7fd39013b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 -- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901fc800 msgr2=0x7fd390159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901fc800 0x7fd390159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 -- 172.21.15.46:0/3852191371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39013b780 con 0x7fd3901527b0 2024-09-17T03:21:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd394a00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 0x7fd39013e4f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00c400 tx=0x7fd37c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0052d0 con 0x7fd3901527b0 2024-09-17T03:21:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd37c007020 con 0x7fd3901527b0 2024-09-17T03:21:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c007450 con 0x7fd3901527b0 2024-09-17T03:21:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39013ea30 con 0x7fd3901527b0 2024-09-17T03:21:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.494+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3901565d0 con 0x7fd3901527b0 2024-09-17T03:21:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.498+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39015bb60 con 0x7fd3901527b0 2024-09-17T03:21:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.498+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd37c0067e0 con 0x7fd3901527b0 2024-09-17T03:21:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.498+0000 7fd38d600640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd370073200 0x7fd370075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.498+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd37c0938a0 con 0x7fd3901527b0 2024-09-17T03:21:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.498+0000 7fd395400640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd370073200 0x7fd370075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.502+0000 7fd395400640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd370073200 0x7fd370075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd384001230 tx=0x7fd384009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.502+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd39013ea30 con 0x7fd3901527b0 2024-09-17T03:21:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.678+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd39014f1e0 con 0x7fd3901527b0 2024-09-17T03:21:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd38d600640 1 -- 172.21.15.46:0/3852191371 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fd37c05e820 con 0x7fd3901527b0 2024-09-17T03:21:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd370073200 msgr2=0x7fd370075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd370073200 0x7fd370075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd384001230 tx=0x7fd384009040 comp rx=0 tx=0).stop 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 msgr2=0x7fd39013e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 0x7fd39013e4f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00c400 tx=0x7fd37c00c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 shutdown_connections 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd370073200 0x7fd370075710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901fc800 0x7fd390159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3901527b0 0x7fd39013e4f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 --2- 172.21.15.46:0/3852191371 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3900b5c70 0x7fd39013b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 >> 172.21.15.46:0/3852191371 conn(0x7fd3900b2040 msgr2=0x7fd3901559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 shutdown_connections 2024-09-17T03:21:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.682+0000 7fd397e00640 1 -- 172.21.15.46:0/3852191371 wait complete. 2024-09-17T03:21:17.706 INFO:teuthology.orchestra.run.smithi046.stdout:[{"version":31,"timestamp":"2024-09-17T03:20:55.729911+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"4247.059402"}]}] 2024-09-17T03:21:17.707 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T03:21:17.707 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T03:21:17.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T03:21:17.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T03:21:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca8e00640 1 -- 172.21.15.46:0/3017991446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 msgr2=0x7f3c88003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/3017991446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3c88003e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400bc00 tx=0x7f3c9401ce70 comp rx=0 tx=0).stop 2024-09-17T03:21:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca2800640 1 -- 172.21.15.46:0/3017991446 reap_dead start 2024-09-17T03:21:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca8e00640 1 -- 172.21.15.46:0/3017991446 shutdown_connections 2024-09-17T03:21:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca8e00640 1 -- 172.21.15.46:0/3017991446 >> 172.21.15.46:0/3017991446 conn(0x7f3ca40b2040 msgr2=0x7f3ca40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca8e00640 1 -- 172.21.15.46:0/3017991446 shutdown_connections 2024-09-17T03:21:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.862+0000 7f3ca8e00640 1 -- 172.21.15.46:0/3017991446 wait complete. 2024-09-17T03:21:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 Processor -- start 2024-09-17T03:21:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 -- start start 2024-09-17T03:21:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3ca40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca415df80 0x7f3ca40af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ca40af820 0x7f3ca40ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4166900 con 0x7f3c88003a10 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4166780 con 0x7f3ca415df80 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca8e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca4166a80 con 0x7f3ca40af820 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3ca40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3ca40cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38366/0 (socket says 172.21.15.46:38366) 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 -- 172.21.15.46:0/1818916658 learned_addr learned my addr 172.21.15.46:0/1818916658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3c9be00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca415df80 0x7f3ca40af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca2800640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ca40af820 0x7f3ca40ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 -- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ca40af820 msgr2=0x7f3ca40ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ca40af820 0x7f3ca40ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 -- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca415df80 msgr2=0x7f3ca40af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca415df80 0x7f3ca40af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca1e00640 1 -- 172.21.15.46:0/1818916658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca40cdb50 con 0x7f3c88003a10 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3c9be00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca415df80 0x7f3ca40af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.866+0000 7f3ca2800640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ca40af820 0x7f3ca40ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3ca1e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3ca40cf550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400bb20 tx=0x7f3c940045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c940bc070 con 0x7f3c88003a10 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c94005030 con 0x7f3c88003a10 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca40cd590 con 0x7f3c88003a10 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c940ab040 con 0x7f3c88003a10 2024-09-17T03:21:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca40ce060 con 0x7f3c88003a10 2024-09-17T03:21:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca415bb60 con 0x7f3c88003a10 2024-09-17T03:21:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3c940bf070 con 0x7f3c88003a10 2024-09-17T03:21:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3c9a000640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c80073200 0x7f3c80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.870+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3c9412dfd0 con 0x7f3c88003a10 2024-09-17T03:21:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.874+0000 7f3c9be00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c80073200 0x7f3c80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.874+0000 7f3c9be00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c80073200 0x7f3c80075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4158b70 tx=0x7f3c8c0011a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:17.874+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3c94016bb0 con 0x7f3c88003a10 2024-09-17T03:21:18.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.098+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3ca415ab90 con 0x7f3c88003a10 2024-09-17T03:21:18.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3c9a000640 1 -- 172.21.15.46:0/1818916658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186454 (secure 0 0 0) 0x7f3c940f8f50 con 0x7f3c88003a10 2024-09-17T03:21:18.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c80073200 msgr2=0x7f3c80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c80073200 0x7f3c80075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4158b70 tx=0x7f3c8c0011a0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 msgr2=0x7f3ca40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3ca40cf550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400bb20 tx=0x7f3c940045d0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 shutdown_connections 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c80073200 0x7f3c80075710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ca40af820 0x7f3ca40ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca415df80 0x7f3ca40af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 --2- 172.21.15.46:0/1818916658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c88003a10 0x7f3ca40cf550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 >> 172.21.15.46:0/1818916658 conn(0x7f3ca40b2040 msgr2=0x7f3ca40b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:18.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 shutdown_connections 2024-09-17T03:21:18.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.102+0000 7f3ca8e00640 1 -- 172.21.15.46:0/1818916658 wait complete. 2024-09-17T03:21:18.128 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"flags":0,"active_gid":4168,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":537437096},{"type":"v1","addr":"172.21.15.46:6801","nonce":537437096}]},"active_addr":"172.21.15.46:6801/537437096","active_change":"2024-09-17T03:20:47.183015+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4195,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3452874668}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2998364696}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":4008587864}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2476410898}]}]} 2024-09-17T03:21:18.133 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T03:21:18.133 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T03:21:18.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 -- 172.21.15.46:0/2213398949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f8159130 msgr2=0x7f05f815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 --2- 172.21.15.46:0/2213398949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f8159130 0x7f05f815bb90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009840 tx=0x7f05e801c700 comp rx=0 tx=0).stop 2024-09-17T03:21:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 -- 172.21.15.46:0/2213398949 shutdown_connections 2024-09-17T03:21:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 --2- 172.21.15.46:0/2213398949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f8159130 0x7f05f815bb90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 -- 172.21.15.46:0/2213398949 >> 172.21.15.46:0/2213398949 conn(0x7f05f80b35f0 msgr2=0x7f05f80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 -- 172.21.15.46:0/2213398949 shutdown_connections 2024-09-17T03:21:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 -- 172.21.15.46:0/2213398949 wait complete. 2024-09-17T03:21:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 Processor -- start 2024-09-17T03:21:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.290+0000 7f05fd800640 1 -- start start 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 0x7f05f81527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f80703a0 0x7f05f815e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05f8159130 0x7f05f815ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8169cb0 con 0x7f05f806cdf0 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8169b30 con 0x7f05f80703a0 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f05f8169e30 con 0x7f05f8159130 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 0x7f05f81527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 0x7f05f81527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38390/0 (socket says 172.21.15.46:38390) 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 -- 172.21.15.46:0/3208517254 learned_addr learned my addr 172.21.15.46:0/3208517254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6000640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f80703a0 0x7f05f815e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f7400640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05f8159130 0x7f05f815ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 -- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05f8159130 msgr2=0x7f05f815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05f8159130 0x7f05f815ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 -- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f80703a0 msgr2=0x7f05f815e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f80703a0 0x7f05f815e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 -- 172.21.15.46:0/3208517254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f8152ce0 con 0x7f05f806cdf0 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6000640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f80703a0 0x7f05f815e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f7400640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05f8159130 0x7f05f815ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05f6a00640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 0x7f05f81527a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f05e80980e0 tx=0x7f05e80a8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e80be070 con 0x7f05f806cdf0 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e80a8a70 con 0x7f05f806cdf0 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f81fc470 con 0x7f05f806cdf0 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8005ca0 con 0x7f05f806cdf0 2024-09-17T03:21:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f81fca30 con 0x7f05f806cdf0 2024-09-17T03:21:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f815bb90 con 0x7f05f806cdf0 2024-09-17T03:21:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.294+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f05e80c1070 con 0x7f05f806cdf0 2024-09-17T03:21:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.298+0000 7f05eea00640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05d8073230 0x7f05d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.298+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05e812e300 con 0x7f05f806cdf0 2024-09-17T03:21:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.298+0000 7f05f6000640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05d8073230 0x7f05d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.298+0000 7f05f6000640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05d8073230 0x7f05d8075740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05f81fd3d0 tx=0x7f05e4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:18.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.302+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f05e80ac030 con 0x7f05f806cdf0 2024-09-17T03:21:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.478+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05f80b2880 con 0x7f05f806cdf0 2024-09-17T03:21:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.478+0000 7f05eea00640 1 -- 172.21.15.46:0/3208517254 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f05e8016410 con 0x7f05f806cdf0 2024-09-17T03:21:18.482 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:18.482 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.478+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05d8073230 msgr2=0x7f05d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.478+0000 7f05fd800640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05d8073230 0x7f05d8075740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05f81fd3d0 tx=0x7f05e4009040 comp rx=0 tx=0).stop 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.478+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 msgr2=0x7f05f81527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.478+0000 7f05fd800640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 0x7f05f81527a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f05e80980e0 tx=0x7f05e80a8590 comp rx=0 tx=0).stop 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 shutdown_connections 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05d8073230 0x7f05d8075740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05f8159130 0x7f05f815ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05f80703a0 0x7f05f815e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 --2- 172.21.15.46:0/3208517254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05f806cdf0 0x7f05f81527a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 >> 172.21.15.46:0/3208517254 conn(0x7f05f80b35f0 msgr2=0x7f05f80b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 shutdown_connections 2024-09-17T03:21:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.482+0000 7f05fd800640 1 -- 172.21.15.46:0/3208517254 wait complete. 2024-09-17T03:21:18.504 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T03:21:18.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:18.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 -- 172.21.15.46:0/449631314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe140039f0 msgr2=0x7efe14003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 --2- 172.21.15.46:0/449631314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe140039f0 0x7efe14003e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c260 tx=0x7efe2001cf30 comp rx=0 tx=0).stop 2024-09-17T03:21:18.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe2e800640 1 -- 172.21.15.46:0/449631314 reap_dead start 2024-09-17T03:21:18.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 -- 172.21.15.46:0/449631314 shutdown_connections 2024-09-17T03:21:18.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 -- 172.21.15.46:0/449631314 >> 172.21.15.46:0/449631314 conn(0x7efe300b2040 msgr2=0x7efe300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:18.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 -- 172.21.15.46:0/449631314 shutdown_connections 2024-09-17T03:21:18.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 -- 172.21.15.46:0/449631314 wait complete. 2024-09-17T03:21:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 Processor -- start 2024-09-17T03:21:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.662+0000 7efe34e00640 1 -- start start 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe140039f0 0x7efe301fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe301fc510 0x7efe30151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 0x7efe30153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe30167b70 con 0x7efe30151b90 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe301679f0 con 0x7efe301fc510 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7efe30167cf0 con 0x7efe140039f0 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 0x7efe30153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 0x7efe30153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38418/0 (socket says 172.21.15.46:38418) 2024-09-17T03:21:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe140039f0 0x7efe301fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 -- 172.21.15.46:0/275609852 learned_addr learned my addr 172.21.15.46:0/275609852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe27e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe301fc510 0x7efe30151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 -- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe140039f0 msgr2=0x7efe301fbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe140039f0 0x7efe301fbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 -- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe301fc510 msgr2=0x7efe30151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe301fc510 0x7efe30151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 -- 172.21.15.46:0/275609852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301fc920 con 0x7efe30151b90 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe2e800640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 0x7efe30153cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe14004790 tx=0x7efe28006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28011900 con 0x7efe30151b90 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe28011aa0 con 0x7efe30151b90 2024-09-17T03:21:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2800f430 con 0x7efe30151b90 2024-09-17T03:21:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301419e0 con 0x7efe30151b90 2024-09-17T03:21:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301534c0 con 0x7efe30151b90 2024-09-17T03:21:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.666+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300b51f0 con 0x7efe30151b90 2024-09-17T03:21:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.670+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efe2800f5d0 con 0x7efe30151b90 2024-09-17T03:21:18.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.670+0000 7efe26000640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe0c073200 0x7efe0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:18.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.670+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efe280949b0 con 0x7efe30151b90 2024-09-17T03:21:18.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.670+0000 7efe2de00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe0c073200 0x7efe0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.670+0000 7efe2de00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe0c073200 0x7efe0c075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe200ab000 tx=0x7efe20019d70 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.674+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efe301fc920 con 0x7efe30151b90 2024-09-17T03:21:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.846+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe3015b310 con 0x7efe30151b90 2024-09-17T03:21:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe26000640 1 -- 172.21.15.46:0/275609852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7efe2805f9c0 con 0x7efe30151b90 2024-09-17T03:21:18.853 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:18.853 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe0c073200 msgr2=0x7efe0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe0c073200 0x7efe0c075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe200ab000 tx=0x7efe20019d70 comp rx=0 tx=0).stop 2024-09-17T03:21:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 msgr2=0x7efe30153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 0x7efe30153cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe14004790 tx=0x7efe28006430 comp rx=0 tx=0).stop 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 shutdown_connections 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe0c073200 0x7efe0c075710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe30151b90 0x7efe30153cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe301fc510 0x7efe30151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 --2- 172.21.15.46:0/275609852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efe140039f0 0x7efe301fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 >> 172.21.15.46:0/275609852 conn(0x7efe300b2040 msgr2=0x7efe300b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 shutdown_connections 2024-09-17T03:21:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:18.850+0000 7efe34e00640 1 -- 172.21.15.46:0/275609852 wait complete. 2024-09-17T03:21:18.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T03:21:18.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T03:21:18.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T03:21:18.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T03:21:18.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T03:21:18.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T03:21:18.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T03:21:18.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T03:21:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 -- 172.21.15.46:0/3629973153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff758036ac0 msgr2=0x7ff75800afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 --2- 172.21.15.46:0/3629973153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff758036ac0 0x7ff75800afd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009900 tx=0x7ff74c01c6d0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 -- 172.21.15.46:0/3629973153 shutdown_connections 2024-09-17T03:21:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 --2- 172.21.15.46:0/3629973153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff758036ac0 0x7ff75800afd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 -- 172.21.15.46:0/1005164162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0df240 msgr2=0x7f3c6c00afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 --2- 172.21.15.46:0/1005164162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0df240 0x7f3c6c00afd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098f0 tx=0x7f3c6801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 -- 172.21.15.46:0/3629973153 >> 172.21.15.46:0/3629973153 conn(0x7ff7580513f0 msgr2=0x7ff758051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 -- 172.21.15.46:0/1005164162 shutdown_connections 2024-09-17T03:21:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 --2- 172.21.15.46:0/1005164162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0df240 0x7f3c6c00afd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 -- 172.21.15.46:0/1005164162 >> 172.21.15.46:0/1005164162 conn(0x7f3c6c0513f0 msgr2=0x7f3c6c051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 -- 172.21.15.46:0/1005164162 shutdown_connections 2024-09-17T03:21:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7f3c7a800640 1 -- 172.21.15.46:0/1005164162 wait complete. 2024-09-17T03:21:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 -- 172.21.15.46:0/1770132190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040538c0 msgr2=0x7fd404056dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 --2- 172.21.15.46:0/1770132190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040538c0 0x7fd404056dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd40c07b050 tx=0x7fd40c098940 comp rx=0 tx=0).stop 2024-09-17T03:21:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 -- 172.21.15.46:0/3629973153 shutdown_connections 2024-09-17T03:21:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 -- 172.21.15.46:0/1770132190 shutdown_connections 2024-09-17T03:21:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 --2- 172.21.15.46:0/1770132190 >> v2:172.21.15.46:3300/0 conn(0x7fd4040e6b10 0x7fd4040e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 --2- 172.21.15.46:0/1770132190 >> v2:172.21.15.99:3300/0 conn(0x7fd4040e34a0 0x7fd4040e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 --2- 172.21.15.46:0/1770132190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040538c0 0x7fd404056dd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7fd412200640 1 -- 172.21.15.46:0/1770132190 >> 172.21.15.46:0/1770132190 conn(0x7fd404058b30 msgr2=0x7fd404058f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.158+0000 7ff764a00640 1 -- 172.21.15.46:0/3629973153 wait complete. 2024-09-17T03:21:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.162+0000 7f3c7a800640 1 Processor -- start 2024-09-17T03:21:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.162+0000 7f3c7a800640 1 -- start start 2024-09-17T03:21:19.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.162+0000 7fd412200640 1 -- 172.21.15.46:0/1770132190 shutdown_connections 2024-09-17T03:21:19.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.162+0000 7fd412200640 1 -- 172.21.15.46:0/1770132190 wait complete. 2024-09-17T03:21:19.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c7a800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c6c0df240 0x7f3c6c0e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c7a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0e2ba0 0x7f3c6c0ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7fd412200640 1 Processor -- start 2024-09-17T03:21:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c7a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c6c17e810 0x7f3c6c17ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c7a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0e6a60 con 0x7f3c6c0e2ba0 2024-09-17T03:21:19.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7fd412200640 1 -- start start 2024-09-17T03:21:19.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c7a800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0e68e0 con 0x7f3c6c17e810 2024-09-17T03:21:19.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c7a800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c0e6be0 con 0x7f3c6c0df240 2024-09-17T03:21:19.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.162+0000 7ff764a00640 1 Processor -- start 2024-09-17T03:21:19.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7ff764a00640 1 -- start start 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c6c17e810 0x7f3c6c17ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c6c17e810 0x7f3c6c17ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49646/0 (socket says 172.21.15.46:49646) 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 -- 172.21.15.46:0/2095299649 learned_addr learned my addr 172.21.15.46:0/2095299649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c78a00640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0e2ba0 0x7f3c6c0ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79400640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c6c0df240 0x7f3c6c0e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 -- 172.21.15.46:0/2095299649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c6c0df240 msgr2=0x7f3c6c0e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c6c0df240 0x7f3c6c0e1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 -- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0e2ba0 msgr2=0x7f3c6c0ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c6c0e2ba0 0x7f3c6c0ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 -- 172.21.15.46:0/2095299649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c17f160 con 0x7f3c6c17e810 2024-09-17T03:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c79e00640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c6c17e810 0x7f3c6c17ec20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3c74075a10 tx=0x7f3c7406aa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74090070 con 0x7f3c6c17e810 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c740754b0 con 0x7f3c6c17e810 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.166+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74063b10 con 0x7f3c6c17e810 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c0de480 con 0x7f3c6c17e810 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd412200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4040538c0 0x7fd4040ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd412200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4040e34a0 0x7fd4040dd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd412200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040e6b10 0x7fd4040de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd412200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd404006e00 con 0x7fd4040e34a0 2024-09-17T03:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd412200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd404006c80 con 0x7fd4040538c0 2024-09-17T03:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd412200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd404006f80 con 0x7fd4040e6b10 2024-09-17T03:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 -- 172.21.15.46:0/2946262678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440003ab0 msgr2=0x7fb440003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 --2- 172.21.15.46:0/2946262678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440003ab0 0x7fb440003f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb448006f90 tx=0x7fb44801f0b0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c0e1110 con 0x7f3c6c17e810 2024-09-17T03:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040e6b10 0x7fd4040de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040e6b10 0x7fd4040de5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56666/0 (socket says 172.21.15.46:56666) 2024-09-17T03:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 -- 172.21.15.46:0/122431647 learned_addr learned my addr 172.21.15.46:0/122431647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd410e00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4040538c0 0x7fd4040ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd40be00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4040e34a0 0x7fd4040dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff758005d70 0x7ff7580e2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff7580e2ba0 0x7ff758053210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff75804b2b0 0x7ff758053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7580edaa0 con 0x7ff758005d70 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7580ed920 con 0x7ff75804b2b0 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff7580edc20 con 0x7ff7580e2ba0 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff758005d70 0x7ff7580e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff758005d70 0x7ff7580e2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38506/0 (socket says 172.21.15.46:38506) 2024-09-17T03:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75de00640 1 -- 172.21.15.46:0/3312812193 learned_addr learned my addr 172.21.15.46:0/3312812193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75e800640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff75804b2b0 0x7ff758053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 -- 172.21.15.46:0/122431647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4040538c0 msgr2=0x7fd4040ebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4040538c0 0x7fd4040ebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 -- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4040e34a0 msgr2=0x7fd4040dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4040e34a0 0x7fd4040dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 -- 172.21.15.46:0/122431647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd404180b50 con 0x7fd4040e6b10 2024-09-17T03:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff7580e2ba0 0x7ff758053210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 -- 172.21.15.46:0/2946262678 shutdown_connections 2024-09-17T03:21:19.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 --2- 172.21.15.46:0/2946262678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440003ab0 0x7fb440003f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 -- 172.21.15.46:0/2946262678 >> 172.21.15.46:0/2946262678 conn(0x7fb45c06e2d0 msgr2=0x7fb45c06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 -- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff75804b2b0 msgr2=0x7ff758053750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff75804b2b0 0x7ff758053750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 -- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff758005d70 msgr2=0x7ff7580e2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff758005d70 0x7ff7580e2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 -- 172.21.15.46:0/3312812193 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7580de9e0 con 0x7ff7580e2ba0 2024-09-17T03:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75e800640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff75804b2b0 0x7ff758053750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 -- 172.21.15.46:0/2946262678 shutdown_connections 2024-09-17T03:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c6c00afd0 con 0x7f3c6c17e810 2024-09-17T03:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fd411800640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040e6b10 0x7fd4040de5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800d610 tx=0x7fd3f800dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff75d400640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff7580e2ba0 0x7ff758053210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff75801d910 tx=0x7ff74c0a55b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0b2040 con 0x7ff7580e2ba0 2024-09-17T03:21:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7f9b4e000640 1 -- 172.21.15.46:0/2551539512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b480cd970 msgr2=0x7f9b480b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7f9b4e000640 1 --2- 172.21.15.46:0/2551539512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b480cd970 0x7f9b480b3340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b480ccc30 tx=0x7f9b3401d910 comp rx=0 tx=0).stop 2024-09-17T03:21:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8017070 con 0x7fd4040e6b10 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7580de310 con 0x7ff7580e2ba0 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff74c0043b0 con 0x7ff7580e2ba0 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0b2040 con 0x7ff7580e2ba0 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f9b4e000640 1 -- 172.21.15.46:0/2551539512 shutdown_connections 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f9b4e000640 1 --2- 172.21.15.46:0/2551539512 >> v2:172.21.15.46:3300/0 conn(0x7f9b480cdd80 0x7f9b480b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f9b4e000640 1 --2- 172.21.15.46:0/2551539512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b480cd970 0x7f9b480b3340 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f9b4e000640 1 -- 172.21.15.46:0/2551539512 >> 172.21.15.46:0/2551539512 conn(0x7f9b480b2040 msgr2=0x7f9b480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f9b4e000640 1 -- 172.21.15.46:0/2551539512 shutdown_connections 2024-09-17T03:21:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f9b4e000640 1 -- 172.21.15.46:0/2551539512 wait complete. 2024-09-17T03:21:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.170+0000 7fb463600640 1 -- 172.21.15.46:0/2946262678 wait complete. 2024-09-17T03:21:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3f8005310 con 0x7fd4040e6b10 2024-09-17T03:21:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80047b0 con 0x7fd4040e6b10 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 -- 172.21.15.46:0/3212745790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d29e0 msgr2=0x7f06ac0d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3212745790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d29e0 0x7f06ac0d37c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0d09e0 tx=0x7f06a401c790 comp rx=0 tx=0).stop 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3c74063cb0 con 0x7f3c6c17e810 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c71600640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c54073300 0x7f3c54075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3c7410ecf0 con 0x7f3c6c17e810 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c71600640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7f3c5407b4f0 0x7f3c5407d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 --> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c5407e010 con 0x7f3c5407b4f0 2024-09-17T03:21:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd404180e30 con 0x7fd4040e6b10 2024-09-17T03:21:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff758048da0 con 0x7ff7580e2ba0 2024-09-17T03:21:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 -- 172.21.15.46:0/3212745790 shutdown_connections 2024-09-17T03:21:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3212745790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d29e0 0x7f06ac0d37c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3212745790 >> v2:172.21.15.46:3300/0 conn(0x7f06ac0d16b0 0x7f06ac0d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 -- 172.21.15.46:0/3212745790 >> 172.21.15.46:0/3212745790 conn(0x7f06ac073e00 msgr2=0x7f06ac06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4041813f0 con 0x7fd4040e6b10 2024-09-17T03:21:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c79400640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c54073300 0x7f3c54075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c740d5e70 con 0x7f3c6c17e810 2024-09-17T03:21:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff75800afd0 con 0x7ff7580e2ba0 2024-09-17T03:21:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fb463600640 1 Processor -- start 2024-09-17T03:21:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f3c78a00640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7f3c5407b4f0 0x7f3c5407d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7fb463600640 1 -- start start 2024-09-17T03:21:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.174+0000 7f06b1e00640 1 -- 172.21.15.46:0/3212745790 shutdown_connections 2024-09-17T03:21:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f06b1e00640 1 -- 172.21.15.46:0/3212745790 wait complete. 2024-09-17T03:21:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f3c79400640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c54073300 0x7f3c54075810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c680af000 tx=0x7f3c6801b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff74c0bf060 con 0x7ff7580e2ba0 2024-09-17T03:21:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f3c78a00640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7f3c5407b4f0 0x7f3c5407d940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 Processor -- start 2024-09-17T03:21:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4040dd6d0 con 0x7fd4040e6b10 2024-09-17T03:21:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 -- start start 2024-09-17T03:21:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 -- 172.21.15.46:0/1383052866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280b3340 msgr2=0x7f46280b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 --2- 172.21.15.46:0/1383052866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280b3340 0x7f46280b3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4620009bc0 tx=0x7f462001cba0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff756000640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff740073400 0x7ff740075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff74c12d680 con 0x7ff7580e2ba0 2024-09-17T03:21:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== osd.0 v2:172.21.15.46:6810/1962470687 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c5407e010 con 0x7f3c5407b4f0 2024-09-17T03:21:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 -- 172.21.15.46:0/1383052866 shutdown_connections 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 --2- 172.21.15.46:0/1383052866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280b3340 0x7f46280b3750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 -- 172.21.15.46:0/1383052866 >> 172.21.15.46:0/1383052866 conn(0x7f46280b2040 msgr2=0x7f46280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff756000640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7ff74007b650 0x7ff74007daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 --> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff74c0b3e50 con 0x7ff74007b650 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff74c134050 con 0x7ff7580e2ba0 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff75de00640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff740073400 0x7ff740075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7ff75e800640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7ff74007b650 0x7ff74007daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b480cd970 0x7f9b48157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b480cdd80 0x7f9b48200cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b4815aa50 0x7f9b481532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4816a250 con 0x7f9b480cd970 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4816a0d0 con 0x7f9b4815aa50 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b4e000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4816a3d0 con 0x7f9b480cdd80 2024-09-17T03:21:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7fb463600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb440003ab0 0x7fb45c150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 -- 172.21.15.46:0/1383052866 shutdown_connections 2024-09-17T03:21:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd3f8004950 con 0x7fd4040e6b10 2024-09-17T03:21:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f9b47e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b4815aa50 0x7f9b481532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7fb460c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb440003ab0 0x7fb45c150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b4815aa50 0x7f9b481532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49676/0 (socket says 172.21.15.46:49676) 2024-09-17T03:21:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 -- 172.21.15.46:0/1651158505 learned_addr learned my addr 172.21.15.46:0/1651158505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb463600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb45c150f30 0x7fb45c1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb463600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb45c1549f0 0x7fb45c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.178+0000 7f4630600640 1 -- 172.21.15.46:0/1383052866 wait complete. 2024-09-17T03:21:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b46a00640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b480cdd80 0x7f9b48200cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb463600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c15b220 con 0x7fb440003ab0 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb463600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c15b0a0 con 0x7fb45c1549f0 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb463600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c15b3a0 con 0x7fb45c150f30 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb440003ab0 0x7fb45c150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38514/0 (socket says 172.21.15.46:38514) 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 -- 172.21.15.46:0/1701138147 learned_addr learned my addr 172.21.15.46:0/1701138147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 -- 172.21.15.46:0/1651158505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b480cdd80 msgr2=0x7f9b48200cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b480cdd80 0x7f9b48200cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 -- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b480cd970 msgr2=0x7f9b48157850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:21:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b480cd970 0x7f9b48157850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 -- 172.21.15.46:0/1651158505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b481551f0 con 0x7f9b4815aa50 2024-09-17T03:21:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb461600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb45c1549f0 0x7fb45c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7ff75e800640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7ff74007b650 0x7ff74007daa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f9b47e00640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b4815aa50 0x7f9b481532a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000cd10 tx=0x7f9b4000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f06b1e00640 1 Processor -- start 2024-09-17T03:21:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb45be00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb45c150f30 0x7fb45c1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd408c00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3ec073200 0x7fd3ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd3f8094110 con 0x7fd4040e6b10 2024-09-17T03:21:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7f06b1e00640 1 -- start start 2024-09-17T03:21:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd410e00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3ec073200 0x7fd3ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd408c00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd3ec07b450 0x7fd3ec07d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 --> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3f8009e50 con 0x7fd3ec07b450 2024-09-17T03:21:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd3f809b050 con 0x7fd4040e6b10 2024-09-17T03:21:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd40be00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd3ec07b450 0x7fd3ec07d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 -- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb45c150f30 msgr2=0x7fb45c1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb45c150f30 0x7fb45c1544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 -- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb45c1549f0 msgr2=0x7fb45c1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb45c1549f0 0x7fb45c1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 -- 172.21.15.46:0/1701138147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c154e00 con 0x7fb440003ab0 2024-09-17T03:21:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb461600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb45c1549f0 0x7fb45c1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd410e00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3ec073200 0x7fd3ec075710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4040e7610 tx=0x7fd40c095780 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== osd.2 v2:172.21.15.46:6826/1373384937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff74c0b3e50 con 0x7ff74007b650 2024-09-17T03:21:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fb460c00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb440003ab0 0x7fb45c150650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb44801f950 tx=0x7fb4480afa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.182+0000 7fd40be00640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd3ec07b450 0x7fd3ec07d8a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448006bc0 con 0x7fb440003ab0 2024-09-17T03:21:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb448006440 con 0x7fb440003ab0 2024-09-17T03:21:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448003600 con 0x7fb440003ab0 2024-09-17T03:21:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c157a50 con 0x7fb440003ab0 2024-09-17T03:21:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06b1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d16b0 0x7f06ac14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000f450 con 0x7f9b4815aa50 2024-09-17T03:21:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b4000f700 con 0x7f9b4815aa50 2024-09-17T03:21:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000fb50 con 0x7f9b4815aa50 2024-09-17T03:21:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7ff75de00640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff740073400 0x7ff740075910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7500018b0 tx=0x7ff750009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c158010 con 0x7fb440003ab0 2024-09-17T03:21:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06ac0d25d0 0x7f06ac0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f3d19600640 1 -- 172.21.15.46:0/4293236730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 msgr2=0x7f3d140cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f3d19600640 1 --2- 172.21.15.46:0/4293236730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 0x7f3d140cd260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009c80 tx=0x7f3d0c01cda0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f4630600640 1 Processor -- start 2024-09-17T03:21:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d16b0 0x7f06ac14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d16b0 0x7f06ac14c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49698/0 (socket says 172.21.15.46:49698) 2024-09-17T03:21:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f4630600640 1 -- start start 2024-09-17T03:21:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b481554d0 con 0x7f9b4815aa50 2024-09-17T03:21:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06aaa00640 1 -- 172.21.15.46:0/3685989671 learned_addr learned my addr 172.21.15.46:0/3685989671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b482004c0 con 0x7f9b4815aa50 2024-09-17T03:21:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== osd.1 v2:172.21.15.46:6802/2558692648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3f8009e50 con 0x7fd3ec07b450 2024-09-17T03:21:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f3d19600640 1 -- 172.21.15.46:0/4293236730 shutdown_connections 2024-09-17T03:21:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb45c06b540 con 0x7fb440003ab0 2024-09-17T03:21:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f3d19600640 1 --2- 172.21.15.46:0/4293236730 >> v2:172.21.15.99:3300/0 conn(0x7f3d140cd7a0 0x7f3d140cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f3d19600640 1 --2- 172.21.15.46:0/4293236730 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 0x7f3d140cd260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b4806b540 con 0x7f9b4815aa50 2024-09-17T03:21:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06ac0d29e0 0x7f06ac0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac16c090 con 0x7f06ac0d25d0 2024-09-17T03:21:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac16bf10 con 0x7f06ac0d16b0 2024-09-17T03:21:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f3d19600640 1 -- 172.21.15.46:0/4293236730 >> 172.21.15.46:0/4293236730 conn(0x7f3d140b2040 msgr2=0x7f3d140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06aa000640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06ac0d25d0 0x7f06ac0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac16c210 con 0x7f06ac0d29e0 2024-09-17T03:21:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.186+0000 7f06ab400640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06ac0d29e0 0x7f06ac0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4630600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46280b3340 0x7f4628158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06aaa00640 1 -- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06ac0d29e0 msgr2=0x7f06ac0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06aaa00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06ac0d29e0 0x7f06ac0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06aaa00640 1 -- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06ac0d25d0 msgr2=0x7f06ac0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06aaa00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06ac0d25d0 0x7f06ac0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06aaa00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac0cc380 con 0x7f06ac0d16b0 2024-09-17T03:21:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4630600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280ce4c0 0x7f4628206db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4630600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f462815aa50 0x7f46280abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06aaa00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d16b0 0x7f06ac14c940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f069c00cb40 tx=0x7f069c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c016020 con 0x7f06ac0d16b0 2024-09-17T03:21:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f069c00ab40 con 0x7f06ac0d16b0 2024-09-17T03:21:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c012420 con 0x7f06ac0d16b0 2024-09-17T03:21:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4630600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f462816b1a0 con 0x7f462815aa50 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4630600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f462816b020 con 0x7f46280ce4c0 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4630600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f462816b320 con 0x7f46280b3340 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46280b3340 0x7f4628158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280ce4c0 0x7f4628206db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46280b3340 0x7f4628158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56718/0 (socket says 172.21.15.46:56718) 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 -- 172.21.15.46:0/3969021385 learned_addr learned my addr 172.21.15.46:0/3969021385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 -- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280ce4c0 msgr2=0x7f4628206db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280ce4c0 0x7f4628206db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 -- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f462815aa50 msgr2=0x7f46280abe70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:21:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb4480b7070 con 0x7fb440003ab0 2024-09-17T03:21:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f462815aa50 0x7f46280abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 -- 172.21.15.46:0/3969021385 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46280aa600 con 0x7f46280b3340 2024-09-17T03:21:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f3d19600640 1 -- 172.21.15.46:0/4293236730 shutdown_connections 2024-09-17T03:21:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f3d19600640 1 -- 172.21.15.46:0/4293236730 wait complete. 2024-09-17T03:21:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b4000fcf0 con 0x7f9b4815aa50 2024-09-17T03:21:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b3f400640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b28073250 0x7f9b28075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9b40095080 con 0x7f9b4815aa50 2024-09-17T03:21:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b3f400640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f9b2807b520 0x7f9b2807d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac0cc660 con 0x7f06ac0d16b0 2024-09-17T03:21:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 --> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b2807e040 con 0x7f9b2807b520 2024-09-17T03:21:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9b40095430 con 0x7f9b4815aa50 2024-09-17T03:21:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac0ccc20 con 0x7f06ac0d16b0 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f462dc00640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46280b3340 0x7f4628158490 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f461c00c440 tx=0x7f461c00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0052d0 con 0x7f46280b3340 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f461c005490 con 0x7f46280b3340 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c012870 con 0x7f46280b3340 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46280aa8a0 con 0x7f46280b3340 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb458c00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb43c073200 0x7fb43c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb44812e690 con 0x7fb440003ab0 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb458c00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7fb43c07b450 0x7fb43c07d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 --> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4480ace50 con 0x7fb43c07b450 2024-09-17T03:21:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.190+0000 7f9b47400640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b28073250 0x7f9b28075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f9b46a00640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f9b2807b520 0x7f9b2807d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb461600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7fb43c07b450 0x7fb43c07d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb461600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7fb43c07b450 0x7fb43c07d8a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb45be00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb43c073200 0x7fb43c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== osd.4 v2:172.21.15.99:6808/2242927697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4480ace50 con 0x7fb43c07b450 2024-09-17T03:21:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f069c019020 con 0x7f06ac0d16b0 2024-09-17T03:21:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06a3400640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f068c073300 0x7f068c075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f069c094170 con 0x7f06ac0d16b0 2024-09-17T03:21:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46280aae60 con 0x7f46280b3340 2024-09-17T03:21:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f3d19600640 1 Processor -- start 2024-09-17T03:21:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f3d19600640 1 -- start start 2024-09-17T03:21:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.194+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f461c012a10 con 0x7f46280b3340 2024-09-17T03:21:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f462806b540 con 0x7f46280b3340 2024-09-17T03:21:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f9b47400640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b28073250 0x7f9b28075760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b38007fd0 tx=0x7f9b38007940 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f4625600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4608073230 0x7f4608075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f0684001660 0x7f0684003b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7fb45be00640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb43c073200 0x7fb43c075710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb45c164fd0 tx=0x7fb454009500 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f461c093700 con 0x7f46280b3340 2024-09-17T03:21:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0684006c10 con 0x7f0684001660 2024-09-17T03:21:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f4625600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f460807b480 0x7f460807d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 --> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f460807dfa0 con 0x7f460807b480 2024-09-17T03:21:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f461c09a050 con 0x7f46280b3340 2024-09-17T03:21:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb448135050 con 0x7fb440003ab0 2024-09-17T03:21:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06aa000640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f068c073300 0x7f068c075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06ab400640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f0684001660 0x7f0684003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d19600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d140cd7a0 0x7f3d14147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f9b46a00640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f9b2807b520 0x7f9b2807d970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d19600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 0x7f3d14154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f462e600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f460807b480 0x7f460807d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d19600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1415e4e0 0x7f3d14155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d19600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14166470 con 0x7f3d140cd7a0 2024-09-17T03:21:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d19600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141662f0 con 0x7f3d1415e4e0 2024-09-17T03:21:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f462d200640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4608073230 0x7f4608075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d19600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3d141665f0 con 0x7f3d1415aa50 2024-09-17T03:21:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d13400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1415e4e0 0x7f3d14155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d13400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1415e4e0 0x7f3d14155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49724/0 (socket says 172.21.15.46:49724) 2024-09-17T03:21:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d13400640 1 -- 172.21.15.46:0/3310379802 learned_addr learned my addr 172.21.15.46:0/3310379802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f3d12000640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 0x7f3d14154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.198+0000 7f06aa000640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f068c073300 0x7f068c075810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06ac161e60 tx=0x7f06980091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d12a00640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d140cd7a0 0x7f3d14147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f462d200640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4608073230 0x7f4608075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4614001180 tx=0x7f4614001110 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d13400640 1 -- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 msgr2=0x7f3d14154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d13400640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d1415aa50 0x7f3d14154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d13400640 1 -- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d140cd7a0 msgr2=0x7f3d14147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d13400640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d140cd7a0 0x7f3d14147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d13400640 1 -- 172.21.15.46:0/3310379802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d140af260 con 0x7f3d1415e4e0 2024-09-17T03:21:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f06ab400640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f0684001660 0x7f0684003b70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d13400640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1415e4e0 0x7f3d14155250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b440 tx=0x7f3d0000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f462e600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f460807b480 0x7f460807d8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== osd.5 v2:172.21.15.99:6800/4253831876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0684006c10 con 0x7f0684001660 2024-09-17T03:21:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00015020 con 0x7f3d1415e4e0 2024-09-17T03:21:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== osd.3 v2:172.21.15.46:6818/2947041972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b2807e040 con 0x7f9b2807b520 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d00011940 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00011d90 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d14148190 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d14074a20 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.202+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cec001040 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.206+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d00018020 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d0aa00640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cf4073200 0x7f3cf4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d000940e0 con 0x7f3d1415e4e0 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d0aa00640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f3cf407b470 0x7f3cf407d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 --> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d0000de50 con 0x7f3cf407b470 2024-09-17T03:21:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3d0009b050 con 0x7f3d1415e4e0 2024-09-17T03:21:19.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== osd.6 v2:172.21.15.99:6816/788789999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f460807dfa0 con 0x7f460807b480 2024-09-17T03:21:19.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 --> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c6c035d30 con 0x7f3c5407b4f0 2024-09-17T03:21:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d12a00640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cf4073200 0x7f3cf4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d12000640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f3cf407b470 0x7f3cf407d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.210+0000 7f3d12a00640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cf4073200 0x7f3cf4075710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0af000 tx=0x7f3d0c01b5e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3d12000640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f3cf407b470 0x7f3cf407d8c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3c71600640 1 -- 172.21.15.46:0/2095299649 <== osd.0 v2:172.21.15.46:6810/1962470687 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3c6c035d30 con 0x7f3c5407b4f0 2024-09-17T03:21:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== osd.7 v2:172.21.15.99:6824/3170570057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d0000de50 con 0x7f3cf407b470 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7f3c5407b4f0 msgr2=0x7f3c5407d940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7f3c5407b4f0 0x7f3c5407d940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c54073300 msgr2=0x7f3c54075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c54073300 0x7f3c54075810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c680af000 tx=0x7f3c6801b120 comp rx=0 tx=0).stop 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.214+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c6c17e810 msgr2=0x7f3c6c17ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2095299649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c6c17e810 0x7f3c6c17ec20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3c74075a10 tx=0x7f3c7406aa80 comp rx=0 tx=0).stop 2024-09-17T03:21:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7f3c79e00640 1 -- 172.21.15.46:0/2095299649 reap_dead start 2024-09-17T03:21:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 --> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7580de0c0 con 0x7ff74007b650 2024-09-17T03:21:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 shutdown_connections 2024-09-17T03:21:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 >> 172.21.15.46:0/2095299649 conn(0x7f3c6c0513f0 msgr2=0x7f3c6c0e3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 shutdown_connections 2024-09-17T03:21:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7ff756000640 1 -- 172.21.15.46:0/3312812193 <== osd.2 v2:172.21.15.46:6826/1373384937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff7580de0c0 con 0x7ff74007b650 2024-09-17T03:21:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7f3c7a800640 1 -- 172.21.15.46:0/2095299649 wait complete. 2024-09-17T03:21:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7ff74007b650 msgr2=0x7ff74007daa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.218+0000 7ff764a00640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7ff74007b650 0x7ff74007daa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 --> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd4040e38b0 con 0x7fd3ec07b450 2024-09-17T03:21:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7fd408c00640 1 -- 172.21.15.46:0/122431647 <== osd.1 v2:172.21.15.46:6802/2558692648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd4040e38b0 con 0x7fd3ec07b450 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff740073400 msgr2=0x7ff740075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff764a00640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff740073400 0x7ff740075910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7500018b0 tx=0x7ff750009040 comp rx=0 tx=0).stop 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff7580e2ba0 msgr2=0x7ff758053210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff764a00640 1 --2- 172.21.15.46:0/3312812193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff7580e2ba0 0x7ff758053210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff75801d910 tx=0x7ff74c0a55b0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff75e800640 1 -- 172.21.15.46:0/3312812193 reap_dead start 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 shutdown_connections 2024-09-17T03:21:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 >> 172.21.15.46:0/3312812193 conn(0x7ff7580513f0 msgr2=0x7ff75804fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd3ec07b450 msgr2=0x7fd3ec07d8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.222+0000 7fd412200640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd3ec07b450 0x7fd3ec07d8a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3ec073200 msgr2=0x7fd3ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3ec073200 0x7fd3ec075710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4040e7610 tx=0x7fd40c095780 comp rx=0 tx=0).stop 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040e6b10 msgr2=0x7fd4040de5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 --2- 172.21.15.46:0/122431647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4040e6b10 0x7fd4040de5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800d610 tx=0x7fd3f800dad0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd411800640 1 -- 172.21.15.46:0/122431647 reap_dead start 2024-09-17T03:21:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 shutdown_connections 2024-09-17T03:21:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 >> 172.21.15.46:0/122431647 conn(0x7fd404058b30 msgr2=0x7fd4040e5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 shutdown_connections 2024-09-17T03:21:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fd412200640 1 -- 172.21.15.46:0/122431647 wait complete. 2024-09-17T03:21:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 --> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb45c06d070 con 0x7fb43c07b450 2024-09-17T03:21:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb458c00640 1 -- 172.21.15.46:0/1701138147 <== osd.4 v2:172.21.15.99:6808/2242927697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb45c06d070 con 0x7fb43c07b450 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7fb43c07b450 msgr2=0x7fb43c07d8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7fb43c07b450 0x7fb43c07d8a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb43c073200 msgr2=0x7fb43c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb43c073200 0x7fb43c075710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb45c164fd0 tx=0x7fb454009500 comp rx=0 tx=0).stop 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb440003ab0 msgr2=0x7fb45c150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb463600640 1 --2- 172.21.15.46:0/1701138147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb440003ab0 0x7fb45c150650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb44801f950 tx=0x7fb4480afa10 comp rx=0 tx=0).stop 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7fb461600640 1 -- 172.21.15.46:0/1701138147 reap_dead start 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.230+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 shutdown_connections 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.230+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 >> 172.21.15.46:0/1701138147 conn(0x7fb45c06e2d0 msgr2=0x7fb45c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.230+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 shutdown_connections 2024-09-17T03:21:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.230+0000 7fb463600640 1 -- 172.21.15.46:0/1701138147 wait complete. 2024-09-17T03:21:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.226+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 shutdown_connections 2024-09-17T03:21:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.230+0000 7ff764a00640 1 -- 172.21.15.46:0/3312812193 wait complete. 2024-09-17T03:21:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.230+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 --> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b480cd260 con 0x7f9b2807b520 2024-09-17T03:21:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b3f400640 1 -- 172.21.15.46:0/1651158505 <== osd.3 v2:172.21.15.46:6818/2947041972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b480cd260 con 0x7f9b2807b520 2024-09-17T03:21:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f9b2807b520 msgr2=0x7f9b2807d970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f9b2807b520 0x7f9b2807d970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 --> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46280b3750 con 0x7f460807b480 2024-09-17T03:21:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4625600640 1 -- 172.21.15.46:0/3969021385 <== osd.6 v2:172.21.15.99:6816/788789999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46280b3750 con 0x7f460807b480 2024-09-17T03:21:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b28073250 msgr2=0x7f9b28075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b28073250 0x7f9b28075760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b38007fd0 tx=0x7f9b38007940 comp rx=0 tx=0).stop 2024-09-17T03:21:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b4815aa50 msgr2=0x7f9b481532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 --2- 172.21.15.46:0/1651158505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b4815aa50 0x7f9b481532a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000cd10 tx=0x7f9b4000a620 comp rx=0 tx=0).stop 2024-09-17T03:21:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b47e00640 1 -- 172.21.15.46:0/1651158505 reap_dead start 2024-09-17T03:21:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 shutdown_connections 2024-09-17T03:21:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 >> 172.21.15.46:0/1651158505 conn(0x7f9b480b2040 msgr2=0x7f9b480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f460807b480 msgr2=0x7f460807d8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f460807b480 0x7f460807d8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 --> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0684005ca0 con 0x7f0684001660 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f06a3400640 1 -- 172.21.15.46:0/3685989671 <== osd.5 v2:172.21.15.99:6800/4253831876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0684005ca0 con 0x7f0684001660 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f0684001660 msgr2=0x7f0684003b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f0684001660 0x7f0684003b70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4608073230 msgr2=0x7f4608075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4608073230 0x7f4608075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4614001180 tx=0x7f4614001110 comp rx=0 tx=0).stop 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46280b3340 msgr2=0x7f4628158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 --2- 172.21.15.46:0/3969021385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46280b3340 0x7f4628158490 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f461c00c440 tx=0x7f461c00c900 comp rx=0 tx=0).stop 2024-09-17T03:21:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f462e600640 1 -- 172.21.15.46:0/3969021385 reap_dead start 2024-09-17T03:21:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 shutdown_connections 2024-09-17T03:21:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 >> 172.21.15.46:0/3969021385 conn(0x7f46280b2040 msgr2=0x7f462815c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f068c073300 msgr2=0x7f068c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 shutdown_connections 2024-09-17T03:21:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f068c073300 0x7f068c075810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06ac161e60 tx=0x7f06980091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.234+0000 7f4630600640 1 -- 172.21.15.46:0/3969021385 wait complete. 2024-09-17T03:21:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d16b0 msgr2=0x7f06ac14c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06b1e00640 1 --2- 172.21.15.46:0/3685989671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06ac0d16b0 0x7f06ac14c940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f069c00cb40 tx=0x7f069c00cc30 comp rx=0 tx=0).stop 2024-09-17T03:21:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 shutdown_connections 2024-09-17T03:21:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f9b4e000640 1 -- 172.21.15.46:0/1651158505 wait complete. 2024-09-17T03:21:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06ab400640 1 -- 172.21.15.46:0/3685989671 reap_dead start 2024-09-17T03:21:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 shutdown_connections 2024-09-17T03:21:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 >> 172.21.15.46:0/3685989671 conn(0x7f06ac073e00 msgr2=0x7f06ac0c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 shutdown_connections 2024-09-17T03:21:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.238+0000 7f06b1e00640 1 -- 172.21.15.46:0/3685989671 wait complete. 2024-09-17T03:21:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.242+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 --> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cec002de0 con 0x7f3cf407b470 2024-09-17T03:21:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.242+0000 7f3d0aa00640 1 -- 172.21.15.46:0/3310379802 <== osd.7 v2:172.21.15.99:6824/3170570057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3cec002de0 con 0x7f3cf407b470 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f3cf407b470 msgr2=0x7f3cf407d8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f3cf407b470 0x7f3cf407d8c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cf4073200 msgr2=0x7f3cf4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cf4073200 0x7f3cf4075710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0af000 tx=0x7f3d0c01b5e0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1415e4e0 msgr2=0x7f3d14155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 --2- 172.21.15.46:0/3310379802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1415e4e0 0x7f3d14155250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b440 tx=0x7f3d0000b900 comp rx=0 tx=0).stop 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d13400640 1 -- 172.21.15.46:0/3310379802 reap_dead start 2024-09-17T03:21:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 shutdown_connections 2024-09-17T03:21:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 >> 172.21.15.46:0/3310379802 conn(0x7f3d140b2040 msgr2=0x7f3d1415e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 shutdown_connections 2024-09-17T03:21:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.246+0000 7f3d19600640 1 -- 172.21.15.46:0/3310379802 wait complete. 2024-09-17T03:21:19.260 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:19.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T03:21:19.279 INFO:teuthology.orchestra.run.smithi046.stdout:60129542151 2024-09-17T03:21:19.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T03:21:19.281 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:19.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T03:21:19.282 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:19.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T03:21:19.285 INFO:teuthology.orchestra.run.smithi046.stdout:60129542151 2024-09-17T03:21:19.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T03:21:19.286 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:19.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T03:21:19.289 INFO:teuthology.orchestra.run.smithi046.stdout:64424509448 2024-09-17T03:21:19.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T03:21:19.295 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:19.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T03:21:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.542+0000 7fc8ad600640 1 -- 172.21.15.46:0/2340636876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 msgr2=0x7fc8a00511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.542+0000 7fc8ad600640 1 --2- 172.21.15.46:0/2340636876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 0x7fc8a00511b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00e6af0 tx=0x7fc89c01c890 comp rx=0 tx=0).stop 2024-09-17T03:21:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 -- 172.21.15.46:0/2340636876 shutdown_connections 2024-09-17T03:21:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 --2- 172.21.15.46:0/2340636876 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a00065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 --2- 172.21.15.46:0/2340636876 >> v2:172.21.15.99:3300/0 conn(0x7fc8a00516f0 0x7fc8a0005c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 --2- 172.21.15.46:0/2340636876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 0x7fc8a00511b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 -- 172.21.15.46:0/2340636876 >> 172.21.15.46:0/2340636876 conn(0x7fc8a001d9b0 msgr2=0x7fc8a0039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 -- 172.21.15.46:0/2340636876 shutdown_connections 2024-09-17T03:21:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.546+0000 7fc8ad600640 1 -- 172.21.15.46:0/2340636876 wait complete. 2024-09-17T03:21:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.550+0000 7fc8ad600640 1 Processor -- start 2024-09-17T03:21:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.550+0000 7fc8ad600640 1 -- start start 2024-09-17T03:21:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a000ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 0x7fc8a00482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8a00516f0 0x7fc8a0048810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a00e34d0 con 0x7fc8a0033bc0 2024-09-17T03:21:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a0050810 con 0x7fc8a00516f0 2024-09-17T03:21:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a0050990 con 0x7fc8a00061a0 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 0x7fc8a00482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a000ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8acc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8a00516f0 0x7fc8a0048810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a000ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56748/0 (socket says 172.21.15.46:56748) 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 -- 172.21.15.46:0/639787143 learned_addr learned my addr 172.21.15.46:0/639787143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 -- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8a00516f0 msgr2=0x7fc8a0048810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8a00516f0 0x7fc8a0048810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 -- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 msgr2=0x7fc8a00482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 0x7fc8a00482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 -- 172.21.15.46:0/639787143 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a00525c0 con 0x7fc8a00061a0 2024-09-17T03:21:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8a7e00640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a000ae10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000c10 tx=0x7fc89c005060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0be070 con 0x7fc8a00061a0 2024-09-17T03:21:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc89c005450 con 0x7fc8a00061a0 2024-09-17T03:21:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0b2040 con 0x7fc8a00061a0 2024-09-17T03:21:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.554+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a0055150 con 0x7fc8a00061a0 2024-09-17T03:21:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 -- 172.21.15.46:0/1874190270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 msgr2=0x7f33480cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 --2- 172.21.15.46:0/1874190270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 0x7f33480cd970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3334009840 tx=0x7f333401c790 comp rx=0 tx=0).stop 2024-09-17T03:21:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 -- 172.21.15.46:0/1874190270 shutdown_connections 2024-09-17T03:21:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 --2- 172.21.15.46:0/1874190270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 0x7f33480cd970 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 -- 172.21.15.46:0/1874190270 >> 172.21.15.46:0/1874190270 conn(0x7f33480b2040 msgr2=0x7f33480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a00cf810 con 0x7fc8a00061a0 2024-09-17T03:21:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 -- 172.21.15.46:0/1874190270 shutdown_connections 2024-09-17T03:21:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.558+0000 7f3350200640 1 -- 172.21.15.46:0/1874190270 wait complete. 2024-09-17T03:21:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a0052330 con 0x7fc8a00061a0 2024-09-17T03:21:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc89c0046d0 con 0x7fc8a00061a0 2024-09-17T03:21:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7f3350200640 1 Processor -- start 2024-09-17T03:21:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7f3350200640 1 -- start start 2024-09-17T03:21:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7fc89be00640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc888073230 0x7fc888075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7fc8a7400640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc888073230 0x7fc888075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.562+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc89c12e030 con 0x7fc8a00061a0 2024-09-17T03:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33480b3340 0x7f3348166ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33480ce4c0 0x7f3348147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 0x7f3348159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f334816afd0 con 0x7f33480b3340 2024-09-17T03:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f334816ae50 con 0x7f334815aa50 2024-09-17T03:21:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f334816b150 con 0x7f33480ce4c0 2024-09-17T03:21:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33480b3340 0x7f3348166ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7fc8a7400640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc888073230 0x7fc888075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00d4ac0 tx=0x7fc894006f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33480b3340 0x7f3348166ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38564/0 (socket says 172.21.15.46:38564) 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 0x7f3348159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334d800640 1 -- 172.21.15.46:0/1620920803 learned_addr learned my addr 172.21.15.46:0/1620920803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33480ce4c0 0x7f3348147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 -- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 msgr2=0x7f3348159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 0x7f3348159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 -- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33480b3340 msgr2=0x7f3348166ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33480b3340 0x7f3348166ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 -- 172.21.15.46:0/1620920803 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3348148120 con 0x7f33480ce4c0 2024-09-17T03:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 -- 172.21.15.46:0/3354202304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a8415f8f0 msgr2=0x7f1a841557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 --2- 172.21.15.46:0/3354202304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a8415f8f0 0x7f1a841557e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a840ced80 tx=0x7f1a7401c6f0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 -- 172.21.15.46:0/3354202304 shutdown_connections 2024-09-17T03:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 --2- 172.21.15.46:0/3354202304 >> v2:172.21.15.99:3300/0 conn(0x7f1a840aeff0 0x7f1a840b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 --2- 172.21.15.46:0/3354202304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a8415f8f0 0x7f1a841557e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 -- 172.21.15.46:0/3354202304 >> 172.21.15.46:0/3354202304 conn(0x7f1a8406e1d0 msgr2=0x7f1a8406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 -- 172.21.15.46:0/3354202304 shutdown_connections 2024-09-17T03:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f334ce00640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33480ce4c0 0x7f3348147be0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f33340ac000 tx=0x7f33340040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc89c0164b0 con 0x7fc8a00061a0 2024-09-17T03:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f1a8b400640 1 -- 172.21.15.46:0/3354202304 wait complete. 2024-09-17T03:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.570+0000 7f1a8b400640 1 Processor -- start 2024-09-17T03:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.570+0000 7f1a8b400640 1 -- start start 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340b2040 con 0x7f33480ce4c0 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3334003600 con 0x7f33480ce4c0 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334003a50 con 0x7f33480ce4c0 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33481672f0 con 0x7f33480ce4c0 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.566+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33481678b0 con 0x7f33480ce4c0 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.570+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3348167060 con 0x7f33480ce4c0 2024-09-17T03:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.570+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33340bd060 con 0x7f33480ce4c0 2024-09-17T03:21:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f3345600640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33280732d0 0x7f33280757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f333412d9b0 con 0x7f33480ce4c0 2024-09-17T03:21:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a8b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a840cf210 0x7f1a84143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a8b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 0x7f1a841f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a8b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a84072d40 0x7f1a8414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a8b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a840ccfb0 con 0x7f1a84072d40 2024-09-17T03:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a8b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a840cce30 con 0x7f1a840cf210 2024-09-17T03:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a8b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1a840cd130 con 0x7f1a8415f8f0 2024-09-17T03:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f334d800640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33280732d0 0x7f33280757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 0x7f1a841f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 0x7f1a841f95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56800/0 (socket says 172.21.15.46:56800) 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 -- 172.21.15.46:0/339471051 learned_addr learned my addr 172.21.15.46:0/339471051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a89400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a84072d40 0x7f1a8414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a88a00640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a840cf210 0x7f1a84143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 -- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a840cf210 msgr2=0x7f1a84143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a840cf210 0x7f1a84143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 -- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a84072d40 msgr2=0x7f1a8414d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a84072d40 0x7f1a8414d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.574+0000 7f1a7be00640 1 -- 172.21.15.46:0/339471051 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a840722a0 con 0x7f1a8415f8f0 2024-09-17T03:21:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1a7be00640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 0x7f1a841f95d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c400 tx=0x7f1a6c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f334d800640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33280732d0 0x7f33280757e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099f0 tx=0x7f333c0090e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e76000640 1 -- 172.21.15.46:0/2702225881 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680062d0 con 0x7f1e5c003a60 2024-09-17T03:21:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c011790 con 0x7f1a8415f8f0 2024-09-17T03:21:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a84074540 con 0x7f1a8415f8f0 2024-09-17T03:21:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a84073d90 con 0x7f1a8415f8f0 2024-09-17T03:21:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3334016410 con 0x7f33480ce4c0 2024-09-17T03:21:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f271ea00640 1 -- 172.21.15.46:0/2899204335 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718009b30 con 0x7f2704005160 2024-09-17T03:21:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a6c017070 con 0x7f1a8415f8f0 2024-09-17T03:21:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c011ca0 con 0x7f1a8415f8f0 2024-09-17T03:21:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2702225881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c003a60 msgr2=0x7f1e5c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2702225881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c003a60 0x7f1e5c003ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800a790 tx=0x7f1e6801d270 comp rx=0 tx=0).stop 2024-09-17T03:21:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2702225881 shutdown_connections 2024-09-17T03:21:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2702225881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c003a60 0x7f1e5c003ed0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800a790 tx=0x7f1e6801d270 comp rx=0 tx=0).stop 2024-09-17T03:21:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2702225881 >> 172.21.15.46:0/2702225881 conn(0x7f1e780b41f0 msgr2=0x7f1e780b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2725400640 1 -- 172.21.15.46:0/2899204335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2704005160 msgr2=0x7f270400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2725400640 1 --2- 172.21.15.46:0/2899204335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2704005160 0x7f270400b9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27200b5700 tx=0x7f271801f660 comp rx=0 tx=0).stop 2024-09-17T03:21:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2724a00640 1 -- 172.21.15.46:0/2899204335 reap_dead start 2024-09-17T03:21:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2725400640 1 -- 172.21.15.46:0/2899204335 shutdown_connections 2024-09-17T03:21:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2725400640 1 -- 172.21.15.46:0/2899204335 >> 172.21.15.46:0/2899204335 conn(0x7f272006ee20 msgr2=0x7f27200d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2702225881 shutdown_connections 2024-09-17T03:21:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2725400640 1 -- 172.21.15.46:0/2899204335 shutdown_connections 2024-09-17T03:21:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2702225881 wait complete. 2024-09-17T03:21:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f2725400640 1 -- 172.21.15.46:0/2899204335 wait complete. 2024-09-17T03:21:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.578+0000 7f46ea000640 1 -- 172.21.15.46:0/4152281056 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d80b2070 con 0x7f46ec070540 2024-09-17T03:21:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5c0052a0 con 0x7f1a8415f8f0 2024-09-17T03:21:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2244400640 1 -- 172.21.15.46:0/2533554325 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c0b5c70 msgr2=0x7f22240052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2244400640 1 --2- 172.21.15.46:0/2533554325 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c0b5c70 0x7f22240052c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22300098a0 tx=0x7f223001c6b0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2242400640 1 -- 172.21.15.46:0/2533554325 reap_dead start 2024-09-17T03:21:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2244400640 1 -- 172.21.15.46:0/2533554325 shutdown_connections 2024-09-17T03:21:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2244400640 1 -- 172.21.15.46:0/2533554325 >> 172.21.15.46:0/2533554325 conn(0x7f223c0b2040 msgr2=0x7f223c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f1e7ce00640 1 Processor -- start 2024-09-17T03:21:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f1e7ce00640 1 -- start start 2024-09-17T03:21:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 -- 172.21.15.46:0/4152281056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec070540 msgr2=0x7f46ec06cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 --2- 172.21.15.46:0/4152281056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec070540 0x7f46ec06cdf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009900 tx=0x7f46d801c900 comp rx=0 tx=0).stop 2024-09-17T03:21:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 -- 172.21.15.46:0/4152281056 shutdown_connections 2024-09-17T03:21:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2725400640 1 Processor -- start 2024-09-17T03:21:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 --2- 172.21.15.46:0/4152281056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec070540 0x7f46ec06cdf0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009900 tx=0x7f46d801c900 comp rx=0 tx=0).stop 2024-09-17T03:21:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 --2- 172.21.15.46:0/4152281056 >> v2:172.21.15.46:3300/0 conn(0x7f46ec15a1f0 0x7f46ec0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1a6c01c460 con 0x7f1a8415f8f0 2024-09-17T03:21:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f2725400640 1 -- start start 2024-09-17T03:21:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 -- 172.21.15.46:0/4152281056 >> 172.21.15.46:0/4152281056 conn(0x7f46ec0d0460 msgr2=0x7f46ec0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 -- 172.21.15.46:0/4152281056 shutdown_connections 2024-09-17T03:21:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f46f2600640 1 -- 172.21.15.46:0/4152281056 wait complete. 2024-09-17T03:21:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f1e7ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7815eb50 0x7f1e7809a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.582+0000 7f1e7ce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e7809a610 0x7f1e78151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e7ce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e78151e90 0x7f1e781522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e7ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7815fb50 con 0x7f1e7815eb50 2024-09-17T03:21:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e7ce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7815f9d0 con 0x7f1e7809a610 2024-09-17T03:21:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e7ce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7815fcd0 con 0x7f1e78151e90 2024-09-17T03:21:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7815eb50 0x7f1e7809a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1a81600640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a60073200 0x7f1a60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2725400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 0x7f2720155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2725400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2720155fa0 0x7f2720154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7815eb50 0x7f1e7809a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38640/0 (socket says 172.21.15.46:38640) 2024-09-17T03:21:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77400640 1 -- 172.21.15.46:0/2459092056 learned_addr learned my addr 172.21.15.46:0/2459092056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e76a00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e7809a610 0x7f1e78151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e78151e90 0x7f1e781522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1a6c093e90 con 0x7f1a8415f8f0 2024-09-17T03:21:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 0x7f2720155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2725400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2720154590 0x7f27201fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2725400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2720165ad0 con 0x7f2720154590 2024-09-17T03:21:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2725400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2720165950 con 0x7f2720155fa0 2024-09-17T03:21:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2725400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2720165c50 con 0x7f272014f8c0 2024-09-17T03:21:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 0x7f2720155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56842/0 (socket says 172.21.15.46:56842) 2024-09-17T03:21:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 -- 172.21.15.46:0/1362558538 learned_addr learned my addr 172.21.15.46:0/1362558538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1a88a00640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a60073200 0x7f1a60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 -- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e7809a610 msgr2=0x7f1e78151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271f400640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2720155fa0 0x7f2720154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e7809a610 0x7f1e78151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 -- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7815eb50 msgr2=0x7f1e7809a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7815eb50 0x7f1e7809a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 -- 172.21.15.46:0/2459092056 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e780c4ab0 con 0x7f1e78151e90 2024-09-17T03:21:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 -- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2720155fa0 msgr2=0x7f2720154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1e77e00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e78151e90 0x7f1e781522a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b1490 tx=0x7f1e70011b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2724a00640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2720154590 0x7f27201fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2720155fa0 0x7f2720154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 -- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2720154590 msgr2=0x7f27201fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f1a88a00640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a60073200 0x7f1a60075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a740af000 tx=0x7f1a7401af30 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2720154590 0x7f27201fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f271fe00640 1 -- 172.21.15.46:0/1362558538 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272015de70 con 0x7f272014f8c0 2024-09-17T03:21:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f2724a00640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2720154590 0x7f27201fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f46f2600640 1 Processor -- start 2024-09-17T03:21:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.586+0000 7f46f2600640 1 -- start start 2024-09-17T03:21:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f271fe00640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 0x7f2720155a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f271000c730 tx=0x7f271000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2244400640 1 -- 172.21.15.46:0/2533554325 shutdown_connections 2024-09-17T03:21:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70004440 con 0x7f1e78151e90 2024-09-17T03:21:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e7000f8f0 con 0x7f1e78151e90 2024-09-17T03:21:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000fd20 con 0x7f1e78151e90 2024-09-17T03:21:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e780c4d90 con 0x7f1e78151e90 2024-09-17T03:21:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e780c5350 con 0x7f1e78151e90 2024-09-17T03:21:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710005480 con 0x7f272014f8c0 2024-09-17T03:21:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f271000ce40 con 0x7f272014f8c0 2024-09-17T03:21:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710003840 con 0x7f272014f8c0 2024-09-17T03:21:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46f2600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 0x7f46ec146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46f2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec070950 0x7f46ec1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e780b08a0 con 0x7f1e78151e90 2024-09-17T03:21:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46f2600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec15a1f0 0x7f46ec152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46eb400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 0x7f46ec146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46eb400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 0x7f46ec146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49854/0 (socket says 172.21.15.46:49854) 2024-09-17T03:21:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46eb400640 1 -- 172.21.15.46:0/2401681792 learned_addr learned my addr 172.21.15.46:0/2401681792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1a6c05efd0 con 0x7f1a8415f8f0 2024-09-17T03:21:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2725400640 1 -- 172.21.15.46:0/1362558538 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27201f8730 con 0x7f272014f8c0 2024-09-17T03:21:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2725400640 1 -- 172.21.15.46:0/1362558538 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201f8c80 con 0x7f272014f8c0 2024-09-17T03:21:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46ebe00640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec15a1f0 0x7f46ec152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46eaa00640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec070950 0x7f46ec1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec16a4c0 con 0x7f46ec070950 2024-09-17T03:21:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec16a340 con 0x7f46ec070540 2024-09-17T03:21:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f2244400640 1 -- 172.21.15.46:0/2533554325 wait complete. 2024-09-17T03:21:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.590+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec16a640 con 0x7f46ec15a1f0 2024-09-17T03:21:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f2244400640 1 Processor -- start 2024-09-17T03:21:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f40052a0 con 0x7f272014f8c0 2024-09-17T03:21:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f2244400640 1 -- start start 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46ebe00640 1 -- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 msgr2=0x7f46ec146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46ebe00640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 0x7f46ec146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46ebe00640 1 -- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec070950 msgr2=0x7f46ec1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46ebe00640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec070950 0x7f46ec1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46ebe00640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec0afff0 con 0x7f46ec15a1f0 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46eb400640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 0x7f46ec146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f46ebe00640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec15a1f0 0x7f46ec152af0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ec0 tx=0x7f46dc00b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e70005020 con 0x7f1e78151e90 2024-09-17T03:21:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f1e6f400640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e58073230 0x7f1e58075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e70094a80 con 0x7f1e78151e90 2024-09-17T03:21:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 -- 172.21.15.46:0/4078363870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 msgr2=0x7f05380b6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 --2- 172.21.15.46:0/4078363870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f05380b6f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f05380b5820 tx=0x7f052801c870 comp rx=0 tx=0).stop 2024-09-17T03:21:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f1e77400640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e58073230 0x7f1e58075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 -- 172.21.15.46:0/4078363870 shutdown_connections 2024-09-17T03:21:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 --2- 172.21.15.46:0/4078363870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f05380b6f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 --2- 172.21.15.46:0/4078363870 >> v2:172.21.15.99:3300/0 conn(0x7f05381554c0 0x7f0538071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 --2- 172.21.15.46:0/4078363870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0538162c30 0x7f053806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 -- 172.21.15.46:0/4078363870 >> 172.21.15.46:0/4078363870 conn(0x7f05380b4560 msgr2=0x7f05380d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 -- 172.21.15.46:0/4078363870 shutdown_connections 2024-09-17T03:21:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f0537200640 1 -- 172.21.15.46:0/4078363870 wait complete. 2024-09-17T03:21:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.594+0000 7f1e77400640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e58073230 0x7f1e58075740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e78152a50 tx=0x7f1e6801a060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e70020030 con 0x7f1e78151e90 2024-09-17T03:21:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2244400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 0x7f223c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc01d070 con 0x7f46ec15a1f0 2024-09-17T03:21:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46dc010a60 con 0x7f46ec15a1f0 2024-09-17T03:21:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec0b0260 con 0x7f46ec15a1f0 2024-09-17T03:21:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc018440 con 0x7f46ec15a1f0 2024-09-17T03:21:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec0b1d60 con 0x7f46ec15a1f0 2024-09-17T03:21:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2241a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 0x7f223c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2241a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 0x7f223c15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38662/0 (socket says 172.21.15.46:38662) 2024-09-17T03:21:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2241a00640 1 -- 172.21.15.46:0/1612776706 learned_addr learned my addr 172.21.15.46:0/1612776706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1f7790 0x7f223c1f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c1f7ba0 0x7f223c0af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec06cdf0 con 0x7f46ec15a1f0 2024-09-17T03:21:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2241000640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1f7790 0x7f223c1f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f223c0b4340 con 0x7f223c0b5c70 2024-09-17T03:21:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.598+0000 7f2242400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c1f7ba0 0x7f223c0af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f223c0b41c0 con 0x7f223c1f7790 2024-09-17T03:21:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f0537200640 1 Processor -- start 2024-09-17T03:21:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f0537200640 1 -- start start 2024-09-17T03:21:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f223c0b44c0 con 0x7f223c1f7ba0 2024-09-17T03:21:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2242400640 1 -- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1f7790 msgr2=0x7f223c1f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2242400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1f7790 0x7f223c1f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27100039e0 con 0x7f272014f8c0 2024-09-17T03:21:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2242400640 1 -- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 msgr2=0x7f223c15a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2242400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 0x7f223c15a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2242400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c1596e0 con 0x7f223c1f7ba0 2024-09-17T03:21:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2241a00640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 0x7f223c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f2242400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c1f7ba0 0x7f223c0af640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f222401ce10 tx=0x7f22300a9de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.602+0000 7f0537200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f0538156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f0538156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f0538156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38666/0 (socket says 172.21.15.46:38666) 2024-09-17T03:21:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 -- 172.21.15.46:0/2093989896 learned_addr learned my addr 172.21.15.46:0/2093989896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2717e00640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27000732d0 0x7f27000757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2710094af0 con 0x7f272014f8c0 2024-09-17T03:21:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05381554c0 0x7f0538145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0538162c30 0x7f053814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f053806d9f0 con 0x7f05380b6b10 2024-09-17T03:21:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f053806d870 con 0x7f0538162c30 2024-09-17T03:21:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f053806db70 con 0x7f05381554c0 2024-09-17T03:21:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27100c2db0 con 0x7f272014f8c0 2024-09-17T03:21:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0536800640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0538162c30 0x7f053814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46dc020070 con 0x7f46ec15a1f0 2024-09-17T03:21:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535400640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05381554c0 0x7f0538145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 -- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05381554c0 msgr2=0x7f0538145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05381554c0 0x7f0538145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 -- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0538162c30 msgr2=0x7f053814e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0538162c30 0x7f053814e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053814f410 con 0x7f05380b6b10 2024-09-17T03:21:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f46e3400640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46cc073200 0x7f46cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f46dc097f90 con 0x7f46ec15a1f0 2024-09-17T03:21:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0535e00640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f0538156900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f052400d780 tx=0x7f052400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524017070 con 0x7f05380b6b10 2024-09-17T03:21:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053814f130 con 0x7f05380b6b10 2024-09-17T03:21:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0538156100 con 0x7f05380b6b10 2024-09-17T03:21:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230009470 con 0x7f223c1f7ba0 2024-09-17T03:21:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22300a9e50 con 0x7f223c1f7ba0 2024-09-17T03:21:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300052a0 con 0x7f223c1f7ba0 2024-09-17T03:21:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f46dc062b70 con 0x7f46ec15a1f0 2024-09-17T03:21:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.610+0000 7f271f400640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27000732d0 0x7f27000757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.610+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0524004310 con 0x7f05380b6b10 2024-09-17T03:21:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.610+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524005280 con 0x7f05380b6b10 2024-09-17T03:21:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.610+0000 7f46eb400640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46cc073200 0x7f46cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c1599c0 con 0x7f223c1f7ba0 2024-09-17T03:21:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.606+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c159f80 con 0x7f223c1f7ba0 2024-09-17T03:21:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.610+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22180052a0 con 0x7f223c1f7ba0 2024-09-17T03:21:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.614+0000 7f46eb400640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46cc073200 0x7f46cc075710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46e400a450 tx=0x7f46e4007560 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.614+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05100052a0 con 0x7f05380b6b10 2024-09-17T03:21:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.614+0000 7f271f400640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27000732d0 0x7f27000757e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f270c00afd0 tx=0x7f270c00c040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2230005440 con 0x7f223c1f7ba0 2024-09-17T03:21:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.614+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0524005420 con 0x7f05380b6b10 2024-09-17T03:21:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f052e000640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0518073200 0x7f0518075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05240944c0 con 0x7f05380b6b10 2024-09-17T03:21:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f2239600640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f221c0732d0 0x7f221c0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f223012e120 con 0x7f223c1f7ba0 2024-09-17T03:21:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f2241a00640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f221c0732d0 0x7f221c0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.618+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f22300ac030 con 0x7f223c1f7ba0 2024-09-17T03:21:19.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.626+0000 7f0535400640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0518073200 0x7f0518075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:19.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.626+0000 7f0535400640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0518073200 0x7f0518075710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05280af000 tx=0x7f052800b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.626+0000 7f2241a00640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f221c0732d0 0x7f221c0757e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f223400c4c0 tx=0x7f223400f040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:19.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.626+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0524020020 con 0x7f05380b6b10 2024-09-17T03:21:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.946+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc8a00511b0 con 0x7fc8a00061a0 2024-09-17T03:21:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7fc89be00640 1 -- 172.21.15.46:0/639787143 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc89c0f8a80 con 0x7fc8a00061a0 2024-09-17T03:21:19.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f33480b3750 con 0x7f33480ce4c0 2024-09-17T03:21:19.953 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-09-17T03:21:19.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7f3345600640 1 -- 172.21.15.46:0/1620920803 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33340f8360 con 0x7f33480ce4c0 2024-09-17T03:21:19.955 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-09-17T03:21:19.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc888073230 msgr2=0x7fc888075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7fc8ad600640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc888073230 0x7fc888075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00d4ac0 tx=0x7fc894006f90 comp rx=0 tx=0).stop 2024-09-17T03:21:19.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 msgr2=0x7fc8a000ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.950+0000 7fc8ad600640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a000ae10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000c10 tx=0x7fc89c005060 comp rx=0 tx=0).stop 2024-09-17T03:21:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 shutdown_connections 2024-09-17T03:21:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc888073230 0x7fc888075740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8a00516f0 0x7fc8a0048810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8a0033bc0 0x7fc8a00482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 --2- 172.21.15.46:0/639787143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc8a00061a0 0x7fc8a000ae10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 >> 172.21.15.46:0/639787143 conn(0x7fc8a001d9b0 msgr2=0x7fc8a00e08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 shutdown_connections 2024-09-17T03:21:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7fc8ad600640 1 -- 172.21.15.46:0/639787143 wait complete. 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33280732d0 msgr2=0x7f33280757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33280732d0 0x7f33280757e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099f0 tx=0x7f333c0090e0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33480ce4c0 msgr2=0x7f3348147be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33480ce4c0 0x7f3348147be0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f33340ac000 tx=0x7f33340040c0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 shutdown_connections 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33280732d0 0x7f33280757e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334815aa50 0x7f3348159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33480ce4c0 0x7f3348147be0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 --2- 172.21.15.46:0/1620920803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33480b3340 0x7f3348166ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 >> 172.21.15.46:0/1620920803 conn(0x7f33480b2040 msgr2=0x7f33480b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 shutdown_connections 2024-09-17T03:21:19.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.954+0000 7f3350200640 1 -- 172.21.15.46:0/1620920803 wait complete. 2024-09-17T03:21:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.966+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1e780ae500 con 0x7f1e78151e90 2024-09-17T03:21:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.966+0000 7f1e6f400640 1 -- 172.21.15.46:0/2459092056 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e7005fb10 con 0x7f1e78151e90 2024-09-17T03:21:19.972 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:19.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1a5c005860 con 0x7f1a8415f8f0 2024-09-17T03:21:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e58073230 msgr2=0x7f1e58075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e58073230 0x7f1e58075740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e78152a50 tx=0x7f1e6801a060 comp rx=0 tx=0).stop 2024-09-17T03:21:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e78151e90 msgr2=0x7f1e781522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e78151e90 0x7f1e781522a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b1490 tx=0x7f1e70011b90 comp rx=0 tx=0).stop 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 shutdown_connections 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e58073230 0x7f1e58075740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e78151e90 0x7f1e781522a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e7809a610 0x7f1e78151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 --2- 172.21.15.46:0/2459092056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7815eb50 0x7f1e7809a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 >> 172.21.15.46:0/2459092056 conn(0x7f1e780b41f0 msgr2=0x7f1e780706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 shutdown_connections 2024-09-17T03:21:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1e7ce00640 1 -- 172.21.15.46:0/2459092056 wait complete. 2024-09-17T03:21:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.974+0000 7f1a81600640 1 -- 172.21.15.46:0/339471051 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1a6c05e8d0 con 0x7f1a8415f8f0 2024-09-17T03:21:19.981 INFO:teuthology.orchestra.run.smithi046.stdout:60129542150 2024-09-17T03:21:19.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f46ec1597b0 con 0x7f46ec15a1f0 2024-09-17T03:21:19.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a60073200 msgr2=0x7f1a60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a60073200 0x7f1a60075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a740af000 tx=0x7f1a7401af30 comp rx=0 tx=0).stop 2024-09-17T03:21:19.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 msgr2=0x7f1a841f95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 0x7f1a841f95d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c400 tx=0x7f1a6c00c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 shutdown_connections 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a60073200 0x7f1a60075710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a84072d40 0x7f1a8414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a8415f8f0 0x7f1a841f95d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 --2- 172.21.15.46:0/339471051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a840cf210 0x7f1a84143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 >> 172.21.15.46:0/339471051 conn(0x7f1a8406e1d0 msgr2=0x7f1a84145370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.985 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-09-17T03:21:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.978+0000 7f46e3400640 1 -- 172.21.15.46:0/2401681792 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46dc0675a0 con 0x7f46ec15a1f0 2024-09-17T03:21:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.982+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 shutdown_connections 2024-09-17T03:21:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.982+0000 7f1a8b400640 1 -- 172.21.15.46:0/339471051 wait complete. 2024-09-17T03:21:19.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-09-17T03:21:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.986+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46cc073200 msgr2=0x7f46cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.986+0000 7f46f2600640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46cc073200 0x7f46cc075710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46e400a450 tx=0x7f46e4007560 comp rx=0 tx=0).stop 2024-09-17T03:21:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec15a1f0 msgr2=0x7f46ec152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec15a1f0 0x7f46ec152af0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ec0 tx=0x7f46dc00b7b0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 shutdown_connections 2024-09-17T03:21:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46cc073200 0x7f46cc075710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46ec15a1f0 0x7f46ec152af0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec070950 0x7f46ec1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 --2- 172.21.15.46:0/2401681792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46ec070540 0x7f46ec146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:19.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.990+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 >> 172.21.15.46:0/2401681792 conn(0x7f46ec0d0460 msgr2=0x7f46ec15c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.994+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 shutdown_connections 2024-09-17T03:21:19.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.994+0000 7f46f2600640 1 -- 172.21.15.46:0/2401681792 wait complete. 2024-09-17T03:21:19.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-17T03:21:20.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.998+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0510005590 con 0x7f05380b6b10 2024-09-17T03:21:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:19.998+0000 7f052e000640 1 -- 172.21.15.46:0/2093989896 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f052405f4d0 con 0x7f05380b6b10 2024-09-17T03:21:20.004 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-09-17T03:21:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0518073200 msgr2=0x7f0518075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0518073200 0x7f0518075710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05280af000 tx=0x7f052800b040 comp rx=0 tx=0).stop 2024-09-17T03:21:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 msgr2=0x7f0538156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f0538156900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f052400d780 tx=0x7f052400db80 comp rx=0 tx=0).stop 2024-09-17T03:21:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 shutdown_connections 2024-09-17T03:21:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0518073200 0x7f0518075710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0538162c30 0x7f053814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05381554c0 0x7f0538145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 --2- 172.21.15.46:0/2093989896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05380b6b10 0x7f0538156900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 >> 172.21.15.46:0/2093989896 conn(0x7f05380b4560 msgr2=0x7f0538154d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:20.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 shutdown_connections 2024-09-17T03:21:20.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.006+0000 7f0537200640 1 -- 172.21.15.46:0/2093989896 wait complete. 2024-09-17T03:21:20.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-09-17T03:21:20.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-17T03:21:20.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.1 2024-09-17T03:21:20.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.026+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f26f4005860 con 0x7f272014f8c0 2024-09-17T03:21:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.026+0000 7f2717e00640 1 -- 172.21.15.46:0/1362558538 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f271005fc10 con 0x7f272014f8c0 2024-09-17T03:21:20.031 INFO:teuthology.orchestra.run.smithi046.stdout:60129542150 2024-09-17T03:21:20.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27000732d0 msgr2=0x7f27000757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27000732d0 0x7f27000757e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f270c00afd0 tx=0x7f270c00c040 comp rx=0 tx=0).stop 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 msgr2=0x7f2720155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 0x7f2720155a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f271000c730 tx=0x7f271000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 shutdown_connections 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27000732d0 0x7f27000757e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2720154590 0x7f27201fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2720155fa0 0x7f2720154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 --2- 172.21.15.46:0/1362558538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f272014f8c0 0x7f2720155a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 >> 172.21.15.46:0/1362558538 conn(0x7f272006ee20 msgr2=0x7f2720150ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 shutdown_connections 2024-09-17T03:21:20.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.030+0000 7f2715600640 1 -- 172.21.15.46:0/1362558538 wait complete. 2024-09-17T03:21:20.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-17T03:21:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2218005590 con 0x7f223c1f7ba0 2024-09-17T03:21:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2239600640 1 -- 172.21.15.46:0/1612776706 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f22300163f0 con 0x7f223c1f7ba0 2024-09-17T03:21:20.049 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-09-17T03:21:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f221c0732d0 msgr2=0x7f221c0757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f221c0732d0 0x7f221c0757e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f223400c4c0 tx=0x7f223400f040 comp rx=0 tx=0).stop 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c1f7ba0 msgr2=0x7f223c0af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c1f7ba0 0x7f223c0af640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f222401ce10 tx=0x7f22300a9de0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 shutdown_connections 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f221c0732d0 0x7f221c0757e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f223c1f7ba0 0x7f223c0af640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1f7790 0x7f223c1f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 --2- 172.21.15.46:0/1612776706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f223c0b5c70 0x7f223c15a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 >> 172.21.15.46:0/1612776706 conn(0x7f223c0b2040 msgr2=0x7f223c1553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 shutdown_connections 2024-09-17T03:21:20.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:20.046+0000 7f2244400640 1 -- 172.21.15.46:0/1612776706 wait complete. 2024-09-17T03:21:20.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.5 2024-09-17T03:21:20.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-17T03:21:20.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T03:21:20.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T03:21:21.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T03:21:21.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T03:21:21.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T03:21:21.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T03:21:21.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T03:21:21.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T03:21:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.234+0000 7f487d800640 1 -- 172.21.15.46:0/3968214841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48700337d0 msgr2=0x7f4870051a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.234+0000 7f487d800640 1 --2- 172.21.15.46:0/3968214841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48700337d0 0x7f4870051a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f487807b050 tx=0x7f4878099b60 comp rx=0 tx=0).stop 2024-09-17T03:21:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.234+0000 7f487d800640 1 -- 172.21.15.46:0/3968214841 shutdown_connections 2024-09-17T03:21:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.234+0000 7f487d800640 1 --2- 172.21.15.46:0/3968214841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48700337d0 0x7f4870051a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.238+0000 7f487d800640 1 -- 172.21.15.46:0/3968214841 >> 172.21.15.46:0/3968214841 conn(0x7f487001d9e0 msgr2=0x7f48700396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.238+0000 7f487d800640 1 -- 172.21.15.46:0/3968214841 shutdown_connections 2024-09-17T03:21:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.238+0000 7f487d800640 1 -- 172.21.15.46:0/3968214841 wait complete. 2024-09-17T03:21:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.242+0000 7f487d800640 1 Processor -- start 2024-09-17T03:21:21.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.242+0000 7f487d800640 1 -- start start 2024-09-17T03:21:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f487d800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 0x7f4870181bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f487d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 0x7f48700db500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f487d800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48700337d0 0x7f48700dba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f4876000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 0x7f48700db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f4876a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 0x7f4870181bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f4876a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 0x7f4870181bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56878/0 (socket says 172.21.15.46:56878) 2024-09-17T03:21:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f4876000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 0x7f48700db500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38680/0 (socket says 172.21.15.46:38680) 2024-09-17T03:21:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f487d800640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48700efe90 con 0x7f4870006270 2024-09-17T03:21:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f487d800640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48700efd10 con 0x7f48700337d0 2024-09-17T03:21:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f487d800640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f48700f0010 con 0x7f4870005e60 2024-09-17T03:21:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f4877400640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48700337d0 0x7f48700dba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4877400640 1 -- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 msgr2=0x7f4870181bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4877400640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 0x7f4870181bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4877400640 1 -- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 msgr2=0x7f48700db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4877400640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 0x7f48700db500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.246+0000 7f4876a00640 1 -- 172.21.15.46:0/2541934075 learned_addr learned my addr 172.21.15.46:0/2541934075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4876a00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 0x7f4870181bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-17T03:21:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4876000640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 0x7f48700db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-17T03:21:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4877400640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487018c100 con 0x7f48700337d0 2024-09-17T03:21:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f4877400640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48700337d0 0x7f48700dba40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f486400b410 tx=0x7f486400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864015030 con 0x7f48700337d0 2024-09-17T03:21:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4864004020 con 0x7f48700337d0 2024-09-17T03:21:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864004470 con 0x7f48700337d0 2024-09-17T03:21:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f487d800640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487018c6f0 con 0x7f48700337d0 2024-09-17T03:21:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.250+0000 7f487d800640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487018ccb0 con 0x7f48700337d0 2024-09-17T03:21:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.254+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4870051a60 con 0x7f48700337d0 2024-09-17T03:21:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.258+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4864018070 con 0x7f48700337d0 2024-09-17T03:21:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.258+0000 7f486ea00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4858073200 0x7f4858075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.258+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4864093f50 con 0x7f48700337d0 2024-09-17T03:21:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.258+0000 7f4876a00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4858073200 0x7f4858075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.262+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f486405efe0 con 0x7f48700337d0 2024-09-17T03:21:21.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.266+0000 7f4876a00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4858073200 0x7f4858075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f487807b410 tx=0x7f48780969a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 -- 172.21.15.46:0/1611659401 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f795415f5c0 msgr2=0x7f795415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 --2- 172.21.15.46:0/1611659401 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f795415f5c0 0x7f795415f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f794c009bc0 tx=0x7f794c01cb30 comp rx=0 tx=0).stop 2024-09-17T03:21:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 -- 172.21.15.46:0/91127670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00b1480 msgr2=0x7f32b00af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 -- 172.21.15.46:0/1611659401 shutdown_connections 2024-09-17T03:21:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 --2- 172.21.15.46:0/1611659401 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f795415f5c0 0x7f795415f9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 --2- 172.21.15.46:0/1611659401 >> v2:172.21.15.46:3300/0 conn(0x7f795406dbc0 0x7f795406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32aca00640 1 -- 172.21.15.46:0/91127670 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0b2070 con 0x7f32b00b1480 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 --2- 172.21.15.46:0/91127670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00b1480 0x7f32b00af7d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f329c009900 tx=0x7f329c01c870 comp rx=0 tx=0).stop 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 -- 172.21.15.46:0/91127670 shutdown_connections 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 --2- 172.21.15.46:0/91127670 >> v2:172.21.15.99:3300/0 conn(0x7f32b006cdf0 0x7f32b00afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 --2- 172.21.15.46:0/91127670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00b1480 0x7f32b00af7d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 -- 172.21.15.46:0/91127670 >> 172.21.15.46:0/91127670 conn(0x7f32b0073450 msgr2=0x7f32b00b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 -- 172.21.15.46:0/91127670 shutdown_connections 2024-09-17T03:21:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f32b5000640 1 -- 172.21.15.46:0/91127670 wait complete. 2024-09-17T03:21:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 -- 172.21.15.46:0/1611659401 >> 172.21.15.46:0/1611659401 conn(0x7f79540b41f0 msgr2=0x7f79540b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.326+0000 7f795aa00640 1 -- 172.21.15.46:0/1611659401 shutdown_connections 2024-09-17T03:21:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.330+0000 7f795aa00640 1 -- 172.21.15.46:0/1611659401 wait complete. 2024-09-17T03:21:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.330+0000 7f32b5000640 1 Processor -- start 2024-09-17T03:21:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.330+0000 7f32b5000640 1 -- start start 2024-09-17T03:21:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f795aa00640 1 Processor -- start 2024-09-17T03:21:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f795aa00640 1 -- start start 2024-09-17T03:21:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 -- 172.21.15.46:0/3641157700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 msgr2=0x7f60a8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 --2- 172.21.15.46:0/3641157700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60a8004c20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60c00d05b0 tx=0x7f60b401f680 comp rx=0 tx=0).stop 2024-09-17T03:21:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 -- 172.21.15.46:0/3641157700 shutdown_connections 2024-09-17T03:21:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 --2- 172.21.15.46:0/3641157700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60a8004c20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 -- 172.21.15.46:0/3641157700 >> 172.21.15.46:0/3641157700 conn(0x7f60c006d1f0 msgr2=0x7f60c006d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 -- 172.21.15.46:0/3641157700 shutdown_connections 2024-09-17T03:21:21.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.334+0000 7f60c6a00640 1 -- 172.21.15.46:0/3641157700 wait complete. 2024-09-17T03:21:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 -- 172.21.15.46:0/1989244126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27ec003a10 msgr2=0x7f27ec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f2803e00640 1 -- 172.21.15.46:0/1989244126 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0003c60 con 0x7f27ec003a10 2024-09-17T03:21:21.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795406dbc0 0x7f795409a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795aa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 0x7f7954164800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 0x7f7954164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 --2- 172.21.15.46:0/1989244126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27ec003a10 0x7f27ec003e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27f0001c00 tx=0x7f27f001d820 comp rx=0 tx=0).stop 2024-09-17T03:21:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7954168ff0 con 0x7f795406dbc0 2024-09-17T03:21:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795aa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7954168e70 con 0x7f795415f5c0 2024-09-17T03:21:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7958c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 0x7f7954164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795aa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7954169170 con 0x7f79540d1140 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 0x7f7954164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7958c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 0x7f7954164800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56920/0 (socket says 172.21.15.46:56920) 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f795a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 0x7f7954164d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49928/0 (socket says 172.21.15.46:49928) 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7958c00640 1 -- 172.21.15.46:0/1705418454 learned_addr learned my addr 172.21.15.46:0/1705418454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 -- 172.21.15.46:0/1989244126 shutdown_connections 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 -- 172.21.15.46:0/686700614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db8007000 msgr2=0x7f8db8036b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 --2- 172.21.15.46:0/686700614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db8007000 0x7f8db8036b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009840 tx=0x7f8dac01c820 comp rx=0 tx=0).stop 2024-09-17T03:21:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 --2- 172.21.15.46:0/1989244126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27ec003a10 0x7f27ec003e40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 -- 172.21.15.46:0/686700614 shutdown_connections 2024-09-17T03:21:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 --2- 172.21.15.46:0/686700614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db8007000 0x7f8db8036b20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 -- 172.21.15.46:0/686700614 >> 172.21.15.46:0/686700614 conn(0x7f8db801e240 msgr2=0x7f8db8039f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 -- 172.21.15.46:0/1989244126 >> 172.21.15.46:0/1989244126 conn(0x7f28040b35f0 msgr2=0x7f28040b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 -- 172.21.15.46:0/686700614 shutdown_connections 2024-09-17T03:21:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7959600640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795406dbc0 0x7f795409a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7958c00640 1 -- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 msgr2=0x7f7954164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7958c00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 0x7f7954164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f7958c00640 1 -- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795406dbc0 msgr2=0x7f795409a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f8dc5400640 1 -- 172.21.15.46:0/686700614 wait complete. 2024-09-17T03:21:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 -- 172.21.15.46:0/1989244126 shutdown_connections 2024-09-17T03:21:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.338+0000 7f280c600640 1 -- 172.21.15.46:0/1989244126 wait complete. 2024-09-17T03:21:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7958c00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795406dbc0 0x7f795409a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7958c00640 1 -- 172.21.15.46:0/1705418454 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7954169790 con 0x7f79540d1140 2024-09-17T03:21:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f795a000640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 0x7f7954164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7959600640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795406dbc0 0x7f795409a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7958c00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 0x7f7954164800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f794800e9a0 tx=0x7f794800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f32b5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 0x7f32b014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f60c6a00640 1 Processor -- start 2024-09-17T03:21:21.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794800cd80 con 0x7f79540d1140 2024-09-17T03:21:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f60c6a00640 1 -- start start 2024-09-17T03:21:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7948004020 con 0x7f79540d1140 2024-09-17T03:21:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948004450 con 0x7f79540d1140 2024-09-17T03:21:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795415cfe0 con 0x7f79540d1140 2024-09-17T03:21:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79540d0470 con 0x7f79540d1140 2024-09-17T03:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f32ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 0x7f32b014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f32ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 0x7f32b014d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38734/0 (socket says 172.21.15.46:38734) 2024-09-17T03:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f32ade00640 1 -- 172.21.15.46:0/1922383223 learned_addr learned my addr 172.21.15.46:0/1922383223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00ae1e0 0x7f32b0201110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795415cd50 con 0x7f79540d1140 2024-09-17T03:21:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7948003600 con 0x7f79540d1140 2024-09-17T03:21:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b0150200 0x7f32b0150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0169400 con 0x7f32b006cdf0 2024-09-17T03:21:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0169280 con 0x7f32b0150200 2024-09-17T03:21:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f32b0169580 con 0x7f32b00ae1e0 2024-09-17T03:21:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f8dc5400640 1 Processor -- start 2024-09-17T03:21:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ade00640 1 -- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00ae1e0 msgr2=0x7f32b0201110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f7951600640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7934073200 0x7f7934075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.342+0000 7f32ad400640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00ae1e0 0x7f32b0201110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ade00640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00ae1e0 0x7f32b0201110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ade00640 1 -- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b0150200 msgr2=0x7f32b0150610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:21:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ade00640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b0150200 0x7f32b0150610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ade00640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b01476f0 con 0x7f32b006cdf0 2024-09-17T03:21:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f8dc5400640 1 -- start start 2024-09-17T03:21:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7948094a90 con 0x7f79540d1140 2024-09-17T03:21:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ade00640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 0x7f32b014d3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c3f0 tx=0x7f32a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32ad400640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00ae1e0 0x7f32b0201110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0017070 con 0x7f32b006cdf0 2024-09-17T03:21:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a00053c0 con 0x7f32b006cdf0 2024-09-17T03:21:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0003860 con 0x7f32b006cdf0 2024-09-17T03:21:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60c0143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b014d910 con 0x7f32b006cdf0 2024-09-17T03:21:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b014bdb0 con 0x7f32b006cdf0 2024-09-17T03:21:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60c01557e0 0x7f60c00b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60c0143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60c0143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49944/0 (socket says 172.21.15.46:49944) 2024-09-17T03:21:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 -- 172.21.15.46:0/1583439140 learned_addr learned my addr 172.21.15.46:0/1583439140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.346+0000 7f7959600640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7934073200 0x7f7934075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f7959600640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7934073200 0x7f7934075710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f79400018b0 tx=0x7f7940009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c0155bf0 0x7f60c00b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60c00d0e30 con 0x7f60c0155bf0 2024-09-17T03:21:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60c00d0cb0 con 0x7f60a8004810 2024-09-17T03:21:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f60c00d0fb0 con 0x7f60c01557e0 2024-09-17T03:21:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c4c00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60c01557e0 0x7f60c00b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc5400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 0x7f8db80dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 -- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60c01557e0 msgr2=0x7f60c00b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60c01557e0 0x7f60c00b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 -- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c0155bf0 msgr2=0x7f60c00b2910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:21:21.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc5400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 0x7f8db8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db80352b0 0x7f8db8005390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 0x7f8db80dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 0x7f8db80dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49952/0 (socket says 172.21.15.46:49952) 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dbf400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 0x7f8db8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dbf400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 0x7f8db8004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56950/0 (socket says 172.21.15.46:56950) 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db80352b0 0x7f8db8005390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db80352b0 0x7f8db8005390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38744/0 (socket says 172.21.15.46:38744) 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dbfe00640 1 -- 172.21.15.46:0/245935490 learned_addr learned my addr 172.21.15.46:0/245935490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80e9770 con 0x7f8db80352b0 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80e95f0 con 0x7f8db8007000 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8db80e98f0 con 0x7f8db8031b30 2024-09-17T03:21:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f8dbfe00640 1 -- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 msgr2=0x7f8db8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c6000640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c0155bf0 0x7f60c00b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c0155bf0 0x7f60c00b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c5600640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c0157650 con 0x7f60a8004810 2024-09-17T03:21:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c4c00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60c01557e0 0x7f60c00b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f60c6000640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c0155bf0 0x7f60c00b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:21.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dbfe00640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 0x7f8db8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.350+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b00cd410 con 0x7f32b006cdf0 2024-09-17T03:21:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f32a001a070 con 0x7f32b006cdf0 2024-09-17T03:21:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dbfe00640 1 -- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db80352b0 msgr2=0x7f8db8005390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dbfe00640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db80352b0 0x7f8db8005390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dbfe00640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db80dd980 con 0x7f8db8007000 2024-09-17T03:21:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dbf400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 0x7f8db8004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60c5600640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60c0143120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60b0001ec0 tx=0x7f60b000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0017070 con 0x7f60a8004810 2024-09-17T03:21:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60b00053c0 con 0x7f60a8004810 2024-09-17T03:21:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0003860 con 0x7f60a8004810 2024-09-17T03:21:21.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c0144750 con 0x7f60a8004810 2024-09-17T03:21:21.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dbfe00640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 0x7f8db80dcb80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0075a10 tx=0x7f8dc006b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f794805fb30 con 0x7f79540d1140 2024-09-17T03:21:21.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 Processor -- start 2024-09-17T03:21:21.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 -- start start 2024-09-17T03:21:21.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0090070 con 0x7f8db8007000 2024-09-17T03:21:21.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c01f9880 con 0x7f60a8004810 2024-09-17T03:21:21.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f32a6000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3290073250 0x7f3290075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f32ad400640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3290073250 0x7f3290075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db81856c0 con 0x7f8db8007000 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db8185c80 con 0x7f8db8007000 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc006b540 con 0x7f8db8007000 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ec003a10 0x7f2804146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 0x7f2804157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f280415ea30 0x7f28041578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f280415bf70 con 0x7f27ec003a10 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f280415bdf0 con 0x7f280414f400 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f280415c0f0 con 0x7f280415ea30 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f2809200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 0x7f2804157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f2809200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 0x7f2804157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49968/0 (socket says 172.21.15.46:49968) 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f2809200640 1 -- 172.21.15.46:0/4185667588 learned_addr learned my addr 172.21.15.46:0/4185667588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f280a600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f280415ea30 0x7f28041578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f2809c00640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ec003a10 0x7f2804146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f2809200640 1 -- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f280415ea30 msgr2=0x7f28041578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f2809200640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f280415ea30 0x7f28041578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f2809200640 1 -- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ec003a10 msgr2=0x7f2804146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f2809200640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ec003a10 0x7f2804146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f2809200640 1 -- 172.21.15.46:0/4185667588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2804150c80 con 0x7f280414f400 2024-09-17T03:21:21.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f32a0093e00 con 0x7f32b006cdf0 2024-09-17T03:21:21.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f2809200640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 0x7f2804157370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d680 tx=0x7f27f800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0090070 con 0x7f8db8007000 2024-09-17T03:21:21.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.354+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c0157820 con 0x7f60a8004810 2024-09-17T03:21:21.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f32ad400640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3290073250 0x7f3290075760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f329c0af000 tx=0x7f329c00b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f60b001a070 con 0x7f60a8004810 2024-09-17T03:21:21.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c0052a0 con 0x7f8db8007000 2024-09-17T03:21:21.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f60bd600640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f60a4073220 0x7f60a4075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.358+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f60b0093ec0 con 0x7f60a8004810 2024-09-17T03:21:21.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f60c4c00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f60a4073220 0x7f60a4075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8017070 con 0x7f280414f400 2024-09-17T03:21:21.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27f8009ad0 con 0x7f280414f400 2024-09-17T03:21:21.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8010460 con 0x7f280414f400 2024-09-17T03:21:21.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28041fa650 con 0x7f280414f400 2024-09-17T03:21:21.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f60b005f080 con 0x7f60a8004810 2024-09-17T03:21:21.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28041fac10 con 0x7f280414f400 2024-09-17T03:21:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f60c4c00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f60a4073220 0x7f60a4075730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60b40b0000 tx=0x7f60b401c4c0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.362+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32b01476f0 con 0x7f32b006cdf0 2024-09-17T03:21:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.366+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8dc00751c0 con 0x7f8db8007000 2024-09-17T03:21:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.366+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28041fa3c0 con 0x7f280414f400 2024-09-17T03:21:21.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.366+0000 7f8db3e00640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8da00733e0 0x7f8da00758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.366+0000 7f8dbf400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8da00733e0 0x7f8da00758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.366+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8dc010da10 con 0x7f8db8007000 2024-09-17T03:21:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.366+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8dc00de970 con 0x7f8db8007000 2024-09-17T03:21:21.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.370+0000 7f8dbf400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8da00733e0 0x7f8da00758f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8db4007f50 tx=0x7f8db4007ee0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.370+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27f801a070 con 0x7f280414f400 2024-09-17T03:21:21.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.374+0000 7f2801600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27e4073200 0x7f27e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.374+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f27f8094600 con 0x7f280414f400 2024-09-17T03:21:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.374+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27f8094ac0 con 0x7f280414f400 2024-09-17T03:21:21.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.374+0000 7f2809c00640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27e4073200 0x7f27e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.378+0000 7f2809c00640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27e4073200 0x7f27e4075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27f0001ec0 tx=0x7f27ec005840 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 -- 172.21.15.46:0/2294164293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 msgr2=0x7efebc0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 --2- 172.21.15.46:0/2294164293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc0cf620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efebc0ceca0 tx=0x7efeac01c780 comp rx=0 tx=0).stop 2024-09-17T03:21:21.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 -- 172.21.15.46:0/2294164293 shutdown_connections 2024-09-17T03:21:21.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 --2- 172.21.15.46:0/2294164293 >> v2:172.21.15.46:3300/0 conn(0x7efebc0b3890 0x7efebc0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 --2- 172.21.15.46:0/2294164293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc0cf620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 -- 172.21.15.46:0/2294164293 >> 172.21.15.46:0/2294164293 conn(0x7efebc06e1d0 msgr2=0x7efebc06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 -- 172.21.15.46:0/2294164293 shutdown_connections 2024-09-17T03:21:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.390+0000 7efec3000640 1 -- 172.21.15.46:0/2294164293 wait complete. 2024-09-17T03:21:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.394+0000 7efec3000640 1 Processor -- start 2024-09-17T03:21:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.394+0000 7efec3000640 1 -- start start 2024-09-17T03:21:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 -- 172.21.15.46:0/2280158471 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d29e0 msgr2=0x7f3f940d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/2280158471 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d29e0 0x7f3f940d37c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009c80 tx=0x7f3f8c01cda0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 -- 172.21.15.46:0/2280158471 shutdown_connections 2024-09-17T03:21:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/2280158471 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d29e0 0x7f3f940d37c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/2280158471 >> v2:172.21.15.99:3300/0 conn(0x7f3f940d16b0 0x7f3f940d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 -- 172.21.15.46:0/2280158471 >> 172.21.15.46:0/2280158471 conn(0x7f3f94073e00 msgr2=0x7f3f9406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec3000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 0x7efebc152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efebc0cd7a0 0x7efebc1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 -- 172.21.15.46:0/2280158471 shutdown_connections 2024-09-17T03:21:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7f3f9ae00640 1 -- 172.21.15.46:0/2280158471 wait complete. 2024-09-17T03:21:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efebbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 0x7efebc152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efebbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 0x7efebc152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56980/0 (socket says 172.21.15.46:56980) 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efebbe00640 1 -- 172.21.15.46:0/1007056166 learned_addr learned my addr 172.21.15.46:0/1007056166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efebc0ccec0 con 0x7efebc0cd7a0 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efebc0ccd40 con 0x7efebc0cf210 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec1000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.398+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7efebc0cd040 con 0x7efebc0b3890 2024-09-17T03:21:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efebbe00640 1 -- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 msgr2=0x7efebc151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efebbe00640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efebbe00640 1 -- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efebc0cd7a0 msgr2=0x7efebc1516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efebbe00640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efebc0cd7a0 0x7efebc1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efebbe00640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1630f0 con 0x7efebc0b3890 2024-09-17T03:21:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efec1000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efebbe00640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 0x7efebc152c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efea400c460 tx=0x7efea400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7f3f9ae00640 1 Processor -- start 2024-09-17T03:21:21.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7f3f9ae00640 1 -- start start 2024-09-17T03:21:21.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4017070 con 0x7efebc0b3890 2024-09-17T03:21:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc153700 con 0x7efebc0b3890 2024-09-17T03:21:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc154990 con 0x7efebc0b3890 2024-09-17T03:21:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efea4003600 con 0x7efebc0b3890 2024-09-17T03:21:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.402+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4017070 con 0x7efebc0b3890 2024-09-17T03:21:21.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc0b6080 con 0x7efebc0b3890 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f9ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 0x7f3f941f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f9ae00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 0x7f3f940b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 0x7f3f941f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 0x7f3f941f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38782/0 (socket says 172.21.15.46:38782) 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f9ae00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f940d29e0 0x7f3f940b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f93400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 0x7f3f940b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.406+0000 7f3f93400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 0x7f3f940b61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56984/0 (socket says 172.21.15.46:56984) 2024-09-17T03:21:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93e00640 1 -- 172.21.15.46:0/1292908414 learned_addr learned my addr 172.21.15.46:0/1292908414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9416c0e0 con 0x7f3f940d16b0 2024-09-17T03:21:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9416bf60 con 0x7f3f940d29e0 2024-09-17T03:21:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9416c260 con 0x7f3f940d25d0 2024-09-17T03:21:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93400640 1 -- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f940d29e0 msgr2=0x7f3f940b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f98e00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f940d29e0 0x7f3f940b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93400640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f940d29e0 0x7f3f940b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93400640 1 -- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 msgr2=0x7f3f941f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93400640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 0x7f3f941f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93400640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f940b9b00 con 0x7f3f940d25d0 2024-09-17T03:21:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f98e00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f940d29e0 0x7f3f940b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93e00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 0x7f3f941f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f93400640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 0x7f3f940b61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400d680 tx=0x7f3f8400db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84017070 con 0x7f3f940d25d0 2024-09-17T03:21:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efea401a070 con 0x7efebc0b3890 2024-09-17T03:21:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.410+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f84005310 con 0x7f3f940d25d0 2024-09-17T03:21:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7efeb8c00640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe98073430 0x7efe98075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efea4093300 con 0x7efebc0b3890 2024-09-17T03:21:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84009820 con 0x7f3f940d25d0 2024-09-17T03:21:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f941f9970 con 0x7f3f940d25d0 2024-09-17T03:21:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efea4093780 con 0x7efebc0b3890 2024-09-17T03:21:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f940b5420 con 0x7f3f940d25d0 2024-09-17T03:21:21.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.414+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f940d0fa0 con 0x7f3f940d25d0 2024-09-17T03:21:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.418+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3f840099c0 con 0x7f3f940d25d0 2024-09-17T03:21:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.422+0000 7efeb3400640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe98073430 0x7efe98075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.422+0000 7f3f8be00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f74073230 0x7f3f74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.422+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3f84094260 con 0x7f3f940d25d0 2024-09-17T03:21:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.422+0000 7f3f93e00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f74073230 0x7f3f74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.422+0000 7f3f93e00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f74073230 0x7f3f74075740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f80001bc0 tx=0x7f3f80009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.422+0000 7efeb3400640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe98073430 0x7efe98075940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efeac0af000 tx=0x7efeac0094a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.430+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f8405eb90 con 0x7f3f940d25d0 2024-09-17T03:21:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.638+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4870007390 con 0x7f48700337d0 2024-09-17T03:21:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.638+0000 7f486ea00640 1 -- 172.21.15.46:0/2541934075 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f486405e8e0 con 0x7f48700337d0 2024-09-17T03:21:21.642 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4858073200 msgr2=0x7f4858075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4858073200 0x7f4858075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f487807b410 tx=0x7f48780969a0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48700337d0 msgr2=0x7f48700dba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48700337d0 0x7f48700dba40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f486400b410 tx=0x7f486400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 shutdown_connections 2024-09-17T03:21:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4858073200 0x7f4858075710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48700337d0 0x7f48700dba40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4870006270 0x7f48700db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 --2- 172.21.15.46:0/2541934075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4870005e60 0x7f4870181bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 >> 172.21.15.46:0/2541934075 conn(0x7f487001d9e0 msgr2=0x7f4870005440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 shutdown_connections 2024-09-17T03:21:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.642+0000 7f4863e00640 1 -- 172.21.15.46:0/2541934075 wait complete. 2024-09-17T03:21:21.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-17T03:21:21.679 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.706+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f28040b56e0 con 0x7f280414f400 2024-09-17T03:21:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f2801600640 1 -- 172.21.15.46:0/4185667588 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27f805f0d0 con 0x7f280414f400 2024-09-17T03:21:21.715 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27e4073200 msgr2=0x7f27e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27e4073200 0x7f27e4075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27f0001ec0 tx=0x7f27ec005840 comp rx=0 tx=0).stop 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 msgr2=0x7f2804157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 0x7f2804157370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d680 tx=0x7f27f800db40 comp rx=0 tx=0).stop 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 shutdown_connections 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27e4073200 0x7f27e4075710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f280415ea30 0x7f28041578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280414f400 0x7f2804157370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 --2- 172.21.15.46:0/4185667588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27ec003a10 0x7f2804146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.710+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 >> 172.21.15.46:0/4185667588 conn(0x7f28040b35f0 msgr2=0x7f280415f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.714+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 shutdown_connections 2024-09-17T03:21:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.714+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f795406dfd0 con 0x7f79540d1140 2024-09-17T03:21:21.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.714+0000 7f280c600640 1 -- 172.21.15.46:0/4185667588 wait complete. 2024-09-17T03:21:21.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.714+0000 7f7951600640 1 -- 172.21.15.46:0/1705418454 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f794805f430 con 0x7f79540d1140 2024-09-17T03:21:21.718 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:21.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7934073200 msgr2=0x7f7934075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7934073200 0x7f7934075710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f79400018b0 tx=0x7f7940009040 comp rx=0 tx=0).stop 2024-09-17T03:21:21.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 msgr2=0x7f7954164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 0x7f7954164800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f794800e9a0 tx=0x7f794800ee60 comp rx=0 tx=0).stop 2024-09-17T03:21:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 shutdown_connections 2024-09-17T03:21:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7934073200 0x7f7934075710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f795415f5c0 0x7f7954164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f79540d1140 0x7f7954164800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 --2- 172.21.15.46:0/1705418454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f795406dbc0 0x7f795409a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 >> 172.21.15.46:0/1705418454 conn(0x7f79540b41f0 msgr2=0x7f79540b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 shutdown_connections 2024-09-17T03:21:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.718+0000 7f795aa00640 1 -- 172.21.15.46:0/1705418454 wait complete. 2024-09-17T03:21:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.734+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f60c00ce4e0 con 0x7f60a8004810 2024-09-17T03:21:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.738+0000 7f60bd600640 1 -- 172.21.15.46:0/1583439140 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f60b005e980 con 0x7f60a8004810 2024-09-17T03:21:21.743 INFO:teuthology.orchestra.run.smithi046.stdout:60129542152 2024-09-17T03:21:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.738+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8d9c005860 con 0x7f8db8007000 2024-09-17T03:21:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f8db3e00640 1 -- 172.21.15.46:0/245935490 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8dc00e3820 con 0x7f8db8007000 2024-09-17T03:21:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f60a4073220 msgr2=0x7f60a4075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f60a4073220 0x7f60a4075730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60b40b0000 tx=0x7f60b401c4c0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 msgr2=0x7f60c0143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60c0143120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60b0001ec0 tx=0x7f60b000c870 comp rx=0 tx=0).stop 2024-09-17T03:21:21.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f32b0154dd0 con 0x7f32b006cdf0 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 shutdown_connections 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f60a4073220 0x7f60a4075730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60c0155bf0 0x7f60c00b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60c01557e0 0x7f60c00b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 --2- 172.21.15.46:0/1583439140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a8004810 0x7f60c0143120 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 >> 172.21.15.46:0/1583439140 conn(0x7f60c006d1f0 msgr2=0x7f60c006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8da00733e0 msgr2=0x7f8da00758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f8dc5400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8da00733e0 0x7f8da00758f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8db4007f50 tx=0x7f8db4007ee0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.742+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 shutdown_connections 2024-09-17T03:21:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f60c6a00640 1 -- 172.21.15.46:0/1583439140 wait complete. 2024-09-17T03:21:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 msgr2=0x7f8db80dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 0x7f8db80dcb80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0075a10 tx=0x7f8dc006b030 comp rx=0 tx=0).stop 2024-09-17T03:21:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f32a6000640 1 -- 172.21.15.46:0/1922383223 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32a005df00 con 0x7f32b006cdf0 2024-09-17T03:21:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 shutdown_connections 2024-09-17T03:21:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8da00733e0 0x7f8da00758f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8db80352b0 0x7f8db8005390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8db8031b30 0x7f8db8004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 --2- 172.21.15.46:0/245935490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8db8007000 0x7f8db80dcb80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 >> 172.21.15.46:0/245935490 conn(0x7f8db801e240 msgr2=0x7f8db80356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.751 INFO:teuthology.orchestra.run.smithi046.stdout:64424509448 2024-09-17T03:21:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 shutdown_connections 2024-09-17T03:21:21.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.746+0000 7f8dc5400640 1 -- 172.21.15.46:0/245935490 wait complete. 2024-09-17T03:21:21.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-17T03:21:21.753 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-17T03:21:21.754 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3290073250 msgr2=0x7f3290075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3290073250 0x7f3290075760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f329c0af000 tx=0x7f329c00b040 comp rx=0 tx=0).stop 2024-09-17T03:21:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 msgr2=0x7f32b014d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 0x7f32b014d3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c3f0 tx=0x7f32a000c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 shutdown_connections 2024-09-17T03:21:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3290073250 0x7f3290075760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b0150200 0x7f32b0150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32b00ae1e0 0x7f32b0201110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 --2- 172.21.15.46:0/1922383223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b006cdf0 0x7f32b014d3d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 >> 172.21.15.46:0/1922383223 conn(0x7f32b0073450 msgr2=0x7f32b0070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 shutdown_connections 2024-09-17T03:21:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.750+0000 7f32b5000640 1 -- 172.21.15.46:0/1922383223 wait complete. 2024-09-17T03:21:21.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.1 2024-09-17T03:21:21.775 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3f940cf490 con 0x7f3f940d25d0 2024-09-17T03:21:21.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-17T03:21:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f8be00640 1 -- 172.21.15.46:0/1292908414 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f840635c0 con 0x7f3f940d25d0 2024-09-17T03:21:21.778 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-09-17T03:21:21.778 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f74073230 msgr2=0x7f3f74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f74073230 0x7f3f74075740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f80001bc0 tx=0x7f3f80009040 comp rx=0 tx=0).stop 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 msgr2=0x7f3f940b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 0x7f3f940b61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400d680 tx=0x7f3f8400db40 comp rx=0 tx=0).stop 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 shutdown_connections 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f74073230 0x7f3f74075740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f940d29e0 0x7f3f940b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f940d25d0 0x7f3f940b61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 --2- 172.21.15.46:0/1292908414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f940d16b0 0x7f3f941f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 >> 172.21.15.46:0/1292908414 conn(0x7f3f94073e00 msgr2=0x7f3f940ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 shutdown_connections 2024-09-17T03:21:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.774+0000 7f3f9ae00640 1 -- 172.21.15.46:0/1292908414 wait complete. 2024-09-17T03:21:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.782+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efebc06b250 con 0x7efebc0b3890 2024-09-17T03:21:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efeb8c00640 1 -- 172.21.15.46:0/1007056166 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efea405de00 con 0x7efebc0b3890 2024-09-17T03:21:21.788 INFO:teuthology.orchestra.run.smithi046.stdout:60129542152 2024-09-17T03:21:21.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-09-17T03:21:21.789 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe98073430 msgr2=0x7efe98075940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe98073430 0x7efe98075940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efeac0af000 tx=0x7efeac0094a0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 msgr2=0x7efebc152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 0x7efebc152c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efea400c460 tx=0x7efea400c920 comp rx=0 tx=0).stop 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 shutdown_connections 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efe98073430 0x7efe98075940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efebc0cf210 0x7efebc151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efebc0cd7a0 0x7efebc1516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 --2- 172.21.15.46:0/1007056166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efebc0b3890 0x7efebc152c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 >> 172.21.15.46:0/1007056166 conn(0x7efebc06e1d0 msgr2=0x7efebc0b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 shutdown_connections 2024-09-17T03:21:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:21.786+0000 7efec3000640 1 -- 172.21.15.46:0/1007056166 wait complete. 2024-09-17T03:21:21.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-17T03:21:21.801 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.5 2024-09-17T03:21:21.813 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:21.814 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:21:21.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.014+0000 7f2156a00640 1 -- 172.21.15.46:0/1889176939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150157030 msgr2=0x7f21500b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.014+0000 7f2156a00640 1 --2- 172.21.15.46:0/1889176939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150157030 0x7f21500b5a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2140009840 tx=0x7f214001c620 comp rx=0 tx=0).stop 2024-09-17T03:21:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.014+0000 7f2156a00640 1 -- 172.21.15.46:0/1889176939 shutdown_connections 2024-09-17T03:21:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.014+0000 7f2156a00640 1 --2- 172.21.15.46:0/1889176939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150157030 0x7f21500b5a80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.014+0000 7f2156a00640 1 -- 172.21.15.46:0/1889176939 >> 172.21.15.46:0/1889176939 conn(0x7f21500708c0 msgr2=0x7f2150070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:22.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 -- 172.21.15.46:0/1889176939 shutdown_connections 2024-09-17T03:21:22.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 -- 172.21.15.46:0/1889176939 wait complete. 2024-09-17T03:21:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 Processor -- start 2024-09-17T03:21:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 -- start start 2024-09-17T03:21:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2150157030 0x7f21501f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 0x7f215014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f215014f840 0x7f215013ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f215015d820 con 0x7f2150160d50 2024-09-17T03:21:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f215015d6a0 con 0x7f215014f840 2024-09-17T03:21:22.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f2156a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f215015d9a0 con 0x7f2150157030 2024-09-17T03:21:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.018+0000 7f214f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 0x7f215014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 0x7f215014e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38804/0 (socket says 172.21.15.46:38804) 2024-09-17T03:21:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 -- 172.21.15.46:0/783537519 learned_addr learned my addr 172.21.15.46:0/783537519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214fe00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2150157030 0x7f21501f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2154a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f215014f840 0x7f215013ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 -- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2150157030 msgr2=0x7f21501f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2150157030 0x7f21501f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 -- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f215014f840 msgr2=0x7f215013ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f215014f840 0x7f215013ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 -- 172.21.15.46:0/783537519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21501f6a30 con 0x7f2150160d50 2024-09-17T03:21:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2154a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f215014f840 0x7f215013ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f214f400640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 0x7f215014e1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f213c00fa70 tx=0x7f213c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c01a070 con 0x7f2150160d50 2024-09-17T03:21:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f213c008570 con 0x7f2150160d50 2024-09-17T03:21:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c00d9a0 con 0x7f2150160d50 2024-09-17T03:21:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21501f6d10 con 0x7f2150160d50 2024-09-17T03:21:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215014e6e0 con 0x7f2150160d50 2024-09-17T03:21:22.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21501f67a0 con 0x7f2150160d50 2024-09-17T03:21:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.022+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f213c01d070 con 0x7f2150160d50 2024-09-17T03:21:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.026+0000 7f2147e00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2130073200 0x7f2130075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.026+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f213c096ad0 con 0x7f2150160d50 2024-09-17T03:21:22.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.026+0000 7f214fe00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2130073200 0x7f2130075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.026+0000 7f214fe00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2130073200 0x7f2130075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21400af000 tx=0x7f21400b2040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:22.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.030+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f213c061490 con 0x7f2150160d50 2024-09-17T03:21:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.202+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f215006b540 con 0x7f2130073200 2024-09-17T03:21:22.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.206+0000 7f2147e00640 1 -- 172.21.15.46:0/783537519 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f21400a6920 con 0x7f2130073200 2024-09-17T03:21:22.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:22.215 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:22.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2130073200 msgr2=0x7f2130075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2130073200 0x7f2130075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21400af000 tx=0x7f21400b2040 comp rx=0 tx=0).stop 2024-09-17T03:21:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 msgr2=0x7f215014e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 0x7f215014e1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f213c00fa70 tx=0x7f213c00ff30 comp rx=0 tx=0).stop 2024-09-17T03:21:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 shutdown_connections 2024-09-17T03:21:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2130073200 0x7f2130075710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f215014f840 0x7f215013ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2150160d50 0x7f215014e1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 --2- 172.21.15.46:0/783537519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2150157030 0x7f21501f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 >> 172.21.15.46:0/783537519 conn(0x7f21500708c0 msgr2=0x7f21500d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:22.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 shutdown_connections 2024-09-17T03:21:22.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.210+0000 7f2156a00640 1 -- 172.21.15.46:0/783537519 wait complete. 2024-09-17T03:21:22.233 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T03:21:21.202262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235756,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738964,"statfs":{"total":773094113280,"available":772852699136,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006682"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:08.073787+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:08.073460+0000","last_became_peered":"2024-09-17T03:21:08.073460+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:56.409644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034270+0000","last_change":"2024-09-17T03:21:08.046022+0000","last_active":"2024-09-17T03:21:09.034270+0000","last_peered":"2024-09-17T03:21:09.034270+0000","last_clean":"2024-09-17T03:21:09.034270+0000","last_became_active":"2024-09-17T03:21:08.045462+0000","last_became_peered":"2024-09-17T03:21:08.045462+0000","last_unstale":"2024-09-17T03:21:09.034270+0000","last_undegraded":"2024-09-17T03:21:09.034270+0000","last_fullsized":"2024-09-17T03:21:09.034270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405803+0000","last_change":"2024-09-17T03:21:08.057050+0000","last_active":"2024-09-17T03:21:10.405803+0000","last_peered":"2024-09-17T03:21:10.405803+0000","last_clean":"2024-09-17T03:21:10.405803+0000","last_became_active":"2024-09-17T03:21:08.056744+0000","last_became_peered":"2024-09-17T03:21:08.056744+0000","last_unstale":"2024-09-17T03:21:10.405803+0000","last_undegraded":"2024-09-17T03:21:10.405803+0000","last_fullsized":"2024-09-17T03:21:10.405803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:52.233846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405049+0000","last_change":"2024-09-17T03:21:08.072759+0000","last_active":"2024-09-17T03:21:10.405049+0000","last_peered":"2024-09-17T03:21:10.405049+0000","last_clean":"2024-09-17T03:21:10.405049+0000","last_became_active":"2024-09-17T03:21:08.072444+0000","last_became_peered":"2024-09-17T03:21:08.072444+0000","last_unstale":"2024-09-17T03:21:10.405049+0000","last_undegraded":"2024-09-17T03:21:10.405049+0000","last_fullsized":"2024-09-17T03:21:10.405049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:09.811529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036226+0000","last_change":"2024-09-17T03:21:08.045612+0000","last_active":"2024-09-17T03:21:09.036226+0000","last_peered":"2024-09-17T03:21:09.036226+0000","last_clean":"2024-09-17T03:21:09.036226+0000","last_became_active":"2024-09-17T03:21:08.044072+0000","last_became_peered":"2024-09-17T03:21:08.044072+0000","last_unstale":"2024-09-17T03:21:09.036226+0000","last_undegraded":"2024-09-17T03:21:09.036226+0000","last_fullsized":"2024-09-17T03:21:09.036226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406443+0000","last_change":"2024-09-17T03:21:08.060425+0000","last_active":"2024-09-17T03:21:10.406443+0000","last_peered":"2024-09-17T03:21:10.406443+0000","last_clean":"2024-09-17T03:21:10.406443+0000","last_became_active":"2024-09-17T03:21:08.060122+0000","last_became_peered":"2024-09-17T03:21:08.060122+0000","last_unstale":"2024-09-17T03:21:10.406443+0000","last_undegraded":"2024-09-17T03:21:10.406443+0000","last_fullsized":"2024-09-17T03:21:10.406443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:49.768654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075521+0000","last_change":"2024-09-17T03:21:08.067905+0000","last_active":"2024-09-17T03:21:09.075521+0000","last_peered":"2024-09-17T03:21:09.075521+0000","last_clean":"2024-09-17T03:21:09.075521+0000","last_became_active":"2024-09-17T03:21:08.066280+0000","last_became_peered":"2024-09-17T03:21:08.066280+0000","last_unstale":"2024-09-17T03:21:09.075521+0000","last_undegraded":"2024-09-17T03:21:09.075521+0000","last_fullsized":"2024-09-17T03:21:09.075521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075384+0000","last_change":"2024-09-17T03:21:08.065069+0000","last_active":"2024-09-17T03:21:09.075384+0000","last_peered":"2024-09-17T03:21:09.075384+0000","last_clean":"2024-09-17T03:21:09.075384+0000","last_became_active":"2024-09-17T03:21:08.064712+0000","last_became_peered":"2024-09-17T03:21:08.064712+0000","last_unstale":"2024-09-17T03:21:09.075384+0000","last_undegraded":"2024-09-17T03:21:09.075384+0000","last_fullsized":"2024-09-17T03:21:09.075384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037730+0000","last_change":"2024-09-17T03:21:08.071508+0000","last_active":"2024-09-17T03:21:09.037730+0000","last_peered":"2024-09-17T03:21:09.037730+0000","last_clean":"2024-09-17T03:21:09.037730+0000","last_became_active":"2024-09-17T03:21:08.071167+0000","last_became_peered":"2024-09-17T03:21:08.071167+0000","last_unstale":"2024-09-17T03:21:09.037730+0000","last_undegraded":"2024-09-17T03:21:09.037730+0000","last_fullsized":"2024-09-17T03:21:09.037730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:05.628131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036578+0000","last_change":"2024-09-17T03:21:08.066995+0000","last_active":"2024-09-17T03:21:09.036578+0000","last_peered":"2024-09-17T03:21:09.036578+0000","last_clean":"2024-09-17T03:21:09.036578+0000","last_became_active":"2024-09-17T03:21:08.066620+0000","last_became_peered":"2024-09-17T03:21:08.066620+0000","last_unstale":"2024-09-17T03:21:09.036578+0000","last_undegraded":"2024-09-17T03:21:09.036578+0000","last_fullsized":"2024-09-17T03:21:09.036578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:53.398923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034599+0000","last_change":"2024-09-17T03:21:08.060591+0000","last_active":"2024-09-17T03:21:09.034599+0000","last_peered":"2024-09-17T03:21:09.034599+0000","last_clean":"2024-09-17T03:21:09.034599+0000","last_became_active":"2024-09-17T03:21:08.056048+0000","last_became_peered":"2024-09-17T03:21:08.056048+0000","last_unstale":"2024-09-17T03:21:09.034599+0000","last_undegraded":"2024-09-17T03:21:09.034599+0000","last_fullsized":"2024-09-17T03:21:09.034599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402958+0000","last_change":"2024-09-17T03:21:08.068237+0000","last_active":"2024-09-17T03:21:10.402958+0000","last_peered":"2024-09-17T03:21:10.402958+0000","last_clean":"2024-09-17T03:21:10.402958+0000","last_became_active":"2024-09-17T03:21:08.067863+0000","last_became_peered":"2024-09-17T03:21:08.067863+0000","last_unstale":"2024-09-17T03:21:10.402958+0000","last_undegraded":"2024-09-17T03:21:10.402958+0000","last_fullsized":"2024-09-17T03:21:10.402958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:18.563998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406412+0000","last_change":"2024-09-17T03:21:08.060741+0000","last_active":"2024-09-17T03:21:10.406412+0000","last_peered":"2024-09-17T03:21:10.406412+0000","last_clean":"2024-09-17T03:21:10.406412+0000","last_became_active":"2024-09-17T03:21:08.060429+0000","last_became_peered":"2024-09-17T03:21:08.060429+0000","last_unstale":"2024-09-17T03:21:10.406412+0000","last_undegraded":"2024-09-17T03:21:10.406412+0000","last_fullsized":"2024-09-17T03:21:10.406412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:20.270467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075420+0000","last_change":"2024-09-17T03:21:08.072066+0000","last_active":"2024-09-17T03:21:09.075420+0000","last_peered":"2024-09-17T03:21:09.075420+0000","last_clean":"2024-09-17T03:21:09.075420+0000","last_became_active":"2024-09-17T03:21:08.070966+0000","last_became_peered":"2024-09-17T03:21:08.070966+0000","last_unstale":"2024-09-17T03:21:09.075420+0000","last_undegraded":"2024-09-17T03:21:09.075420+0000","last_fullsized":"2024-09-17T03:21:09.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034202+0000","last_change":"2024-09-17T03:21:08.049953+0000","last_active":"2024-09-17T03:21:09.034202+0000","last_peered":"2024-09-17T03:21:09.034202+0000","last_clean":"2024-09-17T03:21:09.034202+0000","last_became_active":"2024-09-17T03:21:08.049526+0000","last_became_peered":"2024-09-17T03:21:08.049526+0000","last_unstale":"2024-09-17T03:21:09.034202+0000","last_undegraded":"2024-09-17T03:21:09.034202+0000","last_fullsized":"2024-09-17T03:21:09.034202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:59.117173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036398+0000","last_change":"2024-09-17T03:21:08.051667+0000","last_active":"2024-09-17T03:21:09.036398+0000","last_peered":"2024-09-17T03:21:09.036398+0000","last_clean":"2024-09-17T03:21:09.036398+0000","last_became_active":"2024-09-17T03:21:08.051099+0000","last_became_peered":"2024-09-17T03:21:08.051099+0000","last_unstale":"2024-09-17T03:21:09.036398+0000","last_undegraded":"2024-09-17T03:21:09.036398+0000","last_fullsized":"2024-09-17T03:21:09.036398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:26.245059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037066+0000","last_change":"2024-09-17T03:21:08.052647+0000","last_active":"2024-09-17T03:21:09.037066+0000","last_peered":"2024-09-17T03:21:09.037066+0000","last_clean":"2024-09-17T03:21:09.037066+0000","last_became_active":"2024-09-17T03:21:08.052328+0000","last_became_peered":"2024-09-17T03:21:08.052328+0000","last_unstale":"2024-09-17T03:21:09.037066+0000","last_undegraded":"2024-09-17T03:21:09.037066+0000","last_fullsized":"2024-09-17T03:21:09.037066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:48.836360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075599+0000","last_change":"2024-09-17T03:21:08.068699+0000","last_active":"2024-09-17T03:21:09.075599+0000","last_peered":"2024-09-17T03:21:09.075599+0000","last_clean":"2024-09-17T03:21:09.075599+0000","last_became_active":"2024-09-17T03:21:08.068338+0000","last_became_peered":"2024-09-17T03:21:08.068338+0000","last_unstale":"2024-09-17T03:21:09.075599+0000","last_undegraded":"2024-09-17T03:21:09.075599+0000","last_fullsized":"2024-09-17T03:21:09.075599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036872+0000","last_change":"2024-09-17T03:21:08.059892+0000","last_active":"2024-09-17T03:21:09.036872+0000","last_peered":"2024-09-17T03:21:09.036872+0000","last_clean":"2024-09-17T03:21:09.036872+0000","last_became_active":"2024-09-17T03:21:08.059568+0000","last_became_peered":"2024-09-17T03:21:08.059568+0000","last_unstale":"2024-09-17T03:21:09.036872+0000","last_undegraded":"2024-09-17T03:21:09.036872+0000","last_fullsized":"2024-09-17T03:21:09.036872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:03.752767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034157+0000","last_change":"2024-09-17T03:21:08.049305+0000","last_active":"2024-09-17T03:21:09.034157+0000","last_peered":"2024-09-17T03:21:09.034157+0000","last_clean":"2024-09-17T03:21:09.034157+0000","last_became_active":"2024-09-17T03:21:08.048911+0000","last_became_peered":"2024-09-17T03:21:08.048911+0000","last_unstale":"2024-09-17T03:21:09.034157+0000","last_undegraded":"2024-09-17T03:21:09.034157+0000","last_fullsized":"2024-09-17T03:21:09.034157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402687+0000","last_change":"2024-09-17T03:21:08.060288+0000","last_active":"2024-09-17T03:21:10.402687+0000","last_peered":"2024-09-17T03:21:10.402687+0000","last_clean":"2024-09-17T03:21:10.402687+0000","last_became_active":"2024-09-17T03:21:08.059954+0000","last_became_peered":"2024-09-17T03:21:08.059954+0000","last_unstale":"2024-09-17T03:21:10.402687+0000","last_undegraded":"2024-09-17T03:21:10.402687+0000","last_fullsized":"2024-09-17T03:21:10.402687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075416+0000","last_change":"2024-09-17T03:21:08.067449+0000","last_active":"2024-09-17T03:21:09.075416+0000","last_peered":"2024-09-17T03:21:09.075416+0000","last_clean":"2024-09-17T03:21:09.075416+0000","last_became_active":"2024-09-17T03:21:08.066649+0000","last_became_peered":"2024-09-17T03:21:08.066649+0000","last_unstale":"2024-09-17T03:21:09.075416+0000","last_undegraded":"2024-09-17T03:21:09.075416+0000","last_fullsized":"2024-09-17T03:21:09.075416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037818+0000","last_change":"2024-09-17T03:21:08.055200+0000","last_active":"2024-09-17T03:21:09.037818+0000","last_peered":"2024-09-17T03:21:09.037818+0000","last_clean":"2024-09-17T03:21:09.037818+0000","last_became_active":"2024-09-17T03:21:08.054451+0000","last_became_peered":"2024-09-17T03:21:08.054451+0000","last_unstale":"2024-09-17T03:21:09.037818+0000","last_undegraded":"2024-09-17T03:21:09.037818+0000","last_fullsized":"2024-09-17T03:21:09.037818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:41.720407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036639+0000","last_change":"2024-09-17T03:21:07.042857+0000","last_active":"2024-09-17T03:21:09.036639+0000","last_peered":"2024-09-17T03:21:09.036639+0000","last_clean":"2024-09-17T03:21:09.036639+0000","last_became_active":"2024-09-17T03:21:07.042659+0000","last_became_peered":"2024-09-17T03:21:07.042659+0000","last_unstale":"2024-09-17T03:21:09.036639+0000","last_undegraded":"2024-09-17T03:21:09.036639+0000","last_fullsized":"2024-09-17T03:21:09.036639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034320+0000","last_change":"2024-09-17T03:21:08.054380+0000","last_active":"2024-09-17T03:21:09.034320+0000","last_peered":"2024-09-17T03:21:09.034320+0000","last_clean":"2024-09-17T03:21:09.034320+0000","last_became_active":"2024-09-17T03:21:08.053967+0000","last_became_peered":"2024-09-17T03:21:08.053967+0000","last_unstale":"2024-09-17T03:21:09.034320+0000","last_undegraded":"2024-09-17T03:21:09.034320+0000","last_fullsized":"2024-09-17T03:21:09.034320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074976+0000","last_change":"2024-09-17T03:21:07.037433+0000","last_active":"2024-09-17T03:21:09.074976+0000","last_peered":"2024-09-17T03:21:09.074976+0000","last_clean":"2024-09-17T03:21:09.074976+0000","last_became_active":"2024-09-17T03:21:07.035663+0000","last_became_peered":"2024-09-17T03:21:07.035663+0000","last_unstale":"2024-09-17T03:21:09.074976+0000","last_undegraded":"2024-09-17T03:21:09.074976+0000","last_fullsized":"2024-09-17T03:21:09.074976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036177+0000","last_change":"2024-09-17T03:21:07.033856+0000","last_active":"2024-09-17T03:21:09.036177+0000","last_peered":"2024-09-17T03:21:09.036177+0000","last_clean":"2024-09-17T03:21:09.036177+0000","last_became_active":"2024-09-17T03:21:07.032761+0000","last_became_peered":"2024-09-17T03:21:07.032761+0000","last_unstale":"2024-09-17T03:21:09.036177+0000","last_undegraded":"2024-09-17T03:21:09.036177+0000","last_fullsized":"2024-09-17T03:21:09.036177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036275+0000","last_change":"2024-09-17T03:21:08.046160+0000","last_active":"2024-09-17T03:21:09.036275+0000","last_peered":"2024-09-17T03:21:09.036275+0000","last_clean":"2024-09-17T03:21:09.036275+0000","last_became_active":"2024-09-17T03:21:08.045937+0000","last_became_peered":"2024-09-17T03:21:08.045937+0000","last_unstale":"2024-09-17T03:21:09.036275+0000","last_undegraded":"2024-09-17T03:21:09.036275+0000","last_fullsized":"2024-09-17T03:21:09.036275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:13.549239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075195+0000","last_change":"2024-09-17T03:21:07.042752+0000","last_active":"2024-09-17T03:21:09.075195+0000","last_peered":"2024-09-17T03:21:09.075195+0000","last_clean":"2024-09-17T03:21:09.075195+0000","last_became_active":"2024-09-17T03:21:07.042359+0000","last_became_peered":"2024-09-17T03:21:07.042359+0000","last_unstale":"2024-09-17T03:21:09.075195+0000","last_undegraded":"2024-09-17T03:21:09.075195+0000","last_fullsized":"2024-09-17T03:21:09.075195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034735+0000","last_change":"2024-09-17T03:21:08.060578+0000","last_active":"2024-09-17T03:21:09.034735+0000","last_peered":"2024-09-17T03:21:09.034735+0000","last_clean":"2024-09-17T03:21:09.034735+0000","last_became_active":"2024-09-17T03:21:08.059037+0000","last_became_peered":"2024-09-17T03:21:08.059037+0000","last_unstale":"2024-09-17T03:21:09.034735+0000","last_undegraded":"2024-09-17T03:21:09.034735+0000","last_fullsized":"2024-09-17T03:21:09.034735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037963+0000","last_change":"2024-09-17T03:21:08.052908+0000","last_active":"2024-09-17T03:21:09.037963+0000","last_peered":"2024-09-17T03:21:09.037963+0000","last_clean":"2024-09-17T03:21:09.037963+0000","last_became_active":"2024-09-17T03:21:08.052578+0000","last_became_peered":"2024-09-17T03:21:08.052578+0000","last_unstale":"2024-09-17T03:21:09.037963+0000","last_undegraded":"2024-09-17T03:21:09.037963+0000","last_fullsized":"2024-09-17T03:21:09.037963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:06.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075516+0000","last_change":"2024-09-17T03:21:07.042836+0000","last_active":"2024-09-17T03:21:09.075516+0000","last_peered":"2024-09-17T03:21:09.075516+0000","last_clean":"2024-09-17T03:21:09.075516+0000","last_became_active":"2024-09-17T03:21:07.042380+0000","last_became_peered":"2024-09-17T03:21:07.042380+0000","last_unstale":"2024-09-17T03:21:09.075516+0000","last_undegraded":"2024-09-17T03:21:09.075516+0000","last_fullsized":"2024-09-17T03:21:09.075516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405970+0000","last_change":"2024-09-17T03:21:08.066084+0000","last_active":"2024-09-17T03:21:10.405970+0000","last_peered":"2024-09-17T03:21:10.405970+0000","last_clean":"2024-09-17T03:21:10.405970+0000","last_became_active":"2024-09-17T03:21:08.065807+0000","last_became_peered":"2024-09-17T03:21:08.065807+0000","last_unstale":"2024-09-17T03:21:10.405970+0000","last_undegraded":"2024-09-17T03:21:10.405970+0000","last_fullsized":"2024-09-17T03:21:10.405970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:53.782411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034281+0000","last_change":"2024-09-17T03:21:07.033053+0000","last_active":"2024-09-17T03:21:09.034281+0000","last_peered":"2024-09-17T03:21:09.034281+0000","last_clean":"2024-09-17T03:21:09.034281+0000","last_became_active":"2024-09-17T03:21:07.030140+0000","last_became_peered":"2024-09-17T03:21:07.030140+0000","last_unstale":"2024-09-17T03:21:09.034281+0000","last_undegraded":"2024-09-17T03:21:09.034281+0000","last_fullsized":"2024-09-17T03:21:09.034281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033897+0000","last_change":"2024-09-17T03:21:08.057090+0000","last_active":"2024-09-17T03:21:09.033897+0000","last_peered":"2024-09-17T03:21:09.033897+0000","last_clean":"2024-09-17T03:21:09.033897+0000","last_became_active":"2024-09-17T03:21:08.055419+0000","last_became_peered":"2024-09-17T03:21:08.055419+0000","last_unstale":"2024-09-17T03:21:09.033897+0000","last_undegraded":"2024-09-17T03:21:09.033897+0000","last_fullsized":"2024-09-17T03:21:09.033897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:54.579062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406013+0000","last_change":"2024-09-17T03:21:08.059703+0000","last_active":"2024-09-17T03:21:10.406013+0000","last_peered":"2024-09-17T03:21:10.406013+0000","last_clean":"2024-09-17T03:21:10.406013+0000","last_became_active":"2024-09-17T03:21:08.059411+0000","last_became_peered":"2024-09-17T03:21:08.059411+0000","last_unstale":"2024-09-17T03:21:10.406013+0000","last_undegraded":"2024-09-17T03:21:10.406013+0000","last_fullsized":"2024-09-17T03:21:10.406013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:00.131726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074938+0000","last_change":"2024-09-17T03:21:07.043190+0000","last_active":"2024-09-17T03:21:09.074938+0000","last_peered":"2024-09-17T03:21:09.074938+0000","last_clean":"2024-09-17T03:21:09.074938+0000","last_became_active":"2024-09-17T03:21:07.042868+0000","last_became_peered":"2024-09-17T03:21:07.042868+0000","last_unstale":"2024-09-17T03:21:09.074938+0000","last_undegraded":"2024-09-17T03:21:09.074938+0000","last_fullsized":"2024-09-17T03:21:09.074938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075453+0000","last_change":"2024-09-17T03:21:08.070393+0000","last_active":"2024-09-17T03:21:09.075453+0000","last_peered":"2024-09-17T03:21:09.075453+0000","last_clean":"2024-09-17T03:21:09.075453+0000","last_became_active":"2024-09-17T03:21:08.069695+0000","last_became_peered":"2024-09-17T03:21:08.069695+0000","last_unstale":"2024-09-17T03:21:09.075453+0000","last_undegraded":"2024-09-17T03:21:09.075453+0000","last_fullsized":"2024-09-17T03:21:09.075453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402466+0000","last_change":"2024-09-17T03:21:07.037447+0000","last_active":"2024-09-17T03:21:10.402466+0000","last_peered":"2024-09-17T03:21:10.402466+0000","last_clean":"2024-09-17T03:21:10.402466+0000","last_became_active":"2024-09-17T03:21:07.037133+0000","last_became_peered":"2024-09-17T03:21:07.037133+0000","last_unstale":"2024-09-17T03:21:10.402466+0000","last_undegraded":"2024-09-17T03:21:10.402466+0000","last_fullsized":"2024-09-17T03:21:10.402466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405541+0000","last_change":"2024-09-17T03:21:08.064992+0000","last_active":"2024-09-17T03:21:10.405541+0000","last_peered":"2024-09-17T03:21:10.405541+0000","last_clean":"2024-09-17T03:21:10.405541+0000","last_became_active":"2024-09-17T03:21:08.064705+0000","last_became_peered":"2024-09-17T03:21:08.064705+0000","last_unstale":"2024-09-17T03:21:10.405541+0000","last_undegraded":"2024-09-17T03:21:10.405541+0000","last_fullsized":"2024-09-17T03:21:10.405541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.038518+0000","last_change":"2024-09-17T03:21:07.031511+0000","last_active":"2024-09-17T03:21:09.038518+0000","last_peered":"2024-09-17T03:21:09.038518+0000","last_clean":"2024-09-17T03:21:09.038518+0000","last_became_active":"2024-09-17T03:21:07.031175+0000","last_became_peered":"2024-09-17T03:21:07.031175+0000","last_unstale":"2024-09-17T03:21:09.038518+0000","last_undegraded":"2024-09-17T03:21:09.038518+0000","last_fullsized":"2024-09-17T03:21:09.038518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:23:29.396492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406155+0000","last_change":"2024-09-17T03:21:08.067321+0000","last_active":"2024-09-17T03:21:10.406155+0000","last_peered":"2024-09-17T03:21:10.406155+0000","last_clean":"2024-09-17T03:21:10.406155+0000","last_became_active":"2024-09-17T03:21:08.066953+0000","last_became_peered":"2024-09-17T03:21:08.066953+0000","last_unstale":"2024-09-17T03:21:10.406155+0000","last_undegraded":"2024-09-17T03:21:10.406155+0000","last_fullsized":"2024-09-17T03:21:10.406155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:38.976170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402786+0000","last_change":"2024-09-17T03:21:07.042316+0000","last_active":"2024-09-17T03:21:10.402786+0000","last_peered":"2024-09-17T03:21:10.402786+0000","last_clean":"2024-09-17T03:21:10.402786+0000","last_became_active":"2024-09-17T03:21:07.039133+0000","last_became_peered":"2024-09-17T03:21:07.039133+0000","last_unstale":"2024-09-17T03:21:10.402786+0000","last_undegraded":"2024-09-17T03:21:10.402786+0000","last_fullsized":"2024-09-17T03:21:10.402786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406653+0000","last_change":"2024-09-17T03:21:08.057737+0000","last_active":"2024-09-17T03:21:10.406653+0000","last_peered":"2024-09-17T03:21:10.406653+0000","last_clean":"2024-09-17T03:21:10.406653+0000","last_became_active":"2024-09-17T03:21:08.057560+0000","last_became_peered":"2024-09-17T03:21:08.057560+0000","last_unstale":"2024-09-17T03:21:10.406653+0000","last_undegraded":"2024-09-17T03:21:10.406653+0000","last_fullsized":"2024-09-17T03:21:10.406653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:07.761806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402646+0000","last_change":"2024-09-17T03:21:07.042337+0000","last_active":"2024-09-17T03:21:10.402646+0000","last_peered":"2024-09-17T03:21:10.402646+0000","last_clean":"2024-09-17T03:21:10.402646+0000","last_became_active":"2024-09-17T03:21:07.040846+0000","last_became_peered":"2024-09-17T03:21:07.040846+0000","last_unstale":"2024-09-17T03:21:10.402646+0000","last_undegraded":"2024-09-17T03:21:10.402646+0000","last_fullsized":"2024-09-17T03:21:10.402646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406114+0000","last_change":"2024-09-17T03:21:08.074560+0000","last_active":"2024-09-17T03:21:10.406114+0000","last_peered":"2024-09-17T03:21:10.406114+0000","last_clean":"2024-09-17T03:21:10.406114+0000","last_became_active":"2024-09-17T03:21:08.074267+0000","last_became_peered":"2024-09-17T03:21:08.074267+0000","last_unstale":"2024-09-17T03:21:10.406114+0000","last_undegraded":"2024-09-17T03:21:10.406114+0000","last_fullsized":"2024-09-17T03:21:10.406114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:09.894552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404671+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:10.404671+0000","last_peered":"2024-09-17T03:21:10.404671+0000","last_clean":"2024-09-17T03:21:10.404671+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:10.404671+0000","last_undegraded":"2024-09-17T03:21:10.404671+0000","last_fullsized":"2024-09-17T03:21:10.404671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036442+0000","last_change":"2024-09-17T03:21:08.046702+0000","last_active":"2024-09-17T03:21:09.036442+0000","last_peered":"2024-09-17T03:21:09.036442+0000","last_clean":"2024-09-17T03:21:09.036442+0000","last_became_active":"2024-09-17T03:21:08.046321+0000","last_became_peered":"2024-09-17T03:21:08.046321+0000","last_unstale":"2024-09-17T03:21:09.036442+0000","last_undegraded":"2024-09-17T03:21:09.036442+0000","last_fullsized":"2024-09-17T03:21:09.036442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404543+0000","last_change":"2024-09-17T03:21:07.045789+0000","last_active":"2024-09-17T03:21:10.404543+0000","last_peered":"2024-09-17T03:21:10.404543+0000","last_clean":"2024-09-17T03:21:10.404543+0000","last_became_active":"2024-09-17T03:21:07.045443+0000","last_became_peered":"2024-09-17T03:21:07.045443+0000","last_unstale":"2024-09-17T03:21:10.404543+0000","last_undegraded":"2024-09-17T03:21:10.404543+0000","last_fullsized":"2024-09-17T03:21:10.404543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:00.822415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033526+0000","last_change":"2024-09-17T03:21:08.040522+0000","last_active":"2024-09-17T03:21:09.033526+0000","last_peered":"2024-09-17T03:21:09.033526+0000","last_clean":"2024-09-17T03:21:09.033526+0000","last_became_active":"2024-09-17T03:21:08.037656+0000","last_became_peered":"2024-09-17T03:21:08.037656+0000","last_unstale":"2024-09-17T03:21:09.033526+0000","last_undegraded":"2024-09-17T03:21:09.033526+0000","last_fullsized":"2024-09-17T03:21:09.033526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:58.143540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402609+0000","last_change":"2024-09-17T03:21:07.035825+0000","last_active":"2024-09-17T03:21:10.402609+0000","last_peered":"2024-09-17T03:21:10.402609+0000","last_clean":"2024-09-17T03:21:10.402609+0000","last_became_active":"2024-09-17T03:21:07.035573+0000","last_became_peered":"2024-09-17T03:21:07.035573+0000","last_unstale":"2024-09-17T03:21:10.402609+0000","last_undegraded":"2024-09-17T03:21:10.402609+0000","last_fullsized":"2024-09-17T03:21:10.402609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037938+0000","last_change":"2024-09-17T03:21:08.051692+0000","last_active":"2024-09-17T03:21:09.037938+0000","last_peered":"2024-09-17T03:21:09.037938+0000","last_clean":"2024-09-17T03:21:09.037938+0000","last_became_active":"2024-09-17T03:21:08.050563+0000","last_became_peered":"2024-09-17T03:21:08.050563+0000","last_unstale":"2024-09-17T03:21:09.037938+0000","last_undegraded":"2024-09-17T03:21:09.037938+0000","last_fullsized":"2024-09-17T03:21:09.037938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:07.035552+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:07.035214+0000","last_became_peered":"2024-09-17T03:21:07.035214+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.557724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037050+0000","last_change":"2024-09-17T03:21:08.059028+0000","last_active":"2024-09-17T03:21:09.037050+0000","last_peered":"2024-09-17T03:21:09.037050+0000","last_clean":"2024-09-17T03:21:09.037050+0000","last_became_active":"2024-09-17T03:21:08.058709+0000","last_became_peered":"2024-09-17T03:21:08.058709+0000","last_unstale":"2024-09-17T03:21:09.037050+0000","last_undegraded":"2024-09-17T03:21:09.037050+0000","last_fullsized":"2024-09-17T03:21:09.037050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:21.370502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075234+0000","last_change":"2024-09-17T03:21:07.033103+0000","last_active":"2024-09-17T03:21:09.075234+0000","last_peered":"2024-09-17T03:21:09.075234+0000","last_clean":"2024-09-17T03:21:09.075234+0000","last_became_active":"2024-09-17T03:21:07.032795+0000","last_became_peered":"2024-09-17T03:21:07.032795+0000","last_unstale":"2024-09-17T03:21:09.075234+0000","last_undegraded":"2024-09-17T03:21:09.075234+0000","last_fullsized":"2024-09-17T03:21:09.075234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033821+0000","last_change":"2024-09-17T03:21:08.065685+0000","last_active":"2024-09-17T03:21:09.033821+0000","last_peered":"2024-09-17T03:21:09.033821+0000","last_clean":"2024-09-17T03:21:09.033821+0000","last_became_active":"2024-09-17T03:21:08.065243+0000","last_became_peered":"2024-09-17T03:21:08.065243+0000","last_unstale":"2024-09-17T03:21:09.033821+0000","last_undegraded":"2024-09-17T03:21:09.033821+0000","last_fullsized":"2024-09-17T03:21:09.033821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:23.445102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404390+0000","last_change":"2024-09-17T03:21:07.034832+0000","last_active":"2024-09-17T03:21:10.404390+0000","last_peered":"2024-09-17T03:21:10.404390+0000","last_clean":"2024-09-17T03:21:10.404390+0000","last_became_active":"2024-09-17T03:21:07.034455+0000","last_became_peered":"2024-09-17T03:21:07.034455+0000","last_unstale":"2024-09-17T03:21:10.404390+0000","last_undegraded":"2024-09-17T03:21:10.404390+0000","last_fullsized":"2024-09-17T03:21:10.404390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037637+0000","last_change":"2024-09-17T03:21:08.046797+0000","last_active":"2024-09-17T03:21:09.037637+0000","last_peered":"2024-09-17T03:21:09.037637+0000","last_clean":"2024-09-17T03:21:09.037637+0000","last_became_active":"2024-09-17T03:21:08.046355+0000","last_became_peered":"2024-09-17T03:21:08.046355+0000","last_unstale":"2024-09-17T03:21:09.037637+0000","last_undegraded":"2024-09-17T03:21:09.037637+0000","last_fullsized":"2024-09-17T03:21:09.037637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:41.458145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036511+0000","last_change":"2024-09-17T03:21:08.060646+0000","last_active":"2024-09-17T03:21:09.036511+0000","last_peered":"2024-09-17T03:21:09.036511+0000","last_clean":"2024-09-17T03:21:09.036511+0000","last_became_active":"2024-09-17T03:21:08.060341+0000","last_became_peered":"2024-09-17T03:21:08.060341+0000","last_unstale":"2024-09-17T03:21:09.036511+0000","last_undegraded":"2024-09-17T03:21:09.036511+0000","last_fullsized":"2024-09-17T03:21:09.036511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:06.388228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036654+0000","last_change":"2024-09-17T03:21:08.045211+0000","last_active":"2024-09-17T03:21:09.036654+0000","last_peered":"2024-09-17T03:21:09.036654+0000","last_clean":"2024-09-17T03:21:09.036654+0000","last_became_active":"2024-09-17T03:21:08.043718+0000","last_became_peered":"2024-09-17T03:21:08.043718+0000","last_unstale":"2024-09-17T03:21:09.036654+0000","last_undegraded":"2024-09-17T03:21:09.036654+0000","last_fullsized":"2024-09-17T03:21:09.036654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:41.789200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406079+0000","last_change":"2024-09-17T03:21:07.038659+0000","last_active":"2024-09-17T03:21:10.406079+0000","last_peered":"2024-09-17T03:21:10.406079+0000","last_clean":"2024-09-17T03:21:10.406079+0000","last_became_active":"2024-09-17T03:21:07.038247+0000","last_became_peered":"2024-09-17T03:21:07.038247+0000","last_unstale":"2024-09-17T03:21:10.406079+0000","last_undegraded":"2024-09-17T03:21:10.406079+0000","last_fullsized":"2024-09-17T03:21:10.406079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075179+0000","last_change":"2024-09-17T03:21:08.065090+0000","last_active":"2024-09-17T03:21:09.075179+0000","last_peered":"2024-09-17T03:21:09.075179+0000","last_clean":"2024-09-17T03:21:09.075179+0000","last_became_active":"2024-09-17T03:21:08.064729+0000","last_became_peered":"2024-09-17T03:21:08.064729+0000","last_unstale":"2024-09-17T03:21:09.075179+0000","last_undegraded":"2024-09-17T03:21:09.075179+0000","last_fullsized":"2024-09-17T03:21:09.075179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405784+0000","last_change":"2024-09-17T03:21:07.049981+0000","last_active":"2024-09-17T03:21:10.405784+0000","last_peered":"2024-09-17T03:21:10.405784+0000","last_clean":"2024-09-17T03:21:10.405784+0000","last_became_active":"2024-09-17T03:21:07.049635+0000","last_became_peered":"2024-09-17T03:21:07.049635+0000","last_unstale":"2024-09-17T03:21:10.405784+0000","last_undegraded":"2024-09-17T03:21:10.405784+0000","last_fullsized":"2024-09-17T03:21:10.405784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:11.341893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406212+0000","last_change":"2024-09-17T03:21:07.038577+0000","last_active":"2024-09-17T03:21:10.406212+0000","last_peered":"2024-09-17T03:21:10.406212+0000","last_clean":"2024-09-17T03:21:10.406212+0000","last_became_active":"2024-09-17T03:21:07.038261+0000","last_became_peered":"2024-09-17T03:21:07.038261+0000","last_unstale":"2024-09-17T03:21:10.406212+0000","last_undegraded":"2024-09-17T03:21:10.406212+0000","last_fullsized":"2024-09-17T03:21:10.406212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:02.813101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034518+0000","last_change":"2024-09-17T03:21:08.061875+0000","last_active":"2024-09-17T03:21:09.034518+0000","last_peered":"2024-09-17T03:21:09.034518+0000","last_clean":"2024-09-17T03:21:09.034518+0000","last_became_active":"2024-09-17T03:21:08.057042+0000","last_became_peered":"2024-09-17T03:21:08.057042+0000","last_unstale":"2024-09-17T03:21:09.034518+0000","last_undegraded":"2024-09-17T03:21:09.034518+0000","last_fullsized":"2024-09-17T03:21:09.034518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:18.945802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036926+0000","last_change":"2024-09-17T03:21:07.037271+0000","last_active":"2024-09-17T03:21:09.036926+0000","last_peered":"2024-09-17T03:21:09.036926+0000","last_clean":"2024-09-17T03:21:09.036926+0000","last_became_active":"2024-09-17T03:21:07.036859+0000","last_became_peered":"2024-09-17T03:21:07.036859+0000","last_unstale":"2024-09-17T03:21:09.036926+0000","last_undegraded":"2024-09-17T03:21:09.036926+0000","last_fullsized":"2024-09-17T03:21:09.036926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037073+0000","last_change":"2024-09-17T03:21:08.068273+0000","last_active":"2024-09-17T03:21:09.037073+0000","last_peered":"2024-09-17T03:21:09.037073+0000","last_clean":"2024-09-17T03:21:09.037073+0000","last_became_active":"2024-09-17T03:21:08.066628+0000","last_became_peered":"2024-09-17T03:21:08.066628+0000","last_unstale":"2024-09-17T03:21:09.037073+0000","last_undegraded":"2024-09-17T03:21:09.037073+0000","last_fullsized":"2024-09-17T03:21:09.037073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034788+0000","last_change":"2024-09-17T03:21:07.032795+0000","last_active":"2024-09-17T03:21:09.034788+0000","last_peered":"2024-09-17T03:21:09.034788+0000","last_clean":"2024-09-17T03:21:09.034788+0000","last_became_active":"2024-09-17T03:21:07.032288+0000","last_became_peered":"2024-09-17T03:21:07.032288+0000","last_unstale":"2024-09-17T03:21:09.034788+0000","last_undegraded":"2024-09-17T03:21:09.034788+0000","last_fullsized":"2024-09-17T03:21:09.034788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:41.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:22.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 -- 172.21.15.46:0/3006919238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 msgr2=0x7f0b9c073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/3006919238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c073dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009900 tx=0x7f0b8c01c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 -- 172.21.15.46:0/3006919238 shutdown_connections 2024-09-17T03:21:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/3006919238 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b9c06fa70 0x7f0b9c15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/3006919238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c073dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 -- 172.21.15.46:0/3006919238 >> 172.21.15.46:0/3006919238 conn(0x7f0b9c0b2c40 msgr2=0x7f0b9c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 -- 172.21.15.46:0/3006919238 shutdown_connections 2024-09-17T03:21:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.430+0000 7f0ba1e00640 1 -- 172.21.15.46:0/3006919238 wait complete. 2024-09-17T03:21:22.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 Processor -- start 2024-09-17T03:21:22.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 -- start start 2024-09-17T03:21:22.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b9c06fa70 0x7f0b9c1fc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b9c074300 0x7f0b9c14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c0b3d30 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c15ffd0 con 0x7f0b9c06fa70 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0ba1e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9c168b10 con 0x7f0b9c074300 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c14f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37110/0 (socket says 172.21.15.46:37110) 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 -- 172.21.15.46:0/2335132009 learned_addr learned my addr 172.21.15.46:0/2335132009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9aa00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b9c06fa70 0x7f0b9c1fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b92000640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b9c074300 0x7f0b9c14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 -- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b9c074300 msgr2=0x7f0b9c14d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b9c074300 0x7f0b9c14d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 -- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b9c06fa70 msgr2=0x7f0b9c1fc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b9c06fa70 0x7f0b9c1fc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 -- 172.21.15.46:0/2335132009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c1fcd60 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.434+0000 7f0b9b400640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c14f2a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400eab0 tx=0x7f0b9400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400cea0 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b94004b30 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94005020 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c0cf970 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c0cde10 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c15d7a0 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0b940051c0 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b93e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b78073230 0x7f0b78075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0b94094d60 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.438+0000 7f0b9aa00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b78073230 0x7f0b78075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.442+0000 7f0b9aa00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b78073230 0x7f0b78075740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0af000 tx=0x7f0b8c01b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:22.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.442+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0b9c1fcd60 con 0x7f0b9c15c5d0 2024-09-17T03:21:22.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.614+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b9c06b540 con 0x7f0b78073230 2024-09-17T03:21:22.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0b93e00640 1 -- 172.21.15.46:0/2335132009 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f0b8c0fa7d0 con 0x7f0b78073230 2024-09-17T03:21:22.628 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b78073230 msgr2=0x7f0b78075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b78073230 0x7f0b78075740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0af000 tx=0x7f0b8c01b0f0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 msgr2=0x7f0b9c14f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c14f2a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400eab0 tx=0x7f0b9400ef70 comp rx=0 tx=0).stop 2024-09-17T03:21:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 shutdown_connections 2024-09-17T03:21:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b78073230 0x7f0b78075740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b9c15c5d0 0x7f0b9c14f2a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b9c074300 0x7f0b9c14d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 --2- 172.21.15.46:0/2335132009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b9c06fa70 0x7f0b9c1fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 >> 172.21.15.46:0/2335132009 conn(0x7f0b9c0b2c40 msgr2=0x7f0b9c075f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 shutdown_connections 2024-09-17T03:21:22.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.622+0000 7f0ba1e00640 1 -- 172.21.15.46:0/2335132009 wait complete. 2024-09-17T03:21:22.648 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T03:21:21.202262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235756,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738964,"statfs":{"total":773094113280,"available":772852699136,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006682"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:08.073787+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:08.073460+0000","last_became_peered":"2024-09-17T03:21:08.073460+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:56.409644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071421+0000","last_change":"2024-09-17T03:21:08.065833+0000","last_active":"2024-09-17T03:21:09.071421+0000","last_peered":"2024-09-17T03:21:09.071421+0000","last_clean":"2024-09-17T03:21:09.071421+0000","last_became_active":"2024-09-17T03:21:08.065398+0000","last_became_peered":"2024-09-17T03:21:08.065398+0000","last_unstale":"2024-09-17T03:21:09.071421+0000","last_undegraded":"2024-09-17T03:21:09.071421+0000","last_fullsized":"2024-09-17T03:21:09.071421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:03.526704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034270+0000","last_change":"2024-09-17T03:21:08.046022+0000","last_active":"2024-09-17T03:21:09.034270+0000","last_peered":"2024-09-17T03:21:09.034270+0000","last_clean":"2024-09-17T03:21:09.034270+0000","last_became_active":"2024-09-17T03:21:08.045462+0000","last_became_peered":"2024-09-17T03:21:08.045462+0000","last_unstale":"2024-09-17T03:21:09.034270+0000","last_undegraded":"2024-09-17T03:21:09.034270+0000","last_fullsized":"2024-09-17T03:21:09.034270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404853+0000","last_change":"2024-09-17T03:21:08.069491+0000","last_active":"2024-09-17T03:21:10.404853+0000","last_peered":"2024-09-17T03:21:10.404853+0000","last_clean":"2024-09-17T03:21:10.404853+0000","last_became_active":"2024-09-17T03:21:08.068818+0000","last_became_peered":"2024-09-17T03:21:08.068818+0000","last_unstale":"2024-09-17T03:21:10.404853+0000","last_undegraded":"2024-09-17T03:21:10.404853+0000","last_fullsized":"2024-09-17T03:21:10.404853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.197117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405803+0000","last_change":"2024-09-17T03:21:08.057050+0000","last_active":"2024-09-17T03:21:10.405803+0000","last_peered":"2024-09-17T03:21:10.405803+0000","last_clean":"2024-09-17T03:21:10.405803+0000","last_became_active":"2024-09-17T03:21:08.056744+0000","last_became_peered":"2024-09-17T03:21:08.056744+0000","last_unstale":"2024-09-17T03:21:10.405803+0000","last_undegraded":"2024-09-17T03:21:10.405803+0000","last_fullsized":"2024-09-17T03:21:10.405803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:52.233846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404589+0000","last_change":"2024-09-17T03:21:08.047837+0000","last_active":"2024-09-17T03:21:10.404589+0000","last_peered":"2024-09-17T03:21:10.404589+0000","last_clean":"2024-09-17T03:21:10.404589+0000","last_became_active":"2024-09-17T03:21:08.046940+0000","last_became_peered":"2024-09-17T03:21:08.046940+0000","last_unstale":"2024-09-17T03:21:10.404589+0000","last_undegraded":"2024-09-17T03:21:10.404589+0000","last_fullsized":"2024-09-17T03:21:10.404589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:04.677014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405049+0000","last_change":"2024-09-17T03:21:08.072759+0000","last_active":"2024-09-17T03:21:10.405049+0000","last_peered":"2024-09-17T03:21:10.405049+0000","last_clean":"2024-09-17T03:21:10.405049+0000","last_became_active":"2024-09-17T03:21:08.072444+0000","last_became_peered":"2024-09-17T03:21:08.072444+0000","last_unstale":"2024-09-17T03:21:10.405049+0000","last_undegraded":"2024-09-17T03:21:10.405049+0000","last_fullsized":"2024-09-17T03:21:10.405049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:09.811529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036226+0000","last_change":"2024-09-17T03:21:08.045612+0000","last_active":"2024-09-17T03:21:09.036226+0000","last_peered":"2024-09-17T03:21:09.036226+0000","last_clean":"2024-09-17T03:21:09.036226+0000","last_became_active":"2024-09-17T03:21:08.044072+0000","last_became_peered":"2024-09-17T03:21:08.044072+0000","last_unstale":"2024-09-17T03:21:09.036226+0000","last_undegraded":"2024-09-17T03:21:09.036226+0000","last_fullsized":"2024-09-17T03:21:09.036226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406443+0000","last_change":"2024-09-17T03:21:08.060425+0000","last_active":"2024-09-17T03:21:10.406443+0000","last_peered":"2024-09-17T03:21:10.406443+0000","last_clean":"2024-09-17T03:21:10.406443+0000","last_became_active":"2024-09-17T03:21:08.060122+0000","last_became_peered":"2024-09-17T03:21:08.060122+0000","last_unstale":"2024-09-17T03:21:10.406443+0000","last_undegraded":"2024-09-17T03:21:10.406443+0000","last_fullsized":"2024-09-17T03:21:10.406443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:49.768654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075521+0000","last_change":"2024-09-17T03:21:08.067905+0000","last_active":"2024-09-17T03:21:09.075521+0000","last_peered":"2024-09-17T03:21:09.075521+0000","last_clean":"2024-09-17T03:21:09.075521+0000","last_became_active":"2024-09-17T03:21:08.066280+0000","last_became_peered":"2024-09-17T03:21:08.066280+0000","last_unstale":"2024-09-17T03:21:09.075521+0000","last_undegraded":"2024-09-17T03:21:09.075521+0000","last_fullsized":"2024-09-17T03:21:09.075521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075384+0000","last_change":"2024-09-17T03:21:08.065069+0000","last_active":"2024-09-17T03:21:09.075384+0000","last_peered":"2024-09-17T03:21:09.075384+0000","last_clean":"2024-09-17T03:21:09.075384+0000","last_became_active":"2024-09-17T03:21:08.064712+0000","last_became_peered":"2024-09-17T03:21:08.064712+0000","last_unstale":"2024-09-17T03:21:09.075384+0000","last_undegraded":"2024-09-17T03:21:09.075384+0000","last_fullsized":"2024-09-17T03:21:09.075384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:10.162799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037730+0000","last_change":"2024-09-17T03:21:08.071508+0000","last_active":"2024-09-17T03:21:09.037730+0000","last_peered":"2024-09-17T03:21:09.037730+0000","last_clean":"2024-09-17T03:21:09.037730+0000","last_became_active":"2024-09-17T03:21:08.071167+0000","last_became_peered":"2024-09-17T03:21:08.071167+0000","last_unstale":"2024-09-17T03:21:09.037730+0000","last_undegraded":"2024-09-17T03:21:09.037730+0000","last_fullsized":"2024-09-17T03:21:09.037730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:05.628131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036578+0000","last_change":"2024-09-17T03:21:08.066995+0000","last_active":"2024-09-17T03:21:09.036578+0000","last_peered":"2024-09-17T03:21:09.036578+0000","last_clean":"2024-09-17T03:21:09.036578+0000","last_became_active":"2024-09-17T03:21:08.066620+0000","last_became_peered":"2024-09-17T03:21:08.066620+0000","last_unstale":"2024-09-17T03:21:09.036578+0000","last_undegraded":"2024-09-17T03:21:09.036578+0000","last_fullsized":"2024-09-17T03:21:09.036578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:53.398923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034599+0000","last_change":"2024-09-17T03:21:08.060591+0000","last_active":"2024-09-17T03:21:09.034599+0000","last_peered":"2024-09-17T03:21:09.034599+0000","last_clean":"2024-09-17T03:21:09.034599+0000","last_became_active":"2024-09-17T03:21:08.056048+0000","last_became_peered":"2024-09-17T03:21:08.056048+0000","last_unstale":"2024-09-17T03:21:09.034599+0000","last_undegraded":"2024-09-17T03:21:09.034599+0000","last_fullsized":"2024-09-17T03:21:09.034599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:07.073818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070945+0000","last_change":"2024-09-17T03:21:08.061391+0000","last_active":"2024-09-17T03:21:09.070945+0000","last_peered":"2024-09-17T03:21:09.070945+0000","last_clean":"2024-09-17T03:21:09.070945+0000","last_became_active":"2024-09-17T03:21:08.060838+0000","last_became_peered":"2024-09-17T03:21:08.060838+0000","last_unstale":"2024-09-17T03:21:09.070945+0000","last_undegraded":"2024-09-17T03:21:09.070945+0000","last_fullsized":"2024-09-17T03:21:09.070945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.368850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404503+0000","last_change":"2024-09-17T03:21:08.063409+0000","last_active":"2024-09-17T03:21:10.404503+0000","last_peered":"2024-09-17T03:21:10.404503+0000","last_clean":"2024-09-17T03:21:10.404503+0000","last_became_active":"2024-09-17T03:21:08.062667+0000","last_became_peered":"2024-09-17T03:21:08.062667+0000","last_unstale":"2024-09-17T03:21:10.404503+0000","last_undegraded":"2024-09-17T03:21:10.404503+0000","last_fullsized":"2024-09-17T03:21:10.404503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405077+0000","last_change":"2024-09-17T03:21:08.072107+0000","last_active":"2024-09-17T03:21:10.405077+0000","last_peered":"2024-09-17T03:21:10.405077+0000","last_clean":"2024-09-17T03:21:10.405077+0000","last_became_active":"2024-09-17T03:21:08.066708+0000","last_became_peered":"2024-09-17T03:21:08.066708+0000","last_unstale":"2024-09-17T03:21:10.405077+0000","last_undegraded":"2024-09-17T03:21:10.405077+0000","last_fullsized":"2024-09-17T03:21:10.405077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:40.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402958+0000","last_change":"2024-09-17T03:21:08.068237+0000","last_active":"2024-09-17T03:21:10.402958+0000","last_peered":"2024-09-17T03:21:10.402958+0000","last_clean":"2024-09-17T03:21:10.402958+0000","last_became_active":"2024-09-17T03:21:08.067863+0000","last_became_peered":"2024-09-17T03:21:08.067863+0000","last_unstale":"2024-09-17T03:21:10.402958+0000","last_undegraded":"2024-09-17T03:21:10.402958+0000","last_fullsized":"2024-09-17T03:21:10.402958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:18.563998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406412+0000","last_change":"2024-09-17T03:21:08.060741+0000","last_active":"2024-09-17T03:21:10.406412+0000","last_peered":"2024-09-17T03:21:10.406412+0000","last_clean":"2024-09-17T03:21:10.406412+0000","last_became_active":"2024-09-17T03:21:08.060429+0000","last_became_peered":"2024-09-17T03:21:08.060429+0000","last_unstale":"2024-09-17T03:21:10.406412+0000","last_undegraded":"2024-09-17T03:21:10.406412+0000","last_fullsized":"2024-09-17T03:21:10.406412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:20.270467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075420+0000","last_change":"2024-09-17T03:21:08.072066+0000","last_active":"2024-09-17T03:21:09.075420+0000","last_peered":"2024-09-17T03:21:09.075420+0000","last_clean":"2024-09-17T03:21:09.075420+0000","last_became_active":"2024-09-17T03:21:08.070966+0000","last_became_peered":"2024-09-17T03:21:08.070966+0000","last_unstale":"2024-09-17T03:21:09.075420+0000","last_undegraded":"2024-09-17T03:21:09.075420+0000","last_fullsized":"2024-09-17T03:21:09.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034202+0000","last_change":"2024-09-17T03:21:08.049953+0000","last_active":"2024-09-17T03:21:09.034202+0000","last_peered":"2024-09-17T03:21:09.034202+0000","last_clean":"2024-09-17T03:21:09.034202+0000","last_became_active":"2024-09-17T03:21:08.049526+0000","last_became_peered":"2024-09-17T03:21:08.049526+0000","last_unstale":"2024-09-17T03:21:09.034202+0000","last_undegraded":"2024-09-17T03:21:09.034202+0000","last_fullsized":"2024-09-17T03:21:09.034202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:59.117173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404940+0000","last_change":"2024-09-17T03:21:08.073188+0000","last_active":"2024-09-17T03:21:10.404940+0000","last_peered":"2024-09-17T03:21:10.404940+0000","last_clean":"2024-09-17T03:21:10.404940+0000","last_became_active":"2024-09-17T03:21:08.066652+0000","last_became_peered":"2024-09-17T03:21:08.066652+0000","last_unstale":"2024-09-17T03:21:10.404940+0000","last_undegraded":"2024-09-17T03:21:10.404940+0000","last_fullsized":"2024-09-17T03:21:10.404940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:43.152469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036398+0000","last_change":"2024-09-17T03:21:08.051667+0000","last_active":"2024-09-17T03:21:09.036398+0000","last_peered":"2024-09-17T03:21:09.036398+0000","last_clean":"2024-09-17T03:21:09.036398+0000","last_became_active":"2024-09-17T03:21:08.051099+0000","last_became_peered":"2024-09-17T03:21:08.051099+0000","last_unstale":"2024-09-17T03:21:09.036398+0000","last_undegraded":"2024-09-17T03:21:09.036398+0000","last_fullsized":"2024-09-17T03:21:09.036398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:26.245059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405651+0000","last_change":"2024-09-17T03:21:08.063744+0000","last_active":"2024-09-17T03:21:10.405651+0000","last_peered":"2024-09-17T03:21:10.405651+0000","last_clean":"2024-09-17T03:21:10.405651+0000","last_became_active":"2024-09-17T03:21:08.063062+0000","last_became_peered":"2024-09-17T03:21:08.063062+0000","last_unstale":"2024-09-17T03:21:10.405651+0000","last_undegraded":"2024-09-17T03:21:10.405651+0000","last_fullsized":"2024-09-17T03:21:10.405651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:37.353567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037066+0000","last_change":"2024-09-17T03:21:08.052647+0000","last_active":"2024-09-17T03:21:09.037066+0000","last_peered":"2024-09-17T03:21:09.037066+0000","last_clean":"2024-09-17T03:21:09.037066+0000","last_became_active":"2024-09-17T03:21:08.052328+0000","last_became_peered":"2024-09-17T03:21:08.052328+0000","last_unstale":"2024-09-17T03:21:09.037066+0000","last_undegraded":"2024-09-17T03:21:09.037066+0000","last_fullsized":"2024-09-17T03:21:09.037066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:48.836360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075599+0000","last_change":"2024-09-17T03:21:08.068699+0000","last_active":"2024-09-17T03:21:09.075599+0000","last_peered":"2024-09-17T03:21:09.075599+0000","last_clean":"2024-09-17T03:21:09.075599+0000","last_became_active":"2024-09-17T03:21:08.068338+0000","last_became_peered":"2024-09-17T03:21:08.068338+0000","last_unstale":"2024-09-17T03:21:09.075599+0000","last_undegraded":"2024-09-17T03:21:09.075599+0000","last_fullsized":"2024-09-17T03:21:09.075599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071414+0000","last_change":"2024-09-17T03:21:08.064645+0000","last_active":"2024-09-17T03:21:09.071414+0000","last_peered":"2024-09-17T03:21:09.071414+0000","last_clean":"2024-09-17T03:21:09.071414+0000","last_became_active":"2024-09-17T03:21:08.064086+0000","last_became_peered":"2024-09-17T03:21:08.064086+0000","last_unstale":"2024-09-17T03:21:09.071414+0000","last_undegraded":"2024-09-17T03:21:09.071414+0000","last_fullsized":"2024-09-17T03:21:09.071414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:46.510222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036872+0000","last_change":"2024-09-17T03:21:08.059892+0000","last_active":"2024-09-17T03:21:09.036872+0000","last_peered":"2024-09-17T03:21:09.036872+0000","last_clean":"2024-09-17T03:21:09.036872+0000","last_became_active":"2024-09-17T03:21:08.059568+0000","last_became_peered":"2024-09-17T03:21:08.059568+0000","last_unstale":"2024-09-17T03:21:09.036872+0000","last_undegraded":"2024-09-17T03:21:09.036872+0000","last_fullsized":"2024-09-17T03:21:09.036872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:03.752767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034157+0000","last_change":"2024-09-17T03:21:08.049305+0000","last_active":"2024-09-17T03:21:09.034157+0000","last_peered":"2024-09-17T03:21:09.034157+0000","last_clean":"2024-09-17T03:21:09.034157+0000","last_became_active":"2024-09-17T03:21:08.048911+0000","last_became_peered":"2024-09-17T03:21:08.048911+0000","last_unstale":"2024-09-17T03:21:09.034157+0000","last_undegraded":"2024-09-17T03:21:09.034157+0000","last_fullsized":"2024-09-17T03:21:09.034157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405216+0000","last_change":"2024-09-17T03:21:08.061342+0000","last_active":"2024-09-17T03:21:10.405216+0000","last_peered":"2024-09-17T03:21:10.405216+0000","last_clean":"2024-09-17T03:21:10.405216+0000","last_became_active":"2024-09-17T03:21:08.061025+0000","last_became_peered":"2024-09-17T03:21:08.061025+0000","last_unstale":"2024-09-17T03:21:10.405216+0000","last_undegraded":"2024-09-17T03:21:10.405216+0000","last_fullsized":"2024-09-17T03:21:10.405216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:01.042579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402687+0000","last_change":"2024-09-17T03:21:08.060288+0000","last_active":"2024-09-17T03:21:10.402687+0000","last_peered":"2024-09-17T03:21:10.402687+0000","last_clean":"2024-09-17T03:21:10.402687+0000","last_became_active":"2024-09-17T03:21:08.059954+0000","last_became_peered":"2024-09-17T03:21:08.059954+0000","last_unstale":"2024-09-17T03:21:10.402687+0000","last_undegraded":"2024-09-17T03:21:10.402687+0000","last_fullsized":"2024-09-17T03:21:10.402687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075416+0000","last_change":"2024-09-17T03:21:08.067449+0000","last_active":"2024-09-17T03:21:09.075416+0000","last_peered":"2024-09-17T03:21:09.075416+0000","last_clean":"2024-09-17T03:21:09.075416+0000","last_became_active":"2024-09-17T03:21:08.066649+0000","last_became_peered":"2024-09-17T03:21:08.066649+0000","last_unstale":"2024-09-17T03:21:09.075416+0000","last_undegraded":"2024-09-17T03:21:09.075416+0000","last_fullsized":"2024-09-17T03:21:09.075416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037818+0000","last_change":"2024-09-17T03:21:08.055200+0000","last_active":"2024-09-17T03:21:09.037818+0000","last_peered":"2024-09-17T03:21:09.037818+0000","last_clean":"2024-09-17T03:21:09.037818+0000","last_became_active":"2024-09-17T03:21:08.054451+0000","last_became_peered":"2024-09-17T03:21:08.054451+0000","last_unstale":"2024-09-17T03:21:09.037818+0000","last_undegraded":"2024-09-17T03:21:09.037818+0000","last_fullsized":"2024-09-17T03:21:09.037818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:41.720407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036639+0000","last_change":"2024-09-17T03:21:07.042857+0000","last_active":"2024-09-17T03:21:09.036639+0000","last_peered":"2024-09-17T03:21:09.036639+0000","last_clean":"2024-09-17T03:21:09.036639+0000","last_became_active":"2024-09-17T03:21:07.042659+0000","last_became_peered":"2024-09-17T03:21:07.042659+0000","last_unstale":"2024-09-17T03:21:09.036639+0000","last_undegraded":"2024-09-17T03:21:09.036639+0000","last_fullsized":"2024-09-17T03:21:09.036639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034320+0000","last_change":"2024-09-17T03:21:08.054380+0000","last_active":"2024-09-17T03:21:09.034320+0000","last_peered":"2024-09-17T03:21:09.034320+0000","last_clean":"2024-09-17T03:21:09.034320+0000","last_became_active":"2024-09-17T03:21:08.053967+0000","last_became_peered":"2024-09-17T03:21:08.053967+0000","last_unstale":"2024-09-17T03:21:09.034320+0000","last_undegraded":"2024-09-17T03:21:09.034320+0000","last_fullsized":"2024-09-17T03:21:09.034320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:18.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074976+0000","last_change":"2024-09-17T03:21:07.037433+0000","last_active":"2024-09-17T03:21:09.074976+0000","last_peered":"2024-09-17T03:21:09.074976+0000","last_clean":"2024-09-17T03:21:09.074976+0000","last_became_active":"2024-09-17T03:21:07.035663+0000","last_became_peered":"2024-09-17T03:21:07.035663+0000","last_unstale":"2024-09-17T03:21:09.074976+0000","last_undegraded":"2024-09-17T03:21:09.074976+0000","last_fullsized":"2024-09-17T03:21:09.074976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036177+0000","last_change":"2024-09-17T03:21:07.033856+0000","last_active":"2024-09-17T03:21:09.036177+0000","last_peered":"2024-09-17T03:21:09.036177+0000","last_clean":"2024-09-17T03:21:09.036177+0000","last_became_active":"2024-09-17T03:21:07.032761+0000","last_became_peered":"2024-09-17T03:21:07.032761+0000","last_unstale":"2024-09-17T03:21:09.036177+0000","last_undegraded":"2024-09-17T03:21:09.036177+0000","last_fullsized":"2024-09-17T03:21:09.036177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036275+0000","last_change":"2024-09-17T03:21:08.046160+0000","last_active":"2024-09-17T03:21:09.036275+0000","last_peered":"2024-09-17T03:21:09.036275+0000","last_clean":"2024-09-17T03:21:09.036275+0000","last_became_active":"2024-09-17T03:21:08.045937+0000","last_became_peered":"2024-09-17T03:21:08.045937+0000","last_unstale":"2024-09-17T03:21:09.036275+0000","last_undegraded":"2024-09-17T03:21:09.036275+0000","last_fullsized":"2024-09-17T03:21:09.036275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:13.549239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405281+0000","last_change":"2024-09-17T03:21:07.042939+0000","last_active":"2024-09-17T03:21:10.405281+0000","last_peered":"2024-09-17T03:21:10.405281+0000","last_clean":"2024-09-17T03:21:10.405281+0000","last_became_active":"2024-09-17T03:21:07.042625+0000","last_became_peered":"2024-09-17T03:21:07.042625+0000","last_unstale":"2024-09-17T03:21:10.405281+0000","last_undegraded":"2024-09-17T03:21:10.405281+0000","last_fullsized":"2024-09-17T03:21:10.405281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:03.791087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070951+0000","last_change":"2024-09-17T03:21:08.064569+0000","last_active":"2024-09-17T03:21:09.070951+0000","last_peered":"2024-09-17T03:21:09.070951+0000","last_clean":"2024-09-17T03:21:09.070951+0000","last_became_active":"2024-09-17T03:21:08.064100+0000","last_became_peered":"2024-09-17T03:21:08.064100+0000","last_unstale":"2024-09-17T03:21:09.070951+0000","last_undegraded":"2024-09-17T03:21:09.070951+0000","last_fullsized":"2024-09-17T03:21:09.070951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:54.326600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075195+0000","last_change":"2024-09-17T03:21:07.042752+0000","last_active":"2024-09-17T03:21:09.075195+0000","last_peered":"2024-09-17T03:21:09.075195+0000","last_clean":"2024-09-17T03:21:09.075195+0000","last_became_active":"2024-09-17T03:21:07.042359+0000","last_became_peered":"2024-09-17T03:21:07.042359+0000","last_unstale":"2024-09-17T03:21:09.075195+0000","last_undegraded":"2024-09-17T03:21:09.075195+0000","last_fullsized":"2024-09-17T03:21:09.075195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405109+0000","last_change":"2024-09-17T03:21:08.072460+0000","last_active":"2024-09-17T03:21:10.405109+0000","last_peered":"2024-09-17T03:21:10.405109+0000","last_clean":"2024-09-17T03:21:10.405109+0000","last_became_active":"2024-09-17T03:21:08.070048+0000","last_became_peered":"2024-09-17T03:21:08.070048+0000","last_unstale":"2024-09-17T03:21:10.405109+0000","last_undegraded":"2024-09-17T03:21:10.405109+0000","last_fullsized":"2024-09-17T03:21:10.405109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:45.722536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405253+0000","last_change":"2024-09-17T03:21:07.042935+0000","last_active":"2024-09-17T03:21:10.405253+0000","last_peered":"2024-09-17T03:21:10.405253+0000","last_clean":"2024-09-17T03:21:10.405253+0000","last_became_active":"2024-09-17T03:21:07.042540+0000","last_became_peered":"2024-09-17T03:21:07.042540+0000","last_unstale":"2024-09-17T03:21:10.405253+0000","last_undegraded":"2024-09-17T03:21:10.405253+0000","last_fullsized":"2024-09-17T03:21:10.405253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:41.768327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034735+0000","last_change":"2024-09-17T03:21:08.060578+0000","last_active":"2024-09-17T03:21:09.034735+0000","last_peered":"2024-09-17T03:21:09.034735+0000","last_clean":"2024-09-17T03:21:09.034735+0000","last_became_active":"2024-09-17T03:21:08.059037+0000","last_became_peered":"2024-09-17T03:21:08.059037+0000","last_unstale":"2024-09-17T03:21:09.034735+0000","last_undegraded":"2024-09-17T03:21:09.034735+0000","last_fullsized":"2024-09-17T03:21:09.034735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:08.929102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071271+0000","last_change":"2024-09-17T03:21:07.032070+0000","last_active":"2024-09-17T03:21:09.071271+0000","last_peered":"2024-09-17T03:21:09.071271+0000","last_clean":"2024-09-17T03:21:09.071271+0000","last_became_active":"2024-09-17T03:21:07.029846+0000","last_became_peered":"2024-09-17T03:21:07.029846+0000","last_unstale":"2024-09-17T03:21:09.071271+0000","last_undegraded":"2024-09-17T03:21:09.071271+0000","last_fullsized":"2024-09-17T03:21:09.071271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:00.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037963+0000","last_change":"2024-09-17T03:21:08.052908+0000","last_active":"2024-09-17T03:21:09.037963+0000","last_peered":"2024-09-17T03:21:09.037963+0000","last_clean":"2024-09-17T03:21:09.037963+0000","last_became_active":"2024-09-17T03:21:08.052578+0000","last_became_peered":"2024-09-17T03:21:08.052578+0000","last_unstale":"2024-09-17T03:21:09.037963+0000","last_undegraded":"2024-09-17T03:21:09.037963+0000","last_fullsized":"2024-09-17T03:21:09.037963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:06.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075516+0000","last_change":"2024-09-17T03:21:07.042836+0000","last_active":"2024-09-17T03:21:09.075516+0000","last_peered":"2024-09-17T03:21:09.075516+0000","last_clean":"2024-09-17T03:21:09.075516+0000","last_became_active":"2024-09-17T03:21:07.042380+0000","last_became_peered":"2024-09-17T03:21:07.042380+0000","last_unstale":"2024-09-17T03:21:09.075516+0000","last_undegraded":"2024-09-17T03:21:09.075516+0000","last_fullsized":"2024-09-17T03:21:09.075516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405970+0000","last_change":"2024-09-17T03:21:08.066084+0000","last_active":"2024-09-17T03:21:10.405970+0000","last_peered":"2024-09-17T03:21:10.405970+0000","last_clean":"2024-09-17T03:21:10.405970+0000","last_became_active":"2024-09-17T03:21:08.065807+0000","last_became_peered":"2024-09-17T03:21:08.065807+0000","last_unstale":"2024-09-17T03:21:10.405970+0000","last_undegraded":"2024-09-17T03:21:10.405970+0000","last_fullsized":"2024-09-17T03:21:10.405970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:53.782411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034281+0000","last_change":"2024-09-17T03:21:07.033053+0000","last_active":"2024-09-17T03:21:09.034281+0000","last_peered":"2024-09-17T03:21:09.034281+0000","last_clean":"2024-09-17T03:21:09.034281+0000","last_became_active":"2024-09-17T03:21:07.030140+0000","last_became_peered":"2024-09-17T03:21:07.030140+0000","last_unstale":"2024-09-17T03:21:09.034281+0000","last_undegraded":"2024-09-17T03:21:09.034281+0000","last_fullsized":"2024-09-17T03:21:09.034281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.294676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033897+0000","last_change":"2024-09-17T03:21:08.057090+0000","last_active":"2024-09-17T03:21:09.033897+0000","last_peered":"2024-09-17T03:21:09.033897+0000","last_clean":"2024-09-17T03:21:09.033897+0000","last_became_active":"2024-09-17T03:21:08.055419+0000","last_became_peered":"2024-09-17T03:21:08.055419+0000","last_unstale":"2024-09-17T03:21:09.033897+0000","last_undegraded":"2024-09-17T03:21:09.033897+0000","last_fullsized":"2024-09-17T03:21:09.033897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:54.579062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405382+0000","last_change":"2024-09-17T03:21:07.047300+0000","last_active":"2024-09-17T03:21:10.405382+0000","last_peered":"2024-09-17T03:21:10.405382+0000","last_clean":"2024-09-17T03:21:10.405382+0000","last_became_active":"2024-09-17T03:21:07.047023+0000","last_became_peered":"2024-09-17T03:21:07.047023+0000","last_unstale":"2024-09-17T03:21:10.405382+0000","last_undegraded":"2024-09-17T03:21:10.405382+0000","last_fullsized":"2024-09-17T03:21:10.405382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:03.878193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406013+0000","last_change":"2024-09-17T03:21:08.059703+0000","last_active":"2024-09-17T03:21:10.406013+0000","last_peered":"2024-09-17T03:21:10.406013+0000","last_clean":"2024-09-17T03:21:10.406013+0000","last_became_active":"2024-09-17T03:21:08.059411+0000","last_became_peered":"2024-09-17T03:21:08.059411+0000","last_unstale":"2024-09-17T03:21:10.406013+0000","last_undegraded":"2024-09-17T03:21:10.406013+0000","last_fullsized":"2024-09-17T03:21:10.406013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:00.131726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.074938+0000","last_change":"2024-09-17T03:21:07.043190+0000","last_active":"2024-09-17T03:21:09.074938+0000","last_peered":"2024-09-17T03:21:09.074938+0000","last_clean":"2024-09-17T03:21:09.074938+0000","last_became_active":"2024-09-17T03:21:07.042868+0000","last_became_peered":"2024-09-17T03:21:07.042868+0000","last_unstale":"2024-09-17T03:21:09.074938+0000","last_undegraded":"2024-09-17T03:21:09.074938+0000","last_fullsized":"2024-09-17T03:21:09.074938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075453+0000","last_change":"2024-09-17T03:21:08.070393+0000","last_active":"2024-09-17T03:21:09.075453+0000","last_peered":"2024-09-17T03:21:09.075453+0000","last_clean":"2024-09-17T03:21:09.075453+0000","last_became_active":"2024-09-17T03:21:08.069695+0000","last_became_peered":"2024-09-17T03:21:08.069695+0000","last_unstale":"2024-09-17T03:21:09.075453+0000","last_undegraded":"2024-09-17T03:21:09.075453+0000","last_fullsized":"2024-09-17T03:21:09.075453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:29.806923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402466+0000","last_change":"2024-09-17T03:21:07.037447+0000","last_active":"2024-09-17T03:21:10.402466+0000","last_peered":"2024-09-17T03:21:10.402466+0000","last_clean":"2024-09-17T03:21:10.402466+0000","last_became_active":"2024-09-17T03:21:07.037133+0000","last_became_peered":"2024-09-17T03:21:07.037133+0000","last_unstale":"2024-09-17T03:21:10.402466+0000","last_undegraded":"2024-09-17T03:21:10.402466+0000","last_fullsized":"2024-09-17T03:21:10.402466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:33.126636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405541+0000","last_change":"2024-09-17T03:21:08.064992+0000","last_active":"2024-09-17T03:21:10.405541+0000","last_peered":"2024-09-17T03:21:10.405541+0000","last_clean":"2024-09-17T03:21:10.405541+0000","last_became_active":"2024-09-17T03:21:08.064705+0000","last_became_peered":"2024-09-17T03:21:08.064705+0000","last_unstale":"2024-09-17T03:21:10.405541+0000","last_undegraded":"2024-09-17T03:21:10.405541+0000","last_fullsized":"2024-09-17T03:21:10.405541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.038518+0000","last_change":"2024-09-17T03:21:07.031511+0000","last_active":"2024-09-17T03:21:09.038518+0000","last_peered":"2024-09-17T03:21:09.038518+0000","last_clean":"2024-09-17T03:21:09.038518+0000","last_became_active":"2024-09-17T03:21:07.031175+0000","last_became_peered":"2024-09-17T03:21:07.031175+0000","last_unstale":"2024-09-17T03:21:09.038518+0000","last_undegraded":"2024-09-17T03:21:09.038518+0000","last_fullsized":"2024-09-17T03:21:09.038518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:23:29.396492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406155+0000","last_change":"2024-09-17T03:21:08.067321+0000","last_active":"2024-09-17T03:21:10.406155+0000","last_peered":"2024-09-17T03:21:10.406155+0000","last_clean":"2024-09-17T03:21:10.406155+0000","last_became_active":"2024-09-17T03:21:08.066953+0000","last_became_peered":"2024-09-17T03:21:08.066953+0000","last_unstale":"2024-09-17T03:21:10.406155+0000","last_undegraded":"2024-09-17T03:21:10.406155+0000","last_fullsized":"2024-09-17T03:21:10.406155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:38.976170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402786+0000","last_change":"2024-09-17T03:21:07.042316+0000","last_active":"2024-09-17T03:21:10.402786+0000","last_peered":"2024-09-17T03:21:10.402786+0000","last_clean":"2024-09-17T03:21:10.402786+0000","last_became_active":"2024-09-17T03:21:07.039133+0000","last_became_peered":"2024-09-17T03:21:07.039133+0000","last_unstale":"2024-09-17T03:21:10.402786+0000","last_undegraded":"2024-09-17T03:21:10.402786+0000","last_fullsized":"2024-09-17T03:21:10.402786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:53.869891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406653+0000","last_change":"2024-09-17T03:21:08.057737+0000","last_active":"2024-09-17T03:21:10.406653+0000","last_peered":"2024-09-17T03:21:10.406653+0000","last_clean":"2024-09-17T03:21:10.406653+0000","last_became_active":"2024-09-17T03:21:08.057560+0000","last_became_peered":"2024-09-17T03:21:08.057560+0000","last_unstale":"2024-09-17T03:21:10.406653+0000","last_undegraded":"2024-09-17T03:21:10.406653+0000","last_fullsized":"2024-09-17T03:21:10.406653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:07.761806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071345+0000","last_change":"2024-09-17T03:21:07.021653+0000","last_active":"2024-09-17T03:21:09.071345+0000","last_peered":"2024-09-17T03:21:09.071345+0000","last_clean":"2024-09-17T03:21:09.071345+0000","last_became_active":"2024-09-17T03:21:07.019084+0000","last_became_peered":"2024-09-17T03:21:07.019084+0000","last_unstale":"2024-09-17T03:21:09.071345+0000","last_undegraded":"2024-09-17T03:21:09.071345+0000","last_fullsized":"2024-09-17T03:21:09.071345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:30.978292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405157+0000","last_change":"2024-09-17T03:21:08.069155+0000","last_active":"2024-09-17T03:21:10.405157+0000","last_peered":"2024-09-17T03:21:10.405157+0000","last_clean":"2024-09-17T03:21:10.405157+0000","last_became_active":"2024-09-17T03:21:08.068437+0000","last_became_peered":"2024-09-17T03:21:08.068437+0000","last_unstale":"2024-09-17T03:21:10.405157+0000","last_undegraded":"2024-09-17T03:21:10.405157+0000","last_fullsized":"2024-09-17T03:21:10.405157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:31.177927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402646+0000","last_change":"2024-09-17T03:21:07.042337+0000","last_active":"2024-09-17T03:21:10.402646+0000","last_peered":"2024-09-17T03:21:10.402646+0000","last_clean":"2024-09-17T03:21:10.402646+0000","last_became_active":"2024-09-17T03:21:07.040846+0000","last_became_peered":"2024-09-17T03:21:07.040846+0000","last_unstale":"2024-09-17T03:21:10.402646+0000","last_undegraded":"2024-09-17T03:21:10.402646+0000","last_fullsized":"2024-09-17T03:21:10.402646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406114+0000","last_change":"2024-09-17T03:21:08.074560+0000","last_active":"2024-09-17T03:21:10.406114+0000","last_peered":"2024-09-17T03:21:10.406114+0000","last_clean":"2024-09-17T03:21:10.406114+0000","last_became_active":"2024-09-17T03:21:08.074267+0000","last_became_peered":"2024-09-17T03:21:08.074267+0000","last_unstale":"2024-09-17T03:21:10.406114+0000","last_undegraded":"2024-09-17T03:21:10.406114+0000","last_fullsized":"2024-09-17T03:21:10.406114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:09.894552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404671+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:10.404671+0000","last_peered":"2024-09-17T03:21:10.404671+0000","last_clean":"2024-09-17T03:21:10.404671+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:10.404671+0000","last_undegraded":"2024-09-17T03:21:10.404671+0000","last_fullsized":"2024-09-17T03:21:10.404671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404319+0000","last_change":"2024-09-17T03:21:07.038101+0000","last_active":"2024-09-17T03:21:10.404319+0000","last_peered":"2024-09-17T03:21:10.404319+0000","last_clean":"2024-09-17T03:21:10.404319+0000","last_became_active":"2024-09-17T03:21:07.033430+0000","last_became_peered":"2024-09-17T03:21:07.033430+0000","last_unstale":"2024-09-17T03:21:10.404319+0000","last_undegraded":"2024-09-17T03:21:10.404319+0000","last_fullsized":"2024-09-17T03:21:10.404319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:19.725739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036442+0000","last_change":"2024-09-17T03:21:08.046702+0000","last_active":"2024-09-17T03:21:09.036442+0000","last_peered":"2024-09-17T03:21:09.036442+0000","last_clean":"2024-09-17T03:21:09.036442+0000","last_became_active":"2024-09-17T03:21:08.046321+0000","last_became_peered":"2024-09-17T03:21:08.046321+0000","last_unstale":"2024-09-17T03:21:09.036442+0000","last_undegraded":"2024-09-17T03:21:09.036442+0000","last_fullsized":"2024-09-17T03:21:09.036442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:17.626557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404543+0000","last_change":"2024-09-17T03:21:07.045789+0000","last_active":"2024-09-17T03:21:10.404543+0000","last_peered":"2024-09-17T03:21:10.404543+0000","last_clean":"2024-09-17T03:21:10.404543+0000","last_became_active":"2024-09-17T03:21:07.045443+0000","last_became_peered":"2024-09-17T03:21:07.045443+0000","last_unstale":"2024-09-17T03:21:10.404543+0000","last_undegraded":"2024-09-17T03:21:10.404543+0000","last_fullsized":"2024-09-17T03:21:10.404543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:00.822415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033526+0000","last_change":"2024-09-17T03:21:08.040522+0000","last_active":"2024-09-17T03:21:09.033526+0000","last_peered":"2024-09-17T03:21:09.033526+0000","last_clean":"2024-09-17T03:21:09.033526+0000","last_became_active":"2024-09-17T03:21:08.037656+0000","last_became_peered":"2024-09-17T03:21:08.037656+0000","last_unstale":"2024-09-17T03:21:09.033526+0000","last_undegraded":"2024-09-17T03:21:09.033526+0000","last_fullsized":"2024-09-17T03:21:09.033526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:58.143540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.402609+0000","last_change":"2024-09-17T03:21:07.035825+0000","last_active":"2024-09-17T03:21:10.402609+0000","last_peered":"2024-09-17T03:21:10.402609+0000","last_clean":"2024-09-17T03:21:10.402609+0000","last_became_active":"2024-09-17T03:21:07.035573+0000","last_became_peered":"2024-09-17T03:21:07.035573+0000","last_unstale":"2024-09-17T03:21:10.402609+0000","last_undegraded":"2024-09-17T03:21:10.402609+0000","last_fullsized":"2024-09-17T03:21:10.402609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071006+0000","last_change":"2024-09-17T03:21:08.061617+0000","last_active":"2024-09-17T03:21:09.071006+0000","last_peered":"2024-09-17T03:21:09.071006+0000","last_clean":"2024-09-17T03:21:09.071006+0000","last_became_active":"2024-09-17T03:21:08.059024+0000","last_became_peered":"2024-09-17T03:21:08.059024+0000","last_unstale":"2024-09-17T03:21:09.071006+0000","last_undegraded":"2024-09-17T03:21:09.071006+0000","last_fullsized":"2024-09-17T03:21:09.071006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:08.053589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405475+0000","last_change":"2024-09-17T03:21:07.044634+0000","last_active":"2024-09-17T03:21:10.405475+0000","last_peered":"2024-09-17T03:21:10.405475+0000","last_clean":"2024-09-17T03:21:10.405475+0000","last_became_active":"2024-09-17T03:21:07.040735+0000","last_became_peered":"2024-09-17T03:21:07.040735+0000","last_unstale":"2024-09-17T03:21:10.405475+0000","last_undegraded":"2024-09-17T03:21:10.405475+0000","last_fullsized":"2024-09-17T03:21:10.405475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:02.669163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037938+0000","last_change":"2024-09-17T03:21:08.051692+0000","last_active":"2024-09-17T03:21:09.037938+0000","last_peered":"2024-09-17T03:21:09.037938+0000","last_clean":"2024-09-17T03:21:09.037938+0000","last_became_active":"2024-09-17T03:21:08.050563+0000","last_became_peered":"2024-09-17T03:21:08.050563+0000","last_unstale":"2024-09-17T03:21:09.037938+0000","last_undegraded":"2024-09-17T03:21:09.037938+0000","last_fullsized":"2024-09-17T03:21:09.037938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:03.819375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071215+0000","last_change":"2024-09-17T03:21:07.036505+0000","last_active":"2024-09-17T03:21:09.071215+0000","last_peered":"2024-09-17T03:21:09.071215+0000","last_clean":"2024-09-17T03:21:09.071215+0000","last_became_active":"2024-09-17T03:21:07.032177+0000","last_became_peered":"2024-09-17T03:21:07.032177+0000","last_unstale":"2024-09-17T03:21:09.071215+0000","last_undegraded":"2024-09-17T03:21:09.071215+0000","last_fullsized":"2024-09-17T03:21:09.071215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:45.702142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405276+0000","last_change":"2024-09-17T03:21:08.056241+0000","last_active":"2024-09-17T03:21:10.405276+0000","last_peered":"2024-09-17T03:21:10.405276+0000","last_clean":"2024-09-17T03:21:10.405276+0000","last_became_active":"2024-09-17T03:21:08.055544+0000","last_became_peered":"2024-09-17T03:21:08.055544+0000","last_unstale":"2024-09-17T03:21:10.405276+0000","last_undegraded":"2024-09-17T03:21:10.405276+0000","last_fullsized":"2024-09-17T03:21:10.405276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:37.096010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406358+0000","last_change":"2024-09-17T03:21:07.035552+0000","last_active":"2024-09-17T03:21:10.406358+0000","last_peered":"2024-09-17T03:21:10.406358+0000","last_clean":"2024-09-17T03:21:10.406358+0000","last_became_active":"2024-09-17T03:21:07.035214+0000","last_became_peered":"2024-09-17T03:21:07.035214+0000","last_unstale":"2024-09-17T03:21:10.406358+0000","last_undegraded":"2024-09-17T03:21:10.406358+0000","last_fullsized":"2024-09-17T03:21:10.406358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.557724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071051+0000","last_change":"2024-09-17T03:21:08.057655+0000","last_active":"2024-09-17T03:21:09.071051+0000","last_peered":"2024-09-17T03:21:09.071051+0000","last_clean":"2024-09-17T03:21:09.071051+0000","last_became_active":"2024-09-17T03:21:08.057335+0000","last_became_peered":"2024-09-17T03:21:08.057335+0000","last_unstale":"2024-09-17T03:21:09.071051+0000","last_undegraded":"2024-09-17T03:21:09.071051+0000","last_fullsized":"2024-09-17T03:21:09.071051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:09.104626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.073468+0000","last_change":"2024-09-17T03:21:07.032882+0000","last_active":"2024-09-17T03:21:09.073468+0000","last_peered":"2024-09-17T03:21:09.073468+0000","last_clean":"2024-09-17T03:21:09.073468+0000","last_became_active":"2024-09-17T03:21:07.030560+0000","last_became_peered":"2024-09-17T03:21:07.030560+0000","last_unstale":"2024-09-17T03:21:09.073468+0000","last_undegraded":"2024-09-17T03:21:09.073468+0000","last_fullsized":"2024-09-17T03:21:09.073468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:47.915357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037050+0000","last_change":"2024-09-17T03:21:08.059028+0000","last_active":"2024-09-17T03:21:09.037050+0000","last_peered":"2024-09-17T03:21:09.037050+0000","last_clean":"2024-09-17T03:21:09.037050+0000","last_became_active":"2024-09-17T03:21:08.058709+0000","last_became_peered":"2024-09-17T03:21:08.058709+0000","last_unstale":"2024-09-17T03:21:09.037050+0000","last_undegraded":"2024-09-17T03:21:09.037050+0000","last_fullsized":"2024-09-17T03:21:09.037050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:21.370502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075234+0000","last_change":"2024-09-17T03:21:07.033103+0000","last_active":"2024-09-17T03:21:09.075234+0000","last_peered":"2024-09-17T03:21:09.075234+0000","last_clean":"2024-09-17T03:21:09.075234+0000","last_became_active":"2024-09-17T03:21:07.032795+0000","last_became_peered":"2024-09-17T03:21:07.032795+0000","last_unstale":"2024-09-17T03:21:09.075234+0000","last_undegraded":"2024-09-17T03:21:09.075234+0000","last_fullsized":"2024-09-17T03:21:09.075234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:40.451871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.033821+0000","last_change":"2024-09-17T03:21:08.065685+0000","last_active":"2024-09-17T03:21:09.033821+0000","last_peered":"2024-09-17T03:21:09.033821+0000","last_clean":"2024-09-17T03:21:09.033821+0000","last_became_active":"2024-09-17T03:21:08.065243+0000","last_became_peered":"2024-09-17T03:21:08.065243+0000","last_unstale":"2024-09-17T03:21:09.033821+0000","last_undegraded":"2024-09-17T03:21:09.033821+0000","last_fullsized":"2024-09-17T03:21:09.033821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:23.445102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404390+0000","last_change":"2024-09-17T03:21:07.034832+0000","last_active":"2024-09-17T03:21:10.404390+0000","last_peered":"2024-09-17T03:21:10.404390+0000","last_clean":"2024-09-17T03:21:10.404390+0000","last_became_active":"2024-09-17T03:21:07.034455+0000","last_became_peered":"2024-09-17T03:21:07.034455+0000","last_unstale":"2024-09-17T03:21:10.404390+0000","last_undegraded":"2024-09-17T03:21:10.404390+0000","last_fullsized":"2024-09-17T03:21:10.404390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:40.714133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037637+0000","last_change":"2024-09-17T03:21:08.046797+0000","last_active":"2024-09-17T03:21:09.037637+0000","last_peered":"2024-09-17T03:21:09.037637+0000","last_clean":"2024-09-17T03:21:09.037637+0000","last_became_active":"2024-09-17T03:21:08.046355+0000","last_became_peered":"2024-09-17T03:21:08.046355+0000","last_unstale":"2024-09-17T03:21:09.037637+0000","last_undegraded":"2024-09-17T03:21:09.037637+0000","last_fullsized":"2024-09-17T03:21:09.037637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:41.458145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.071075+0000","last_change":"2024-09-17T03:21:07.034292+0000","last_active":"2024-09-17T03:21:09.071075+0000","last_peered":"2024-09-17T03:21:09.071075+0000","last_clean":"2024-09-17T03:21:09.071075+0000","last_became_active":"2024-09-17T03:21:07.029326+0000","last_became_peered":"2024-09-17T03:21:07.029326+0000","last_unstale":"2024-09-17T03:21:09.071075+0000","last_undegraded":"2024-09-17T03:21:09.071075+0000","last_fullsized":"2024-09-17T03:21:09.071075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:04.742753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036511+0000","last_change":"2024-09-17T03:21:08.060646+0000","last_active":"2024-09-17T03:21:09.036511+0000","last_peered":"2024-09-17T03:21:09.036511+0000","last_clean":"2024-09-17T03:21:09.036511+0000","last_became_active":"2024-09-17T03:21:08.060341+0000","last_became_peered":"2024-09-17T03:21:08.060341+0000","last_unstale":"2024-09-17T03:21:09.036511+0000","last_undegraded":"2024-09-17T03:21:09.036511+0000","last_fullsized":"2024-09-17T03:21:09.036511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:06.388228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.070990+0000","last_change":"2024-09-17T03:21:07.039765+0000","last_active":"2024-09-17T03:21:09.070990+0000","last_peered":"2024-09-17T03:21:09.070990+0000","last_clean":"2024-09-17T03:21:09.070990+0000","last_became_active":"2024-09-17T03:21:07.039435+0000","last_became_peered":"2024-09-17T03:21:07.039435+0000","last_unstale":"2024-09-17T03:21:09.070990+0000","last_undegraded":"2024-09-17T03:21:09.070990+0000","last_fullsized":"2024-09-17T03:21:09.070990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:12.516594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036654+0000","last_change":"2024-09-17T03:21:08.045211+0000","last_active":"2024-09-17T03:21:09.036654+0000","last_peered":"2024-09-17T03:21:09.036654+0000","last_clean":"2024-09-17T03:21:09.036654+0000","last_became_active":"2024-09-17T03:21:08.043718+0000","last_became_peered":"2024-09-17T03:21:08.043718+0000","last_unstale":"2024-09-17T03:21:09.036654+0000","last_undegraded":"2024-09-17T03:21:09.036654+0000","last_fullsized":"2024-09-17T03:21:09.036654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:41.789200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406079+0000","last_change":"2024-09-17T03:21:07.038659+0000","last_active":"2024-09-17T03:21:10.406079+0000","last_peered":"2024-09-17T03:21:10.406079+0000","last_clean":"2024-09-17T03:21:10.406079+0000","last_became_active":"2024-09-17T03:21:07.038247+0000","last_became_peered":"2024-09-17T03:21:07.038247+0000","last_unstale":"2024-09-17T03:21:10.406079+0000","last_undegraded":"2024-09-17T03:21:10.406079+0000","last_fullsized":"2024-09-17T03:21:10.406079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:42.133587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.075179+0000","last_change":"2024-09-17T03:21:08.065090+0000","last_active":"2024-09-17T03:21:09.075179+0000","last_peered":"2024-09-17T03:21:09.075179+0000","last_clean":"2024-09-17T03:21:09.075179+0000","last_became_active":"2024-09-17T03:21:08.064729+0000","last_became_peered":"2024-09-17T03:21:08.064729+0000","last_unstale":"2024-09-17T03:21:09.075179+0000","last_undegraded":"2024-09-17T03:21:09.075179+0000","last_fullsized":"2024-09-17T03:21:09.075179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.253313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404929+0000","last_change":"2024-09-17T03:21:08.055905+0000","last_active":"2024-09-17T03:21:10.404929+0000","last_peered":"2024-09-17T03:21:10.404929+0000","last_clean":"2024-09-17T03:21:10.404929+0000","last_became_active":"2024-09-17T03:21:08.051807+0000","last_became_peered":"2024-09-17T03:21:08.051807+0000","last_unstale":"2024-09-17T03:21:10.404929+0000","last_undegraded":"2024-09-17T03:21:10.404929+0000","last_fullsized":"2024-09-17T03:21:10.404929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:16.949588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.405784+0000","last_change":"2024-09-17T03:21:07.049981+0000","last_active":"2024-09-17T03:21:10.405784+0000","last_peered":"2024-09-17T03:21:10.405784+0000","last_clean":"2024-09-17T03:21:10.405784+0000","last_became_active":"2024-09-17T03:21:07.049635+0000","last_became_peered":"2024-09-17T03:21:07.049635+0000","last_unstale":"2024-09-17T03:21:10.405784+0000","last_undegraded":"2024-09-17T03:21:10.405784+0000","last_fullsized":"2024-09-17T03:21:10.405784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:11.341893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.404932+0000","last_change":"2024-09-17T03:21:08.058582+0000","last_active":"2024-09-17T03:21:10.404932+0000","last_peered":"2024-09-17T03:21:10.404932+0000","last_clean":"2024-09-17T03:21:10.404932+0000","last_became_active":"2024-09-17T03:21:08.058228+0000","last_became_peered":"2024-09-17T03:21:08.058228+0000","last_unstale":"2024-09-17T03:21:10.404932+0000","last_undegraded":"2024-09-17T03:21:10.404932+0000","last_fullsized":"2024-09-17T03:21:10.404932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:10.406212+0000","last_change":"2024-09-17T03:21:07.038577+0000","last_active":"2024-09-17T03:21:10.406212+0000","last_peered":"2024-09-17T03:21:10.406212+0000","last_clean":"2024-09-17T03:21:10.406212+0000","last_became_active":"2024-09-17T03:21:07.038261+0000","last_became_peered":"2024-09-17T03:21:07.038261+0000","last_unstale":"2024-09-17T03:21:10.406212+0000","last_undegraded":"2024-09-17T03:21:10.406212+0000","last_fullsized":"2024-09-17T03:21:10.406212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:02.813101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034518+0000","last_change":"2024-09-17T03:21:08.061875+0000","last_active":"2024-09-17T03:21:09.034518+0000","last_peered":"2024-09-17T03:21:09.034518+0000","last_clean":"2024-09-17T03:21:09.034518+0000","last_became_active":"2024-09-17T03:21:08.057042+0000","last_became_peered":"2024-09-17T03:21:08.057042+0000","last_unstale":"2024-09-17T03:21:09.034518+0000","last_undegraded":"2024-09-17T03:21:09.034518+0000","last_fullsized":"2024-09-17T03:21:09.034518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:18.945802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.036926+0000","last_change":"2024-09-17T03:21:07.037271+0000","last_active":"2024-09-17T03:21:09.036926+0000","last_peered":"2024-09-17T03:21:09.036926+0000","last_clean":"2024-09-17T03:21:09.036926+0000","last_became_active":"2024-09-17T03:21:07.036859+0000","last_became_peered":"2024-09-17T03:21:07.036859+0000","last_unstale":"2024-09-17T03:21:09.036926+0000","last_undegraded":"2024-09-17T03:21:09.036926+0000","last_fullsized":"2024-09-17T03:21:09.036926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:13.462508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.037073+0000","last_change":"2024-09-17T03:21:08.068273+0000","last_active":"2024-09-17T03:21:09.037073+0000","last_peered":"2024-09-17T03:21:09.037073+0000","last_clean":"2024-09-17T03:21:09.037073+0000","last_became_active":"2024-09-17T03:21:08.066628+0000","last_became_peered":"2024-09-17T03:21:08.066628+0000","last_unstale":"2024-09-17T03:21:09.037073+0000","last_undegraded":"2024-09-17T03:21:09.037073+0000","last_fullsized":"2024-09-17T03:21:09.037073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:07.005025+0000","last_clean_scrub_stamp":"2024-09-17T03:21:07.005025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:03.676328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T03:21:09.034788+0000","last_change":"2024-09-17T03:21:07.032795+0000","last_active":"2024-09-17T03:21:09.034788+0000","last_peered":"2024-09-17T03:21:09.034788+0000","last_clean":"2024-09-17T03:21:09.034788+0000","last_became_active":"2024-09-17T03:21:07.032288+0000","last_became_peered":"2024-09-17T03:21:07.032288+0000","last_unstale":"2024-09-17T03:21:09.034788+0000","last_undegraded":"2024-09-17T03:21:09.034788+0000","last_fullsized":"2024-09-17T03:21:09.034788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:05.998750+0000","last_clean_scrub_stamp":"2024-09-17T03:21:05.998750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:41.226611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:22.652 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:21:22.652 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T03:21:22.652 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T03:21:22.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 -- 172.21.15.46:0/4229705711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 msgr2=0x7fa69800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 --2- 172.21.15.46:0/4229705711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa69800b9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40cca90 tx=0x7fa6ac01d630 comp rx=0 tx=0).stop 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b3400640 1 -- 172.21.15.46:0/4229705711 reap_dead start 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 -- 172.21.15.46:0/4229705711 shutdown_connections 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 -- 172.21.15.46:0/4229705711 >> 172.21.15.46:0/4229705711 conn(0x7fa6b40b2040 msgr2=0x7fa6b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 -- 172.21.15.46:0/4229705711 shutdown_connections 2024-09-17T03:21:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 -- 172.21.15.46:0/4229705711 wait complete. 2024-09-17T03:21:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 Processor -- start 2024-09-17T03:21:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.850+0000 7fa6b9600640 1 -- start start 2024-09-17T03:21:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa6b41e7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6b414fdc0 0x7fa6b41e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4140a70 0x7fa6b4140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b415cba0 con 0x7fa698005220 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b415bf00 con 0x7fa6b4140a70 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b415cd20 con 0x7fa6b414fdc0 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa6b41e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa6b41e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37128/0 (socket says 172.21.15.46:37128) 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 -- 172.21.15.46:0/2840558591 learned_addr learned my addr 172.21.15.46:0/2840558591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6abe00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6b414fdc0 0x7fa6b41e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b3400640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4140a70 0x7fa6b4140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 -- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6b414fdc0 msgr2=0x7fa6b41e7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6b414fdc0 0x7fa6b41e7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 -- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4140a70 msgr2=0x7fa6b4140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4140a70 0x7fa6b4140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 -- 172.21.15.46:0/2840558591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40aa570 con 0x7fa698005220 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6abe00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6b414fdc0 0x7fa6b41e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b3400640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4140a70 0x7fa6b4140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b2a00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa6b41e7930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4003eb0 tx=0x7fa6a4001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4015070 con 0x7fa698005220 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6a4011940 con 0x7fa698005220 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b41f43f0 con 0x7fa698005220 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4011d90 con 0x7fa698005220 2024-09-17T03:21:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b40aaaf0 con 0x7fa698005220 2024-09-17T03:21:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.854+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b4073cf0 con 0x7fa698005220 2024-09-17T03:21:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.858+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa6a4018070 con 0x7fa698005220 2024-09-17T03:21:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.858+0000 7fa6aaa00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa690073200 0x7fa690075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.858+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6a40950a0 con 0x7fa698005220 2024-09-17T03:21:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.858+0000 7fa6abe00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa690073200 0x7fa690075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:22.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.858+0000 7fa6abe00640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa690073200 0x7fa690075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0019f0 tx=0x7fa69c001b00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:22.862+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6a4060020 con 0x7fa698005220 2024-09-17T03:21:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa6b41e83b0 con 0x7fa698005220 2024-09-17T03:21:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6aaa00640 1 -- 172.21.15.46:0/2840558591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa6a405f920 con 0x7fa698005220 2024-09-17T03:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:23.078 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T03:21:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa690073200 msgr2=0x7fa690075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa690073200 0x7fa690075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0019f0 tx=0x7fa69c001b00 comp rx=0 tx=0).stop 2024-09-17T03:21:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 msgr2=0x7fa6b41e7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa6b41e7930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4003eb0 tx=0x7fa6a4001810 comp rx=0 tx=0).stop 2024-09-17T03:21:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 shutdown_connections 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa690073200 0x7fa690075710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4140a70 0x7fa6b4140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6b414fdc0 0x7fa6b41e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 --2- 172.21.15.46:0/2840558591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa698005220 0x7fa6b41e7930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.074+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 >> 172.21.15.46:0/2840558591 conn(0x7fa6b40b2040 msgr2=0x7fa6b4155790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.082+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 shutdown_connections 2024-09-17T03:21:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.082+0000 7fa6b9600640 1 -- 172.21.15.46:0/2840558591 wait complete. 2024-09-17T03:21:23.101 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T03:21:23.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.258+0000 7f3131e00640 1 -- 172.21.15.46:0/986992401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c157030 msgr2=0x7f312c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.258+0000 7f3131e00640 1 --2- 172.21.15.46:0/986992401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c157030 0x7f312c0b5a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f311c009840 tx=0x7f311c01c790 comp rx=0 tx=0).stop 2024-09-17T03:21:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- 172.21.15.46:0/986992401 shutdown_connections 2024-09-17T03:21:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 --2- 172.21.15.46:0/986992401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c157030 0x7f312c0b5a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- 172.21.15.46:0/986992401 >> 172.21.15.46:0/986992401 conn(0x7f312c0708c0 msgr2=0x7f312c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- 172.21.15.46:0/986992401 shutdown_connections 2024-09-17T03:21:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- 172.21.15.46:0/986992401 wait complete. 2024-09-17T03:21:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 Processor -- start 2024-09-17T03:21:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- start start 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f312c157030 0x7f312c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 0x7f312c1361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f312c154a20 0x7f312c136700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f312c16b8d0 con 0x7f312c160d50 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f312c16b750 con 0x7f312c154a20 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.262+0000 7f3131e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f312c16ba50 con 0x7f312c157030 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 0x7f312c1361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 0x7f312c1361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37148/0 (socket says 172.21.15.46:37148) 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 -- 172.21.15.46:0/3448002356 learned_addr learned my addr 172.21.15.46:0/3448002356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f312aa00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f312c157030 0x7f312c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f312b400640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f312c154a20 0x7f312c136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 -- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f312c157030 msgr2=0x7f312c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f312c157030 0x7f312c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 -- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f312c154a20 msgr2=0x7f312c136700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f312c154a20 0x7f312c136700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 -- 172.21.15.46:0/3448002356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c0cf860 con 0x7f312c160d50 2024-09-17T03:21:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 0x7f312c1361c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f311400c400 tx=0x7f311400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114017070 con 0x7f312c160d50 2024-09-17T03:21:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3114003600 con 0x7f312c160d50 2024-09-17T03:21:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114003a50 con 0x7f312c160d50 2024-09-17T03:21:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c135cc0 con 0x7f312c160d50 2024-09-17T03:21:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1554b0 con 0x7f312c160d50 2024-09-17T03:21:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c0d2260 con 0x7f312c160d50 2024-09-17T03:21:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.266+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f311401a070 con 0x7f312c160d50 2024-09-17T03:21:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.270+0000 7f3123400640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3108073200 0x7f3108075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.270+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3114093480 con 0x7f312c160d50 2024-09-17T03:21:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.270+0000 7f312aa00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3108073200 0x7f3108075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.270+0000 7f312aa00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3108073200 0x7f3108075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f311c0ae000 tx=0x7f311c01afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.274+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f311405dee0 con 0x7f312c160d50 2024-09-17T03:21:23.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.490+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f312c154f60 con 0x7f312c160d50 2024-09-17T03:21:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3123400640 1 -- 172.21.15.46:0/3448002356 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7f3114062910 con 0x7f312c160d50 2024-09-17T03:21:23.492 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:23.493 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:23.493 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3108073200 msgr2=0x7f3108075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3108073200 0x7f3108075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f311c0ae000 tx=0x7f311c01afd0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 msgr2=0x7f312c1361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 0x7f312c1361c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f311400c400 tx=0x7f311400c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 shutdown_connections 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3108073200 0x7f3108075710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f312c154a20 0x7f312c136700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c160d50 0x7f312c1361c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 --2- 172.21.15.46:0/3448002356 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f312c157030 0x7f312c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 >> 172.21.15.46:0/3448002356 conn(0x7f312c0708c0 msgr2=0x7f312c15e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 shutdown_connections 2024-09-17T03:21:23.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.494+0000 7f3131e00640 1 -- 172.21.15.46:0/3448002356 wait complete. 2024-09-17T03:21:23.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 -- 172.21.15.46:0/475051138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 msgr2=0x7fb2c815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 --2- 172.21.15.46:0/475051138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c815bb60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80cca90 tx=0x7fb2b001c800 comp rx=0 tx=0).stop 2024-09-17T03:21:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 -- 172.21.15.46:0/475051138 shutdown_connections 2024-09-17T03:21:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 --2- 172.21.15.46:0/475051138 >> v2:172.21.15.99:3300/0 conn(0x7fb2c80b4de0 0x7fb2c80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 --2- 172.21.15.46:0/475051138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c815bb60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 -- 172.21.15.46:0/475051138 >> 172.21.15.46:0/475051138 conn(0x7fb2c80b2040 msgr2=0x7fb2c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 -- 172.21.15.46:0/475051138 shutdown_connections 2024-09-17T03:21:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 -- 172.21.15.46:0/475051138 wait complete. 2024-09-17T03:21:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 Processor -- start 2024-09-17T03:21:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.674+0000 7fb2cd600640 1 -- start start 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2c80b4de0 0x7fb2c81fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2c80b5c70 0x7fb2c81f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c8153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8168920 con 0x7fb2c81527b0 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81687a0 con 0x7fb2c80b4de0 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c8168aa0 con 0x7fb2c80b5c70 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c8153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c8153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37166/0 (socket says 172.21.15.46:37166) 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 -- 172.21.15.46:0/483535462 learned_addr learned my addr 172.21.15.46:0/483535462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:23.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c6a00640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2c80b4de0 0x7fb2c81fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2bfe00640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2c80b5c70 0x7fb2c81f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 -- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2c80b5c70 msgr2=0x7fb2c81f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2c80b5c70 0x7fb2c81f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 -- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2c80b4de0 msgr2=0x7fb2c81fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2c80b4de0 0x7fb2c81fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 -- 172.21.15.46:0/483535462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81fc770 con 0x7fb2c81527b0 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2c7400640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c8153060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e910 tx=0x7fb2c000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0004d30 con 0x7fb2c81527b0 2024-09-17T03:21:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c000cb20 con 0x7fb2c81527b0 2024-09-17T03:21:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81fca50 con 0x7fb2c81527b0 2024-09-17T03:21:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0005020 con 0x7fb2c81527b0 2024-09-17T03:21:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81f8a40 con 0x7fb2c81527b0 2024-09-17T03:21:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.678+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c81fc4e0 con 0x7fb2c81527b0 2024-09-17T03:21:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.682+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb2c00051c0 con 0x7fb2c81527b0 2024-09-17T03:21:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.682+0000 7fb2bea00640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2a4073200 0x7fb2a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.682+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c0094970 con 0x7fb2c81527b0 2024-09-17T03:21:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.682+0000 7fb2c6a00640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2a4073200 0x7fb2a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.682+0000 7fb2c6a00640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2a4073200 0x7fb2a4075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8007fd0 tx=0x7fb2b8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:23.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.686+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb2c0016030 con 0x7fb2c81527b0 2024-09-17T03:21:23.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2c80b51f0 con 0x7fb2c81527b0 2024-09-17T03:21:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2bea00640 1 -- 172.21.15.46:0/483535462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7fb2c005f330 con 0x7fb2c81527b0 2024-09-17T03:21:23.898 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:23.898 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2a4073200 msgr2=0x7fb2a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2a4073200 0x7fb2a4075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8007fd0 tx=0x7fb2b8009040 comp rx=0 tx=0).stop 2024-09-17T03:21:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 msgr2=0x7fb2c8153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c8153060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e910 tx=0x7fb2c000edd0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 shutdown_connections 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2a4073200 0x7fb2a4075710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c81527b0 0x7fb2c8153060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2c80b5c70 0x7fb2c81f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 --2- 172.21.15.46:0/483535462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2c80b4de0 0x7fb2c81fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.898+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 >> 172.21.15.46:0/483535462 conn(0x7fb2c80b2040 msgr2=0x7fb2c815a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.902+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 shutdown_connections 2024-09-17T03:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:23.902+0000 7fb2cd600640 1 -- 172.21.15.46:0/483535462 wait complete. 2024-09-17T03:21:23.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 -- 172.21.15.46:0/1335484455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 msgr2=0x7fc7c015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 --2- 172.21.15.46:0/1335484455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c015cf90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009c80 tx=0x7fc7b801cda0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 -- 172.21.15.46:0/1335484455 shutdown_connections 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 --2- 172.21.15.46:0/1335484455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c015cf90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 --2- 172.21.15.46:0/1335484455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c00b5a80 0x7fc7c0074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 -- 172.21.15.46:0/1335484455 >> 172.21.15.46:0/1335484455 conn(0x7fc7c00708c0 msgr2=0x7fc7c0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 -- 172.21.15.46:0/1335484455 shutdown_connections 2024-09-17T03:21:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.078+0000 7fc7c6600640 1 -- 172.21.15.46:0/1335484455 wait complete. 2024-09-17T03:21:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 Processor -- start 2024-09-17T03:21:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 -- start start 2024-09-17T03:21:24.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c0144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c00b5a80 0x7fc7c015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c0157030 0x7fc7c015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c016a360 con 0x7fc7c0074950 2024-09-17T03:21:24.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c016a1e0 con 0x7fc7c00b5a80 2024-09-17T03:21:24.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7c6600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c016a4e0 con 0x7fc7c0157030 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c0144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c0144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37184/0 (socket says 172.21.15.46:37184) 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7bf400640 1 -- 172.21.15.46:0/753079861 learned_addr learned my addr 172.21.15.46:0/753079861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7b6a00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c00b5a80 0x7fc7c015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.082+0000 7fc7bfe00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c0157030 0x7fc7c015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bf400640 1 -- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c0157030 msgr2=0x7fc7c015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bf400640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c0157030 0x7fc7c015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bf400640 1 -- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c00b5a80 msgr2=0x7fc7c015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bf400640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c00b5a80 0x7fc7c015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bf400640 1 -- 172.21.15.46:0/753079861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c0144e50 con 0x7fc7c0074950 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7b6a00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c00b5a80 0x7fc7c015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bfe00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c0157030 0x7fc7c015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7bf400640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c0144910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c3f0 tx=0x7fc7b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0017070 con 0x7fc7c0074950 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b00053c0 con 0x7fc7c0074950 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c01fa830 con 0x7fc7c0074950 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0003860 con 0x7fc7c0074950 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c01fadf0 con 0x7fc7c0074950 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c015cf90 con 0x7fc7c0074950 2024-09-17T03:21:24.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7b001a070 con 0x7fc7c0074950 2024-09-17T03:21:24.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.086+0000 7fc7b7e00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc79c073200 0x7fc79c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.090+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc7b0093590 con 0x7fc7c0074950 2024-09-17T03:21:24.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.090+0000 7fc7b6a00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc79c073200 0x7fc79c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.090+0000 7fc7b6a00640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc79c073200 0x7fc79c075710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00b4db0 tx=0x7fc7a80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:24.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.094+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc7c0144e50 con 0x7fc7c0074950 2024-09-17T03:21:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.270+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7c00d2260 con 0x7fc7c0074950 2024-09-17T03:21:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.270+0000 7fc7b7e00640 1 -- 172.21.15.46:0/753079861 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7fc7c0074d60 con 0x7fc7c0074950 2024-09-17T03:21:24.270 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:24.270 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc79c073200 msgr2=0x7fc79c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc79c073200 0x7fc79c075710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00b4db0 tx=0x7fc7a80091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 msgr2=0x7fc7c0144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c0144910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c3f0 tx=0x7fc7b000c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 shutdown_connections 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc79c073200 0x7fc79c075710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7c0157030 0x7fc7c015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c00b5a80 0x7fc7c015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 --2- 172.21.15.46:0/753079861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c0074950 0x7fc7c0144910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 >> 172.21.15.46:0/753079861 conn(0x7fc7c00708c0 msgr2=0x7fc7c01540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 shutdown_connections 2024-09-17T03:21:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.274+0000 7fc7c6600640 1 -- 172.21.15.46:0/753079861 wait complete. 2024-09-17T03:21:24.294 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 -- 172.21.15.46:0/1126010898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 msgr2=0x7f70f4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 --2- 172.21.15.46:0/1126010898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 0x7f70f4074e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009840 tx=0x7f70e001c680 comp rx=0 tx=0).stop 2024-09-17T03:21:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 -- 172.21.15.46:0/1126010898 shutdown_connections 2024-09-17T03:21:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 --2- 172.21.15.46:0/1126010898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 0x7f70f4074e30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 --2- 172.21.15.46:0/1126010898 >> v2:172.21.15.99:3300/0 conn(0x7f70f40b5c70 0x7f70f415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 -- 172.21.15.46:0/1126010898 >> 172.21.15.46:0/1126010898 conn(0x7f70f40b2040 msgr2=0x7f70f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 -- 172.21.15.46:0/1126010898 shutdown_connections 2024-09-17T03:21:24.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 -- 172.21.15.46:0/1126010898 wait complete. 2024-09-17T03:21:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 Processor -- start 2024-09-17T03:21:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.454+0000 7f70fb600640 1 -- start start 2024-09-17T03:21:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 0x7f70f4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f40b5c70 0x7f70f4143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 0x7f70f4159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4168c50 con 0x7f70f40b4de0 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4168ad0 con 0x7f70f40b5c70 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f70f4168dd0 con 0x7f70f41527b0 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 0x7f70f4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 0x7f70f4157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37196/0 (socket says 172.21.15.46:37196) 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 -- 172.21.15.46:0/143344860 learned_addr learned my addr 172.21.15.46:0/143344860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f3e00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f40b5c70 0x7f70f4143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f9600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 0x7f70f4159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 -- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 msgr2=0x7f70f4159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 0x7f70f4159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 -- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f40b5c70 msgr2=0x7f70f4143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f40b5c70 0x7f70f4143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 -- 172.21.15.46:0/143344860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f4158650 con 0x7f70f40b4de0 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f8c00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 0x7f70f4157d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e7c0 tx=0x7f70e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e800ccc0 con 0x7f70f40b4de0 2024-09-17T03:21:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70e800ce80 con 0x7f70f40b4de0 2024-09-17T03:21:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f4158930 con 0x7f70f40b4de0 2024-09-17T03:21:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8005280 con 0x7f70f40b4de0 2024-09-17T03:21:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f4158ef0 con 0x7f70f40b4de0 2024-09-17T03:21:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.458+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f415bb60 con 0x7f70f40b4de0 2024-09-17T03:21:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.462+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70e8005420 con 0x7f70f40b4de0 2024-09-17T03:21:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.462+0000 7f70f0c00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f70d4073200 0x7f70d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.462+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f70e8094b30 con 0x7f70f40b4de0 2024-09-17T03:21:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.462+0000 7f70f3e00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f70d4073200 0x7f70d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.462+0000 7f70f3e00640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f70d4073200 0x7f70d4075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f70e00af000 tx=0x7f70e001aec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:24.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.466+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f70e805f4f0 con 0x7f70f40b4de0 2024-09-17T03:21:24.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70f40afd10 con 0x7f70f40b4de0 2024-09-17T03:21:24.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70f0c00640 1 -- 172.21.15.46:0/143344860 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7f70e8063f20 con 0x7f70f40b4de0 2024-09-17T03:21:24.677 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:24.677 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:24.677 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f70d4073200 msgr2=0x7f70d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f70d4073200 0x7f70d4075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f70e00af000 tx=0x7f70e001aec0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 msgr2=0x7f70f4157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 0x7f70f4157d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e7c0 tx=0x7f70e800ec80 comp rx=0 tx=0).stop 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 shutdown_connections 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f70d4073200 0x7f70d4075710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f70f41527b0 0x7f70f4159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f40b5c70 0x7f70f4143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 --2- 172.21.15.46:0/143344860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70f40b4de0 0x7f70f4157d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 >> 172.21.15.46:0/143344860 conn(0x7f70f40b2040 msgr2=0x7f70f40ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 shutdown_connections 2024-09-17T03:21:24.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:24.678+0000 7f70fb600640 1 -- 172.21.15.46:0/143344860 wait complete. 2024-09-17T03:21:25.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:21:07.435943+0000', 'modified': '2024-09-17T03:21:10.471393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 244315}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_244315': {'gid': 244315, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6833/2638881791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 2638881791}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 2638881791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 244315, 'qdb_cluster': [244315]} 2024-09-17T03:21:25.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:21:25.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T03:21:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 -- 172.21.15.46:0/913098904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd015cc60 msgr2=0x7febd015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 --2- 172.21.15.46:0/913098904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd015cc60 0x7febd015d070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febc8009840 tx=0x7febc801c710 comp rx=0 tx=0).stop 2024-09-17T03:21:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 -- 172.21.15.46:0/913098904 shutdown_connections 2024-09-17T03:21:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 --2- 172.21.15.46:0/913098904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd015cc60 0x7febd015d070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 -- 172.21.15.46:0/913098904 >> 172.21.15.46:0/913098904 conn(0x7febd00b2040 msgr2=0x7febd00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 -- 172.21.15.46:0/913098904 shutdown_connections 2024-09-17T03:21:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.266+0000 7febd6c00640 1 -- 172.21.15.46:0/913098904 wait complete. 2024-09-17T03:21:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 Processor -- start 2024-09-17T03:21:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 -- start start 2024-09-17T03:21:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd00738b0 0x7febd01f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febd00b4de0 0x7febd0157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 0x7febd0158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febd0169fd0 con 0x7febd015cc60 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7febd0169e50 con 0x7febd00738b0 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd6c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7febd016a150 con 0x7febd00b4de0 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 0x7febd0158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 0x7febd0158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37198/0 (socket says 172.21.15.46:37198) 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 -- 172.21.15.46:0/3526566353 learned_addr learned my addr 172.21.15.46:0/3526566353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febcfe00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd00738b0 0x7febd01f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 -- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febd00b4de0 msgr2=0x7febd0157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febc7400640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febd00b4de0 0x7febd0157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febd00b4de0 0x7febd0157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 -- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd00738b0 msgr2=0x7febd01f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd00738b0 0x7febd01f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febd4c00640 1 -- 172.21.15.46:0/3526566353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd0151e40 con 0x7febd015cc60 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febcfe00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd00738b0 0x7febd01f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.270+0000 7febc7400640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febd00b4de0 0x7febd0157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febd4c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 0x7febd0158b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febc8019320 tx=0x7febc8004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febccc00640 1 -- 172.21.15.46:0/3526566353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80bb070 con 0x7febd015cc60 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febccc00640 1 -- 172.21.15.46:0/3526566353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc8005320 con 0x7febd015cc60 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd0158210 con 0x7febd015cc60 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febccc00640 1 -- 172.21.15.46:0/3526566353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80b2040 con 0x7febd015cc60 2024-09-17T03:21:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd00c5800 con 0x7febd015cc60 2024-09-17T03:21:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.274+0000 7febccc00640 1 -- 172.21.15.46:0/3526566353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7febc80046d0 con 0x7febd015cc60 2024-09-17T03:21:25.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febccc00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7febac073230 0x7febac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 Processor -- start 2024-09-17T03:21:25.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febccc00640 1 -- 172.21.15.46:0/3526566353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7febc812db20 con 0x7febd015cc60 2024-09-17T03:21:25.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 -- start start 2024-09-17T03:21:25.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7febac073230 0x7febac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7febac073230 0x7febac075740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febc00014b0 tx=0x7febc0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e424eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e4253f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e425930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56485e2f1410 con 0x56485dedb460 2024-09-17T03:21:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56485e22eb40 con 0x56485e2cb950 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56485e21df70 con 0x56485e2b4ca0 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e424eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e424eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37212/0 (socket says 172.21.15.46:37212) 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 -- 172.21.15.46:0/527962002 learned_addr learned my addr 172.21.15.46:0/527962002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febd4c00640 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e425930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febc7400640 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e4253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 -- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 msgr2=0x56485e425930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e425930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 -- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 msgr2=0x56485e4253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e4253f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 -- 172.21.15.46:0/527962002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485e1e55b0 con 0x56485dedb460 2024-09-17T03:21:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febcfe00640 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e424eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febc000dbe0 tx=0x7febc0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964bdd64c43b13e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febbfe00640 1 -- 172.21.15.46:0/527962002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0024070 con 0x56485dedb460 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febbfe00640 1 -- 172.21.15.46:0/527962002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc001f400 con 0x56485dedb460 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.278+0000 7febbfe00640 1 -- 172.21.15.46:0/527962002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc001f7b0 con 0x56485dedb460 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- 172.21.15.46:0/527962002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 msgr2=0x56485e424eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e424eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febc000dbe0 tx=0x7febc0010000 comp rx=0 tx=0).stop 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- 172.21.15.46:0/527962002 shutdown_connections 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e425930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e4253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- 172.21.15.46:0/527962002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e424eb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- 172.21.15.46:0/527962002 >> 172.21.15.46:0/527962002 conn(0x56485e3a1d90 msgr2=0x56485e3d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- 172.21.15.46:0/527962002 shutdown_connections 2024-09-17T03:21:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- 172.21.15.46:0/527962002 wait complete. 2024-09-17T03:21:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- start start 2024-09-17T03:21:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 Processor -- start 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485dedb460 0x56485e3d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e3d8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 0x56485e410780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56485e1b85f0 con 0x56485e2cb950 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56485e221d80 con 0x56485dedb460 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56485e1c4520 con 0x56485e2b4ca0 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febde34e000 10 client.0 _ll_register_callbacks cb 0x56485e1c8830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febd4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 0x56485e410780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febd4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 0x56485e410780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37228/0 (socket says 172.21.15.46:37228) 2024-09-17T03:21:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febcfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485dedb460 0x56485e3d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febd4c00640 1 -- 172.21.15.46:0/1873334728 learned_addr learned my addr 172.21.15.46:0/1873334728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.282+0000 7febc7400640 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e3d8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febd4c00640 1 -- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 msgr2=0x56485e3d8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febd4c00640 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e3d8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febd4c00640 1 -- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485dedb460 msgr2=0x56485e3d7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febd4c00640 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485dedb460 0x56485e3d7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febd4c00640 1 -- 172.21.15.46:0/1873334728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485e1bc190 con 0x56485e2cb950 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febd4c00640 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 0x56485e410780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febc80982f0 tx=0x7febc80f7b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78c4d293a0e087f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 1 -- 172.21.15.46:0/1873334728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8101b50 con 0x56485e2cb950 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56485e1b6110 con 0x56485e2cb950 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 1 -- 172.21.15.46:0/1873334728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7febc80f7620 con 0x56485e2cb950 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 1 -- 172.21.15.46:0/1873334728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7febc80b7800 con 0x56485e2cb950 2024-09-17T03:21:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 10 client.244626 fetch_fsmap learned FSMap version 8 2024-09-17T03:21:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x56485e2a2f10 con 0x56485e2cb950 2024-09-17T03:21:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 1 -- 172.21.15.46:0/1873334728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56485e1b6110 con 0x56485e2cb950 2024-09-17T03:21:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 1 -- 172.21.15.46:0/1873334728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7febc80b5030 con 0x56485e2cb950 2024-09-17T03:21:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 10 client.244626 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T03:21:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 10 client.244626 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 20 client.244626 populate_metadata read hostname 'smithi046' 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485dec07d0 0x56485e42c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 4 client.244626 mds_command: new command op to 244315 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 --> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56485e230a90 con 0x56485dec07d0 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febcfe00640 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485dec07d0 0x56485e42c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febcfe00640 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485dec07d0 0x56485e42c130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b9b854f304a66 server_cookie=b7ebdb1fdb518119 in_seq=0 out_seq=0 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 10 client.244626 ms_handle_connect on v2:172.21.15.99:6832/2638881791 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 1 -- 172.21.15.46:0/1873334728 <== mds.0 v2:172.21.15.99:6832/2638881791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56485e230a90 con 0x56485dec07d0 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 10 client.244626 handle_command_reply: tid=0 2024-09-17T03:21:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 client.244626 shutdown 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 msgr2=0x56485e410780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 0x56485e410780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febc80982f0 tx=0x7febc80f7b30 comp rx=0 tx=0).stop 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 shutdown_connections 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485dec07d0 0x56485e42c130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485e2cb950 0x56485e410780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e3d8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 --2- 172.21.15.46:0/1873334728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485dedb460 0x56485e3d7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febbcc00640 0 client.244626 ms_handle_reset on v2:172.21.15.99:6832/2638881791 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.286+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 >> 172.21.15.46:0/1873334728 conn(0x56485e3a1d90 msgr2=0x56485e3d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.290+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 shutdown_connections 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.290+0000 7febde34e000 1 -- 172.21.15.46:0/1873334728 wait complete. 2024-09-17T03:21:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.290+0000 7febde34e000 20 client.244626 trim_cache size 0 max 16384 2024-09-17T03:21:25.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.306+0000 7febde34e000 1 Processor -- start 2024-09-17T03:21:25.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.306+0000 7febde34e000 1 -- start start 2024-09-17T03:21:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e43caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e43d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e3d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febd00ac760 con 0x56485dedb460 2024-09-17T03:21:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7febd00ab780 con 0x56485e2cb950 2024-09-17T03:21:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7febd01fc3a0 con 0x56485e2b4ca0 2024-09-17T03:21:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e43caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e43caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37238/0 (socket says 172.21.15.46:37238) 2024-09-17T03:21:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 -- 172.21.15.46:0/324507167 learned_addr learned my addr 172.21.15.46:0/324507167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:25.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febd4c00640 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e3d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 -- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 msgr2=0x56485e43d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e43d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 -- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 msgr2=0x56485e3d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e3d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 -- 172.21.15.46:0/324507167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485e279db0 con 0x56485dedb460 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febd4c00640 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e3d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febcfe00640 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e43caf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x56485e2287e0 tx=0x7febc0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc40031bcc21406 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febbf400640 1 -- 172.21.15.46:0/324507167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0038070 con 0x56485dedb460 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febbf400640 1 -- 172.21.15.46:0/324507167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc001fc40 con 0x56485dedb460 2024-09-17T03:21:25.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febbf400640 1 -- 172.21.15.46:0/324507167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0024040 con 0x56485dedb460 2024-09-17T03:21:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- 172.21.15.46:0/324507167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 msgr2=0x56485e43caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e43caf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x56485e2287e0 tx=0x7febc0007c80 comp rx=0 tx=0).stop 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- 172.21.15.46:0/324507167 shutdown_connections 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2cb950 0x56485e3d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2b4ca0 0x56485e43d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 --2- 172.21.15.46:0/324507167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e43caf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- 172.21.15.46:0/324507167 >> 172.21.15.46:0/324507167 conn(0x56485e3a1d90 msgr2=0x56485e424580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- 172.21.15.46:0/324507167 shutdown_connections 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- 172.21.15.46:0/324507167 wait complete. 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 -- start start 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.310+0000 7febde34e000 1 Processor -- start 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e3d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2b4ca0 0x56485e3d4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2cb950 0x56485e457670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febd01fc9f0 con 0x56485dedb460 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56485e1e9ee0 con 0x56485e2b4ca0 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56485e221090 con 0x56485e2cb950 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 10 client.0 _ll_register_callbacks cb 0x7febc80f8190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e3d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e3d48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37246/0 (socket says 172.21.15.46:37246) 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 -- 172.21.15.46:0/470735417 learned_addr learned my addr 172.21.15.46:0/470735417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febc7400640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2b4ca0 0x56485e3d4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febd4c00640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2cb950 0x56485e457670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 -- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2cb950 msgr2=0x56485e457670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2cb950 0x56485e457670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 -- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2b4ca0 msgr2=0x56485e3d4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2b4ca0 0x56485e3d4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 -- 172.21.15.46:0/470735417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485e1e4ab0 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febcfe00640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e3d48a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7febc0001b70 tx=0x7febc0001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2476e2d9609e5b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febbcc00640 1 -- 172.21.15.46:0/470735417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0038070 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febbcc00640 1 -- 172.21.15.46:0/470735417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7febc001fa40 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc001fc40 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febbcc00640 1 -- 172.21.15.46:0/470735417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7febc0024040 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 10 client.244632 fetch_fsmap learned FSMap version 8 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x56485e1c3070 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febbcc00640 1 -- 172.21.15.46:0/470735417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc001fc40 con 0x56485dedb460 2024-09-17T03:21:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febbcc00640 1 -- 172.21.15.46:0/470735417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7febc0006b60 con 0x56485dedb460 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 10 client.244632 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 10 client.244632 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 20 client.244632 populate_metadata read hostname 'smithi046' 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485deae830 0x56485e458000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 4 client.244632 mds_command: new command op to 244315 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 --> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] -- command(tid 0: {"prefix": "status"}) -- 0x56485e22c540 con 0x56485deae830 2024-09-17T03:21:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.314+0000 7febc7400640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485deae830 0x56485e458000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febbcc00640 10 client.244632 ms_handle_connect on v2:172.21.15.99:6832/2638881791 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febc7400640 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485deae830 0x56485e458000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f970bae4ca3d0732 server_cookie=8fddc16b90164ab0 in_seq=0 out_seq=0 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febbcc00640 1 -- 172.21.15.46:0/470735417 <== mds.0 v2:172.21.15.99:6832/2638881791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56485e22c540 con 0x56485deae830 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febbcc00640 10 client.244632 handle_command_reply: tid=0 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 client.244632 shutdown 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 msgr2=0x56485e3d48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e3d48a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7febc0001b70 tx=0x7febc0001740 comp rx=0 tx=0).stop 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 shutdown_connections 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:6832/2638881791,v1:172.21.15.99:6833/2638881791] conn(0x56485deae830 0x56485e458000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56485e2cb950 0x56485e457670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febbcc00640 0 client.244632 ms_handle_reset on v2:172.21.15.99:6832/2638881791 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56485e2b4ca0 0x56485e3d4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 --2- 172.21.15.46:0/470735417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56485dedb460 0x56485e3d48a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 >> 172.21.15.46:0/470735417 conn(0x56485e3a1d90 msgr2=0x56485e424350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 shutdown_connections 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 1 -- 172.21.15.46:0/470735417 wait complete. 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febde34e000 20 client.244632 trim_cache size 0 max 16384 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 244315, 2024-09-17T03:21:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 17.265593750000001, 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 8, 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 21, 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 21, 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 23.362985262999999 2024-09-17T03:21:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7febac073230 msgr2=0x7febac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7febac073230 0x7febac075740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febc00014b0 tx=0x7febc0009040 comp rx=0 tx=0).stop 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 msgr2=0x7febd0158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 0x7febd0158b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febc8019320 tx=0x7febc8004620 comp rx=0 tx=0).stop 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 shutdown_connections 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7febac073230 0x7febac075740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd015cc60 0x7febd0158b40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febd00b4de0 0x7febd0157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 --2- 172.21.15.46:0/3526566353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd00738b0 0x7febd01f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 >> 172.21.15.46:0/3526566353 conn(0x7febd00b2040 msgr2=0x7febd00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 shutdown_connections 2024-09-17T03:21:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.318+0000 7febd6c00640 1 -- 172.21.15.46:0/3526566353 wait complete. 2024-09-17T03:21:25.339 INFO:teuthology.run_tasks:Running task kclient... 2024-09-17T03:21:25.367 INFO:tasks.kclient:Mounting kernel clients... 2024-09-17T03:21:25.367 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T03:21:25.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.368 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:21:25.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-17T03:21:25.447 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-09-17T03:21:25.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:21:25.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.449 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:21:25.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-17T03:21:25.566 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-09-17T03:21:25.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:21:25.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.568 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-17T03:21:25.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-09-17T03:21:25.649 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-09-17T03:21:25.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:21:25.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.651 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-17T03:21:25.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:25.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-09-17T03:21:25.771 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-09-17T03:21:25.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:21:25.773 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-17T03:21:25.774 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:21:25.774 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:21:25.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:21:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 -- 172.21.15.46:0/1390358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015bc20 msgr2=0x7f3db015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1390358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015bc20 0x7f3db015d280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3db00b3040 tx=0x7f3da801cd80 comp rx=0 tx=0).stop 2024-09-17T03:21:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 -- 172.21.15.46:0/1390358835 shutdown_connections 2024-09-17T03:21:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1390358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015bc20 0x7f3db015d280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1390358835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3db015b2d0 0x7f3db015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1390358835 >> v2:172.21.15.99:3300/0 conn(0x7f3db015e8c0 0x7f3db015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 -- 172.21.15.46:0/1390358835 >> 172.21.15.46:0/1390358835 conn(0x7f3db00b2040 msgr2=0x7f3db00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 -- 172.21.15.46:0/1390358835 shutdown_connections 2024-09-17T03:21:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 -- 172.21.15.46:0/1390358835 wait complete. 2024-09-17T03:21:25.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.934+0000 7f3db7e00640 1 Processor -- start 2024-09-17T03:21:25.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 -- start start 2024-09-17T03:21:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 0x7f3db0165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3db015bc20 0x7f3db00cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3db015e8c0 0x7f3db01fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0166bd0 con 0x7f3db015b2d0 2024-09-17T03:21:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0166a50 con 0x7f3db015bc20 2024-09-17T03:21:25.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3db0166d50 con 0x7f3db015e8c0 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 0x7f3db0165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 0x7f3db0165cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37264/0 (socket says 172.21.15.46:37264) 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 -- 172.21.15.46:0/1869847997 learned_addr learned my addr 172.21.15.46:0/1869847997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db4a00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3db015bc20 0x7f3db00cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3db015e8c0 0x7f3db01fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 -- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3db015e8c0 msgr2=0x7f3db01fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3db015e8c0 0x7f3db01fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 -- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3db015bc20 msgr2=0x7f3db00cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3db015bc20 0x7f3db00cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 -- 172.21.15.46:0/1869847997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db01575c0 con 0x7f3db015b2d0 2024-09-17T03:21:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db5400640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 0x7f3db0165cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e7d0 tx=0x7f3da400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400ccc0 con 0x7f3db015b2d0 2024-09-17T03:21:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3da4010430 con 0x7f3db015b2d0 2024-09-17T03:21:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4010860 con 0x7f3db015b2d0 2024-09-17T03:21:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db01578a0 con 0x7f3db015b2d0 2024-09-17T03:21:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.938+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db0157e60 con 0x7f3db015b2d0 2024-09-17T03:21:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.942+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db015d280 con 0x7f3db015b2d0 2024-09-17T03:21:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.942+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3da4010a00 con 0x7f3db015b2d0 2024-09-17T03:21:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.942+0000 7f3dad600640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d90073230 0x7f3d90075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.942+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3da4094a80 con 0x7f3db015b2d0 2024-09-17T03:21:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.942+0000 7f3db4a00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d90073230 0x7f3d90075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.942+0000 7f3db4a00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d90073230 0x7f3d90075740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3db00ce030 tx=0x7f3d9c010040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:25.946+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3da405f570 con 0x7f3db015b2d0 2024-09-17T03:21:26.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3db0159570 con 0x7f3db015b2d0 2024-09-17T03:21:26.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3dad600640 1 -- 172.21.15.46:0/1869847997 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3da4063fa0 con 0x7f3db015b2d0 2024-09-17T03:21:26.158 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d90073230 msgr2=0x7f3d90075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d90073230 0x7f3d90075740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3db00ce030 tx=0x7f3d9c010040 comp rx=0 tx=0).stop 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 msgr2=0x7f3db0165cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 0x7f3db0165cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e7d0 tx=0x7f3da400ec90 comp rx=0 tx=0).stop 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 shutdown_connections 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d90073230 0x7f3d90075740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3db015e8c0 0x7f3db01fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3db015bc20 0x7f3db00cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 --2- 172.21.15.46:0/1869847997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db015b2d0 0x7f3db0165cd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 >> 172.21.15.46:0/1869847997 conn(0x7f3db00b2040 msgr2=0x7f3db00b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:26.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 shutdown_connections 2024-09-17T03:21:26.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26.158+0000 7f3db7e00640 1 -- 172.21.15.46:0/1869847997 wait complete. 2024-09-17T03:21:26.179 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:21:26.180 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:21:26.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:26.180 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:21:26.189 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42402sec preferred_lft 42402sec 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:21:26.191 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:26.192 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T03:21:26.193 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:26.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:26.350 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:21:26.369 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:21:26.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:26.371 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:21:26.442 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:21:26.442 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:21:26.442 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:21:26.442 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:21:26.442 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:21:26.442 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:21:26.443 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:26.443 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:26.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:21:26.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:21:26.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:21:26.443 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:26.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:26.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:26.580 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:21:26.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:26.590 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-17T03:21:26.641 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:26.642 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:26.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:21:26.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T03:21:26.642 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:26.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:26.769 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T03:21:26.770 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:26.770 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:26.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T03:21:26.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T03:21:26.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T03:21:26.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T03:21:26.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T03:21:26.772 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:26.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:26.981 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:26.981 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:26.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:21:26.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T03:21:26.982 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:27.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:27.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:27.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:27.069 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:21:27.120 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:21:27.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:27.120 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:21:27.172 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:21:27.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:27.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:27.278 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:27.278 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:21:27.978 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:21:27.982 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:21:27.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:27.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:21:28.005 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:28.143 INFO:teuthology.orchestra.run.smithi046.stdout:client244624 2024-09-17T03:21:28.143 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:28.143 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:28.153 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:28.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:28.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:28.268 INFO:teuthology.orchestra.run.smithi046.stdout:client244624 2024-09-17T03:21:28.269 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:28.337 INFO:teuthology.orchestra.run.smithi046.stdout:client244624 2024-09-17T03:21:28.337 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/6c25af6b-3689-4426-acb2-6ebe6de6efd5.client244624/status') 2024-09-17T03:21:28.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:28.406 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.244624 (3)172.21.15.46:0/1189128156 2024-09-17T03:21:28.406 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-09-17T03:21:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-09-17T03:21:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-09-17T03:21:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:72 bytes copied, 5.1365e-05 s, 1.4 MB/s 2024-09-17T03:21:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:28.414 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-17T03:21:28.414 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:21:28.415 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T03:21:28.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.569+0000 7fe89d800640 1 -- 172.21.15.46:0/4222781754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8980b5a80 msgr2=0x7fe898074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.569+0000 7fe89d800640 1 --2- 172.21.15.46:0/4222781754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8980b5a80 0x7fe898074410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe8980b6a50 tx=0x7fe88001c830 comp rx=0 tx=0).stop 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- 172.21.15.46:0/4222781754 shutdown_connections 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 --2- 172.21.15.46:0/4222781754 >> v2:172.21.15.99:3300/0 conn(0x7fe898074950 0x7fe89815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 --2- 172.21.15.46:0/4222781754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8980b5a80 0x7fe898074410 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- 172.21.15.46:0/4222781754 >> 172.21.15.46:0/4222781754 conn(0x7fe8980708c0 msgr2=0x7fe898070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- 172.21.15.46:0/4222781754 shutdown_connections 2024-09-17T03:21:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- 172.21.15.46:0/4222781754 wait complete. 2024-09-17T03:21:28.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 Processor -- start 2024-09-17T03:21:28.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- start start 2024-09-17T03:21:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898074950 0x7fe89813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8980b5a80 0x7fe89815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 0x7fe89815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe89816a310 con 0x7fe898157030 2024-09-17T03:21:28.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe89816a190 con 0x7fe898074950 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.573+0000 7fe89d800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe89816a490 con 0x7fe8980b5a80 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 0x7fe89815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 0x7fe89815f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37280/0 (socket says 172.21.15.46:37280) 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 -- 172.21.15.46:0/1649388249 learned_addr learned my addr 172.21.15.46:0/1649388249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe896a00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898074950 0x7fe89813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe88fe00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8980b5a80 0x7fe89815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 -- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8980b5a80 msgr2=0x7fe89815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8980b5a80 0x7fe89815ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 -- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898074950 msgr2=0x7fe89813ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898074950 0x7fe89813ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 -- 172.21.15.46:0/1649388249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89814a310 con 0x7fe898157030 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe88fe00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8980b5a80 0x7fe89815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe896a00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898074950 0x7fe89813ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe897400640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 0x7fe89815f3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe89000eab0 tx=0x7fe89000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000cea0 con 0x7fe898157030 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe890004b30 con 0x7fe898157030 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890005020 con 0x7fe898157030 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe898149270 con 0x7fe898157030 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe898149830 con 0x7fe898157030 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89815cf90 con 0x7fe898157030 2024-09-17T03:21:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.577+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe8900051c0 con 0x7fe898157030 2024-09-17T03:21:28.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.581+0000 7fe88ea00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe874073200 0x7fe874075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.581+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe890094fb0 con 0x7fe898157030 2024-09-17T03:21:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.581+0000 7fe896a00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe874073200 0x7fe874075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:28.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.581+0000 7fe896a00640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe874073200 0x7fe874075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe888001630 tx=0x7fe888009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:28.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.585+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe89005ff30 con 0x7fe898157030 2024-09-17T03:21:28.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:21:28.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:21:28.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.793+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8980d2260 con 0x7fe898157030 2024-09-17T03:21:28.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.793+0000 7fe88ea00640 1 -- 172.21.15.46:0/1649388249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe89005f830 con 0x7fe898157030 2024-09-17T03:21:28.796 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:21:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe874073200 msgr2=0x7fe874075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe874073200 0x7fe874075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe888001630 tx=0x7fe888009040 comp rx=0 tx=0).stop 2024-09-17T03:21:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 msgr2=0x7fe89815f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 0x7fe89815f3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe89000eab0 tx=0x7fe89000ef70 comp rx=0 tx=0).stop 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 shutdown_connections 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe874073200 0x7fe874075710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe898157030 0x7fe89815f3a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8980b5a80 0x7fe89815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 --2- 172.21.15.46:0/1649388249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe898074950 0x7fe89813ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 >> 172.21.15.46:0/1649388249 conn(0x7fe8980708c0 msgr2=0x7fe8981561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 shutdown_connections 2024-09-17T03:21:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:28.797+0000 7fe89d800640 1 -- 172.21.15.46:0/1649388249 wait complete. 2024-09-17T03:21:28.818 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:21:28.819 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T03:21:28.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:28.820 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:21:28.869 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:21:28.870 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:28.871 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:21:28.872 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.873 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:21:28.873 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:21:28.873 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:21:28.873 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:28.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:28.873 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:21:28.934 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:21:28.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:28.937 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:21:28.989 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:21:28.989 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:21:28.990 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:21:28.990 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:21:28.990 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:21:28.990 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:21:28.990 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:28.990 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:28.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:21:28.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:21:28.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:21:28.991 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:29.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:29.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:29.131 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:21:29.140 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:21:29.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:29.141 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-17T03:21:29.196 INFO:teuthology.orchestra.run.smithi046.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T03:21:29.197 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:29.197 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:29.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:21:29.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T03:21:29.197 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:29.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:29.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T03:21:29.388 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 76:fa:12:60:cb:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::74fa:12ff:fe60:cb2a/64 scope link 2024-09-17T03:21:29.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:29.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T03:21:29.456 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:29.456 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 76:fa:12:60:cb:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::74fa:12ff:fe60:cb2a/64 scope link 2024-09-17T03:21:29.457 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:29.458 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T03:21:29.459 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:29.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:29.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:29.644 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:29.645 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:21:29.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.1 up 2024-09-17T03:21:29.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T03:21:29.645 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:21:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:29.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:29.731 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T03:21:29.740 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T03:21:29.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:29.740 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T03:21:29.792 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-17T03:21:29.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:29.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:29.898 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:29.898 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:21:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 2@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:21:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:21:29.917 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:21:29.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:29.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T03:21:29.937 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:30.039 INFO:teuthology.orchestra.run.smithi046.stdout:client244630 2024-09-17T03:21:30.039 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:30.039 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:30.048 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:30.150 INFO:teuthology.orchestra.run.smithi046.stdout:client244630 2024-09-17T03:21:30.150 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:30.254 INFO:teuthology.orchestra.run.smithi046.stdout:client244630 2024-09-17T03:21:30.254 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/6c25af6b-3689-4426-acb2-6ebe6de6efd5.client244630/status') 2024-09-17T03:21:30.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:30.315 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.244630 (3)172.21.15.46:0/1132327726 2024-09-17T03:21:30.315 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-09-17T03:21:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-09-17T03:21:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-09-17T03:21:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:72 bytes copied, 7.0247e-05 s, 1.0 MB/s 2024-09-17T03:21:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:30.323 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-17T03:21:30.323 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:21:30.324 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T03:21:30.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:21:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 -- 172.21.15.46:0/3015669003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef88160af0 msgr2=0x7fef88070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 --2- 172.21.15.46:0/3015669003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef88160af0 0x7fef88070690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fef80009bc0 tx=0x7fef8001cb10 comp rx=0 tx=0).stop 2024-09-17T03:21:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 -- 172.21.15.46:0/3015669003 shutdown_connections 2024-09-17T03:21:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 --2- 172.21.15.46:0/3015669003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef88160af0 0x7fef88070690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 --2- 172.21.15.46:0/3015669003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef8815c7b0 0x7fef88068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 -- 172.21.15.46:0/3015669003 >> 172.21.15.46:0/3015669003 conn(0x7fef880b35f0 msgr2=0x7fef880b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 -- 172.21.15.46:0/3015669003 shutdown_connections 2024-09-17T03:21:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.525+0000 7fef8ec00640 1 -- 172.21.15.46:0/3015669003 wait complete. 2024-09-17T03:21:30.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 Processor -- start 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 -- start start 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef88069330 0x7fef8814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 0x7fef8814b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef88160af0 0x7fef881530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef88169db0 con 0x7fef8815c7b0 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef88169c30 con 0x7fef88160af0 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fef88169f30 con 0x7fef88069330 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 0x7fef8814b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 0x7fef8814b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37306/0 (socket says 172.21.15.46:37306) 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 -- 172.21.15.46:0/4064951191 learned_addr learned my addr 172.21.15.46:0/4064951191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87e00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef88069330 0x7fef8814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8cc00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef88160af0 0x7fef881530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 -- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef88069330 msgr2=0x7fef8814a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef88069330 0x7fef8814a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 -- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef88160af0 msgr2=0x7fef881530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef88160af0 0x7fef881530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 -- 172.21.15.46:0/4064951191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8814b690 con 0x7fef8815c7b0 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef8cc00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef88160af0 0x7fef881530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.529+0000 7fef87400640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 0x7fef8814b8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c370 tx=0x7fef7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74017070 con 0x7fef8815c7b0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef74003600 con 0x7fef8815c7b0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74003ab0 con 0x7fef8815c7b0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef88152010 con 0x7fef8815c7b0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef881525d0 con 0x7fef8815c7b0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef881583f0 con 0x7fef8815c7b0 2024-09-17T03:21:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fef7401a070 con 0x7fef8815c7b0 2024-09-17T03:21:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef7fe00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef68073200 0x7fef68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef74093490 con 0x7fef8815c7b0 2024-09-17T03:21:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.533+0000 7fef87e00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef68073200 0x7fef68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.537+0000 7fef87e00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef68073200 0x7fef68075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef78004630 tx=0x7fef7800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.537+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fef7405de40 con 0x7fef8815c7b0 2024-09-17T03:21:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fef88068df0 con 0x7fef8815c7b0 2024-09-17T03:21:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef7fe00640 1 -- 172.21.15.46:0/4064951191 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fef74062870 con 0x7fef8815c7b0 2024-09-17T03:21:30.750 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:21:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef68073200 msgr2=0x7fef68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef68073200 0x7fef68075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef78004630 tx=0x7fef7800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 msgr2=0x7fef8814b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 0x7fef8814b8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c370 tx=0x7fef7400c830 comp rx=0 tx=0).stop 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 shutdown_connections 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef68073200 0x7fef68075710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef88160af0 0x7fef881530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8815c7b0 0x7fef8814b8f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 --2- 172.21.15.46:0/4064951191 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef88069330 0x7fef8814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 >> 172.21.15.46:0/4064951191 conn(0x7fef880b35f0 msgr2=0x7fef8814e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 shutdown_connections 2024-09-17T03:21:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:30.749+0000 7fef8ec00640 1 -- 172.21.15.46:0/4064951191 wait complete. 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:21:30.772 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T03:21:30.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:30.773 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-17T03:21:30.782 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:30.782 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:30.783 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-17T03:21:30.784 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-09-17T03:21:30.785 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:30.786 INFO:teuthology.orchestra.run.smithi099.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:21:30.786 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 02:42:4e:d9:c6:76 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:30.786 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:21:30.786 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:30.787 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T03:21:30.787 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:30.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:30.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:30.950 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:21:30.969 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-17T03:21:30.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:30.971 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-17T03:21:31.044 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-17T03:21:31.044 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:21:31.044 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T03:21:31.045 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:21:31.045 INFO:teuthology.orchestra.run.smithi099.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T03:21:31.045 INFO:teuthology.orchestra.run.smithi099.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:21:31.045 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:21:31.045 INFO:teuthology.orchestra.run.smithi099.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T03:21:31.045 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T03:21:31.046 INFO:teuthology.orchestra.run.smithi099.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T03:21:31.046 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:21:31.046 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:31.046 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:31.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:21:31.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:21:31.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:21:31.046 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:31.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:31.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:31.181 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-17T03:21:31.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:31.191 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-09-17T03:21:31.240 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:31.241 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:31.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:21:31.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T03:21:31.241 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:31.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:31.367 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T03:21:31.367 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:31.367 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:31.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T03:21:31.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T03:21:31.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T03:21:31.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T03:21:31.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T03:21:31.368 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:31.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:31.582 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:31.582 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:31.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-17T03:21:31.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T03:21:31.582 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:31.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:31.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:31.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:31.665 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T03:21:31.674 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T03:21:31.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:31.675 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T03:21:31.727 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-17T03:21:31.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:31.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:31.831 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:31.831 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:32.425 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:32.425 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:32.425 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:21:32.426 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 1@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:21:32.426 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:21:32.429 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:21:32.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:32.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T03:21:32.449 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:32.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:32.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:32.562 INFO:teuthology.orchestra.run.smithi099.stdout:client244636 2024-09-17T03:21:32.562 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:32.563 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:32.571 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:32.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:32.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:32.666 INFO:teuthology.orchestra.run.smithi099.stdout:client244636 2024-09-17T03:21:32.667 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:32.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:32.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:32.721 INFO:teuthology.orchestra.run.smithi099.stdout:client244636 2024-09-17T03:21:32.722 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/6c25af6b-3689-4426-acb2-6ebe6de6efd5.client244636/status') 2024-09-17T03:21:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:32.780 INFO:teuthology.orchestra.run.smithi099.stdout:instance: client.244636 (3)192.168.0.1:0/2564122988 2024-09-17T03:21:32.780 INFO:teuthology.orchestra.run.smithi099.stdout:blocklisted: false 2024-09-17T03:21:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records in 2024-09-17T03:21:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records out 2024-09-17T03:21:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:71 bytes copied, 4.9237e-05 s, 1.4 MB/s 2024-09-17T03:21:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:32.788 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-17T03:21:32.788 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:21:32.789 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T03:21:32.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:21:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 -- 172.21.15.46:0/1924215211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 msgr2=0x7f3fe406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/1924215211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe406e520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009840 tx=0x7f3fd401c820 comp rx=0 tx=0).stop 2024-09-17T03:21:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 -- 172.21.15.46:0/1924215211 shutdown_connections 2024-09-17T03:21:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/1924215211 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fe415f180 0x7f3fe415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/1924215211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe406e520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 -- 172.21.15.46:0/1924215211 >> 172.21.15.46:0/1924215211 conn(0x7f3fe40708c0 msgr2=0x7f3fe4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 -- 172.21.15.46:0/1924215211 shutdown_connections 2024-09-17T03:21:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.961+0000 7f3fe9e00640 1 -- 172.21.15.46:0/1924215211 wait complete. 2024-09-17T03:21:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 Processor -- start 2024-09-17T03:21:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 -- start start 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe40d29c0 0x7f3fe41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe41435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fe415f180 0x7f3fe41474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe416c4a0 con 0x7f3fe4155ad0 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe416c320 con 0x7f3fe40d29c0 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe9e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe416c620 con 0x7f3fe415f180 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe41435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe41435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56250/0 (socket says 172.21.15.46:56250) 2024-09-17T03:21:32.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fdbe00640 1 -- 172.21.15.46:0/458322142 learned_addr learned my addr 172.21.15.46:0/458322142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:32.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe2a00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe40d29c0 0x7f3fe41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:32.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.965+0000 7f3fe3400640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fe415f180 0x7f3fe41474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdbe00640 1 -- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fe415f180 msgr2=0x7f3fe41474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdbe00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fe415f180 0x7f3fe41474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdbe00640 1 -- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe40d29c0 msgr2=0x7f3fe41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdbe00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe40d29c0 0x7f3fe41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdbe00640 1 -- 172.21.15.46:0/458322142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe40b2b00 con 0x7f3fe4155ad0 2024-09-17T03:21:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdbe00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe41435b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c3a0 tx=0x7f3fcc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc017070 con 0x7f3fe4155ad0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fcc003600 con 0x7f3fe4155ad0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc003a50 con 0x7f3fe4155ad0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe4148550 con 0x7f3fe4155ad0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe41501c0 con 0x7f3fe4155ad0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe415f590 con 0x7f3fe4155ad0 2024-09-17T03:21:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.969+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3fcc01a070 con 0x7f3fe4155ad0 2024-09-17T03:21:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.973+0000 7f3fdb400640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fc0073200 0x7f3fc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.973+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3fcc093930 con 0x7f3fe4155ad0 2024-09-17T03:21:32.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.973+0000 7f3fe2a00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fc0073200 0x7f3fc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:32.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.973+0000 7f3fe2a00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fc0073200 0x7f3fc0075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40af000 tx=0x7f3fd401b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:32.973+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3fcc021020 con 0x7f3fe4155ad0 2024-09-17T03:21:33.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.181+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3fe40d3150 con 0x7f3fe4155ad0 2024-09-17T03:21:33.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fdb400640 1 -- 172.21.15.46:0/458322142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3fcc05e2e0 con 0x7f3fe4155ad0 2024-09-17T03:21:33.185 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fc0073200 msgr2=0x7f3fc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fc0073200 0x7f3fc0075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40af000 tx=0x7f3fd401b060 comp rx=0 tx=0).stop 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 msgr2=0x7f3fe41435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe41435b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c3a0 tx=0x7f3fcc00c860 comp rx=0 tx=0).stop 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 shutdown_connections 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fc0073200 0x7f3fc0075710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fe415f180 0x7f3fe41474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe4155ad0 0x7f3fe41435b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 --2- 172.21.15.46:0/458322142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fe40d29c0 0x7f3fe41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 >> 172.21.15.46:0/458322142 conn(0x7f3fe40708c0 msgr2=0x7f3fe4152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 shutdown_connections 2024-09-17T03:21:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:33.185+0000 7f3fe9e00640 1 -- 172.21.15.46:0/458322142 wait complete. 2024-09-17T03:21:33.207 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:21:33.208 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T03:21:33.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.208 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-17T03:21:33.217 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.217 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:33.218 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42376sec preferred_lft 42376sec 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 02:42:4e:d9:c6:76 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:21:33.219 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:21:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:21:33.220 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.221 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:21:33.285 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-17T03:21:33.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.288 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-17T03:21:33.339 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-17T03:21:33.339 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:21:33.339 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T03:21:33.340 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:21:33.341 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:33.341 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:33.341 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:21:33.341 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:21:33.341 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:21:33.341 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:33.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:33.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:33.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.473 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-17T03:21:33.482 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T03:21:33.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.483 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-09-17T03:21:33.532 INFO:teuthology.orchestra.run.smithi099.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T03:21:33.532 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:33.532 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:33.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:21:33.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T03:21:33.532 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:33.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:33.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.657 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether be:08:07:42:67:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::bc08:7ff:fe42:67a9/64 scope link 2024-09-17T03:21:33.678 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:33.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:21:33.743 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether be:08:07:42:67:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:21:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:21:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::bc08:7ff:fe42:67a9/64 scope link 2024-09-17T03:21:33.744 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:21:33.745 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T03:21:33.745 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:33.745 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:33.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T03:21:33.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T03:21:33.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T03:21:33.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T03:21:33.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T03:21:33.746 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:33.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:33.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:33.924 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:21:33.925 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-17T03:21:33.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.1 up 2024-09-17T03:21:33.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T03:21:33.925 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-17T03:21:33.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:34.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:34.008 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T03:21:34.059 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T03:21:34.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:21:34.060 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T03:21:34.111 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-17T03:21:34.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:34.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:34.211 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:34.211 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:34.225 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:21:34.225 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:21:34.226 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:21:34.226 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 3@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:21:34.226 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:21:34.229 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:21:34.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:34.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T03:21:34.249 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:34.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:34.347 INFO:teuthology.orchestra.run.smithi099.stdout:client244481 2024-09-17T03:21:34.348 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:34.348 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:34.400 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:34.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:34.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:34.502 INFO:teuthology.orchestra.run.smithi099.stdout:client244481 2024-09-17T03:21:34.502 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:21:34.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:21:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:34.598 INFO:teuthology.orchestra.run.smithi099.stdout:client244481 2024-09-17T03:21:34.598 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/6c25af6b-3689-4426-acb2-6ebe6de6efd5.client244481/status') 2024-09-17T03:21:34.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:21:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:instance: client.244481 (3)192.168.0.2:0/54891795 2024-09-17T03:21:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:blocklisted: false 2024-09-17T03:21:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records in 2024-09-17T03:21:34.657 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records out 2024-09-17T03:21:34.657 INFO:teuthology.orchestra.run.smithi099.stderr:69 bytes copied, 9.2638e-05 s, 745 kB/s 2024-09-17T03:21:34.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-17T03:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:21:34.664 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T03:21:34.757 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-17T03:21:34.758 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:21:34.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 -- 172.21.15.46:0/4045130864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 msgr2=0x7f30a415dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 --2- 172.21.15.46:0/4045130864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a415dfe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f309c009c80 tx=0x7f309c01cda0 comp rx=0 tx=0).stop 2024-09-17T03:21:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 -- 172.21.15.46:0/4045130864 shutdown_connections 2024-09-17T03:21:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 --2- 172.21.15.46:0/4045130864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a415dfe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 --2- 172.21.15.46:0/4045130864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30a415d280 0x7f30a415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 -- 172.21.15.46:0/4045130864 >> 172.21.15.46:0/4045130864 conn(0x7f30a40b2040 msgr2=0x7f30a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 -- 172.21.15.46:0/4045130864 shutdown_connections 2024-09-17T03:21:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 -- 172.21.15.46:0/4045130864 wait complete. 2024-09-17T03:21:34.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 Processor -- start 2024-09-17T03:21:34.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.929+0000 7f30a8c00640 1 -- start start 2024-09-17T03:21:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30a415d280 0x7f30a4143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a41f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30a415e8c0 0x7f30a4140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41681b0 con 0x7f30a415dbd0 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4168030 con 0x7f30a415e8c0 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f30a4168330 con 0x7f30a415d280 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a41f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a41f8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56272/0 (socket says 172.21.15.46:56272) 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a1e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30a415d280 0x7f30a4143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 -- 172.21.15.46:0/2797274612 learned_addr learned my addr 172.21.15.46:0/2797274612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a2800640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30a415e8c0 0x7f30a4140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 -- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30a415d280 msgr2=0x7f30a4143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30a415d280 0x7f30a4143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 -- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30a415e8c0 msgr2=0x7f30a4140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30a415e8c0 0x7f30a4140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 -- 172.21.15.46:0/2797274612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a4163e00 con 0x7f30a415dbd0 2024-09-17T03:21:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309be00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a41f8c10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f308c00db40 tx=0x7f308c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c004020 con 0x7f30a415dbd0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f308c0042b0 con 0x7f30a415dbd0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c003860 con 0x7f30a415dbd0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41443f0 con 0x7f30a415dbd0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.933+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41643a0 con 0x7f30a415dbd0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.937+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a415d690 con 0x7f30a415dbd0 2024-09-17T03:21:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.937+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f308c003a00 con 0x7f30a415dbd0 2024-09-17T03:21:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.937+0000 7f309a000640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3080073200 0x7f3080075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.937+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f308c094750 con 0x7f30a415dbd0 2024-09-17T03:21:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.937+0000 7f30a1e00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3080073200 0x7f3080075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.937+0000 7f30a1e00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3080073200 0x7f3080075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3094007fd0 tx=0x7f3094009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:34.941+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f308c05f110 con 0x7f30a415dbd0 2024-09-17T03:21:35.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.117+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30a4144180 con 0x7f30a415dbd0 2024-09-17T03:21:35.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.117+0000 7f309a000640 1 -- 172.21.15.46:0/2797274612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f308c063b40 con 0x7f30a415dbd0 2024-09-17T03:21:35.119 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:35.120 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3080073200 msgr2=0x7f3080075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3080073200 0x7f3080075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3094007fd0 tx=0x7f3094009040 comp rx=0 tx=0).stop 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 msgr2=0x7f30a41f8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a41f8c10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f308c00db40 tx=0x7f308c00a250 comp rx=0 tx=0).stop 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 shutdown_connections 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3080073200 0x7f3080075710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30a415e8c0 0x7f30a4140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a415dbd0 0x7f30a41f8c10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 --2- 172.21.15.46:0/2797274612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30a415d280 0x7f30a4143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 >> 172.21.15.46:0/2797274612 conn(0x7f30a40b2040 msgr2=0x7f30a4165f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:35.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 shutdown_connections 2024-09-17T03:21:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.121+0000 7f30a8c00640 1 -- 172.21.15.46:0/2797274612 wait complete. 2024-09-17T03:21:35.142 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T03:21:05.388964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T03:21:06.409730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:21:35.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 -- 172.21.15.46:0/654426307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a41527b0 msgr2=0x7f99a4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 --2- 172.21.15.46:0/654426307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a41527b0 0x7f99a4074e30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99a4071cb0 tx=0x7f998c01c800 comp rx=0 tx=0).stop 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 -- 172.21.15.46:0/654426307 shutdown_connections 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 --2- 172.21.15.46:0/654426307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a41527b0 0x7f99a4074e30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 --2- 172.21.15.46:0/654426307 >> v2:172.21.15.99:3300/0 conn(0x7f99a40b5c70 0x7f99a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 -- 172.21.15.46:0/654426307 >> 172.21.15.46:0/654426307 conn(0x7f99a40b2040 msgr2=0x7f99a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:35.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 -- 172.21.15.46:0/654426307 shutdown_connections 2024-09-17T03:21:35.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 -- 172.21.15.46:0/654426307 wait complete. 2024-09-17T03:21:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 Processor -- start 2024-09-17T03:21:35.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.305+0000 7f99a9c00640 1 -- start start 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a40b4de0 0x7f99a41fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 0x7f99a41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99a41527b0 0x7f99a4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4168b50 con 0x7f99a40b5c70 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41689d0 con 0x7f99a40b4de0 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f99a4168cd0 con 0x7f99a41527b0 2024-09-17T03:21:35.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 0x7f99a41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a2a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a40b4de0 0x7f99a41fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 0x7f99a41599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56282/0 (socket says 172.21.15.46:56282) 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 -- 172.21.15.46:0/3813102292 learned_addr learned my addr 172.21.15.46:0/3813102292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a3400640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99a41527b0 0x7f99a4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 -- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99a41527b0 msgr2=0x7f99a4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99a41527b0 0x7f99a4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 -- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a40b4de0 msgr2=0x7f99a41fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a40b4de0 0x7f99a41fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 -- 172.21.15.46:0/3813102292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a414ecc0 con 0x7f99a40b5c70 2024-09-17T03:21:35.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999a000640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 0x7f99a41599b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f998c01aeb0 tx=0x7f998c0a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0bb070 con 0x7f99a40b5c70 2024-09-17T03:21:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f998c0044b0 con 0x7f99a40b5c70 2024-09-17T03:21:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0b2040 con 0x7f99a40b5c70 2024-09-17T03:21:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a414efa0 con 0x7f99a40b5c70 2024-09-17T03:21:35.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a414f560 con 0x7f99a40b5c70 2024-09-17T03:21:35.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.309+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a41503b0 con 0x7f99a40b5c70 2024-09-17T03:21:35.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.313+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f998c0be070 con 0x7f99a40b5c70 2024-09-17T03:21:35.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.313+0000 7f999b400640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9980073200 0x7f9980075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.313+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f998c12dbd0 con 0x7f99a40b5c70 2024-09-17T03:21:35.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.313+0000 7f99a2a00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9980073200 0x7f9980075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.313+0000 7f99a2a00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9980073200 0x7f9980075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9994004630 tx=0x7f999400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:35.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.317+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f99a4075580 con 0x7f99a40b5c70 2024-09-17T03:21:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.493+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99a415bb60 con 0x7f99a40b5c70 2024-09-17T03:21:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.493+0000 7f999b400640 1 -- 172.21.15.46:0/3813102292 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f998c0164b0 con 0x7f99a40b5c70 2024-09-17T03:21:35.496 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9980073200 msgr2=0x7f9980075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9980073200 0x7f9980075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9994004630 tx=0x7f999400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 msgr2=0x7f99a41599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 0x7f99a41599b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f998c01aeb0 tx=0x7f998c0a5800 comp rx=0 tx=0).stop 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 shutdown_connections 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9980073200 0x7f9980075710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99a41527b0 0x7f99a4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99a40b5c70 0x7f99a41599b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 --2- 172.21.15.46:0/3813102292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a40b4de0 0x7f99a41fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 >> 172.21.15.46:0/3813102292 conn(0x7f99a40b2040 msgr2=0x7f99a40b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 shutdown_connections 2024-09-17T03:21:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.497+0000 7f99a9c00640 1 -- 172.21.15.46:0/3813102292 wait complete. 2024-09-17T03:21:35.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 -- 172.21.15.46:0/616019717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 msgr2=0x7f08a80cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 --2- 172.21.15.46:0/616019717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80cf5c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009c80 tx=0x7f08a001cda0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 -- 172.21.15.46:0/616019717 shutdown_connections 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 --2- 172.21.15.46:0/616019717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80cf5c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 --2- 172.21.15.46:0/616019717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08a81602a0 0x7f08a81606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 -- 172.21.15.46:0/616019717 >> 172.21.15.46:0/616019717 conn(0x7f08a80b2c40 msgr2=0x7f08a80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 -- 172.21.15.46:0/616019717 shutdown_connections 2024-09-17T03:21:35.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 -- 172.21.15.46:0/616019717 wait complete. 2024-09-17T03:21:35.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 Processor -- start 2024-09-17T03:21:35.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.681+0000 7f08af400640 1 -- start start 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08a80cd2e0 0x7f08a80b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a81602a0 0x7f08a80ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8169ed0 con 0x7f08a80cf1b0 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8169d50 con 0x7f08a81602a0 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f08a816a050 con 0x7f08a80cd2e0 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56296/0 (socket says 172.21.15.46:56296) 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 -- 172.21.15.46:0/972897999 learned_addr learned my addr 172.21.15.46:0/972897999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08aca00640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08a80cd2e0 0x7f08a80b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08ad400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a81602a0 0x7f08a80ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 -- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08a80cd2e0 msgr2=0x7f08a80b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08a80cd2e0 0x7f08a80b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 -- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a81602a0 msgr2=0x7f08a80ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a81602a0 0x7f08a80ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 -- 172.21.15.46:0/972897999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a81fc8b0 con 0x7f08a80cf1b0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f089fe00640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80ac040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f089000e7a0 tx=0x7f089000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089000ccc0 con 0x7f08a80cf1b0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f089000ce60 con 0x7f08a80cf1b0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a81fcb90 con 0x7f08a80cf1b0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890005260 con 0x7f08a80cf1b0 2024-09-17T03:21:35.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81fd150 con 0x7f08a80cf1b0 2024-09-17T03:21:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a80cf5c0 con 0x7f08a80cf1b0 2024-09-17T03:21:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.685+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0890005400 con 0x7f08a80cf1b0 2024-09-17T03:21:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.689+0000 7f08a5600640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0884073200 0x7f0884075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.689+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0890094a70 con 0x7f08a80cf1b0 2024-09-17T03:21:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.689+0000 7f08aca00640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0884073200 0x7f0884075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:35.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.689+0000 7f08aca00640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0884073200 0x7f0884075710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0898007fd0 tx=0x7f0898009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.693+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f089005f430 con 0x7f08a80cf1b0 2024-09-17T03:21:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.869+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08a81fc610 con 0x7f08a80cf1b0 2024-09-17T03:21:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.869+0000 7f08a5600640 1 -- 172.21.15.46:0/972897999 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f0890063e60 con 0x7f08a80cf1b0 2024-09-17T03:21:35.870 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.869+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0884073200 msgr2=0x7f0884075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.869+0000 7f08af400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0884073200 0x7f0884075710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0898007fd0 tx=0x7f0898009040 comp rx=0 tx=0).stop 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 msgr2=0x7f08a80ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80ac040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f089000e7a0 tx=0x7f089000ec60 comp rx=0 tx=0).stop 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 shutdown_connections 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0884073200 0x7f0884075710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a81602a0 0x7f08a80ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08a80cf1b0 0x7f08a80ac040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 --2- 172.21.15.46:0/972897999 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08a80cd2e0 0x7f08a80b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 >> 172.21.15.46:0/972897999 conn(0x7f08a80b2c40 msgr2=0x7f08a80d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 shutdown_connections 2024-09-17T03:21:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:35.873+0000 7f08af400640 1 -- 172.21.15.46:0/972897999 wait complete. 2024-09-17T03:21:35.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T03:21:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 -- 172.21.15.46:0/2905618137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 msgr2=0x7feb9815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 --2- 172.21.15.46:0/2905618137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb9815bb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb98072e30 tx=0x7feb8801c800 comp rx=0 tx=0).stop 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 -- 172.21.15.46:0/2905618137 shutdown_connections 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 --2- 172.21.15.46:0/2905618137 >> v2:172.21.15.99:3300/0 conn(0x7feb980b4de0 0x7feb980b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 --2- 172.21.15.46:0/2905618137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb981527b0 0x7feb98074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 --2- 172.21.15.46:0/2905618137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb9815bb60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 -- 172.21.15.46:0/2905618137 >> 172.21.15.46:0/2905618137 conn(0x7feb980b2040 msgr2=0x7feb980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 -- 172.21.15.46:0/2905618137 shutdown_connections 2024-09-17T03:21:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 -- 172.21.15.46:0/2905618137 wait complete. 2024-09-17T03:21:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.053+0000 7feb9f200640 1 Processor -- start 2024-09-17T03:21:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- start start 2024-09-17T03:21:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb980b4de0 0x7feb980adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb98150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb981527b0 0x7feb980ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb9815f700 con 0x7feb980b5c70 2024-09-17T03:21:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb9815f580 con 0x7feb980b4de0 2024-09-17T03:21:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feb9815f880 con 0x7feb981527b0 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb98150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb98150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56302/0 (socket says 172.21.15.46:56302) 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 -- 172.21.15.46:0/2302975539 learned_addr learned my addr 172.21.15.46:0/2302975539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb97e00640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb980b4de0 0x7feb980adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9d200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb981527b0 0x7feb980ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 -- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb981527b0 msgr2=0x7feb980ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb981527b0 0x7feb980ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 -- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb980b4de0 msgr2=0x7feb980adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb980b4de0 0x7feb980adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 -- 172.21.15.46:0/2302975539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb98151390 con 0x7feb980b5c70 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb97e00640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb980b4de0 0x7feb980adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb8f400640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb98150e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb8000e7c0 tx=0x7feb8000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8000cc60 con 0x7feb980b5c70 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb8000ce00 con 0x7feb980b5c70 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb980abe90 con 0x7feb980b5c70 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80005280 con 0x7feb980b5c70 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb980ac450 con 0x7feb980b5c70 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.057+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9815fda0 con 0x7feb980b5c70 2024-09-17T03:21:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.061+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb80005420 con 0x7feb980b5c70 2024-09-17T03:21:36.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.061+0000 7feb94c00640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb74073200 0x7feb74075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.061+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feb80094c30 con 0x7feb980b5c70 2024-09-17T03:21:36.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.061+0000 7feb97e00640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb74073200 0x7feb74075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.061+0000 7feb97e00640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb74073200 0x7feb74075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb880af000 tx=0x7feb8801b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.065+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb8005fbb0 con 0x7feb980b5c70 2024-09-17T03:21:36.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.241+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb980abc20 con 0x7feb980b5c70 2024-09-17T03:21:36.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb94c00640 1 -- 172.21.15.46:0/2302975539 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7feb8005f4b0 con 0x7feb980b5c70 2024-09-17T03:21:36.245 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-09-17T03:21:36.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb74073200 msgr2=0x7feb74075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb74073200 0x7feb74075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb880af000 tx=0x7feb8801b040 comp rx=0 tx=0).stop 2024-09-17T03:21:36.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 msgr2=0x7feb98150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb98150e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb8000e7c0 tx=0x7feb8000ec80 comp rx=0 tx=0).stop 2024-09-17T03:21:36.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 shutdown_connections 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb74073200 0x7feb74075710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb981527b0 0x7feb980ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb980b5c70 0x7feb98150e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 --2- 172.21.15.46:0/2302975539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb980b4de0 0x7feb980adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 >> 172.21.15.46:0/2302975539 conn(0x7feb980b2040 msgr2=0x7feb980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 shutdown_connections 2024-09-17T03:21:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.245+0000 7feb9f200640 1 -- 172.21.15.46:0/2302975539 wait complete. 2024-09-17T03:21:36.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-17T03:21:36.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 -- 172.21.15.46:0/703862521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b4de0 msgr2=0x7f22380b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 --2- 172.21.15.46:0/703862521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b4de0 0x7f22380b51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2238071cb0 tx=0x7f223001cae0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 -- 172.21.15.46:0/703862521 shutdown_connections 2024-09-17T03:21:36.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 --2- 172.21.15.46:0/703862521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b4de0 0x7f22380b51f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 --2- 172.21.15.46:0/703862521 >> v2:172.21.15.99:3300/0 conn(0x7f22380b5c70 0x7f223815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 -- 172.21.15.46:0/703862521 >> 172.21.15.46:0/703862521 conn(0x7f22380b2040 msgr2=0x7f22380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 -- 172.21.15.46:0/703862521 shutdown_connections 2024-09-17T03:21:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 -- 172.21.15.46:0/703862521 wait complete. 2024-09-17T03:21:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.425+0000 7f223f200640 1 Processor -- start 2024-09-17T03:21:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 -- start start 2024-09-17T03:21:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22380b4de0 0x7f2238149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 0x7f2238156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22381527b0 0x7f22380ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22381689a0 con 0x7f22380b5c70 2024-09-17T03:21:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2238168820 con 0x7f22380b4de0 2024-09-17T03:21:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2238168b20 con 0x7f22381527b0 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 0x7f2238156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 0x7f2238156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56330/0 (socket says 172.21.15.46:56330) 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 -- 172.21.15.46:0/2444575468 learned_addr learned my addr 172.21.15.46:0/2444575468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f2237e00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22380b4de0 0x7f2238149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223d200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22381527b0 0x7f22380ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 -- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22381527b0 msgr2=0x7f22380ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22381527b0 0x7f22380ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 -- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22380b4de0 msgr2=0x7f2238149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22380b4de0 0x7f2238149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 -- 172.21.15.46:0/2444575468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2238157310 con 0x7f22380b5c70 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f222fe00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 0x7f2238156dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f222000b3e0 tx=0x7f222000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:36.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220015020 con 0x7f22380b5c70 2024-09-17T03:21:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2220011940 con 0x7f22380b5c70 2024-09-17T03:21:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220011d90 con 0x7f22380b5c70 2024-09-17T03:21:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22380ce080 con 0x7f22380b5c70 2024-09-17T03:21:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.429+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2238150130 con 0x7f22380b5c70 2024-09-17T03:21:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.433+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22380b51f0 con 0x7f22380b5c70 2024-09-17T03:21:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.433+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2220018020 con 0x7f22380b5c70 2024-09-17T03:21:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.433+0000 7f2234c00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2214073200 0x7f2214075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.433+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2220094090 con 0x7f22380b5c70 2024-09-17T03:21:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.433+0000 7f2237e00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2214073200 0x7f2214075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.433+0000 7f2237e00640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2214073200 0x7f2214075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22280098f0 tx=0x7f2228006f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.437+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f222001f020 con 0x7f22380b5c70 2024-09-17T03:21:36.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:36.673+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f22380cdca0 con 0x7f22380b5c70 2024-09-17T03:21:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f2234c00640 1 -- 172.21.15.46:0/2444575468 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v54) ==== 187+0+0 (secure 0 0 0) 0x7f222005ea60 con 0x7f22380b5c70 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2214073200 msgr2=0x7f2214075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2214073200 0x7f2214075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22280098f0 tx=0x7f2228006f90 comp rx=0 tx=0).stop 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 msgr2=0x7f2238156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 0x7f2238156dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f222000b3e0 tx=0x7f222000b8a0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 shutdown_connections 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2214073200 0x7f2214075710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f22381527b0 0x7f22380ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22380b5c70 0x7f2238156dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 --2- 172.21.15.46:0/2444575468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22380b4de0 0x7f2238149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 >> 172.21.15.46:0/2444575468 conn(0x7f22380b2040 msgr2=0x7f223814d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 shutdown_connections 2024-09-17T03:21:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.181+0000 7f223f200640 1 -- 172.21.15.46:0/2444575468 wait complete. 2024-09-17T03:21:37.206 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 -- 172.21.15.46:0/1050766775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364070050 msgr2=0x7f0364072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 --2- 172.21.15.46:0/1050766775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364070050 0x7f0364072ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03640d07f0 tx=0x7f035001c750 comp rx=0 tx=0).stop 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 -- 172.21.15.46:0/1050766775 shutdown_connections 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 --2- 172.21.15.46:0/1050766775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0364070050 0x7f0364072ba0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 --2- 172.21.15.46:0/1050766775 >> v2:172.21.15.99:3300/0 conn(0x7f03640730e0 0x7f03640734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 --2- 172.21.15.46:0/1050766775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03640af5b0 0x7f036406fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 -- 172.21.15.46:0/1050766775 >> 172.21.15.46:0/1050766775 conn(0x7f036406d3f0 msgr2=0x7f03640b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 -- 172.21.15.46:0/1050766775 shutdown_connections 2024-09-17T03:21:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.365+0000 7f036ae00640 1 -- 172.21.15.46:0/1050766775 wait complete. 2024-09-17T03:21:37.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 Processor -- start 2024-09-17T03:21:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 -- start start 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 0x7f0364157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03640af5b0 0x7f036415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f036406eba0 0x7f036406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f036415f960 con 0x7f03640730e0 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f036415f7e0 con 0x7f036406eba0 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f036ae00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f036415fae0 con 0x7f03640af5b0 2024-09-17T03:21:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 0x7f0364157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 0x7f0364157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56358/0 (socket says 172.21.15.46:56358) 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 -- 172.21.15.46:0/2960393404 learned_addr learned my addr 172.21.15.46:0/2960393404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 -- 172.21.15.46:0/2960393404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03640af5b0 msgr2=0x7f036415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03640af5b0 0x7f036415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 -- 172.21.15.46:0/2960393404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f036406eba0 msgr2=0x7f036406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f036406eba0 0x7f036406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.369+0000 7f0369a00640 1 -- 172.21.15.46:0/2960393404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036406f4f0 con 0x7f03640730e0 2024-09-17T03:21:37.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0369a00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 0x7f0364157fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f035800d720 tx=0x7f035800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358017070 con 0x7f03640730e0 2024-09-17T03:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0358009cc0 con 0x7f03640730e0 2024-09-17T03:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358005280 con 0x7f03640730e0 2024-09-17T03:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03641584f0 con 0x7f03640730e0 2024-09-17T03:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641fbee0 con 0x7f03640730e0 2024-09-17T03:21:37.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364072ba0 con 0x7f03640730e0 2024-09-17T03:21:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0358005420 con 0x7f03640730e0 2024-09-17T03:21:37.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0361600640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0344073230 0x7f0344075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0358094350 con 0x7f03640730e0 2024-09-17T03:21:37.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.373+0000 7f0369000640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0344073230 0x7f0344075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:37.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.377+0000 7f0369000640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0344073230 0x7f0344075740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f03500af000 tx=0x7f03500194f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.377+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f036406f4f0 con 0x7f03640730e0 2024-09-17T03:21:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.553+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0364073580 con 0x7f03640730e0 2024-09-17T03:21:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.553+0000 7f0361600640 1 -- 172.21.15.46:0/2960393404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f035805f360 con 0x7f03640730e0 2024-09-17T03:21:37.556 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:21:37.556 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:21:37.556 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:21:37.557 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:21:37.558 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:21:37.559 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:21:37.559 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:21:37.559 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0344073230 msgr2=0x7f0344075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0344073230 0x7f0344075740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f03500af000 tx=0x7f03500194f0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 msgr2=0x7f0364157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 0x7f0364157fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f035800d720 tx=0x7f035800dbe0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 shutdown_connections 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0344073230 0x7f0344075740 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f036406eba0 0x7f036406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03640af5b0 0x7f036415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 --2- 172.21.15.46:0/2960393404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03640730e0 0x7f0364157fb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 >> 172.21.15.46:0/2960393404 conn(0x7f036406d3f0 msgr2=0x7f03640cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 shutdown_connections 2024-09-17T03:21:37.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.557+0000 7f036ae00640 1 -- 172.21.15.46:0/2960393404 wait complete. 2024-09-17T03:21:37.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 -- 172.21.15.46:0/484813655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b415f6b0 msgr2=0x7f24b415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 --2- 172.21.15.46:0/484813655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b415f6b0 0x7f24b415fac0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f24b40b5f20 tx=0x7f24ac01ccd0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 -- 172.21.15.46:0/484813655 shutdown_connections 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 --2- 172.21.15.46:0/484813655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b415f6b0 0x7f24b415fac0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 --2- 172.21.15.46:0/484813655 >> v2:172.21.15.99:3300/0 conn(0x7f24b4074300 0x7f24b415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 --2- 172.21.15.46:0/484813655 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24b415c5d0 0x7f24b4073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 -- 172.21.15.46:0/484813655 >> 172.21.15.46:0/484813655 conn(0x7f24b40b2c40 msgr2=0x7f24b40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 -- 172.21.15.46:0/484813655 shutdown_connections 2024-09-17T03:21:37.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.737+0000 7f24bb800640 1 -- 172.21.15.46:0/484813655 wait complete. 2024-09-17T03:21:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 Processor -- start 2024-09-17T03:21:37.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 -- start start 2024-09-17T03:21:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 0x7f24b413d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24b415c5d0 0x7f24b413d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24b415f6b0 0x7f24b4073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24b406f3f0 con 0x7f24b4074300 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24b406f270 con 0x7f24b415f6b0 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24bb800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f24b406f570 con 0x7f24b415c5d0 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 0x7f24b413d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 0x7f24b413d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56376/0 (socket says 172.21.15.46:56376) 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 -- 172.21.15.46:0/3668057586 learned_addr learned my addr 172.21.15.46:0/3668057586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b9800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24b415f6b0 0x7f24b4073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24abe00640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24b415c5d0 0x7f24b413d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:37.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 -- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24b415c5d0 msgr2=0x7f24b413d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24b415c5d0 0x7f24b413d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 -- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24b415f6b0 msgr2=0x7f24b4073550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24b415f6b0 0x7f24b4073550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b8e00640 1 -- 172.21.15.46:0/3668057586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b40aab70 con 0x7f24b4074300 2024-09-17T03:21:37.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.741+0000 7f24b9800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24b415f6b0 0x7f24b4073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:37.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b8e00640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 0x7f24b413d2d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24a40058d0 tx=0x7f24a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4017070 con 0x7f24b4074300 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24a4009cc0 con 0x7f24b4074300 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40106c0 con 0x7f24b4074300 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b40aa890 con 0x7f24b4074300 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b4157660 con 0x7f24b4074300 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b406fb00 con 0x7f24b4074300 2024-09-17T03:21:37.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24a401a070 con 0x7f24b4074300 2024-09-17T03:21:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b1600640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2490073200 0x7f2490075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.745+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f24a4094750 con 0x7f24b4074300 2024-09-17T03:21:37.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.749+0000 7f24abe00640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2490073200 0x7f2490075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:37.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.749+0000 7f24abe00640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2490073200 0x7f2490075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f24b4155710 tx=0x7f249c00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:37.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.749+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f24b40aab70 con 0x7f24b4074300 2024-09-17T03:21:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.921+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f24b4155140 con 0x7f24b4074300 2024-09-17T03:21:37.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24b1600640 1 -- 172.21.15.46:0/3668057586 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f24a405f100 con 0x7f24b4074300 2024-09-17T03:21:37.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2490073200 msgr2=0x7f2490075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2490073200 0x7f2490075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f24b4155710 tx=0x7f249c00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 msgr2=0x7f24b413d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 0x7f24b413d2d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24a40058d0 tx=0x7f24a400dde0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 shutdown_connections 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2490073200 0x7f2490075710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24b415f6b0 0x7f24b4073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24b415c5d0 0x7f24b413d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 --2- 172.21.15.46:0/3668057586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b4074300 0x7f24b413d2d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 >> 172.21.15.46:0/3668057586 conn(0x7f24b40b2c40 msgr2=0x7f24b40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 shutdown_connections 2024-09-17T03:21:37.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:37.929+0000 7f24bb800640 1 -- 172.21.15.46:0/3668057586 wait complete. 2024-09-17T03:21:37.952 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:21:37.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:21:38.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:21:38.167 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:21:38.167 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:38.167 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:38.176 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:38.176 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:38.225 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:38.225 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:38.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:21:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 -- 172.21.15.46:0/3439301880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac003a10 msgr2=0x7f68ac003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 --2- 172.21.15.46:0/3439301880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac003a10 0x7f68ac003e80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c470 tx=0x7f68b801cf70 comp rx=0 tx=0).stop 2024-09-17T03:21:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cd400640 1 -- 172.21.15.46:0/3439301880 reap_dead start 2024-09-17T03:21:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 -- 172.21.15.46:0/3439301880 shutdown_connections 2024-09-17T03:21:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 -- 172.21.15.46:0/3439301880 >> 172.21.15.46:0/3439301880 conn(0x7f68c80b2040 msgr2=0x7f68c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 -- 172.21.15.46:0/3439301880 shutdown_connections 2024-09-17T03:21:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 -- 172.21.15.46:0/3439301880 wait complete. 2024-09-17T03:21:38.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 Processor -- start 2024-09-17T03:21:38.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.481+0000 7f68cf400640 1 -- start start 2024-09-17T03:21:38.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68ac003a10 0x7f68c80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:38.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68c815d820 0x7f68c81f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:38.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 0x7f68c8153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:38.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8166900 con 0x7f68c8152c20 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8166780 con 0x7f68ac003a10 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f68c8166a80 con 0x7f68c815d820 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 0x7f68c8153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 0x7f68c8153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56398/0 (socket says 172.21.15.46:56398) 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 -- 172.21.15.46:0/158266011 learned_addr learned my addr 172.21.15.46:0/158266011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68bfe00640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68c815d820 0x7f68c81f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cca00640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68ac003a10 0x7f68c80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 -- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68c815d820 msgr2=0x7f68c81f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68c815d820 0x7f68c81f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 -- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68ac003a10 msgr2=0x7f68c80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68ac003a10 0x7f68c80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 -- 172.21.15.46:0/158266011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c81fcd40 con 0x7f68c8152c20 2024-09-17T03:21:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68bfe00640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68c815d820 0x7f68c81f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cd400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 0x7f68c8153030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68c81596f0 tx=0x7f68c000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004b30 con 0x7f68c8152c20 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68c0004cd0 con 0x7f68c8152c20 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0003be0 con 0x7f68c8152c20 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c81fd020 con 0x7f68c8152c20 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c8154c70 con 0x7f68c8152c20 2024-09-17T03:21:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.485+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c815bb60 con 0x7f68c8152c20 2024-09-17T03:21:38.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.489+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68c0004030 con 0x7f68c8152c20 2024-09-17T03:21:38.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.489+0000 7f68c5600640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f68a4073200 0x7f68a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:38.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.489+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68c0094870 con 0x7f68c8152c20 2024-09-17T03:21:38.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.489+0000 7f68cca00640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f68a4073200 0x7f68a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:38.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.489+0000 7f68cca00640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f68a4073200 0x7f68a4075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c470 tx=0x7f68b8019db0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:38.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.493+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f68c005f7f0 con 0x7f68c8152c20 2024-09-17T03:21:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68c81592b0 con 0x7f68c8152c20 2024-09-17T03:21:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68c5600640 1 -- 172.21.15.46:0/158266011 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f68c005f0f0 con 0x7f68c8152c20 2024-09-17T03:21:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f68a4073200 msgr2=0x7f68a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f68a4073200 0x7f68a4075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c470 tx=0x7f68b8019db0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 msgr2=0x7f68c8153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 0x7f68c8153030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68c81596f0 tx=0x7f68c000f3c0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 shutdown_connections 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f68a4073200 0x7f68a4075710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c8152c20 0x7f68c8153030 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f68c815d820 0x7f68c81f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 --2- 172.21.15.46:0/158266011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68ac003a10 0x7f68c80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 >> 172.21.15.46:0/158266011 conn(0x7f68c80b2040 msgr2=0x7f68c81510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 shutdown_connections 2024-09-17T03:21:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:38.685+0000 7f68cf400640 1 -- 172.21.15.46:0/158266011 wait complete. 2024-09-17T03:21:38.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:21:38.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:38.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T03:21:38.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:21:38.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:38.741 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:21:38.804 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:38.804 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:38.814 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:38.814 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:38.864 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:38.865 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:38.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 -- 172.21.15.46:0/2769525259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f01527b0 msgr2=0x7f67f015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 --2- 172.21.15.46:0/2769525259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f01527b0 0x7f67f015bb60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67f0074b20 tx=0x7f67d801c820 comp rx=0 tx=0).stop 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 -- 172.21.15.46:0/2769525259 shutdown_connections 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 --2- 172.21.15.46:0/2769525259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f01527b0 0x7f67f015bb60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 -- 172.21.15.46:0/2769525259 >> 172.21.15.46:0/2769525259 conn(0x7f67f00b2040 msgr2=0x7f67f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 -- 172.21.15.46:0/2769525259 shutdown_connections 2024-09-17T03:21:39.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.077+0000 7f67f6200640 1 -- 172.21.15.46:0/2769525259 wait complete. 2024-09-17T03:21:39.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 Processor -- start 2024-09-17T03:21:39.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 -- start start 2024-09-17T03:21:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00b4de0 0x7f67f013c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 0x7f67f00af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67f01527b0 0x7f67f01f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0169b30 con 0x7f67f00b5c70 2024-09-17T03:21:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01699b0 con 0x7f67f00b4de0 2024-09-17T03:21:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f67f0169cb0 con 0x7f67f01527b0 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 0x7f67f00af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 0x7f67f00af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56412/0 (socket says 172.21.15.46:56412) 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 -- 172.21.15.46:0/2834459415 learned_addr learned my addr 172.21.15.46:0/2834459415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67ef400640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00b4de0 0x7f67f013c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67efe00640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67f01527b0 0x7f67f01f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 -- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67f01527b0 msgr2=0x7f67f01f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67f01527b0 0x7f67f01f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 -- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00b4de0 msgr2=0x7f67f013c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00b4de0 0x7f67f013c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 -- 172.21.15.46:0/2834459415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f00afc90 con 0x7f67f00b5c70 2024-09-17T03:21:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7e00640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 0x7f67f00af750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009850 tx=0x7f67d80a5850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d80b2040 con 0x7f67f00b5c70 2024-09-17T03:21:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67d80bd070 con 0x7f67f00b5c70 2024-09-17T03:21:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8003860 con 0x7f67f00b5c70 2024-09-17T03:21:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f0150e30 con 0x7f67f00b5c70 2024-09-17T03:21:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.081+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01513f0 con 0x7f67f00b5c70 2024-09-17T03:21:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.085+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f015bb60 con 0x7f67f00b5c70 2024-09-17T03:21:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.085+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67d8003a00 con 0x7f67f00b5c70 2024-09-17T03:21:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.085+0000 7f67e7400640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f67cc073200 0x7f67cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.085+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67d812ee10 con 0x7f67f00b5c70 2024-09-17T03:21:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.085+0000 7f67ef400640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f67cc073200 0x7f67cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.085+0000 7f67ef400640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f67cc073200 0x7f67cc075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67e0007fd0 tx=0x7f67e0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.089+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f67d8132030 con 0x7f67f00b5c70 2024-09-17T03:21:39.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.277+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67f00b2780 con 0x7f67f00b5c70 2024-09-17T03:21:39.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.277+0000 7f67e7400640 1 -- 172.21.15.46:0/2834459415 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f67d80164d0 con 0x7f67f00b5c70 2024-09-17T03:21:39.279 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:21:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f67cc073200 msgr2=0x7f67cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f67cc073200 0x7f67cc075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67e0007fd0 tx=0x7f67e0009040 comp rx=0 tx=0).stop 2024-09-17T03:21:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 msgr2=0x7f67f00af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 0x7f67f00af750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009850 tx=0x7f67d80a5850 comp rx=0 tx=0).stop 2024-09-17T03:21:39.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 shutdown_connections 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f67cc073200 0x7f67cc075710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f67f01527b0 0x7f67f01f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67f00b5c70 0x7f67f00af750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 --2- 172.21.15.46:0/2834459415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00b4de0 0x7f67f013c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 >> 172.21.15.46:0/2834459415 conn(0x7f67f00b2040 msgr2=0x7f67f015af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 shutdown_connections 2024-09-17T03:21:39.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.281+0000 7f67f6200640 1 -- 172.21.15.46:0/2834459415 wait complete. 2024-09-17T03:21:39.302 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T03:21:39.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:39.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-17T03:21:39.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:21:39.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:39.345 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T03:21:39.403 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:39.404 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:39.413 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:39.413 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:39.464 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:21:39.465 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:39.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:21:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 -- 172.21.15.46:0/3762021256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c8003c90 msgr2=0x7f52c8004100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 --2- 172.21.15.46:0/3762021256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c8003c90 0x7f52c8004100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f52d4001860 tx=0x7f52d401c800 comp rx=0 tx=0).stop 2024-09-17T03:21:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 -- 172.21.15.46:0/3762021256 shutdown_connections 2024-09-17T03:21:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 --2- 172.21.15.46:0/3762021256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c8003c90 0x7f52c8004100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 -- 172.21.15.46:0/3762021256 >> 172.21.15.46:0/3762021256 conn(0x7f52e40b2040 msgr2=0x7f52e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:39.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 -- 172.21.15.46:0/3762021256 shutdown_connections 2024-09-17T03:21:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 -- 172.21.15.46:0/3762021256 wait complete. 2024-09-17T03:21:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 Processor -- start 2024-09-17T03:21:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.721+0000 7f52eaa00640 1 -- start start 2024-09-17T03:21:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c8003c90 0x7f52e4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 0x7f52e41f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52e4159da0 0x7f52e415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52e415d300 con 0x7f52e41fd110 2024-09-17T03:21:39.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52e415d180 con 0x7f52c8003c90 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f52e415d480 con 0x7f52e4159da0 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 0x7f52e41f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 0x7f52e41f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56424/0 (socket says 172.21.15.46:56424) 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 -- 172.21.15.46:0/2527974184 learned_addr learned my addr 172.21.15.46:0/2527974184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52e3e00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c8003c90 0x7f52e4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52e8a00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52e4159da0 0x7f52e415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 -- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52e4159da0 msgr2=0x7f52e415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52e4159da0 0x7f52e415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 -- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c8003c90 msgr2=0x7f52e4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c8003c90 0x7f52e4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 -- 172.21.15.46:0/2527974184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e40ccb30 con 0x7f52e41fd110 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52db400640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 0x7f52e41f2d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006400 tx=0x7f52cc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:39.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc015070 con 0x7f52e41fd110 2024-09-17T03:21:39.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52cc011840 con 0x7f52e41fd110 2024-09-17T03:21:39.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc011c50 con 0x7f52e41fd110 2024-09-17T03:21:39.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e415a6f0 con 0x7f52e41fd110 2024-09-17T03:21:39.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41fc630 con 0x7f52e41fd110 2024-09-17T03:21:39.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.725+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e415d9e0 con 0x7f52e41fd110 2024-09-17T03:21:39.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.729+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f52cc018070 con 0x7f52e41fd110 2024-09-17T03:21:39.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.729+0000 7f52e0c00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52c0073230 0x7f52c0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:39.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.729+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f52cc094dd0 con 0x7f52e41fd110 2024-09-17T03:21:39.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.729+0000 7f52e3e00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52c0073230 0x7f52c0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:39.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.729+0000 7f52e3e00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52c0073230 0x7f52c0075740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52d4001830 tx=0x7f52d4019640 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:39.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.733+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52e415a6f0 con 0x7f52e41fd110 2024-09-17T03:21:39.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.917+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52e40cd360 con 0x7f52e41fd110 2024-09-17T03:21:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.917+0000 7f52e0c00640 1 -- 172.21.15.46:0/2527974184 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f52cc05f700 con 0x7f52e41fd110 2024-09-17T03:21:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:21:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52c0073230 msgr2=0x7f52c0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52c0073230 0x7f52c0075740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52d4001830 tx=0x7f52d4019640 comp rx=0 tx=0).stop 2024-09-17T03:21:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 msgr2=0x7f52e41f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:39.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 0x7f52e41f2d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006400 tx=0x7f52cc0039d0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 shutdown_connections 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52c0073230 0x7f52c0075740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52e4159da0 0x7f52e415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52e41fd110 0x7f52e41f2d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 --2- 172.21.15.46:0/2527974184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c8003c90 0x7f52e4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 >> 172.21.15.46:0/2527974184 conn(0x7f52e40b2040 msgr2=0x7f52e40ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 shutdown_connections 2024-09-17T03:21:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:39.921+0000 7f52eaa00640 1 -- 172.21.15.46:0/2527974184 wait complete. 2024-09-17T03:21:39.942 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-17T03:21:39.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:39.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-17T03:21:39.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:21:39.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:39.967 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T03:21:40.021 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:40.021 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:40.031 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:40.031 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:40.084 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:21:40.084 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:21:40.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:21:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 -- 172.21.15.46:0/746796523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ce4c0 msgr2=0x7faa5415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 --2- 172.21.15.46:0/746796523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ce4c0 0x7faa5415aa50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa44009840 tx=0x7faa4401c820 comp rx=0 tx=0).stop 2024-09-17T03:21:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 -- 172.21.15.46:0/746796523 shutdown_connections 2024-09-17T03:21:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 --2- 172.21.15.46:0/746796523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa540cdd80 0x7faa54165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 --2- 172.21.15.46:0/746796523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540ce4c0 0x7faa5415aa50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 -- 172.21.15.46:0/746796523 >> 172.21.15.46:0/746796523 conn(0x7faa540b2040 msgr2=0x7faa540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:40.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 -- 172.21.15.46:0/746796523 shutdown_connections 2024-09-17T03:21:40.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.293+0000 7faa59400640 1 -- 172.21.15.46:0/746796523 wait complete. 2024-09-17T03:21:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 Processor -- start 2024-09-17T03:21:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 -- start start 2024-09-17T03:21:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540cd260 0x7faa541fa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa540cdd80 0x7faa5413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 0x7faa541f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa54168c00 con 0x7faa540ce4c0 2024-09-17T03:21:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa54168a80 con 0x7faa540cd260 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa59400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faa54168d80 con 0x7faa540cdd80 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 0x7faa541f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa52800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540cd260 0x7faa541fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 0x7faa541f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56426/0 (socket says 172.21.15.46:56426) 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa53200640 1 -- 172.21.15.46:0/3430645791 learned_addr learned my addr 172.21.15.46:0/3430645791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.297+0000 7faa49e00640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa540cdd80 0x7faa5413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa53200640 1 -- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa540cdd80 msgr2=0x7faa5413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa53200640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa540cdd80 0x7faa5413eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa53200640 1 -- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540cd260 msgr2=0x7faa541fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa53200640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540cd260 0x7faa541fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa53200640 1 -- 172.21.15.46:0/3430645791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5413f070 con 0x7faa540ce4c0 2024-09-17T03:21:40.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa53200640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 0x7faa541f77b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009fd0 tx=0x7faa4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00ce20 con 0x7faa540ce4c0 2024-09-17T03:21:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa4c004b30 con 0x7faa540ce4c0 2024-09-17T03:21:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa541f7210 con 0x7faa540ce4c0 2024-09-17T03:21:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c005020 con 0x7faa540ce4c0 2024-09-17T03:21:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa541f9f60 con 0x7faa540ce4c0 2024-09-17T03:21:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa5415aa50 con 0x7faa540ce4c0 2024-09-17T03:21:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.301+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faa4c0051c0 con 0x7faa540ce4c0 2024-09-17T03:21:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.305+0000 7faa4b400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa30073200 0x7faa30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.305+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faa4c094d40 con 0x7faa540ce4c0 2024-09-17T03:21:40.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.305+0000 7faa52800640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa30073200 0x7faa30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.305+0000 7faa52800640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa30073200 0x7faa30075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa440af000 tx=0x7faa4401b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.309+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faa4c05f710 con 0x7faa540ce4c0 2024-09-17T03:21:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.493+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faa540b4010 con 0x7faa540ce4c0 2024-09-17T03:21:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.493+0000 7faa4b400640 1 -- 172.21.15.46:0/3430645791 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7faa4c064140 con 0x7faa540ce4c0 2024-09-17T03:21:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa30073200 msgr2=0x7faa30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa30073200 0x7faa30075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa440af000 tx=0x7faa4401b060 comp rx=0 tx=0).stop 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 msgr2=0x7faa541f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 0x7faa541f77b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009fd0 tx=0x7faa4c00ef10 comp rx=0 tx=0).stop 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 shutdown_connections 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa30073200 0x7faa30075710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa540ce4c0 0x7faa541f77b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa540cdd80 0x7faa5413eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 --2- 172.21.15.46:0/3430645791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa540cd260 0x7faa541fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 >> 172.21.15.46:0/3430645791 conn(0x7faa540b2040 msgr2=0x7faa540cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 shutdown_connections 2024-09-17T03:21:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.497+0000 7faa59400640 1 -- 172.21.15.46:0/3430645791 wait complete. 2024-09-17T03:21:40.517 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-17T03:21:40.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:40.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-17T03:21:40.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:21:40.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:21:40.544 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T03:21:40.601 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:21:40.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 -- 172.21.15.46:0/4033442917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 msgr2=0x7f73c415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 --2- 172.21.15.46:0/4033442917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c415bb90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73c40ce1a0 tx=0x7f73b801c650 comp rx=0 tx=0).stop 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 -- 172.21.15.46:0/4033442917 shutdown_connections 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 --2- 172.21.15.46:0/4033442917 >> v2:172.21.15.99:3300/0 conn(0x7f73c40703a0 0x7f73c406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 --2- 172.21.15.46:0/4033442917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c415bb90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 -- 172.21.15.46:0/4033442917 >> 172.21.15.46:0/4033442917 conn(0x7f73c40b35f0 msgr2=0x7f73c40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 -- 172.21.15.46:0/4033442917 shutdown_connections 2024-09-17T03:21:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.757+0000 7f73cc200640 1 -- 172.21.15.46:0/4033442917 wait complete. 2024-09-17T03:21:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 Processor -- start 2024-09-17T03:21:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 -- start start 2024-09-17T03:21:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c406cdf0 0x7f73c4154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c40703a0 0x7f73c41f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4168d40 con 0x7f73c4159130 2024-09-17T03:21:40.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4168bc0 con 0x7f73c40703a0 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73cc200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f73c4168ec0 con 0x7f73c406cdf0 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73ca200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73c9800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c406cdf0 0x7f73c4154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73c9800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c406cdf0 0x7f73c4154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73ca200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56438/0 (socket says 172.21.15.46:56438) 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73c9800640 1 -- 172.21.15.46:0/1624709914 learned_addr learned my addr 172.21.15.46:0/1624709914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:40.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.761+0000 7f73c8e00640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c40703a0 0x7f73c41f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73ca200640 1 -- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c406cdf0 msgr2=0x7f73c4154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73ca200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c406cdf0 0x7f73c4154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73ca200640 1 -- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c40703a0 msgr2=0x7f73c41f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73ca200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c40703a0 0x7f73c41f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73ca200640 1 -- 172.21.15.46:0/1624709914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c4155410 con 0x7f73c4159130 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73ca200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c414da30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e990 tx=0x7f73bc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc004d30 con 0x7f73c4159130 2024-09-17T03:21:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73bc00caa0 con 0x7f73c4159130 2024-09-17T03:21:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc005020 con 0x7f73c4159130 2024-09-17T03:21:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c41f6c70 con 0x7f73c4159130 2024-09-17T03:21:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c4155be0 con 0x7f73c4159130 2024-09-17T03:21:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c406d200 con 0x7f73c4159130 2024-09-17T03:21:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.765+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f73bc0051c0 con 0x7f73c4159130 2024-09-17T03:21:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.769+0000 7f73c1600640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73a4073200 0x7f73a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.769+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f73bc094ab0 con 0x7f73c4159130 2024-09-17T03:21:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.769+0000 7f73c9800640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73a4073200 0x7f73a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.769+0000 7f73c9800640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73a4073200 0x7f73a4075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009230 tx=0x7f73b801ae90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.773+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f73bc016030 con 0x7f73c4159130 2024-09-17T03:21:40.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.985+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c4153e00 con 0x7f73c4159130 2024-09-17T03:21:40.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.985+0000 7f73c1600640 1 -- 172.21.15.46:0/1624709914 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7f73bc05f3e0 con 0x7f73c4159130 2024-09-17T03:21:40.988 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:40.988 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:40.989 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:40.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73a4073200 msgr2=0x7f73a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73a4073200 0x7f73a4075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009230 tx=0x7f73b801ae90 comp rx=0 tx=0).stop 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 msgr2=0x7f73c414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c414da30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e990 tx=0x7f73bc00ee50 comp rx=0 tx=0).stop 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 shutdown_connections 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73a4073200 0x7f73a4075710 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4159130 0x7f73c414da30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73c40703a0 0x7f73c41f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 --2- 172.21.15.46:0/1624709914 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73c406cdf0 0x7f73c4154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 >> 172.21.15.46:0/1624709914 conn(0x7f73c40b35f0 msgr2=0x7f73c4075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 shutdown_connections 2024-09-17T03:21:40.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:40.989+0000 7f73cc200640 1 -- 172.21.15.46:0/1624709914 wait complete. 2024-09-17T03:21:41.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 -- 172.21.15.46:0/3444397829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4157030 msgr2=0x7f5bc40b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/3444397829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4157030 0x7f5bc40b5a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009900 tx=0x7f5bb401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 -- 172.21.15.46:0/3444397829 shutdown_connections 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/3444397829 >> v2:172.21.15.99:3300/0 conn(0x7f5bc4160d50 0x7f5bc4074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/3444397829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4157030 0x7f5bc40b5a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 -- 172.21.15.46:0/3444397829 >> 172.21.15.46:0/3444397829 conn(0x7f5bc40708c0 msgr2=0x7f5bc4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 -- 172.21.15.46:0/3444397829 shutdown_connections 2024-09-17T03:21:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 -- 172.21.15.46:0/3444397829 wait complete. 2024-09-17T03:21:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.169+0000 7f5bc8e00640 1 Processor -- start 2024-09-17T03:21:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- start start 2024-09-17T03:21:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 0x7f5bc41fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4160d50 0x7f5bc4147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc4161160 0x7f5bc41463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc416a780 con 0x7f5bc4157030 2024-09-17T03:21:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc416a600 con 0x7f5bc4161160 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc416a900 con 0x7f5bc4160d50 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 0x7f5bc41fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 0x7f5bc41fdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56452/0 (socket says 172.21.15.46:56452) 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 -- 172.21.15.46:0/81087739 learned_addr learned my addr 172.21.15.46:0/81087739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc2800640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc4161160 0x7f5bc41463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bbbe00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4160d50 0x7f5bc4147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 -- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4160d50 msgr2=0x7f5bc4147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4160d50 0x7f5bc4147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 -- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc4161160 msgr2=0x7f5bc41463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc4161160 0x7f5bc41463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 -- 172.21.15.46:0/81087739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc41fd2c0 con 0x7f5bc4157030 2024-09-17T03:21:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc2800640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc4161160 0x7f5bc41463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bbbe00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4160d50 0x7f5bc4147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc1e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 0x7f5bc41fdd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009850 tx=0x7f5bb4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40b2040 con 0x7f5bc4157030 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb4004330 con 0x7f5bc4157030 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4005280 con 0x7f5bc4157030 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc4147e50 con 0x7f5bc4157030 2024-09-17T03:21:41.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc41fd530 con 0x7f5bc4157030 2024-09-17T03:21:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.173+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc40d1c90 con 0x7f5bc4157030 2024-09-17T03:21:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.177+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5bb4005420 con 0x7f5bc4157030 2024-09-17T03:21:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.177+0000 7f5bba000640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba0073200 0x7f5ba0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.177+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb412dca0 con 0x7f5bc4157030 2024-09-17T03:21:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.177+0000 7f5bbbe00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba0073200 0x7f5ba0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.177+0000 7f5bbbe00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba0073200 0x7f5ba0075710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5bc41520b0 tx=0x7f5bac0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:41.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.181+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5bb40bd070 con 0x7f5bc4157030 2024-09-17T03:21:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.393+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bc40d19b0 con 0x7f5bc4157030 2024-09-17T03:21:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.393+0000 7f5bba000640 1 -- 172.21.15.46:0/81087739 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7f5bb40164d0 con 0x7f5bc4157030 2024-09-17T03:21:41.395 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:41.395 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba0073200 msgr2=0x7f5ba0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba0073200 0x7f5ba0075710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5bc41520b0 tx=0x7f5bac0091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 msgr2=0x7f5bc41fdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 0x7f5bc41fdd30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009850 tx=0x7f5bb4005e00 comp rx=0 tx=0).stop 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 shutdown_connections 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba0073200 0x7f5ba0075710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc4161160 0x7f5bc41463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc4160d50 0x7f5bc4147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 --2- 172.21.15.46:0/81087739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc4157030 0x7f5bc41fdd30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 >> 172.21.15.46:0/81087739 conn(0x7f5bc40708c0 msgr2=0x7f5bc414c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 shutdown_connections 2024-09-17T03:21:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.397+0000 7f5bc8e00640 1 -- 172.21.15.46:0/81087739 wait complete. 2024-09-17T03:21:41.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bfa00640 1 -- 172.21.15.46:0/1108810779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a0004d50 msgr2=0x7f07a0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bfa00640 1 --2- 172.21.15.46:0/1108810779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a0004d50 0x7f07a0005160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c640 tx=0x7f07b001fa00 comp rx=0 tx=0).stop 2024-09-17T03:21:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bda00640 1 -- 172.21.15.46:0/1108810779 reap_dead start 2024-09-17T03:21:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bfa00640 1 -- 172.21.15.46:0/1108810779 shutdown_connections 2024-09-17T03:21:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bfa00640 1 -- 172.21.15.46:0/1108810779 >> 172.21.15.46:0/1108810779 conn(0x7f07b80b2040 msgr2=0x7f07b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bfa00640 1 -- 172.21.15.46:0/1108810779 shutdown_connections 2024-09-17T03:21:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.577+0000 7f07bfa00640 1 -- 172.21.15.46:0/1108810779 wait complete. 2024-09-17T03:21:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 Processor -- start 2024-09-17T03:21:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 -- start start 2024-09-17T03:21:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07a0004d50 0x7f07b81e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07b80aa4d0 0x7f07b81e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 0x7f07b814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07b815cf30 con 0x7f07b81e8220 2024-09-17T03:21:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07b815cdb0 con 0x7f07a0004d50 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f07b815d0b0 con 0x7f07b80aa4d0 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 0x7f07b814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 0x7f07b814fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56462/0 (socket says 172.21.15.46:56462) 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 -- 172.21.15.46:0/4168180682 learned_addr learned my addr 172.21.15.46:0/4168180682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07b7e00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07b80aa4d0 0x7f07b81e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bd000640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07a0004d50 0x7f07b81e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 -- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07b80aa4d0 msgr2=0x7f07b81e7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07b80aa4d0 0x7f07b81e7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 -- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07a0004d50 msgr2=0x7f07b81e3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07a0004d50 0x7f07b81e3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 -- 172.21.15.46:0/4168180682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b81e2610 con 0x7f07b81e8220 2024-09-17T03:21:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bda00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 0x7f07b814fc00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c670 tx=0x7f07b001fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0006d50 con 0x7f07b81e8220 2024-09-17T03:21:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07b0004e40 con 0x7f07b81e8220 2024-09-17T03:21:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0003980 con 0x7f07b81e8220 2024-09-17T03:21:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b81f96d0 con 0x7f07b81e8220 2024-09-17T03:21:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.581+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81f9c90 con 0x7f07b81e8220 2024-09-17T03:21:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.585+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b80768c0 con 0x7f07b81e8220 2024-09-17T03:21:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.585+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f07b00bb030 con 0x7f07b81e8220 2024-09-17T03:21:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.585+0000 7f07b4c00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0798073200 0x7f0798075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.585+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f07b01312f0 con 0x7f07b81e8220 2024-09-17T03:21:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.585+0000 7f07bd000640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0798073200 0x7f0798075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.585+0000 7f07bd000640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0798073200 0x7f0798075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009be0 tx=0x7f07ac0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.589+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07b00197b0 con 0x7f07b81e8220 2024-09-17T03:21:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.769+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07b815d640 con 0x7f07b81e8220 2024-09-17T03:21:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07b4c00640 1 -- 172.21.15.46:0/4168180682 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f07b00fbc20 con 0x7f07b81e8220 2024-09-17T03:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:41.773 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0798073200 msgr2=0x7f0798075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0798073200 0x7f0798075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009be0 tx=0x7f07ac0091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 msgr2=0x7f07b814fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 0x7f07b814fc00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c670 tx=0x7f07b001fc50 comp rx=0 tx=0).stop 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 shutdown_connections 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0798073200 0x7f0798075710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b81e8220 0x7f07b814fc00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07b80aa4d0 0x7f07b81e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 --2- 172.21.15.46:0/4168180682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07a0004d50 0x7f07b81e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 >> 172.21.15.46:0/4168180682 conn(0x7f07b80b2040 msgr2=0x7f07b80cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 shutdown_connections 2024-09-17T03:21:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:41.773+0000 7f07bfa00640 1 -- 172.21.15.46:0/4168180682 wait complete. 2024-09-17T03:21:41.855 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:21:41.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 -- 172.21.15.46:0/4264917839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 msgr2=0x7f580415d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 --2- 172.21.15.46:0/4264917839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 0x7f580415d070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009c80 tx=0x7f57fc01cd70 comp rx=0 tx=0).stop 2024-09-17T03:21:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 -- 172.21.15.46:0/4264917839 shutdown_connections 2024-09-17T03:21:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 --2- 172.21.15.46:0/4264917839 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 0x7f580415d070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 --2- 172.21.15.46:0/4264917839 >> v2:172.21.15.99:3300/0 conn(0x7f58040b4de0 0x7f58040b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 -- 172.21.15.46:0/4264917839 >> 172.21.15.46:0/4264917839 conn(0x7f58040b2040 msgr2=0x7f58040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 -- 172.21.15.46:0/4264917839 shutdown_connections 2024-09-17T03:21:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.013+0000 7f5809a00640 1 -- 172.21.15.46:0/4264917839 wait complete. 2024-09-17T03:21:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 Processor -- start 2024-09-17T03:21:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 -- start start 2024-09-17T03:21:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 0x7f5804150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58040b4de0 0x7f58041f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 0x7f58041f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5804168eb0 con 0x7f58040738b0 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5804168d30 con 0x7f58040b4de0 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5804169030 con 0x7f580415cc60 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 0x7f5804150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 0x7f5804150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56466/0 (socket says 172.21.15.46:56466) 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 -- 172.21.15.46:0/1205342427 learned_addr learned my addr 172.21.15.46:0/1205342427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5803400640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 0x7f58041f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f57fa000640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58040b4de0 0x7f58041f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 -- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 msgr2=0x7f58041f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 0x7f58041f7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 -- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58040b4de0 msgr2=0x7f58041f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58040b4de0 0x7f58041f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 -- 172.21.15.46:0/1205342427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58041f8190 con 0x7f58040738b0 2024-09-17T03:21:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5802a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 0x7f5804150ce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c3f0 tx=0x7f57f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4017070 con 0x7f58040738b0 2024-09-17T03:21:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57f40053c0 con 0x7f58040738b0 2024-09-17T03:21:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003860 con 0x7f58040738b0 2024-09-17T03:21:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58040b0ba0 con 0x7f58040738b0 2024-09-17T03:21:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.017+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58040b1160 con 0x7f58040738b0 2024-09-17T03:21:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.021+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58040b51f0 con 0x7f58040738b0 2024-09-17T03:21:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.021+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57f401a070 con 0x7f58040738b0 2024-09-17T03:21:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.021+0000 7f57fb400640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f57e0073200 0x7f57e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.021+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f57f4093550 con 0x7f58040738b0 2024-09-17T03:21:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.021+0000 7f57fa000640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f57e0073200 0x7f57e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.021+0000 7f57fa000640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f57e0073200 0x7f57e0075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58040b4870 tx=0x7f57ec0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.025+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f58040b5730 con 0x7f58040738b0 2024-09-17T03:21:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58040d0050 con 0x7f58040738b0 2024-09-17T03:21:42.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f57fb400640 1 -- 172.21.15.46:0/1205342427 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7f57f405de90 con 0x7f58040738b0 2024-09-17T03:21:42.242 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:42.242 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f57e0073200 msgr2=0x7f57e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f57e0073200 0x7f57e0075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58040b4870 tx=0x7f57ec0091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 msgr2=0x7f5804150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 0x7f5804150ce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c3f0 tx=0x7f57f400c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 shutdown_connections 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f57e0073200 0x7f57e0075710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f580415cc60 0x7f58041f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58040b4de0 0x7f58041f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 --2- 172.21.15.46:0/1205342427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58040738b0 0x7f5804150ce0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 >> 172.21.15.46:0/1205342427 conn(0x7f58040b2040 msgr2=0x7f58041540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 shutdown_connections 2024-09-17T03:21:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.241+0000 7f5809a00640 1 -- 172.21.15.46:0/1205342427 wait complete. 2024-09-17T03:21:42.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 -- 172.21.15.46:0/3781361706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19480b4de0 msgr2=0x7f19480b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 --2- 172.21.15.46:0/3781361706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19480b4de0 0x7f19480b51f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1940009c80 tx=0x7f194001cd70 comp rx=0 tx=0).stop 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 -- 172.21.15.46:0/3781361706 shutdown_connections 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 --2- 172.21.15.46:0/3781361706 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19480b4de0 0x7f19480b51f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 --2- 172.21.15.46:0/3781361706 >> v2:172.21.15.99:3300/0 conn(0x7f19481527b0 0x7f194815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 -- 172.21.15.46:0/3781361706 >> 172.21.15.46:0/3781361706 conn(0x7f19480b2040 msgr2=0x7f19480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:42.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 -- 172.21.15.46:0/3781361706 shutdown_connections 2024-09-17T03:21:42.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.421+0000 7f194f800640 1 -- 172.21.15.46:0/3781361706 wait complete. 2024-09-17T03:21:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 Processor -- start 2024-09-17T03:21:42.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 -- start start 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19480b4de0 0x7f194814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 0x7f194814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19481527b0 0x7f19480ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19481689d0 con 0x7f19480b5c70 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1948168850 con 0x7f19480b4de0 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194f800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1948168b50 con 0x7f19481527b0 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 0x7f194814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 0x7f194814ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43508/0 (socket says 172.21.15.46:43508) 2024-09-17T03:21:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 -- 172.21.15.46:0/2428431725 learned_addr learned my addr 172.21.15.46:0/2428431725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194ce00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19480b4de0 0x7f194814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f194d800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19481527b0 0x7f19480ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 -- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19481527b0 msgr2=0x7f19480ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19481527b0 0x7f19480ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 -- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19480b4de0 msgr2=0x7f194814b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19480b4de0 0x7f194814b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.425+0000 7f1947e00640 1 -- 172.21.15.46:0/2428431725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194814edd0 con 0x7f19480b5c70 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f1947e00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 0x7f194814ca40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f193400e970 tx=0x7f193400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193400cd80 con 0x7f19480b5c70 2024-09-17T03:21:42.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1934004020 con 0x7f19480b5c70 2024-09-17T03:21:42.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194814f870 con 0x7f19480b5c70 2024-09-17T03:21:42.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934004470 con 0x7f19480b5c70 2024-09-17T03:21:42.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194814d690 con 0x7f19480b5c70 2024-09-17T03:21:42.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19480b51f0 con 0x7f19480b5c70 2024-09-17T03:21:42.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1934003600 con 0x7f19480b5c70 2024-09-17T03:21:42.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.429+0000 7f1944c00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1928073200 0x7f1928075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.433+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1934094b70 con 0x7f19480b5c70 2024-09-17T03:21:42.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.433+0000 7f194ce00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1928073200 0x7f1928075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.433+0000 7f194ce00640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1928073200 0x7f1928075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f193c007fd0 tx=0x7f193c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:42.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.437+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f193405f4b0 con 0x7f19480b5c70 2024-09-17T03:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1948152ed0 con 0x7f19480b5c70 2024-09-17T03:21:42.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f1944c00640 1 -- 172.21.15.46:0/2428431725 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5230 (secure 0 0 0) 0x7f1934063ee0 con 0x7f19480b5c70 2024-09-17T03:21:42.646 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:42.646 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"btime":"2024-09-17T03:21:11:051652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2074076336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2074076336},{"type":"v1","addr":"172.21.15.99:6835","nonce":2074076336}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":244293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/659380554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":659380554},{"type":"v1","addr":"172.21.15.46:6835","nonce":659380554}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":244296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3517029698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3517029698},{"type":"v1","addr":"172.21.15.46:6837","nonce":3517029698}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:07.435943+0000","modified":"2024-09-17T03:21:10.471393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244315":{"gid":244315,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6833/2638881791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2638881791},{"type":"v1","addr":"172.21.15.99:6833","nonce":2638881791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244315,"qdb_cluster":[244315]},"id":1}]} 2024-09-17T03:21:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-09-17T03:21:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1928073200 msgr2=0x7f1928075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1928073200 0x7f1928075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f193c007fd0 tx=0x7f193c009040 comp rx=0 tx=0).stop 2024-09-17T03:21:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 msgr2=0x7f194814ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 0x7f194814ca40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f193400e970 tx=0x7f193400ee30 comp rx=0 tx=0).stop 2024-09-17T03:21:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 shutdown_connections 2024-09-17T03:21:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1928073200 0x7f1928075710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19481527b0 0x7f19480ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19480b5c70 0x7f194814ca40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 --2- 172.21.15.46:0/2428431725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19480b4de0 0x7f194814b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 >> 172.21.15.46:0/2428431725 conn(0x7f19480b2040 msgr2=0x7f19480b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 shutdown_connections 2024-09-17T03:21:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.645+0000 7f194f800640 1 -- 172.21.15.46:0/2428431725 wait complete. 2024-09-17T03:21:42.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 -- 172.21.15.46:0/1496427354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9840cf7f0 msgr2=0x7fd98415d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 --2- 172.21.15.46:0/1496427354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9840cf7f0 0x7fd98415d180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009840 tx=0x7fd96c01c820 comp rx=0 tx=0).stop 2024-09-17T03:21:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 -- 172.21.15.46:0/1496427354 shutdown_connections 2024-09-17T03:21:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 --2- 172.21.15.46:0/1496427354 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd98415d6c0 0x7fd98415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 --2- 172.21.15.46:0/1496427354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9840cf7f0 0x7fd98415d180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 -- 172.21.15.46:0/1496427354 >> 172.21.15.46:0/1496427354 conn(0x7fd9840b2040 msgr2=0x7fd9840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 -- 172.21.15.46:0/1496427354 shutdown_connections 2024-09-17T03:21:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 -- 172.21.15.46:0/1496427354 wait complete. 2024-09-17T03:21:42.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 Processor -- start 2024-09-17T03:21:42.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.825+0000 7fd989200640 1 -- start start 2024-09-17T03:21:42.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9840cf7f0 0x7fd98413d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 0x7fd9840ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd98415eeb0 0x7fd9840aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd984169490 con 0x7fd98415d6c0 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd984169310 con 0x7fd98415eeb0 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd984169610 con 0x7fd9840cf7f0 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 0x7fd9840ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 0x7fd9840ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43522/0 (socket says 172.21.15.46:43522) 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 -- 172.21.15.46:0/2968158561 learned_addr learned my addr 172.21.15.46:0/2968158561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd982800640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd98415eeb0 0x7fd9840aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd981e00640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9840cf7f0 0x7fd98413d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 -- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9840cf7f0 msgr2=0x7fd98413d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9840cf7f0 0x7fd98413d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 -- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd98415eeb0 msgr2=0x7fd9840aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:42.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd98415eeb0 0x7fd9840aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:42.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 -- 172.21.15.46:0/2968158561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9840cefd0 con 0x7fd98415d6c0 2024-09-17T03:21:42.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97be00640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 0x7fd9840ae8e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd96c01cd00 tx=0x7fd96c0a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:42.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c004240 con 0x7fd98415d6c0 2024-09-17T03:21:42.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd96c0bf070 con 0x7fd98415d6c0 2024-09-17T03:21:42.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9841f9de0 con 0x7fd98415d6c0 2024-09-17T03:21:42.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c003820 con 0x7fd98415d6c0 2024-09-17T03:21:42.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9841f9ab0 con 0x7fd98415d6c0 2024-09-17T03:21:42.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98415d180 con 0x7fd98415d6c0 2024-09-17T03:21:42.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.829+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd96c0039c0 con 0x7fd98415d6c0 2024-09-17T03:21:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.833+0000 7fd97a000640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd960073200 0x7fd960075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.833+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd96c12f3c0 con 0x7fd98415d6c0 2024-09-17T03:21:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.833+0000 7fd981e00640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd960073200 0x7fd960075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.833+0000 7fd981e00640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd960073200 0x7fd960075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd974001640 tx=0x7fd974009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:42.837+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd96c0c9020 con 0x7fd98415d6c0 2024-09-17T03:21:43.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.009+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9840b1e20 con 0x7fd98415d6c0 2024-09-17T03:21:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd97a000640 1 -- 172.21.15.46:0/2968158561 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7fd96c016410 con 0x7fd98415d6c0 2024-09-17T03:21:43.014 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:43.014 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:09.029016+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:05.388964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:06.409730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd960073200 msgr2=0x7fd960075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd960073200 0x7fd960075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd974001640 tx=0x7fd974009040 comp rx=0 tx=0).stop 2024-09-17T03:21:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 msgr2=0x7fd9840ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 0x7fd9840ae8e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd96c01cd00 tx=0x7fd96c0a6600 comp rx=0 tx=0).stop 2024-09-17T03:21:43.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 shutdown_connections 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd960073200 0x7fd960075710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd98415eeb0 0x7fd9840aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415d6c0 0x7fd9840ae8e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 --2- 172.21.15.46:0/2968158561 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9840cf7f0 0x7fd98413d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 >> 172.21.15.46:0/2968158561 conn(0x7fd9840b2040 msgr2=0x7fd984075020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 shutdown_connections 2024-09-17T03:21:43.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.013+0000 7fd989200640 1 -- 172.21.15.46:0/2968158561 wait complete. 2024-09-17T03:21:43.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:21:43.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 -- 172.21.15.46:0/2780626345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 msgr2=0x7f7320074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 --2- 172.21.15.46:0/2780626345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f7320074db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f73200cfc90 tx=0x7f731401c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 -- 172.21.15.46:0/2780626345 shutdown_connections 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 --2- 172.21.15.46:0/2780626345 >> v2:172.21.15.99:3300/0 conn(0x7f73200b3340 0x7f73200b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 --2- 172.21.15.46:0/2780626345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f7320074db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 -- 172.21.15.46:0/2780626345 >> 172.21.15.46:0/2780626345 conn(0x7f73200b2040 msgr2=0x7f73200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 -- 172.21.15.46:0/2780626345 shutdown_connections 2024-09-17T03:21:43.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.193+0000 7f7328400640 1 -- 172.21.15.46:0/2780626345 wait complete. 2024-09-17T03:21:43.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 Processor -- start 2024-09-17T03:21:43.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 -- start start 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f73201fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73200b3340 0x7f73200cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73200cefa0 0x7f732013c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7320169d20 con 0x7f73200749a0 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7320169ba0 con 0x7f73200cefa0 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7328400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7320169ea0 con 0x7f73200b3340 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7325a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f73201fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7325a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f73201fc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43554/0 (socket says 172.21.15.46:43554) 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7325a00640 1 -- 172.21.15.46:0/2945868037 learned_addr learned my addr 172.21.15.46:0/2945868037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.197+0000 7f7325000640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73200b3340 0x7f73200cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7326400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73200cefa0 0x7f732013c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7325a00640 1 -- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73200b3340 msgr2=0x7f73200cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7325a00640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73200b3340 0x7f73200cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7325a00640 1 -- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73200cefa0 msgr2=0x7f732013c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7325a00640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73200cefa0 0x7f732013c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7325a00640 1 -- 172.21.15.46:0/2945868037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200cdc00 con 0x7f73200749a0 2024-09-17T03:21:43.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7325a00640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f73201fc1e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f73140194d0 tx=0x7f73140a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140bb070 con 0x7f73200749a0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73140044b0 con 0x7f73200749a0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140b2040 con 0x7f73200749a0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73201fcae0 con 0x7f73200749a0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73200ccec0 con 0x7f73200749a0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320074db0 con 0x7f73200749a0 2024-09-17T03:21:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f73140052d0 con 0x7f73200749a0 2024-09-17T03:21:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.201+0000 7f731d600640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7300073200 0x7f7300075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.205+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f731412dbd0 con 0x7f73200749a0 2024-09-17T03:21:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.205+0000 7f7325000640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7300073200 0x7f7300075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.205+0000 7f7325000640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7300073200 0x7f7300075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73200aee90 tx=0x7f730c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.209+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f73140f1db0 con 0x7f73200749a0 2024-09-17T03:21:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.421+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f73200b11b0 con 0x7f73200749a0 2024-09-17T03:21:43.421 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:21:43.421+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:21:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.421+0000 7f731d600640 1 -- 172.21.15.46:0/2945868037 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f73140164b0 con 0x7f73200749a0 2024-09-17T03:21:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:21:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7300073200 msgr2=0x7f7300075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7300073200 0x7f7300075710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73200aee90 tx=0x7f730c009040 comp rx=0 tx=0).stop 2024-09-17T03:21:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 msgr2=0x7f73201fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f73201fc1e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f73140194d0 tx=0x7f73140a57f0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 shutdown_connections 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7300073200 0x7f7300075710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73200cefa0 0x7f732013c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73200b3340 0x7f73200cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 --2- 172.21.15.46:0/2945868037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73200749a0 0x7f73201fc1e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 >> 172.21.15.46:0/2945868037 conn(0x7f73200b2040 msgr2=0x7f73200b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 shutdown_connections 2024-09-17T03:21:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.425+0000 7f7328400640 1 -- 172.21.15.46:0/2945868037 wait complete. 2024-09-17T03:21:43.440 INFO:tasks.ceph.mds.b.smithi099.stderr: -7088> 2024-09-17T03:21:12.944+0000 7fcb4fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:21:43.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:21:43.561 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 -- 172.21.15.46:0/3852282938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94006f940 msgr2=0x7fb94006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 --2- 172.21.15.46:0/3852282938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94006f940 0x7fb94006fd50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009840 tx=0x7fb92c01c700 comp rx=0 tx=0).stop 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 -- 172.21.15.46:0/3852282938 shutdown_connections 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 --2- 172.21.15.46:0/3852282938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94006f940 0x7fb94006fd50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 --2- 172.21.15.46:0/3852282938 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400af5b0 0x7fb94015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 -- 172.21.15.46:0/3852282938 >> 172.21.15.46:0/3852282938 conn(0x7fb94006d3f0 msgr2=0x7fb9400b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 -- 172.21.15.46:0/3852282938 shutdown_connections 2024-09-17T03:21:43.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 -- 172.21.15.46:0/3852282938 wait complete. 2024-09-17T03:21:43.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 Processor -- start 2024-09-17T03:21:43.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.609+0000 7fb945e00640 1 -- start start 2024-09-17T03:21:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94006f940 0x7fb940140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400af5b0 0x7fb9400cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 0x7fb940152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb940169290 con 0x7fb94015f130 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb940169110 con 0x7fb94006f940 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb940169410 con 0x7fb9400af5b0 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 0x7fb940152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 0x7fb940152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43568/0 (socket says 172.21.15.46:43568) 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 -- 172.21.15.46:0/2609381438 learned_addr learned my addr 172.21.15.46:0/2609381438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb944a00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94006f940 0x7fb940140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb93fe00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400af5b0 0x7fb9400cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 -- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400af5b0 msgr2=0x7fb9400cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400af5b0 0x7fb9400cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 -- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94006f940 msgr2=0x7fb940140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94006f940 0x7fb940140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 -- 172.21.15.46:0/2609381438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400cc9c0 con 0x7fb94015f130 2024-09-17T03:21:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945400640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 0x7fb940152240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c6b0 tx=0x7fb93800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938017070 con 0x7fb94015f130 2024-09-17T03:21:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb938003c70 con 0x7fb94015f130 2024-09-17T03:21:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb940140960 con 0x7fb94015f130 2024-09-17T03:21:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938005350 con 0x7fb94015f130 2024-09-17T03:21:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94015c320 con 0x7fb94015f130 2024-09-17T03:21:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.613+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94015bd90 con 0x7fb94015f130 2024-09-17T03:21:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.617+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb93801a070 con 0x7fb94015f130 2024-09-17T03:21:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.617+0000 7fb93cc00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb920073200 0x7fb920075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.617+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7fb9380937e0 con 0x7fb94015f130 2024-09-17T03:21:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.617+0000 7fb944a00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb920073200 0x7fb920075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.617+0000 7fb944a00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb920073200 0x7fb920075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb934009b40 tx=0x7fb9340091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.621+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb940070290 con 0x7fb94015f130 2024-09-17T03:21:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:43.829+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb94006fd50 con 0x7fb94015f130 2024-09-17T03:21:44.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb93cc00640 1 -- 172.21.15.46:0/2609381438 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fb93805e270 con 0x7fb94015f130 2024-09-17T03:21:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb920073200 msgr2=0x7fb920075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb920073200 0x7fb920075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb934009b40 tx=0x7fb9340091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 msgr2=0x7fb940152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 0x7fb940152240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c6b0 tx=0x7fb93800cb70 comp rx=0 tx=0).stop 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 shutdown_connections 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb920073200 0x7fb920075710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb94015f130 0x7fb940152240 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9400af5b0 0x7fb9400cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 --2- 172.21.15.46:0/2609381438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94006f940 0x7fb940140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 >> 172.21.15.46:0/2609381438 conn(0x7fb94006d3f0 msgr2=0x7fb940075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.425+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 shutdown_connections 2024-09-17T03:21:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.429+0000 7fb945e00640 1 -- 172.21.15.46:0/2609381438 wait complete. 2024-09-17T03:21:44.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 -- 172.21.15.46:0/3258306282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40749a0 msgr2=0x7f7cc4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 --2- 172.21.15.46:0/3258306282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40749a0 0x7f7cc4074db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40b4f30 tx=0x7f7cb401c6e0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 -- 172.21.15.46:0/3258306282 shutdown_connections 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 --2- 172.21.15.46:0/3258306282 >> v2:172.21.15.99:3300/0 conn(0x7f7cc40cefa0 0x7f7cc40cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 --2- 172.21.15.46:0/3258306282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40749a0 0x7f7cc4074db0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 -- 172.21.15.46:0/3258306282 >> 172.21.15.46:0/3258306282 conn(0x7f7cc40b2040 msgr2=0x7f7cc40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 -- 172.21.15.46:0/3258306282 shutdown_connections 2024-09-17T03:21:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.617+0000 7f7cca600640 1 -- 172.21.15.46:0/3258306282 wait complete. 2024-09-17T03:21:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 Processor -- start 2024-09-17T03:21:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 -- start start 2024-09-17T03:21:44.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cc40749a0 0x7f7cc4145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 0x7f7cc415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cc4160330 0x7f7cc4072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc416a060 con 0x7f7cc40cefa0 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4169ee0 con 0x7f7cc40749a0 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cca600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc416a1e0 con 0x7f7cc4160330 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 0x7f7cc415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 0x7f7cc415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43590/0 (socket says 172.21.15.46:43590) 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 -- 172.21.15.46:0/1570827468 learned_addr learned my addr 172.21.15.46:0/1570827468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc3400640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cc40749a0 0x7f7cc4145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc3e00640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cc4160330 0x7f7cc4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 -- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cc4160330 msgr2=0x7f7cc4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cc4160330 0x7f7cc4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 -- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cc40749a0 msgr2=0x7f7cc4145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cc40749a0 0x7f7cc4145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 -- 172.21.15.46:0/1570827468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc4146180 con 0x7f7cc40cefa0 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc3400640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cc40749a0 0x7f7cc4145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.621+0000 7f7cc2a00640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 0x7f7cc415a030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000d720 tx=0x7f7cb000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0017070 con 0x7f7cc40cefa0 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cb0009d20 con 0x7f7cc40cefa0 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0005280 con 0x7f7cc40cefa0 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc413bd80 con 0x7f7cc40cefa0 2024-09-17T03:21:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc413c340 con 0x7f7cc40cefa0 2024-09-17T03:21:44.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc4074db0 con 0x7f7cc40cefa0 2024-09-17T03:21:44.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7cb0005420 con 0x7f7cc40cefa0 2024-09-17T03:21:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.625+0000 7f7cbb400640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ca4073200 0x7f7ca4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.629+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f7cb00943c0 con 0x7f7cc40cefa0 2024-09-17T03:21:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.629+0000 7f7cc3400640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ca4073200 0x7f7ca4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.629+0000 7f7cc3400640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ca4073200 0x7f7ca4075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40ae000 tx=0x7f7cb401af20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.629+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7cb005f370 con 0x7f7cc40cefa0 2024-09-17T03:21:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:44.805+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7cc415a570 con 0x7f7cc40cefa0 2024-09-17T03:21:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.433+0000 7f7cbb400640 1 -- 172.21.15.46:0/1570827468 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f7cb005ec70 con 0x7f7cc40cefa0 2024-09-17T03:21:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:21:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ca4073200 msgr2=0x7f7ca4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:45.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ca4073200 0x7f7ca4075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40ae000 tx=0x7f7cb401af20 comp rx=0 tx=0).stop 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 msgr2=0x7f7cc415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 0x7f7cc415a030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000d720 tx=0x7f7cb000dbe0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 shutdown_connections 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ca4073200 0x7f7ca4075710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cc4160330 0x7f7cc4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc40cefa0 0x7f7cc415a030 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 --2- 172.21.15.46:0/1570827468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cc40749a0 0x7f7cc4145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.437+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 >> 172.21.15.46:0/1570827468 conn(0x7f7cc40b2040 msgr2=0x7f7cc40b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:45.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.441+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 shutdown_connections 2024-09-17T03:21:45.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.441+0000 7f7cca600640 1 -- 172.21.15.46:0/1570827468 wait complete. 2024-09-17T03:21:45.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:21:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2713243001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 msgr2=0x7f6a480b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2713243001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a480b51f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a48070710 tx=0x7f6a4001ccc0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2713243001 shutdown_connections 2024-09-17T03:21:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2713243001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a480b51f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2713243001 >> v2:172.21.15.99:3300/0 conn(0x7f6a481527b0 0x7f6a4815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2713243001 >> 172.21.15.46:0/2713243001 conn(0x7f6a480b2040 msgr2=0x7f6a480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2713243001 shutdown_connections 2024-09-17T03:21:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.633+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2713243001 wait complete. 2024-09-17T03:21:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 Processor -- start 2024-09-17T03:21:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 -- start start 2024-09-17T03:21:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a4813ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a480b5c70 0x7f6a480b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a481527b0 0x7f6a480b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48168920 con 0x7f6a480b4de0 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481687a0 con 0x7f6a481527b0 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6a48168aa0 con 0x7f6a480b5c70 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a4813ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a4813ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43604/0 (socket says 172.21.15.46:43604) 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 -- 172.21.15.46:0/2897757066 learned_addr learned my addr 172.21.15.46:0/2897757066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ce00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a481527b0 0x7f6a480b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 -- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a480b5c70 msgr2=0x7f6a480b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a480b5c70 0x7f6a480b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 -- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a481527b0 msgr2=0x7f6a480b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a481527b0 0x7f6a480b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 -- 172.21.15.46:0/2897757066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a481574a0 con 0x7f6a480b4de0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ce00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a481527b0 0x7f6a480b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a47e00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a4813ea50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c3f0 tx=0x7f6a3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38017070 con 0x7f6a480b4de0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a380053c0 con 0x7f6a480b4de0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38003860 con 0x7f6a480b4de0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a48157780 con 0x7f6a480b4de0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.637+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a48157d40 con 0x7f6a480b4de0 2024-09-17T03:21:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.641+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a480b51f0 con 0x7f6a480b4de0 2024-09-17T03:21:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.641+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6a3801a070 con 0x7f6a480b4de0 2024-09-17T03:21:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.641+0000 7f6a44c00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6a24073200 0x7f6a24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.641+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7f6a38093470 con 0x7f6a480b4de0 2024-09-17T03:21:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.641+0000 7f6a3fe00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6a24073200 0x7f6a24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:45.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.641+0000 7f6a3fe00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6a24073200 0x7f6a24075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a300058d0 tx=0x7f6a30005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:45.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.645+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6a480b5730 con 0x7f6a480b4de0 2024-09-17T03:21:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:45.821+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a480b16b0 con 0x7f6a480b4de0 2024-09-17T03:21:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.437+0000 7f6a44c00640 1 -- 172.21.15.46:0/2897757066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f6a3805e550 con 0x7f6a480b4de0 2024-09-17T03:21:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6a24073200 msgr2=0x7f6a24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6a24073200 0x7f6a24075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a300058d0 tx=0x7f6a30005860 comp rx=0 tx=0).stop 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 msgr2=0x7f6a4813ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a4813ea50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c3f0 tx=0x7f6a3800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 shutdown_connections 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6a24073200 0x7f6a24075710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a481527b0 0x7f6a480b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6a480b5c70 0x7f6a480b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 --2- 172.21.15.46:0/2897757066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a480b4de0 0x7f6a4813ea50 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c3f0 tx=0x7f6a3800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 >> 172.21.15.46:0/2897757066 conn(0x7f6a480b2040 msgr2=0x7f6a481519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 shutdown_connections 2024-09-17T03:21:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.441+0000 7f6a4ee00640 1 -- 172.21.15.46:0/2897757066 wait complete. 2024-09-17T03:21:46.475 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:21:46.475 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:21:46.475 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:21:46.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:21:46.476 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:21:46.476 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:21:46.476 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:21:46.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:21:46.476 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:21:46.476 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:21:46.477 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:21:46.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:21:46.477 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:21:46.477 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:21:46.477 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:21:46.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:21:46.490 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:21:46.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:21:46.491 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:21:46.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:21:46.496 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:21:46.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:21:46.504 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:21:46.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:21:46.535 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:21:46.536 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:46.537 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:21:46.537 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:46.549 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:21:46.550 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:46.552 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:21:46.552 DEBUG:teuthology.parallel:result is None 2024-09-17T03:21:46.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:21:46.636 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:21:46.641 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:21:46.654 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:21:46.655 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:21:46.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 -- 172.21.15.46:0/3600761677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8878159130 msgr2=0x7f887815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:46.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 --2- 172.21.15.46:0/3600761677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8878159130 0x7f887815bb90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88780ce1a0 tx=0x7f886801caf0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 -- 172.21.15.46:0/3600761677 shutdown_connections 2024-09-17T03:21:46.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 --2- 172.21.15.46:0/3600761677 >> v2:172.21.15.99:3300/0 conn(0x7f88780703a0 0x7f887806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 --2- 172.21.15.46:0/3600761677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8878159130 0x7f887815bb90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 -- 172.21.15.46:0/3600761677 >> 172.21.15.46:0/3600761677 conn(0x7f88780b35f0 msgr2=0x7f88780b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 -- 172.21.15.46:0/3600761677 shutdown_connections 2024-09-17T03:21:46.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.721+0000 7f887e800640 1 -- 172.21.15.46:0/3600761677 wait complete. 2024-09-17T03:21:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 Processor -- start 2024-09-17T03:21:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 -- start start 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f887806cdf0 0x7f88781f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 0x7f88781556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8878159130 0x7f88781496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8878168cb0 con 0x7f88780703a0 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8878168b30 con 0x7f887806cdf0 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8878168e30 con 0x7f8878159130 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 0x7f88781556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 0x7f88781556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43636/0 (socket says 172.21.15.46:43636) 2024-09-17T03:21:46.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 -- 172.21.15.46:0/1367083531 learned_addr learned my addr 172.21.15.46:0/1367083531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f8877400640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f887806cdf0 0x7f88781f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f8877e00640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8878159130 0x7f88781496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 -- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8878159130 msgr2=0x7f88781496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8878159130 0x7f88781496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 -- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f887806cdf0 msgr2=0x7f88781f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f887806cdf0 0x7f88781f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 -- 172.21.15.46:0/1367083531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88781519a0 con 0x7f88780703a0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886ea00640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 0x7f88781556d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f886000c760 tx=0x7f886000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860016020 con 0x7f88780703a0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f886000a950 con 0x7f88780703a0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886000ada0 con 0x7f88780703a0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8878151c80 con 0x7f88780703a0 2024-09-17T03:21:46.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.725+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8878154aa0 con 0x7f88780703a0 2024-09-17T03:21:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.729+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887806d200 con 0x7f88780703a0 2024-09-17T03:21:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.729+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8860019020 con 0x7f88780703a0 2024-09-17T03:21:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.729+0000 7f886fe00640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8854073200 0x7f8854075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.729+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f8860093db0 con 0x7f88780703a0 2024-09-17T03:21:46.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.729+0000 7f8877400640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8854073200 0x7f8854075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.733+0000 7f8877400640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8854073200 0x7f8854075710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88680af000 tx=0x7f886801b330 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:46.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.737+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8860098070 con 0x7f88780703a0 2024-09-17T03:21:46.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:46.925+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f887815b6b0 con 0x7f88780703a0 2024-09-17T03:21:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.449+0000 7f886fe00640 1 -- 172.21.15.46:0/1367083531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f8860020020 con 0x7f88780703a0 2024-09-17T03:21:47.450 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:21:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8854073200 msgr2=0x7f8854075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8854073200 0x7f8854075710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88680af000 tx=0x7f886801b330 comp rx=0 tx=0).stop 2024-09-17T03:21:47.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 msgr2=0x7f88781556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 0x7f88781556d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f886000c760 tx=0x7f886000cc20 comp rx=0 tx=0).stop 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 shutdown_connections 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8854073200 0x7f8854075710 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8878159130 0x7f88781496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88780703a0 0x7f88781556d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 --2- 172.21.15.46:0/1367083531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f887806cdf0 0x7f88781f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 >> 172.21.15.46:0/1367083531 conn(0x7f88780b35f0 msgr2=0x7f88780b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 shutdown_connections 2024-09-17T03:21:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.453+0000 7f887e800640 1 -- 172.21.15.46:0/1367083531 wait complete. 2024-09-17T03:21:47.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:21:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 -- 172.21.15.46:0/1393529670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b4de0 msgr2=0x7f34080b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 --2- 172.21.15.46:0/1393529670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b4de0 0x7f34080b51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3400009bc0 tx=0x7f340001cae0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 -- 172.21.15.46:0/1393529670 shutdown_connections 2024-09-17T03:21:47.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 --2- 172.21.15.46:0/1393529670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b4de0 0x7f34080b51f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 --2- 172.21.15.46:0/1393529670 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34081527b0 0x7f340815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 -- 172.21.15.46:0/1393529670 >> 172.21.15.46:0/1393529670 conn(0x7f34080b2040 msgr2=0x7f34080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 -- 172.21.15.46:0/1393529670 shutdown_connections 2024-09-17T03:21:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.633+0000 7f340fa00640 1 -- 172.21.15.46:0/1393529670 wait complete. 2024-09-17T03:21:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 Processor -- start 2024-09-17T03:21:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 -- start start 2024-09-17T03:21:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34080b4de0 0x7f340813d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b5c70 0x7f34081404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 0x7f340815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3408168a10 con 0x7f34081527b0 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3408168890 con 0x7f34080b5c70 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3408168b90 con 0x7f34080b4de0 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 0x7f340815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 0x7f340815a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43642/0 (socket says 172.21.15.46:43642) 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 -- 172.21.15.46:0/183068593 learned_addr learned my addr 172.21.15.46:0/183068593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f3407e00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b5c70 0x7f34081404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340d000640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34080b4de0 0x7f340813d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:47.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 -- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34080b4de0 msgr2=0x7f340813d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34080b4de0 0x7f340813d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 -- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b5c70 msgr2=0x7f34081404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b5c70 0x7f34081404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 -- 172.21.15.46:0/183068593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340813e380 con 0x7f34081527b0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340da00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 0x7f340815a9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f340001c550 tx=0x7f3400003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000b2040 con 0x7f34081527b0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34000bb070 con 0x7f34081527b0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400004370 con 0x7f34081527b0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.637+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340813e660 con 0x7f34081527b0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.641+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340813ec20 con 0x7f34081527b0 2024-09-17T03:21:47.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.641+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3408140f80 con 0x7f34081527b0 2024-09-17T03:21:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.641+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3400004510 con 0x7f34081527b0 2024-09-17T03:21:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.641+0000 7f3404c00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33e8073230 0x7f33e8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.645+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f340012e850 con 0x7f34081527b0 2024-09-17T03:21:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.645+0000 7f340d000640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33e8073230 0x7f33e8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.645+0000 7f340d000640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33e8073230 0x7f33e8075740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007d70 tx=0x7f33fc007d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:47.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.649+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3400016790 con 0x7f34081527b0 2024-09-17T03:21:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f34080b51f0 con 0x7f34081527b0 2024-09-17T03:21:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f3404c00640 1 -- 172.21.15.46:0/183068593 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+6645 (secure 0 0 0) 0x7f34000f9450 con 0x7f34081527b0 2024-09-17T03:21:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:47.881 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:21:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:21:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:21:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:21:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:21:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCJ9ehmzZSDBxAArOUqxwe1+EEbBCNoKuOgXw==", 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCJ9ehmwaCDBxAAMVWJz/Cqrjcs1nHFd8dpZg==", 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T03:21:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCJ9ehmz6mDBxAAUgEDEyDI3hB/OrxhBPoFAQ==", 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCJ9ehmjrKDBxAAEacQmr/88JtUiKsQYSzQSQ==", 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T03:21:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCJ9ehmlbuDBxAAB29ItfSAu7+UBOHgUOhHXw==", 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCJ9ehm8sSDBxAA255EOQ9gENbcYpTQrmjGKQ==", 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:21:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:21:47.898 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:21:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33e8073230 msgr2=0x7f33e8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33e8073230 0x7f33e8075740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007d70 tx=0x7f33fc007d00 comp rx=0 tx=0).stop 2024-09-17T03:21:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 msgr2=0x7f340815a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 0x7f340815a9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f340001c550 tx=0x7f3400003f40 comp rx=0 tx=0).stop 2024-09-17T03:21:47.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 shutdown_connections 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33e8073230 0x7f33e8075740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34081527b0 0x7f340815a9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34080b5c70 0x7f34081404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 --2- 172.21.15.46:0/183068593 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34080b4de0 0x7f340813d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 >> 172.21.15.46:0/183068593 conn(0x7f34080b2040 msgr2=0x7f34080ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 shutdown_connections 2024-09-17T03:21:47.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:47.881+0000 7f340fa00640 1 -- 172.21.15.46:0/183068593 wait complete. 2024-09-17T03:21:47.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T03:21:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 -- 172.21.15.46:0/1493416671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 msgr2=0x7f443815e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 --2- 172.21.15.46:0/1493416671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f443815e890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4420009930 tx=0x7f442001c850 comp rx=0 tx=0).stop 2024-09-17T03:21:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 -- 172.21.15.46:0/1493416671 shutdown_connections 2024-09-17T03:21:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 --2- 172.21.15.46:0/1493416671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f443815e890 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 --2- 172.21.15.46:0/1493416671 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f443813e790 0x7f44380cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 -- 172.21.15.46:0/1493416671 >> 172.21.15.46:0/1493416671 conn(0x7f44380b2040 msgr2=0x7f44380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 -- 172.21.15.46:0/1493416671 shutdown_connections 2024-09-17T03:21:48.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.061+0000 7f443e400640 1 -- 172.21.15.46:0/1493416671 wait complete. 2024-09-17T03:21:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 Processor -- start 2024-09-17T03:21:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 -- start start 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f44381530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443813e790 0x7f44381f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f443815edd0 0x7f4438150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44381407a0 con 0x7f44380cc860 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44380b5690 con 0x7f443813e790 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f443e400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4438140920 con 0x7f443815edd0 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f4437400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f44381530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f4437400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f44381530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43668/0 (socket says 172.21.15.46:43668) 2024-09-17T03:21:48.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f4437400640 1 -- 172.21.15.46:0/706000294 learned_addr learned my addr 172.21.15.46:0/706000294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f442ea00640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443813e790 0x7f44381f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.065+0000 7f4437e00640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f443815edd0 0x7f4438150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f4437400640 1 -- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f443815edd0 msgr2=0x7f4438150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f4437400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f443815edd0 0x7f4438150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f4437400640 1 -- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443813e790 msgr2=0x7f44381f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f4437400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443813e790 0x7f44381f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f4437400640 1 -- 172.21.15.46:0/706000294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44381539e0 con 0x7f44380cc860 2024-09-17T03:21:48.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f4437400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f44381530e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f442800db90 tx=0x7f442800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428005020 con 0x7f44380cc860 2024-09-17T03:21:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44280052b0 con 0x7f44380cc860 2024-09-17T03:21:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4438153cc0 con 0x7f44380cc860 2024-09-17T03:21:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280047b0 con 0x7f44380cc860 2024-09-17T03:21:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4438154280 con 0x7f44380cc860 2024-09-17T03:21:48.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438140e40 con 0x7f44380cc860 2024-09-17T03:21:48.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.069+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4428004950 con 0x7f44380cc860 2024-09-17T03:21:48.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.073+0000 7f442fe00640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4414073200 0x7f4414075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.073+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f4428094480 con 0x7f44380cc860 2024-09-17T03:21:48.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.073+0000 7f442ea00640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4414073200 0x7f4414075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.073+0000 7f442ea00640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4414073200 0x7f4414075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f44200af000 tx=0x7f442001b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:48.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.077+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f442805f190 con 0x7f44380cc860 2024-09-17T03:21:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.313+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f44380b1c20 con 0x7f44380cc860 2024-09-17T03:21:48.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.317+0000 7f442fe00640 1 -- 172.21.15.46:0/706000294 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7f4428063bc0 con 0x7f44380cc860 2024-09-17T03:21:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4414073200 msgr2=0x7f4414075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4414073200 0x7f4414075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f44200af000 tx=0x7f442001b090 comp rx=0 tx=0).stop 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 msgr2=0x7f44381530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f44381530e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f442800db90 tx=0x7f442800a620 comp rx=0 tx=0).stop 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 shutdown_connections 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4414073200 0x7f4414075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f443815edd0 0x7f4438150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443813e790 0x7f44381f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 --2- 172.21.15.46:0/706000294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44380cc860 0x7f44381530e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 >> 172.21.15.46:0/706000294 conn(0x7f44380b2040 msgr2=0x7f4438073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 shutdown_connections 2024-09-17T03:21:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.321+0000 7f443e400640 1 -- 172.21.15.46:0/706000294 wait complete. 2024-09-17T03:21:48.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 -- 172.21.15.46:0/1169085181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 msgr2=0x7f6124004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 --2- 172.21.15.46:0/1169085181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f6124004c40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6128006f80 tx=0x7f612801ceb0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 -- 172.21.15.46:0/1169085181 shutdown_connections 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 --2- 172.21.15.46:0/1169085181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f6124004c40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 -- 172.21.15.46:0/1169085181 >> 172.21.15.46:0/1169085181 conn(0x7f61400b2040 msgr2=0x7f61400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 -- 172.21.15.46:0/1169085181 shutdown_connections 2024-09-17T03:21:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.505+0000 7f6145e00640 1 -- 172.21.15.46:0/1169085181 wait complete. 2024-09-17T03:21:48.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 Processor -- start 2024-09-17T03:21:48.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 -- start start 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f61401f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6140149eb0 0x7f614014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f614014b2a0 0x7f61401424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f61400b5580 con 0x7f6124004810 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61400b5400 con 0x7f6140149eb0 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f61400b5700 con 0x7f614014b2a0 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f61401f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f61401f96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43676/0 (socket says 172.21.15.46:43676) 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 -- 172.21.15.46:0/828865743 learned_addr learned my addr 172.21.15.46:0/828865743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613f400640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f614014b2a0 0x7f61401424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6137e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6140149eb0 0x7f614014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 -- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f614014b2a0 msgr2=0x7f61401424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f614014b2a0 0x7f61401424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 -- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6140149eb0 msgr2=0x7f614014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6140149eb0 0x7f614014ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 -- 172.21.15.46:0/828865743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61401f6930 con 0x7f6124004810 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f613ea00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f61401f96d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6130011a40 tx=0x7f6130011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613001e070 con 0x7f6124004810 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f613000fe70 con 0x7f6124004810 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6140141440 con 0x7f6124004810 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300042b0 con 0x7f6124004810 2024-09-17T03:21:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.509+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614014a680 con 0x7f6124004810 2024-09-17T03:21:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.513+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614015f500 con 0x7f6124004810 2024-09-17T03:21:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.513+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6130021070 con 0x7f6124004810 2024-09-17T03:21:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.513+0000 7f6137400640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f611c073230 0x7f611c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.513+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f6130094990 con 0x7f6124004810 2024-09-17T03:21:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.513+0000 7f6137e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f611c073230 0x7f611c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.513+0000 7f6137e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f611c073230 0x7f611c075740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f61280ab000 tx=0x7f6128019cf0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:48.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.517+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f613005f590 con 0x7f6124004810 2024-09-17T03:21:48.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.753+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f61401598c0 con 0x7f6124004810 2024-09-17T03:21:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.753+0000 7f6137400640 1 -- 172.21.15.46:0/828865743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v29) ==== 90+0+0 (secure 0 0 0) 0x7f6130063fc0 con 0x7f6124004810 2024-09-17T03:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f611c073230 msgr2=0x7f611c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f611c073230 0x7f611c075740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f61280ab000 tx=0x7f6128019cf0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 msgr2=0x7f61401f96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f61401f96d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6130011a40 tx=0x7f6130011f00 comp rx=0 tx=0).stop 2024-09-17T03:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 shutdown_connections 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f611c073230 0x7f611c075740 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f614014b2a0 0x7f61401424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6140149eb0 0x7f614014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 --2- 172.21.15.46:0/828865743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6124004810 0x7f61401f96d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 >> 172.21.15.46:0/828865743 conn(0x7f61400b2040 msgr2=0x7f614014df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 shutdown_connections 2024-09-17T03:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.757+0000 7f6145e00640 1 -- 172.21.15.46:0/828865743 wait complete. 2024-09-17T03:21:48.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 -- 172.21.15.46:0/2799900851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 msgr2=0x7f2ae4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 --2- 172.21.15.46:0/2799900851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae4074db0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009900 tx=0x7f2ad801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 -- 172.21.15.46:0/2799900851 shutdown_connections 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 --2- 172.21.15.46:0/2799900851 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ae40cefa0 0x7f2ae40cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 --2- 172.21.15.46:0/2799900851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae4074db0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 -- 172.21.15.46:0/2799900851 >> 172.21.15.46:0/2799900851 conn(0x7f2ae40b2040 msgr2=0x7f2ae40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 -- 172.21.15.46:0/2799900851 shutdown_connections 2024-09-17T03:21:48.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.937+0000 7f2aec200640 1 -- 172.21.15.46:0/2799900851 wait complete. 2024-09-17T03:21:48.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 Processor -- start 2024-09-17T03:21:48.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 -- start start 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae41453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ae40cefa0 0x7f2ae40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4160330 0x7f2ae40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae416a060 con 0x7f2ae40749a0 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4169ee0 con 0x7f2ae4160330 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2aec200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae416a1e0 con 0x7f2ae40cefa0 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2ae9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae41453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2ae9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae41453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43696/0 (socket says 172.21.15.46:43696) 2024-09-17T03:21:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2ae9800640 1 -- 172.21.15.46:0/215544639 learned_addr learned my addr 172.21.15.46:0/215544639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.941+0000 7f2ae8e00640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ae40cefa0 0x7f2ae40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2aea200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4160330 0x7f2ae40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae9800640 1 -- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ae40cefa0 msgr2=0x7f2ae40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae9800640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ae40cefa0 0x7f2ae40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae9800640 1 -- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4160330 msgr2=0x7f2ae40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae9800640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4160330 0x7f2ae40ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae9800640 1 -- 172.21.15.46:0/215544639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41fc480 con 0x7f2ae40749a0 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2aea200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4160330 0x7f2ae40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:48.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae9800640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae41453b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009850 tx=0x7f2ad80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80b2040 con 0x7f2ae40749a0 2024-09-17T03:21:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ad8005030 con 0x7f2ae40749a0 2024-09-17T03:21:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8005440 con 0x7f2ae40749a0 2024-09-17T03:21:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41fc760 con 0x7f2ae40749a0 2024-09-17T03:21:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41fcd20 con 0x7f2ae40749a0 2024-09-17T03:21:48.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4074db0 con 0x7f2ae40749a0 2024-09-17T03:21:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.945+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ad80bd060 con 0x7f2ae40749a0 2024-09-17T03:21:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.949+0000 7f2ae1600640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2ac4073200 0x7f2ac4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.949+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f2ad812d410 con 0x7f2ae40749a0 2024-09-17T03:21:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.949+0000 7f2ae8e00640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2ac4073200 0x7f2ac4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:48.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.949+0000 7f2ae8e00640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2ac4073200 0x7f2ac4075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4133530 tx=0x7f2ad000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:48.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:48.953+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ad80164d0 con 0x7f2ae40749a0 2024-09-17T03:21:49.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.185+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f2ae4073250 con 0x7f2ae40749a0 2024-09-17T03:21:49.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.189+0000 7f2ae1600640 1 -- 172.21.15.46:0/215544639 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v30) ==== 90+0+0 (secure 0 0 0) 0x7f2ad80f8800 con 0x7f2ae40749a0 2024-09-17T03:21:49.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2ac4073200 msgr2=0x7f2ac4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2ac4073200 0x7f2ac4075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4133530 tx=0x7f2ad000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 msgr2=0x7f2ae41453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae41453b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009850 tx=0x7f2ad80040c0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 shutdown_connections 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2ac4073200 0x7f2ac4075710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ae4160330 0x7f2ae40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ae40cefa0 0x7f2ae40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 --2- 172.21.15.46:0/215544639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ae40749a0 0x7f2ae41453b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 >> 172.21.15.46:0/215544639 conn(0x7f2ae40b2040 msgr2=0x7f2ae415aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 shutdown_connections 2024-09-17T03:21:49.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.193+0000 7f2aec200640 1 -- 172.21.15.46:0/215544639 wait complete. 2024-09-17T03:21:49.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T03:21:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 -- 172.21.15.46:0/2025386559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5700051b0 msgr2=0x7fd57000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 --2- 172.21.15.46:0/2025386559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5700051b0 0x7fd57000b9d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd5880cf890 tx=0x7fd58001cfa0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58e000640 1 -- 172.21.15.46:0/2025386559 reap_dead start 2024-09-17T03:21:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 -- 172.21.15.46:0/2025386559 shutdown_connections 2024-09-17T03:21:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 -- 172.21.15.46:0/2025386559 >> 172.21.15.46:0/2025386559 conn(0x7fd5880b4560 msgr2=0x7fd5880d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 -- 172.21.15.46:0/2025386559 shutdown_connections 2024-09-17T03:21:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 -- 172.21.15.46:0/2025386559 wait complete. 2024-09-17T03:21:49.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 Processor -- start 2024-09-17T03:21:49.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.373+0000 7fd58ea00640 1 -- start start 2024-09-17T03:21:49.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5700051b0 0x7fd5881f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd588152eb0 0x7fd5880b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 0x7fd5881f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5880b5260 con 0x7fd5881584f0 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd588160740 con 0x7fd5700051b0 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd5880b53e0 con 0x7fd588152eb0 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 0x7fd5881f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 0x7fd5881f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43704/0 (socket says 172.21.15.46:43704) 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 -- 172.21.15.46:0/463574534 learned_addr learned my addr 172.21.15.46:0/463574534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58cc00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd588152eb0 0x7fd5880b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58d600640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5700051b0 0x7fd5881f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 -- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd588152eb0 msgr2=0x7fd5880b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd588152eb0 0x7fd5880b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 -- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5700051b0 msgr2=0x7fd5881f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5700051b0 0x7fd5881f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 -- 172.21.15.46:0/463574534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5880d2820 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58d600640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5700051b0 0x7fd5881f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58e000640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 0x7fd5881f3620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd58001cdd0 tx=0x7fd5800b09f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800ab070 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5800b0c70 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580004c70 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5880aad70 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5880aaa90 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.377+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5880b7280 con 0x7fd5881584f0 2024-09-17T03:21:49.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.381+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd580005030 con 0x7fd5881584f0 2024-09-17T03:21:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.381+0000 7fd585600640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd568073200 0x7fd568075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.381+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd58012e390 con 0x7fd5881584f0 2024-09-17T03:21:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.381+0000 7fd58d600640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd568073200 0x7fd568075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.381+0000 7fd58d600640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd568073200 0x7fd568075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009bc0 tx=0x7fd57c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.385+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd5880aad70 con 0x7fd5881584f0 2024-09-17T03:21:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.621+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fd5880b6fa0 con 0x7fd5881584f0 2024-09-17T03:21:49.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd585600640 1 -- 172.21.15.46:0/463574534 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v31) ==== 90+0+0 (secure 0 0 0) 0x7fd580016cb0 con 0x7fd5881584f0 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd568073200 msgr2=0x7fd568075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd568073200 0x7fd568075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009bc0 tx=0x7fd57c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 msgr2=0x7fd5881f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 0x7fd5881f3620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd58001cdd0 tx=0x7fd5800b09f0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 shutdown_connections 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd568073200 0x7fd568075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881584f0 0x7fd5881f3620 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd588152eb0 0x7fd5880b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 --2- 172.21.15.46:0/463574534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5700051b0 0x7fd5881f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 >> 172.21.15.46:0/463574534 conn(0x7fd5880b4560 msgr2=0x7fd5880d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 shutdown_connections 2024-09-17T03:21:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.625+0000 7fd58ea00640 1 -- 172.21.15.46:0/463574534 wait complete. 2024-09-17T03:21:49.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 -- 172.21.15.46:0/3011161293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb40744d0 msgr2=0x7feeb406ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3011161293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb40744d0 0x7feeb406ea60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feeb40b0750 tx=0x7fee9c01c7d0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 -- 172.21.15.46:0/3011161293 shutdown_connections 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3011161293 >> v2:172.21.15.99:3300/0 conn(0x7feeb406efa0 0x7feeb406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3011161293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb40744d0 0x7feeb406ea60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 -- 172.21.15.46:0/3011161293 >> 172.21.15.46:0/3011161293 conn(0x7feeb40b2c40 msgr2=0x7feeb40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 -- 172.21.15.46:0/3011161293 shutdown_connections 2024-09-17T03:21:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 -- 172.21.15.46:0/3011161293 wait complete. 2024-09-17T03:21:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 Processor -- start 2024-09-17T03:21:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.805+0000 7feeb9a00640 1 -- start start 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeb406efa0 0x7feeb413ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feeb415c5d0 0x7feeb41f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 0x7feeb40b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4168c90 con 0x7feeb41fa220 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4168b10 con 0x7feeb406efa0 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feeb4168e10 con 0x7feeb415c5d0 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 0x7feeb40b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 0x7feeb40b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43726/0 (socket says 172.21.15.46:43726) 2024-09-17T03:21:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 -- 172.21.15.46:0/3006484294 learned_addr learned my addr 172.21.15.46:0/3006484294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeabe00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feeb415c5d0 0x7feeb41f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb2a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeb406efa0 0x7feeb413ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 -- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feeb415c5d0 msgr2=0x7feeb41f9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feeb415c5d0 0x7feeb41f9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 -- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeb406efa0 msgr2=0x7feeb413ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeb406efa0 0x7feeb413ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 -- 172.21.15.46:0/3006484294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb41f7490 con 0x7feeb41fa220 2024-09-17T03:21:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb2a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeb406efa0 0x7feeb413ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb3400640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 0x7feeb40b0f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feeac009fd0 tx=0x7feeac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00ce20 con 0x7feeb41fa220 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeac004b30 con 0x7feeb41fa220 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb41faa50 con 0x7feeb41fa220 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac005020 con 0x7feeb41fa220 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb41f69b0 con 0x7feeb41fa220 2024-09-17T03:21:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.809+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb406f3b0 con 0x7feeb41fa220 2024-09-17T03:21:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.813+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feeac0051c0 con 0x7feeb41fa220 2024-09-17T03:21:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.813+0000 7feeaaa00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee90073200 0x7fee90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.813+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7feeac094be0 con 0x7feeb41fa220 2024-09-17T03:21:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.813+0000 7feeb2a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee90073200 0x7fee90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.813+0000 7feeb2a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee90073200 0x7fee90075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feea4007a60 tx=0x7feea4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:49.817+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feeac05fe30 con 0x7feeb41fa220 2024-09-17T03:21:50.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.057+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7feeb40b0b60 con 0x7feeb41fa220 2024-09-17T03:21:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.061+0000 7feeaaa00640 1 -- 172.21.15.46:0/3006484294 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v32) ==== 97+0+0 (secure 0 0 0) 0x7feeac05f730 con 0x7feeb41fa220 2024-09-17T03:21:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee90073200 msgr2=0x7fee90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee90073200 0x7fee90075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feea4007a60 tx=0x7feea4009040 comp rx=0 tx=0).stop 2024-09-17T03:21:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 msgr2=0x7feeb40b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 0x7feeb40b0f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feeac009fd0 tx=0x7feeac00ef10 comp rx=0 tx=0).stop 2024-09-17T03:21:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 shutdown_connections 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee90073200 0x7fee90075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feeb41fa220 0x7feeb40b0f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feeb415c5d0 0x7feeb41f9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 --2- 172.21.15.46:0/3006484294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feeb406efa0 0x7feeb413ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 >> 172.21.15.46:0/3006484294 conn(0x7feeb40b2c40 msgr2=0x7feeb415d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 shutdown_connections 2024-09-17T03:21:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.065+0000 7feeb9a00640 1 -- 172.21.15.46:0/3006484294 wait complete. 2024-09-17T03:21:50.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 -- 172.21.15.46:0/1500572685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 msgr2=0x7fa6f8004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 --2- 172.21.15.46:0/1500572685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa6f8004070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa704001860 tx=0x7fa70401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa713400640 1 -- 172.21.15.46:0/1500572685 reap_dead start 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 -- 172.21.15.46:0/1500572685 shutdown_connections 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 -- 172.21.15.46:0/1500572685 >> 172.21.15.46:0/1500572685 conn(0x7fa7140b35f0 msgr2=0x7fa7140b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 -- 172.21.15.46:0/1500572685 shutdown_connections 2024-09-17T03:21:50.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 -- 172.21.15.46:0/1500572685 wait complete. 2024-09-17T03:21:50.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 Processor -- start 2024-09-17T03:21:50.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.249+0000 7fa719c00640 1 -- start start 2024-09-17T03:21:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa71415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7140b2110 0x7fa7141fb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7141fbdc0 0x7fa7141fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa714168780 con 0x7fa6f8003c20 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa714168900 con 0x7fa7141fbdc0 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa714168a80 con 0x7fa7140b2110 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa71415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa71415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43740/0 (socket says 172.21.15.46:43740) 2024-09-17T03:21:50.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 -- 172.21.15.46:0/1749839601 learned_addr learned my addr 172.21.15.46:0/1749839601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712000640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7140b2110 0x7fa7141fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa713400640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7141fbdc0 0x7fa7141fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 -- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7140b2110 msgr2=0x7fa7141fb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7140b2110 0x7fa7141fb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 -- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7141fbdc0 msgr2=0x7fa7141fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7141fbdc0 0x7fa7141fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 -- 172.21.15.46:0/1749839601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7141526c0 con 0x7fa6f8003c20 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa712a00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa71415b4f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa704005040 tx=0x7fa7040a5db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704001480 con 0x7fa6f8003c20 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7040b4030 con 0x7fa6f8003c20 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7040068d0 con 0x7fa6f8003c20 2024-09-17T03:21:50.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7140b2c70 con 0x7fa6f8003c20 2024-09-17T03:21:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7140b26b0 con 0x7fa6f8003c20 2024-09-17T03:21:50.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.253+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7141603e0 con 0x7fa6f8003c20 2024-09-17T03:21:50.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.257+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa704003bb0 con 0x7fa6f8003c20 2024-09-17T03:21:50.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.257+0000 7fa70aa00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa6f4073200 0x7fa6f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.257+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fa70412e240 con 0x7fa6f8003c20 2024-09-17T03:21:50.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.257+0000 7fa712000640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa6f4073200 0x7fa6f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.257+0000 7fa712000640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa6f4073200 0x7fa6f4075710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa700001870 tx=0x7fa700001e00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:50.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.261+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7141526c0 con 0x7fa6f8003c20 2024-09-17T03:21:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.497+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa714155ad0 con 0x7fa6f8003c20 2024-09-17T03:21:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.497+0000 7fa70aa00640 1 -- 172.21.15.46:0/1749839601 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v33) ==== 90+0+0 (secure 0 0 0) 0x7fa704016690 con 0x7fa6f8003c20 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa6f4073200 msgr2=0x7fa6f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa6f4073200 0x7fa6f4075710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa700001870 tx=0x7fa700001e00 comp rx=0 tx=0).stop 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 msgr2=0x7fa71415b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa71415b4f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa704005040 tx=0x7fa7040a5db0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 shutdown_connections 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa6f4073200 0x7fa6f4075710 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7141fbdc0 0x7fa7141fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa7140b2110 0x7fa7141fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 --2- 172.21.15.46:0/1749839601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f8003c20 0x7fa71415b4f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 >> 172.21.15.46:0/1749839601 conn(0x7fa7140b35f0 msgr2=0x7fa7140b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:50.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 shutdown_connections 2024-09-17T03:21:50.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.501+0000 7fa719c00640 1 -- 172.21.15.46:0/1749839601 wait complete. 2024-09-17T03:21:50.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:21:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 -- 172.21.15.46:0/2633570584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448157030 msgr2=0x7fe4480b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 --2- 172.21.15.46:0/2633570584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448157030 0x7fe4480b5a80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe438009840 tx=0x7fe43801c820 comp rx=0 tx=0).stop 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 -- 172.21.15.46:0/2633570584 shutdown_connections 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 --2- 172.21.15.46:0/2633570584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448160d50 0x7fe448074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 --2- 172.21.15.46:0/2633570584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448157030 0x7fe4480b5a80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 -- 172.21.15.46:0/2633570584 >> 172.21.15.46:0/2633570584 conn(0x7fe4480708c0 msgr2=0x7fe448070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 -- 172.21.15.46:0/2633570584 shutdown_connections 2024-09-17T03:21:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.677+0000 7fe44e400640 1 -- 172.21.15.46:0/2633570584 wait complete. 2024-09-17T03:21:50.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 Processor -- start 2024-09-17T03:21:50.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 -- start start 2024-09-17T03:21:50.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448157030 0x7fe448145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 0x7fe4481f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe448161160 0x7fe44815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe44816a7d0 con 0x7fe448160d50 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe44816a650 con 0x7fe448161160 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe44e400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe44816a950 con 0x7fe448157030 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 0x7fe4481f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 0x7fe4481f9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43756/0 (socket says 172.21.15.46:43756) 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.681+0000 7fe43fe00640 1 -- 172.21.15.46:0/2293957727 learned_addr learned my addr 172.21.15.46:0/2293957727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe447400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448157030 0x7fe448145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43fe00640 1 -- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448157030 msgr2=0x7fe448145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43fe00640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448157030 0x7fe448145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43fe00640 1 -- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe448161160 msgr2=0x7fe44815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe447e00640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe448161160 0x7fe44815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43fe00640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe448161160 0x7fe44815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43fe00640 1 -- 172.21.15.46:0/2293957727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44814fc20 con 0x7fe448160d50 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe447400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448157030 0x7fe448145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe447e00640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe448161160 0x7fe44815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43fe00640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 0x7fe4481f9f50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe43000d6a0 tx=0x7fe43000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430017070 con 0x7fe448160d50 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe430005310 con 0x7fe448160d50 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430009820 con 0x7fe448160d50 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44814ff00 con 0x7fe448160d50 2024-09-17T03:21:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481504c0 con 0x7fe448160d50 2024-09-17T03:21:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4480d2260 con 0x7fe448160d50 2024-09-17T03:21:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.685+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4300099c0 con 0x7fe448160d50 2024-09-17T03:21:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.689+0000 7fe43f400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe424073200 0x7fe424075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.689+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fe430094420 con 0x7fe448160d50 2024-09-17T03:21:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.689+0000 7fe447400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe424073200 0x7fe424075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.689+0000 7fe447400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe424073200 0x7fe424075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe4380af000 tx=0x7fe43801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:50.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.689+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe43005f800 con 0x7fe448160d50 2024-09-17T03:21:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.865+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe4480b1a90 con 0x7fe448160d50 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.865+0000 7fe43f400640 1 -- 172.21.15.46:0/2293957727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7fe43005f100 con 0x7fe448160d50 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:21:50.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:21:50.869 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:21:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe424073200 msgr2=0x7fe424075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe424073200 0x7fe424075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe4380af000 tx=0x7fe43801b060 comp rx=0 tx=0).stop 2024-09-17T03:21:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 msgr2=0x7fe4481f9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 0x7fe4481f9f50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe43000d6a0 tx=0x7fe43000db60 comp rx=0 tx=0).stop 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 shutdown_connections 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe424073200 0x7fe424075710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe448161160 0x7fe44815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe448160d50 0x7fe4481f9f50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 --2- 172.21.15.46:0/2293957727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe448157030 0x7fe448145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 >> 172.21.15.46:0/2293957727 conn(0x7fe4480708c0 msgr2=0x7fe44815e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 shutdown_connections 2024-09-17T03:21:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:50.869+0000 7fe44e400640 1 -- 172.21.15.46:0/2293957727 wait complete. 2024-09-17T03:21:50.890 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:21:50.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:21:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 -- 172.21.15.46:0/832867256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e44004810 msgr2=0x7f6e44004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 --2- 172.21.15.46:0/832867256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e44004810 0x7f6e44004c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6e60072120 tx=0x7f6e4801c820 comp rx=0 tx=0).stop 2024-09-17T03:21:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e64a00640 1 -- 172.21.15.46:0/832867256 reap_dead start 2024-09-17T03:21:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 -- 172.21.15.46:0/832867256 shutdown_connections 2024-09-17T03:21:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 -- 172.21.15.46:0/832867256 >> 172.21.15.46:0/832867256 conn(0x7f6e600b35f0 msgr2=0x7f6e600b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 -- 172.21.15.46:0/832867256 shutdown_connections 2024-09-17T03:21:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 -- 172.21.15.46:0/832867256 wait complete. 2024-09-17T03:21:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 Processor -- start 2024-09-17T03:21:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.053+0000 7f6e66a00640 1 -- start start 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e44004810 0x7f6e601fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 0x7f6e6006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e601567c0 0x7f6e6006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6006d380 con 0x7f6e601563b0 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6006d200 con 0x7f6e601567c0 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6006d500 con 0x7f6e44004810 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 0x7f6e6006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 0x7f6e6006d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43766/0 (socket says 172.21.15.46:43766) 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 -- 172.21.15.46:0/1631056784 learned_addr learned my addr 172.21.15.46:0/1631056784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e5fe00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e44004810 0x7f6e601fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e64a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e601567c0 0x7f6e6006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 -- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e44004810 msgr2=0x7f6e601fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e44004810 0x7f6e601fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 -- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e601567c0 msgr2=0x7f6e6006dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e601567c0 0x7f6e6006dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 -- 172.21.15.46:0/1631056784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e600af4c0 con 0x7f6e601563b0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e57400640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 0x7f6e6006d960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6e4801af20 tx=0x7f6e48003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e480ae070 con 0x7f6e601563b0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e48003e50 con 0x7f6e601563b0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e600af7a0 con 0x7f6e601563b0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48001480 con 0x7f6e601563b0 2024-09-17T03:21:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e600afd60 con 0x7f6e601563b0 2024-09-17T03:21:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.057+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e60069930 con 0x7f6e601563b0 2024-09-17T03:21:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.061+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6e480bd060 con 0x7f6e601563b0 2024-09-17T03:21:51.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.061+0000 7f6e5cc00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e3c073200 0x7f6e3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.061+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f6e4812da20 con 0x7f6e601563b0 2024-09-17T03:21:51.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.061+0000 7f6e5fe00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e3c073200 0x7f6e3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.061+0000 7f6e5fe00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e3c073200 0x7f6e3c075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009420 tx=0x7f6e50009500 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.065+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6e480165d0 con 0x7f6e601563b0 2024-09-17T03:21:51.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.237+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6e6015b690 con 0x7f6e601563b0 2024-09-17T03:21:51.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.501+0000 7f6e5cc00640 1 -- 172.21.15.46:0/1631056784 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f6e480f86a0 con 0x7f6e601563b0 2024-09-17T03:21:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:21:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e3c073200 msgr2=0x7f6e3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e3c073200 0x7f6e3c075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009420 tx=0x7f6e50009500 comp rx=0 tx=0).stop 2024-09-17T03:21:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 msgr2=0x7f6e6006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 0x7f6e6006d960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6e4801af20 tx=0x7f6e48003c60 comp rx=0 tx=0).stop 2024-09-17T03:21:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 shutdown_connections 2024-09-17T03:21:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e3c073200 0x7f6e3c075710 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009420 tx=0x7f6e50009500 comp rx=0 tx=0).stop 2024-09-17T03:21:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e601567c0 0x7f6e6006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e601563b0 0x7f6e6006d960 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 --2- 172.21.15.46:0/1631056784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e44004810 0x7f6e601fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.505+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 >> 172.21.15.46:0/1631056784 conn(0x7f6e600b35f0 msgr2=0x7f6e601597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.509+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 shutdown_connections 2024-09-17T03:21:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.509+0000 7f6e66a00640 1 -- 172.21.15.46:0/1631056784 wait complete. 2024-09-17T03:21:51.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:21:51.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5ec00640 1 -- 172.21.15.46:0/997184570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c003a10 msgr2=0x7f7c3c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/997184570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c003a10 0x7f7c3c003e80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009d10 tx=0x7f7c4801d440 comp rx=0 tx=0).stop 2024-09-17T03:21:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5cc00640 1 -- 172.21.15.46:0/997184570 reap_dead start 2024-09-17T03:21:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5ec00640 1 -- 172.21.15.46:0/997184570 shutdown_connections 2024-09-17T03:21:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5ec00640 1 -- 172.21.15.46:0/997184570 >> 172.21.15.46:0/997184570 conn(0x7f7c580b2040 msgr2=0x7f7c580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5ec00640 1 -- 172.21.15.46:0/997184570 shutdown_connections 2024-09-17T03:21:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.701+0000 7f7c5ec00640 1 -- 172.21.15.46:0/997184570 wait complete. 2024-09-17T03:21:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 Processor -- start 2024-09-17T03:21:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 -- start start 2024-09-17T03:21:51.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c3c003a10 0x7f7c580aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58157630 0x7f7c581f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 0x7f7c581fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58166900 con 0x7f7c58157a40 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58166780 con 0x7f7c58157630 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7c58166a80 con 0x7f7c3c003a10 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 0x7f7c581fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 0x7f7c581fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43786/0 (socket says 172.21.15.46:43786) 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 -- 172.21.15.46:0/1651702120 learned_addr learned my addr 172.21.15.46:0/1651702120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c4fe00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58157630 0x7f7c581f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c57e00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c3c003a10 0x7f7c580aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 -- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c3c003a10 msgr2=0x7f7c580aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c3c003a10 0x7f7c580aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 -- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58157630 msgr2=0x7f7c581f9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:51.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58157630 0x7f7c581f9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 -- 172.21.15.46:0/1651702120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c58156590 con 0x7f7c58157a40 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c57e00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c3c003a10 0x7f7c580aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5cc00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 0x7f7c581fa4e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c580b3680 tx=0x7f7c5000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50003b80 con 0x7f7c58157a40 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c5000de90 con 0x7f7c58157a40 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c58156870 con 0x7f7c58157a40 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004290 con 0x7f7c58157a40 2024-09-17T03:21:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.705+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c58156e30 con 0x7f7c58157a40 2024-09-17T03:21:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.709+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c58140400 con 0x7f7c58157a40 2024-09-17T03:21:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.709+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7c50004430 con 0x7f7c58157a40 2024-09-17T03:21:51.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.709+0000 7f7c54c00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c34073230 0x7f7c34075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:51.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.709+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f7c50095270 con 0x7f7c58157a40 2024-09-17T03:21:51.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.709+0000 7f7c57e00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c34073230 0x7f7c34075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.713+0000 7f7c57e00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c34073230 0x7f7c34075740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009d10 tx=0x7f7c4801a1e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.713+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c58140400 con 0x7f7c58157a40 2024-09-17T03:21:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:51.889+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7c5815bb60 con 0x7f7c58157a40 2024-09-17T03:21:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.509+0000 7f7c54c00640 1 -- 172.21.15.46:0/1651702120 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f7c5005fe90 con 0x7f7c58157a40 2024-09-17T03:21:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:21:52.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.513+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c34073230 msgr2=0x7f7c34075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:52.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.513+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c34073230 0x7f7c34075740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009d10 tx=0x7f7c4801a1e0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.513+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 msgr2=0x7f7c581fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:52.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.513+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 0x7f7c581fa4e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c580b3680 tx=0x7f7c5000d6b0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 shutdown_connections 2024-09-17T03:21:52.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c34073230 0x7f7c34075740 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c58157a40 0x7f7c581fa4e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58157630 0x7f7c581f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 --2- 172.21.15.46:0/1651702120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c3c003a10 0x7f7c580aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 >> 172.21.15.46:0/1651702120 conn(0x7f7c580b2040 msgr2=0x7f7c580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:52.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 shutdown_connections 2024-09-17T03:21:52.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.517+0000 7f7c5ec00640 1 -- 172.21.15.46:0/1651702120 wait complete. 2024-09-17T03:21:52.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:21:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.721+0000 7f3875200640 1 -- 172.21.15.46:0/3996455178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 msgr2=0x7f3854004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.721+0000 7f3875200640 1 --2- 172.21.15.46:0/3996455178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f3854004c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f385c006f00 tx=0x7f385c01ce30 comp rx=0 tx=0).stop 2024-09-17T03:21:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- 172.21.15.46:0/3996455178 shutdown_connections 2024-09-17T03:21:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 --2- 172.21.15.46:0/3996455178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f3854004c40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- 172.21.15.46:0/3996455178 >> 172.21.15.46:0/3996455178 conn(0x7f38700b2040 msgr2=0x7f38700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:52.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- 172.21.15.46:0/3996455178 shutdown_connections 2024-09-17T03:21:52.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- 172.21.15.46:0/3996455178 wait complete. 2024-09-17T03:21:52.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 Processor -- start 2024-09-17T03:21:52.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- start start 2024-09-17T03:21:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f387014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38700cf310 0x7f38700cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3870150a30 0x7f3870150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38700b55d0 con 0x7f3854004810 2024-09-17T03:21:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38700b5450 con 0x7f38700cf310 2024-09-17T03:21:52.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.725+0000 7f3875200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f38700b5750 con 0x7f3870150a30 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f387014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f387014fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46084/0 (socket says 172.21.15.46:46084) 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 -- 172.21.15.46:0/164538136 learned_addr learned my addr 172.21.15.46:0/164538136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386e800640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3870150a30 0x7f3870150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386d400640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38700cf310 0x7f38700cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 -- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3870150a30 msgr2=0x7f3870150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3870150a30 0x7f3870150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 -- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38700cf310 msgr2=0x7f38700cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38700cf310 0x7f38700cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 -- 172.21.15.46:0/164538136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870151380 con 0x7f3854004810 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386d400640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38700cf310 0x7f38700cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f386de00640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f387014fb40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3860011a40 tx=0x7f3860011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860004460 con 0x7f3854004810 2024-09-17T03:21:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f386000f9a0 con 0x7f3854004810 2024-09-17T03:21:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386000fdb0 con 0x7f3854004810 2024-09-17T03:21:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38701f9e60 con 0x7f3854004810 2024-09-17T03:21:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38701f90a0 con 0x7f3854004810 2024-09-17T03:21:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387015f560 con 0x7f3854004810 2024-09-17T03:21:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.729+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3860005020 con 0x7f3854004810 2024-09-17T03:21:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.733+0000 7f3866000640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3850073200 0x7f3850075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.733+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f3860095670 con 0x7f3854004810 2024-09-17T03:21:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.733+0000 7f386d400640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3850073200 0x7f3850075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.733+0000 7f386d400640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3850073200 0x7f3850075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ab000 tx=0x7f385c019c70 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.737+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3870151380 con 0x7f3854004810 2024-09-17T03:21:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:52.945+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3870150080 con 0x7f3854004810 2024-09-17T03:21:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.513+0000 7f3866000640 1 -- 172.21.15.46:0/164538136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f3860060140 con 0x7f3854004810 2024-09-17T03:21:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 4 and data pool 5 2024-09-17T03:21:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.517+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3850073200 msgr2=0x7f3850075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.517+0000 7f3875200640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3850073200 0x7f3850075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ab000 tx=0x7f385c019c70 comp rx=0 tx=0).stop 2024-09-17T03:21:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.517+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 msgr2=0x7f387014fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.517+0000 7f3875200640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f387014fb40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3860011a40 tx=0x7f3860011f00 comp rx=0 tx=0).stop 2024-09-17T03:21:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 shutdown_connections 2024-09-17T03:21:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3850073200 0x7f3850075710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3870150a30 0x7f3870150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38700cf310 0x7f38700cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 --2- 172.21.15.46:0/164538136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3854004810 0x7f387014fb40 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3860011a40 tx=0x7f3860011f00 comp rx=0 tx=0).stop 2024-09-17T03:21:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 >> 172.21.15.46:0/164538136 conn(0x7f38700b2040 msgr2=0x7f3870159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 shutdown_connections 2024-09-17T03:21:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.521+0000 7f3875200640 1 -- 172.21.15.46:0/164538136 wait complete. 2024-09-17T03:21:53.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 -- 172.21.15.46:0/424441342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b54f0 msgr2=0x7f12bc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 --2- 172.21.15.46:0/424441342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b54f0 0x7f12bc15bb60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009bc0 tx=0x7f12b401cae0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 -- 172.21.15.46:0/424441342 shutdown_connections 2024-09-17T03:21:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 --2- 172.21.15.46:0/424441342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b54f0 0x7f12bc15bb60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 -- 172.21.15.46:0/424441342 >> 172.21.15.46:0/424441342 conn(0x7f12bc0b2040 msgr2=0x7f12bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 -- 172.21.15.46:0/424441342 shutdown_connections 2024-09-17T03:21:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.729+0000 7f12c1200640 1 -- 172.21.15.46:0/424441342 wait complete. 2024-09-17T03:21:53.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 Processor -- start 2024-09-17T03:21:53.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 -- start start 2024-09-17T03:21:53.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 0x7f12bc151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:53.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12bc1527b0 0x7f12bc1f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:53.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc158c00 0x7f12bc1f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc169b30 con 0x7f12bc0b5c70 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1699b0 con 0x7f12bc158c00 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12c1200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc169cb0 con 0x7f12bc1527b0 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 0x7f12bc151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 0x7f12bc151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46104/0 (socket says 172.21.15.46:46104) 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12b9e00640 1 -- 172.21.15.46:0/2780732822 learned_addr learned my addr 172.21.15.46:0/2780732822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12ba800640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc158c00 0x7f12bc1f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.733+0000 7f12b1400640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12bc1527b0 0x7f12bc1f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b9e00640 1 -- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12bc1527b0 msgr2=0x7f12bc1f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b9e00640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12bc1527b0 0x7f12bc1f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b9e00640 1 -- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc158c00 msgr2=0x7f12bc1f9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b9e00640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc158c00 0x7f12bc1f9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b9e00640 1 -- 172.21.15.46:0/2780732822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc0c5df0 con 0x7f12bc0b5c70 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b1400640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12bc1527b0 0x7f12bc1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b9e00640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 0x7f12bc151550 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c400 tx=0x7f12ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac011790 con 0x7f12bc0b5c70 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12ac017070 con 0x7f12bc0b5c70 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc158120 con 0x7f12bc0b5c70 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac011cc0 con 0x7f12bc0b5c70 2024-09-17T03:21:53.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc1fa570 con 0x7f12bc0b5c70 2024-09-17T03:21:53.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc1fa7d0 con 0x7f12bc0b5c70 2024-09-17T03:21:53.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.737+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f12ac01c460 con 0x7f12bc0b5c70 2024-09-17T03:21:53.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.741+0000 7f12b2a00640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298073200 0x7f1298075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:53.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.741+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f12ac093f50 con 0x7f12bc0b5c70 2024-09-17T03:21:53.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.741+0000 7f12b1400640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298073200 0x7f1298075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:53.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.741+0000 7f12b1400640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298073200 0x7f1298075710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12bc157560 tx=0x7f12a4008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:53.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.745+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12ac05eff0 con 0x7f12bc0b5c70 2024-09-17T03:21:53.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.917+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12bc15bb60 con 0x7f12bc0b5c70 2024-09-17T03:21:53.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12b2a00640 1 -- 172.21.15.46:0/2780732822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16706 (secure 0 0 0) 0x7f12ac05e8f0 con 0x7f12bc0b5c70 2024-09-17T03:21:53.921 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:53.921 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:52.949222+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:51.251673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:51.902649+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298073200 msgr2=0x7f1298075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298073200 0x7f1298075710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12bc157560 tx=0x7f12a4008040 comp rx=0 tx=0).stop 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 msgr2=0x7f12bc151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 0x7f12bc151550 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c400 tx=0x7f12ac00c8c0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 shutdown_connections 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298073200 0x7f1298075710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc158c00 0x7f12bc1f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12bc1527b0 0x7f12bc1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 --2- 172.21.15.46:0/2780732822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc0b5c70 0x7f12bc151550 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 >> 172.21.15.46:0/2780732822 conn(0x7f12bc0b2040 msgr2=0x7f12bc0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 shutdown_connections 2024-09-17T03:21:53.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:53.921+0000 7f12c1200640 1 -- 172.21.15.46:0/2780732822 wait complete. 2024-09-17T03:21:53.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f36a00640 1 -- 172.21.15.46:0/1737689207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 msgr2=0x7f2f14003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1737689207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f14003e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c470 tx=0x7f2f2001cf70 comp rx=0 tx=0).stop 2024-09-17T03:21:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f34a00640 1 -- 172.21.15.46:0/1737689207 reap_dead start 2024-09-17T03:21:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f36a00640 1 -- 172.21.15.46:0/1737689207 shutdown_connections 2024-09-17T03:21:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f36a00640 1 -- 172.21.15.46:0/1737689207 >> 172.21.15.46:0/1737689207 conn(0x7f2f300b2040 msgr2=0x7f2f300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f36a00640 1 -- 172.21.15.46:0/1737689207 shutdown_connections 2024-09-17T03:21:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.101+0000 7f2f36a00640 1 -- 172.21.15.46:0/1737689207 wait complete. 2024-09-17T03:21:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 Processor -- start 2024-09-17T03:21:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 -- start start 2024-09-17T03:21:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f30156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f30157c70 0x7f2f3013fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f300ae8e0 0x7f2f300aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30166900 con 0x7f2f14003a10 2024-09-17T03:21:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30166780 con 0x7f2f300ae8e0 2024-09-17T03:21:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2f30166a80 con 0x7f2f30157c70 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f30156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f30156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46120/0 (socket says 172.21.15.46:46120) 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 -- 172.21.15.46:0/1709368983 learned_addr learned my addr 172.21.15.46:0/1709368983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f34a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f300ae8e0 0x7f2f300aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f27400640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f30157c70 0x7f2f3013fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 -- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f30157c70 msgr2=0x7f2f3013fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f30157c70 0x7f2f3013fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 -- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f300ae8e0 msgr2=0x7f2f300aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f300ae8e0 0x7f2f300aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 -- 172.21.15.46:0/1709368983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f300af230 con 0x7f2f14003a10 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f34a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f300ae8e0 0x7f2f300aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f27400640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f30157c70 0x7f2f3013fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2fe00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f30156180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f20098a30 tx=0x7f2f20000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20001690 con 0x7f2f14003a10 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f20001920 con 0x7f2f14003a10 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003840 con 0x7f2f14003a10 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f301566c0 con 0x7f2f14003a10 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.105+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f30156eb0 con 0x7f2f14003a10 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.109+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3015bb60 con 0x7f2f14003a10 2024-09-17T03:21:54.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.109+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2f200039e0 con 0x7f2f14003a10 2024-09-17T03:21:54.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.109+0000 7f2f2cc00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2f0c073230 0x7f2f0c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.109+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f2f2012ea20 con 0x7f2f14003a10 2024-09-17T03:21:54.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.109+0000 7f2f27400640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2f0c073230 0x7f2f0c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.109+0000 7f2f27400640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2f0c073230 0x7f2f0c075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f3015b1b0 tx=0x7f2f180091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.113+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2f300af230 con 0x7f2f14003a10 2024-09-17T03:21:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.289+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f30076150 con 0x7f2f14003a10 2024-09-17T03:21:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.289+0000 7f2f2cc00640 1 -- 172.21.15.46:0/1709368983 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16706 (secure 0 0 0) 0x7f2f300b5730 con 0x7f2f14003a10 2024-09-17T03:21:54.296 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:54.296 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:52.949222+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:51.251673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:51.902649+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2f0c073230 msgr2=0x7f2f0c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2f0c073230 0x7f2f0c075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f3015b1b0 tx=0x7f2f180091e0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 msgr2=0x7f2f30156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f30156180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f20098a30 tx=0x7f2f20000f70 comp rx=0 tx=0).stop 2024-09-17T03:21:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 shutdown_connections 2024-09-17T03:21:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2f0c073230 0x7f2f0c075740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f300ae8e0 0x7f2f300aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2f30157c70 0x7f2f3013fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 --2- 172.21.15.46:0/1709368983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f14003a10 0x7f2f30156180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 >> 172.21.15.46:0/1709368983 conn(0x7f2f300b2040 msgr2=0x7f2f30152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 shutdown_connections 2024-09-17T03:21:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.293+0000 7f2f36a00640 1 -- 172.21.15.46:0/1709368983 wait complete. 2024-09-17T03:21:54.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:21:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 -- 172.21.15.46:0/2803658040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740749a0 msgr2=0x7f3c74074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2803658040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740749a0 0x7f3c74074db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009bc0 tx=0x7f3c6401cba0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 -- 172.21.15.46:0/2803658040 shutdown_connections 2024-09-17T03:21:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2803658040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740749a0 0x7f3c74074db0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 -- 172.21.15.46:0/2803658040 >> 172.21.15.46:0/2803658040 conn(0x7f3c740b2040 msgr2=0x7f3c740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 -- 172.21.15.46:0/2803658040 shutdown_connections 2024-09-17T03:21:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 -- 172.21.15.46:0/2803658040 wait complete. 2024-09-17T03:21:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 Processor -- start 2024-09-17T03:21:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.477+0000 7f3c7a800640 1 -- start start 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c740749a0 0x7f3c74143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 0x7f3c74145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c740cefa0 0x7f3c741f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7416b350 con 0x7f3c740b3340 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7416b1d0 con 0x7f3c740cefa0 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7416b4d0 con 0x7f3c740749a0 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 0x7f3c74145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 0x7f3c74145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46132/0 (socket says 172.21.15.46:46132) 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 -- 172.21.15.46:0/2898368702 learned_addr learned my addr 172.21.15.46:0/2898368702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c73400640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c740749a0 0x7f3c74143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c73e00640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c740cefa0 0x7f3c741f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 -- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c740749a0 msgr2=0x7f3c74143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c740749a0 0x7f3c74143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 -- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c740cefa0 msgr2=0x7f3c741f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c740cefa0 0x7f3c741f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 -- 172.21.15.46:0/2898368702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c741f7570 con 0x7f3c740b3340 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6aa00640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 0x7f3c74145cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c3f0 tx=0x7f3c5c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x7f3c740b3340 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c5c0053c0 con 0x7f3c740b3340 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c741f7850 con 0x7f3c740b3340 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c003860 con 0x7f3c740b3340 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c741435a0 con 0x7f3c740b3340 2024-09-17T03:21:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.481+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c74074db0 con 0x7f3c740b3340 2024-09-17T03:21:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.485+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3c5c01a070 con 0x7f3c740b3340 2024-09-17T03:21:54.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.485+0000 7f3c6be00640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c50073230 0x7f3c50075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:54.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.485+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f3c5c093570 con 0x7f3c740b3340 2024-09-17T03:21:54.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.485+0000 7f3c73400640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c50073230 0x7f3c50075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:54.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.485+0000 7f3c73400640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c50073230 0x7f3c50075740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c640af000 tx=0x7f3c6401b3e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.489+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3c5c05dea0 con 0x7f3c740b3340 2024-09-17T03:21:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:54.705+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3c741f7300 con 0x7f3c740b3340 2024-09-17T03:21:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.537+0000 7f3c6be00640 1 -- 172.21.15.46:0/2898368702 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f3c5c091e20 con 0x7f3c740b3340 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c50073230 msgr2=0x7f3c50075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c50073230 0x7f3c50075740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c640af000 tx=0x7f3c6401b3e0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 msgr2=0x7f3c74145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 0x7f3c74145cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c3f0 tx=0x7f3c5c00c8b0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 shutdown_connections 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3c50073230 0x7f3c50075740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c740cefa0 0x7f3c741f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c740b3340 0x7f3c74145cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 --2- 172.21.15.46:0/2898368702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3c740749a0 0x7f3c74143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 >> 172.21.15.46:0/2898368702 conn(0x7f3c740b2040 msgr2=0x7f3c7415af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:55.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 shutdown_connections 2024-09-17T03:21:55.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.541+0000 7f3c7a800640 1 -- 172.21.15.46:0/2898368702 wait complete. 2024-09-17T03:21:55.566 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:21:55.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:21:55.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 -- 172.21.15.46:0/3380267763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0160d50 msgr2=0x7fd0e0074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:55.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 --2- 172.21.15.46:0/3380267763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0160d50 0x7fd0e0074410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009900 tx=0x7fd0cc01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 -- 172.21.15.46:0/3380267763 shutdown_connections 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 --2- 172.21.15.46:0/3380267763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0160d50 0x7fd0e0074410 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 --2- 172.21.15.46:0/3380267763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0e0157030 0x7fd0e00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 -- 172.21.15.46:0/3380267763 >> 172.21.15.46:0/3380267763 conn(0x7fd0e00708c0 msgr2=0x7fd0e0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 -- 172.21.15.46:0/3380267763 shutdown_connections 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 -- 172.21.15.46:0/3380267763 wait complete. 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 Processor -- start 2024-09-17T03:21:55.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.729+0000 7fd0e7000640 1 -- start start 2024-09-17T03:21:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e0157030 0x7fd0e0150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0e0160d50 0x7fd0e0141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 0x7fd0e013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e016a8a0 con 0x7fd0e0161160 2024-09-17T03:21:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e016a720 con 0x7fd0e0157030 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e016aa20 con 0x7fd0e0160d50 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 0x7fd0e013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 0x7fd0e013e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46142/0 (socket says 172.21.15.46:46142) 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 -- 172.21.15.46:0/4080827208 learned_addr learned my addr 172.21.15.46:0/4080827208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0dfe00640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e0157030 0x7fd0e0150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0df400640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0e0160d50 0x7fd0e0141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 -- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0e0160d50 msgr2=0x7fd0e0141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0e0160d50 0x7fd0e0141d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 -- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e0157030 msgr2=0x7fd0e0150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e0157030 0x7fd0e0150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 -- 172.21.15.46:0/4080827208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e01510f0 con 0x7fd0e0161160 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e5000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 0x7fd0e013e6d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b9a0 tx=0x7fd0d800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8019070 con 0x7fd0e0161160 2024-09-17T03:21:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d800cc90 con 0x7fd0e0161160 2024-09-17T03:21:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e01513d0 con 0x7fd0e0161160 2024-09-17T03:21:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80146a0 con 0x7fd0e0161160 2024-09-17T03:21:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e0158d80 con 0x7fd0e0161160 2024-09-17T03:21:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e0150e60 con 0x7fd0e0161160 2024-09-17T03:21:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.733+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd0d801c070 con 0x7fd0e0161160 2024-09-17T03:21:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.737+0000 7fd0d7e00640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd0c0073230 0x7fd0c0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.737+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d8093e00 con 0x7fd0e0161160 2024-09-17T03:21:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.737+0000 7fd0dfe00640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd0c0073230 0x7fd0c0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:55.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.737+0000 7fd0dfe00640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd0c0073230 0x7fd0c0075740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000ab40 tx=0x7fd0d000a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.741+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd0d805e7c0 con 0x7fd0e0161160 2024-09-17T03:21:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:55.953+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd0e00d2260 con 0x7fd0e0161160 2024-09-17T03:21:56.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.541+0000 7fd0d7e00640 1 -- 172.21.15.46:0/4080827208 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fd0d80631f0 con 0x7fd0e0161160 2024-09-17T03:21:56.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd0c0073230 msgr2=0x7fd0c0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd0c0073230 0x7fd0c0075740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000ab40 tx=0x7fd0d000a250 comp rx=0 tx=0).stop 2024-09-17T03:21:56.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 msgr2=0x7fd0e013e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 0x7fd0e013e6d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b9a0 tx=0x7fd0d800be60 comp rx=0 tx=0).stop 2024-09-17T03:21:56.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 shutdown_connections 2024-09-17T03:21:56.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd0c0073230 0x7fd0c0075740 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e0161160 0x7fd0e013e6d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0e0160d50 0x7fd0e0141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 --2- 172.21.15.46:0/4080827208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e0157030 0x7fd0e0150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 >> 172.21.15.46:0/4080827208 conn(0x7fd0e00708c0 msgr2=0x7fd0e0152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:56.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 shutdown_connections 2024-09-17T03:21:56.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.545+0000 7fd0e7000640 1 -- 172.21.15.46:0/4080827208 wait complete. 2024-09-17T03:21:56.571 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:21:56.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:56.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 -- 172.21.15.46:0/1313423096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b80b54f0 msgr2=0x7fa4b8155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 --2- 172.21.15.46:0/1313423096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b80b54f0 0x7fa4b8155b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009900 tx=0x7fa4a001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 -- 172.21.15.46:0/1313423096 shutdown_connections 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 --2- 172.21.15.46:0/1313423096 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4b8073a90 0x7fa4b8073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 --2- 172.21.15.46:0/1313423096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b80b54f0 0x7fa4b8155b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 -- 172.21.15.46:0/1313423096 >> 172.21.15.46:0/1313423096 conn(0x7fa4b80b2040 msgr2=0x7fa4b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 -- 172.21.15.46:0/1313423096 shutdown_connections 2024-09-17T03:21:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 -- 172.21.15.46:0/1313423096 wait complete. 2024-09-17T03:21:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 Processor -- start 2024-09-17T03:21:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.733+0000 7fa4bde00640 1 -- start start 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4b8072100 0x7fa4b81f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b8073a90 0x7fa4b8154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 0x7fa4b814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8168f30 con 0x7fa4b8154f80 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8168db0 con 0x7fa4b8073a90 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b81690b0 con 0x7fa4b8072100 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 0x7fa4b814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 0x7fa4b814c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46166/0 (socket says 172.21.15.46:46166) 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 -- 172.21.15.46:0/2309457253 learned_addr learned my addr 172.21.15.46:0/2309457253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4afe00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b8073a90 0x7fa4b8154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b6a00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4b8072100 0x7fa4b81f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 -- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4b8072100 msgr2=0x7fa4b81f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4b8072100 0x7fa4b81f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 -- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b8073a90 msgr2=0x7fa4b8154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b8073a90 0x7fa4b8154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 -- 172.21.15.46:0/2309457253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80af790 con 0x7fa4b8154f80 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4afe00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b8073a90 0x7fa4b8154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4b7400640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 0x7fa4b814c580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b9a0 tx=0x7fa4b000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0019070 con 0x7fa4b8154f80 2024-09-17T03:21:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4b000cc50 con 0x7fa4b8154f80 2024-09-17T03:21:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00146a0 con 0x7fa4b8154f80 2024-09-17T03:21:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b80afa70 con 0x7fa4b8154f80 2024-09-17T03:21:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b814fe50 con 0x7fa4b8154f80 2024-09-17T03:21:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.737+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b8073ea0 con 0x7fa4b8154f80 2024-09-17T03:21:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.741+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4b001c070 con 0x7fa4b8154f80 2024-09-17T03:21:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.741+0000 7fa4af400640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa494073200 0x7fa494075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.741+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa4b0093ff0 con 0x7fa4b8154f80 2024-09-17T03:21:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.741+0000 7fa4b6a00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa494073200 0x7fa494075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.741+0000 7fa4b6a00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa494073200 0x7fa494075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800b810 tx=0x7fa4a8008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.745+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa4b005f000 con 0x7fa4b8154f80 2024-09-17T03:21:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.953+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4b80b3740 con 0x7fa4b8154f80 2024-09-17T03:21:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.953+0000 7fa4af400640 1 -- 172.21.15.46:0/2309457253 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5236 (secure 0 0 0) 0x7fa4b005e900 con 0x7fa4b8154f80 2024-09-17T03:21:56.957 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:56.957 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-17T03:21:56:541550+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:52.949184+0000","modified":"2024-09-17T03:21:55.955106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244821":{"gid":244821,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3646914597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3646914597},{"type":"v1","addr":"172.21.15.99:6835","nonce":3646914597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244821,"qdb_cluster":[244821]},"id":2}]} 2024-09-17T03:21:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa494073200 msgr2=0x7fa494075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa494073200 0x7fa494075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800b810 tx=0x7fa4a8008040 comp rx=0 tx=0).stop 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 msgr2=0x7fa4b814c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 0x7fa4b814c580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b9a0 tx=0x7fa4b000be60 comp rx=0 tx=0).stop 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 shutdown_connections 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa494073200 0x7fa494075710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4b8154f80 0x7fa4b814c580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b8073a90 0x7fa4b8154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 --2- 172.21.15.46:0/2309457253 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4b8072100 0x7fa4b81f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 >> 172.21.15.46:0/2309457253 conn(0x7fa4b80b2040 msgr2=0x7fa4b80afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 shutdown_connections 2024-09-17T03:21:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:56.957+0000 7fa4bde00640 1 -- 172.21.15.46:0/2309457253 wait complete. 2024-09-17T03:21:56.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:21:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 -- 172.21.15.46:0/794417826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300ae8e0 msgr2=0x7f21300aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 --2- 172.21.15.46:0/794417826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300ae8e0 0x7f21300aecf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2120009840 tx=0x7f212001c820 comp rx=0 tx=0).stop 2024-09-17T03:21:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 -- 172.21.15.46:0/794417826 shutdown_connections 2024-09-17T03:21:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 --2- 172.21.15.46:0/794417826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21300732c0 0x7f21300736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 --2- 172.21.15.46:0/794417826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300ae8e0 0x7f21300aecf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 -- 172.21.15.46:0/794417826 >> 172.21.15.46:0/794417826 conn(0x7f21300b2c40 msgr2=0x7f21300b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 -- 172.21.15.46:0/794417826 shutdown_connections 2024-09-17T03:21:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.137+0000 7f2136600640 1 -- 172.21.15.46:0/794417826 wait complete. 2024-09-17T03:21:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 Processor -- start 2024-09-17T03:21:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 -- start start 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21300732c0 0x7f21300709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21300ae8e0 0x7f21301fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 0x7f21301490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f213016c2d0 con 0x7f21300b4630 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f213016c150 con 0x7f21300732c0 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f213016c450 con 0x7f21300ae8e0 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 0x7f21301490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 0x7f21301490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46186/0 (socket says 172.21.15.46:46186) 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 -- 172.21.15.46:0/81879283 learned_addr learned my addr 172.21.15.46:0/81879283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:57.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212f400640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21300732c0 0x7f21300709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212ea00640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21300ae8e0 0x7f21301fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 -- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21300ae8e0 msgr2=0x7f21301fcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21300ae8e0 0x7f21301fcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 -- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21300732c0 msgr2=0x7f21300709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21300732c0 0x7f21300709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 -- 172.21.15.46:0/81879283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213015f740 con 0x7f21300b4630 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f212fe00640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 0x7f21301490b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f212800ea80 tx=0x7f212800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212800ce40 con 0x7f21300b4630 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2128004b30 con 0x7f21300b4630 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128005020 con 0x7f21300b4630 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2130149970 con 0x7f21300b4630 2024-09-17T03:21:57.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.141+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2130149f30 con 0x7f21300b4630 2024-09-17T03:21:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.145+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2130071010 con 0x7f21300b4630 2024-09-17T03:21:57.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.145+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21280051c0 con 0x7f21300b4630 2024-09-17T03:21:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.145+0000 7f2127400640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2110073200 0x7f2110075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.145+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2128094cc0 con 0x7f21300b4630 2024-09-17T03:21:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.145+0000 7f212f400640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2110073200 0x7f2110075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.149+0000 7f212f400640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2110073200 0x7f2110075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21200af000 tx=0x7f212001b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.149+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f212809b080 con 0x7f21300b4630 2024-09-17T03:21:57.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.329+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21300736d0 con 0x7f21300b4630 2024-09-17T03:21:57.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.329+0000 7f2127400640 1 -- 172.21.15.46:0/81879283 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16693 (secure 0 0 0) 0x7f212805f680 con 0x7f21300b4630 2024-09-17T03:21:57.332 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:57.332 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:54.529415+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:51.251673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:51.902649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2110073200 msgr2=0x7f2110075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2110073200 0x7f2110075710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21200af000 tx=0x7f212001b060 comp rx=0 tx=0).stop 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 msgr2=0x7f21301490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 0x7f21301490b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f212800ea80 tx=0x7f212800ef40 comp rx=0 tx=0).stop 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 shutdown_connections 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2110073200 0x7f2110075710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21300b4630 0x7f21301490b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21300ae8e0 0x7f21301fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 --2- 172.21.15.46:0/81879283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21300732c0 0x7f21300709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 >> 172.21.15.46:0/81879283 conn(0x7f21300b2c40 msgr2=0x7f21301600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 shutdown_connections 2024-09-17T03:21:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.333+0000 7f2136600640 1 -- 172.21.15.46:0/81879283 wait complete. 2024-09-17T03:21:57.355 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:21:57.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 -- 172.21.15.46:0/830448512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b5c70 msgr2=0x7f33c015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 --2- 172.21.15.46:0/830448512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b5c70 0x7f33c015bb60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33c00cca90 tx=0x7f33b001c740 comp rx=0 tx=0).stop 2024-09-17T03:21:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 -- 172.21.15.46:0/830448512 shutdown_connections 2024-09-17T03:21:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 --2- 172.21.15.46:0/830448512 >> v2:172.21.15.99:3300/0 conn(0x7f33c00b4de0 0x7f33c00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 --2- 172.21.15.46:0/830448512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b5c70 0x7f33c015bb60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 -- 172.21.15.46:0/830448512 >> 172.21.15.46:0/830448512 conn(0x7f33c00b2040 msgr2=0x7f33c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 -- 172.21.15.46:0/830448512 shutdown_connections 2024-09-17T03:21:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 -- 172.21.15.46:0/830448512 wait complete. 2024-09-17T03:21:57.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 Processor -- start 2024-09-17T03:21:57.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.513+0000 7f33c4c00640 1 -- start start 2024-09-17T03:21:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 0x7f33c013fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c00b5c70 0x7f33c01f9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33c01527b0 0x7f33c01f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0168970 con 0x7f33c00b4de0 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01687f0 con 0x7f33c00b5c70 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f33c0168af0 con 0x7f33c01527b0 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 0x7f33c013fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 0x7f33c013fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46200/0 (socket says 172.21.15.46:46200) 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 -- 172.21.15.46:0/3646683448 learned_addr learned my addr 172.21.15.46:0/3646683448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33be800640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33c01527b0 0x7f33c01f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33b7e00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c00b5c70 0x7f33c01f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 -- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33c01527b0 msgr2=0x7f33c01f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33c01527b0 0x7f33c01f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 -- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c00b5c70 msgr2=0x7f33c01f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c00b5c70 0x7f33c01f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 -- 172.21.15.46:0/3646683448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c015da70 con 0x7f33c00b4de0 2024-09-17T03:21:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33bde00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 0x7f33c013fad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f33b0000c10 tx=0x7f33b00a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00bb070 con 0x7f33c00b4de0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b0005030 con 0x7f33c00b4de0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c01fa5e0 con 0x7f33c00b4de0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00b2040 con 0x7f33c00b4de0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c0140010 con 0x7f33c00b4de0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.517+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c00ad710 con 0x7f33c00b4de0 2024-09-17T03:21:57.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.521+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33b00be070 con 0x7f33c00b4de0 2024-09-17T03:21:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.521+0000 7f33b6000640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f339c073200 0x7f339c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.521+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f33b012de10 con 0x7f33c00b4de0 2024-09-17T03:21:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.521+0000 7f33b7e00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f339c073200 0x7f339c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.521+0000 7f33b7e00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f339c073200 0x7f339c075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33c00b2950 tx=0x7f33a8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:57.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.525+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33b0131030 con 0x7f33c00b4de0 2024-09-17T03:21:57.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.697+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33c006b540 con 0x7f339c073200 2024-09-17T03:21:57.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.705+0000 7f33b6000640 1 -- 172.21.15.46:0/3646683448 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7f33a80050d0 con 0x7f339c073200 2024-09-17T03:21:57.708 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:57.710 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:57.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f339c073200 msgr2=0x7f339c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f339c073200 0x7f339c075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33c00b2950 tx=0x7f33a8009040 comp rx=0 tx=0).stop 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 msgr2=0x7f33c013fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 0x7f33c013fad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f33b0000c10 tx=0x7f33b00a5710 comp rx=0 tx=0).stop 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 shutdown_connections 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f339c073200 0x7f339c075710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33c01527b0 0x7f33c01f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33c00b5c70 0x7f33c01f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 --2- 172.21.15.46:0/3646683448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33c00b4de0 0x7f33c013fad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 >> 172.21.15.46:0/3646683448 conn(0x7f33c00b2040 msgr2=0x7f33c0151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 shutdown_connections 2024-09-17T03:21:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.709+0000 7f33c4c00640 1 -- 172.21.15.46:0/3646683448 wait complete. 2024-09-17T03:21:57.730 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T03:21:57.210583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241004,"kb_used_data":3952,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733716,"statfs":{"total":773094113280,"available":772847325184,"internally_reserved":0,"allocated":4046848,"data_stored":2255988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.704118"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213251+0000","last_change":"2024-09-17T03:21:53.554223+0000","last_active":"2024-09-17T03:21:56.213251+0000","last_peered":"2024-09-17T03:21:56.213251+0000","last_clean":"2024-09-17T03:21:56.213251+0000","last_became_active":"2024-09-17T03:21:53.553879+0000","last_became_peered":"2024-09-17T03:21:53.553879+0000","last_unstale":"2024-09-17T03:21:56.213251+0000","last_undegraded":"2024-09-17T03:21:56.213251+0000","last_fullsized":"2024-09-17T03:21:56.213251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:25.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539783+0000","last_change":"2024-09-17T03:21:53.546731+0000","last_active":"2024-09-17T03:21:54.539783+0000","last_peered":"2024-09-17T03:21:54.539783+0000","last_clean":"2024-09-17T03:21:54.539783+0000","last_became_active":"2024-09-17T03:21:53.546359+0000","last_became_peered":"2024-09-17T03:21:53.546359+0000","last_unstale":"2024-09-17T03:21:54.539783+0000","last_undegraded":"2024-09-17T03:21:54.539783+0000","last_fullsized":"2024-09-17T03:21:54.539783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:28.689089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.556598+0000","last_change":"2024-09-17T03:21:53.547245+0000","last_active":"2024-09-17T03:21:53.556598+0000","last_peered":"2024-09-17T03:21:53.556598+0000","last_clean":"2024-09-17T03:21:53.556598+0000","last_became_active":"2024-09-17T03:21:53.545526+0000","last_became_peered":"2024-09-17T03:21:53.545526+0000","last_unstale":"2024-09-17T03:21:53.556598+0000","last_undegraded":"2024-09-17T03:21:53.556598+0000","last_fullsized":"2024-09-17T03:21:53.556598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:53.045996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538707+0000","last_change":"2024-09-17T03:21:53.540990+0000","last_active":"2024-09-17T03:21:54.538707+0000","last_peered":"2024-09-17T03:21:54.538707+0000","last_clean":"2024-09-17T03:21:54.538707+0000","last_became_active":"2024-09-17T03:21:53.540663+0000","last_became_peered":"2024-09-17T03:21:53.540663+0000","last_unstale":"2024-09-17T03:21:54.538707+0000","last_undegraded":"2024-09-17T03:21:54.538707+0000","last_fullsized":"2024-09-17T03:21:54.538707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:30.953926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209242+0000","last_change":"2024-09-17T03:21:53.559972+0000","last_active":"2024-09-17T03:21:56.209242+0000","last_peered":"2024-09-17T03:21:56.209242+0000","last_clean":"2024-09-17T03:21:56.209242+0000","last_became_active":"2024-09-17T03:21:53.559648+0000","last_became_peered":"2024-09-17T03:21:53.559648+0000","last_unstale":"2024-09-17T03:21:56.209242+0000","last_undegraded":"2024-09-17T03:21:56.209242+0000","last_fullsized":"2024-09-17T03:21:56.209242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:55.395879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538733+0000","last_change":"2024-09-17T03:21:53.564719+0000","last_active":"2024-09-17T03:21:54.538733+0000","last_peered":"2024-09-17T03:21:54.538733+0000","last_clean":"2024-09-17T03:21:54.538733+0000","last_became_active":"2024-09-17T03:21:53.564405+0000","last_became_peered":"2024-09-17T03:21:53.564405+0000","last_unstale":"2024-09-17T03:21:54.538733+0000","last_undegraded":"2024-09-17T03:21:54.538733+0000","last_fullsized":"2024-09-17T03:21:54.538733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:23.432463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213924+0000","last_change":"2024-09-17T03:21:53.549854+0000","last_active":"2024-09-17T03:21:56.213924+0000","last_peered":"2024-09-17T03:21:56.213924+0000","last_clean":"2024-09-17T03:21:56.213924+0000","last_became_active":"2024-09-17T03:21:53.549430+0000","last_became_peered":"2024-09-17T03:21:53.549430+0000","last_unstale":"2024-09-17T03:21:56.213924+0000","last_undegraded":"2024-09-17T03:21:56.213924+0000","last_fullsized":"2024-09-17T03:21:56.213924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:03.890254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.211674+0000","last_change":"2024-09-17T03:21:53.562656+0000","last_active":"2024-09-17T03:21:56.211674+0000","last_peered":"2024-09-17T03:21:56.211674+0000","last_clean":"2024-09-17T03:21:56.211674+0000","last_became_active":"2024-09-17T03:21:53.562136+0000","last_became_peered":"2024-09-17T03:21:53.562136+0000","last_unstale":"2024-09-17T03:21:56.211674+0000","last_undegraded":"2024-09-17T03:21:56.211674+0000","last_fullsized":"2024-09-17T03:21:56.211674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:13.791865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542282+0000","last_change":"2024-09-17T03:21:53.537342+0000","last_active":"2024-09-17T03:21:54.542282+0000","last_peered":"2024-09-17T03:21:54.542282+0000","last_clean":"2024-09-17T03:21:54.542282+0000","last_became_active":"2024-09-17T03:21:53.536959+0000","last_became_peered":"2024-09-17T03:21:53.536959+0000","last_unstale":"2024-09-17T03:21:54.542282+0000","last_undegraded":"2024-09-17T03:21:54.542282+0000","last_fullsized":"2024-09-17T03:21:54.542282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:42.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539789+0000","last_change":"2024-09-17T03:21:53.544212+0000","last_active":"2024-09-17T03:21:54.539789+0000","last_peered":"2024-09-17T03:21:54.539789+0000","last_clean":"2024-09-17T03:21:54.539789+0000","last_became_active":"2024-09-17T03:21:53.543735+0000","last_became_peered":"2024-09-17T03:21:53.543735+0000","last_unstale":"2024-09-17T03:21:54.539789+0000","last_undegraded":"2024-09-17T03:21:54.539789+0000","last_fullsized":"2024-09-17T03:21:54.539789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:41.198957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213404+0000","last_change":"2024-09-17T03:21:53.556648+0000","last_active":"2024-09-17T03:21:56.213404+0000","last_peered":"2024-09-17T03:21:56.213404+0000","last_clean":"2024-09-17T03:21:56.213404+0000","last_became_active":"2024-09-17T03:21:53.556328+0000","last_became_peered":"2024-09-17T03:21:53.556328+0000","last_unstale":"2024-09-17T03:21:56.213404+0000","last_undegraded":"2024-09-17T03:21:56.213404+0000","last_fullsized":"2024-09-17T03:21:56.213404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:50.900224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.567953+0000","last_change":"2024-09-17T03:21:53.562375+0000","last_active":"2024-09-17T03:21:53.567953+0000","last_peered":"2024-09-17T03:21:53.567953+0000","last_clean":"2024-09-17T03:21:53.567953+0000","last_became_active":"2024-09-17T03:21:53.562031+0000","last_became_peered":"2024-09-17T03:21:53.562031+0000","last_unstale":"2024-09-17T03:21:53.567953+0000","last_undegraded":"2024-09-17T03:21:53.567953+0000","last_fullsized":"2024-09-17T03:21:53.567953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:37.815602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209245+0000","last_change":"2024-09-17T03:21:53.549799+0000","last_active":"2024-09-17T03:21:56.209245+0000","last_peered":"2024-09-17T03:21:56.209245+0000","last_clean":"2024-09-17T03:21:56.209245+0000","last_became_active":"2024-09-17T03:21:53.549448+0000","last_became_peered":"2024-09-17T03:21:53.549448+0000","last_unstale":"2024-09-17T03:21:56.209245+0000","last_undegraded":"2024-09-17T03:21:56.209245+0000","last_fullsized":"2024-09-17T03:21:56.209245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:02.073151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538955+0000","last_change":"2024-09-17T03:21:53.546744+0000","last_active":"2024-09-17T03:21:54.538955+0000","last_peered":"2024-09-17T03:21:54.538955+0000","last_clean":"2024-09-17T03:21:54.538955+0000","last_became_active":"2024-09-17T03:21:53.541722+0000","last_became_peered":"2024-09-17T03:21:53.541722+0000","last_unstale":"2024-09-17T03:21:54.538955+0000","last_undegraded":"2024-09-17T03:21:54.538955+0000","last_fullsized":"2024-09-17T03:21:54.538955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:57.520193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.556674+0000","last_change":"2024-09-17T03:21:53.548206+0000","last_active":"2024-09-17T03:21:53.556674+0000","last_peered":"2024-09-17T03:21:53.556674+0000","last_clean":"2024-09-17T03:21:53.556674+0000","last_became_active":"2024-09-17T03:21:53.547914+0000","last_became_peered":"2024-09-17T03:21:53.547914+0000","last_unstale":"2024-09-17T03:21:53.556674+0000","last_undegraded":"2024-09-17T03:21:53.556674+0000","last_fullsized":"2024-09-17T03:21:53.556674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:58.325382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.214075+0000","last_change":"2024-09-17T03:21:53.570916+0000","last_active":"2024-09-17T03:21:56.214075+0000","last_peered":"2024-09-17T03:21:56.214075+0000","last_clean":"2024-09-17T03:21:56.214075+0000","last_became_active":"2024-09-17T03:21:53.570608+0000","last_became_peered":"2024-09-17T03:21:53.570608+0000","last_unstale":"2024-09-17T03:21:56.214075+0000","last_undegraded":"2024-09-17T03:21:56.214075+0000","last_fullsized":"2024-09-17T03:21:56.214075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:39.924918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213467+0000","last_change":"2024-09-17T03:21:53.551557+0000","last_active":"2024-09-17T03:21:56.213467+0000","last_peered":"2024-09-17T03:21:56.213467+0000","last_clean":"2024-09-17T03:21:56.213467+0000","last_became_active":"2024-09-17T03:21:53.548597+0000","last_became_peered":"2024-09-17T03:21:53.548597+0000","last_unstale":"2024-09-17T03:21:56.213467+0000","last_undegraded":"2024-09-17T03:21:56.213467+0000","last_fullsized":"2024-09-17T03:21:56.213467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:13.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539992+0000","last_change":"2024-09-17T03:21:53.538855+0000","last_active":"2024-09-17T03:21:54.539992+0000","last_peered":"2024-09-17T03:21:54.539992+0000","last_clean":"2024-09-17T03:21:54.539992+0000","last_became_active":"2024-09-17T03:21:53.538535+0000","last_became_peered":"2024-09-17T03:21:53.538535+0000","last_unstale":"2024-09-17T03:21:54.539992+0000","last_undegraded":"2024-09-17T03:21:54.539992+0000","last_fullsized":"2024-09-17T03:21:54.539992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:59.464205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539616+0000","last_change":"2024-09-17T03:21:53.538613+0000","last_active":"2024-09-17T03:21:54.539616+0000","last_peered":"2024-09-17T03:21:54.539616+0000","last_clean":"2024-09-17T03:21:54.539616+0000","last_became_active":"2024-09-17T03:21:53.538257+0000","last_became_peered":"2024-09-17T03:21:53.538257+0000","last_unstale":"2024-09-17T03:21:54.539616+0000","last_undegraded":"2024-09-17T03:21:54.539616+0000","last_fullsized":"2024-09-17T03:21:54.539616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:05.207778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209401+0000","last_change":"2024-09-17T03:21:53.559462+0000","last_active":"2024-09-17T03:21:56.209401+0000","last_peered":"2024-09-17T03:21:56.209401+0000","last_clean":"2024-09-17T03:21:56.209401+0000","last_became_active":"2024-09-17T03:21:53.559104+0000","last_became_peered":"2024-09-17T03:21:53.559104+0000","last_unstale":"2024-09-17T03:21:56.209401+0000","last_undegraded":"2024-09-17T03:21:56.209401+0000","last_fullsized":"2024-09-17T03:21:56.209401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:53.263133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541784+0000","last_change":"2024-09-17T03:21:53.554251+0000","last_active":"2024-09-17T03:21:54.541784+0000","last_peered":"2024-09-17T03:21:54.541784+0000","last_clean":"2024-09-17T03:21:54.541784+0000","last_became_active":"2024-09-17T03:21:53.553896+0000","last_became_peered":"2024-09-17T03:21:53.553896+0000","last_unstale":"2024-09-17T03:21:54.541784+0000","last_undegraded":"2024-09-17T03:21:54.541784+0000","last_fullsized":"2024-09-17T03:21:54.541784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:44.459862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541967+0000","last_change":"2024-09-17T03:21:53.541374+0000","last_active":"2024-09-17T03:21:54.541967+0000","last_peered":"2024-09-17T03:21:54.541967+0000","last_clean":"2024-09-17T03:21:54.541967+0000","last_became_active":"2024-09-17T03:21:53.541026+0000","last_became_peered":"2024-09-17T03:21:53.541026+0000","last_unstale":"2024-09-17T03:21:54.541967+0000","last_undegraded":"2024-09-17T03:21:54.541967+0000","last_fullsized":"2024-09-17T03:21:54.541967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:21.499182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530523+0000","last_change":"2024-09-17T03:21:53.550640+0000","last_active":"2024-09-17T03:21:55.530523+0000","last_peered":"2024-09-17T03:21:55.530523+0000","last_clean":"2024-09-17T03:21:55.530523+0000","last_became_active":"2024-09-17T03:21:53.550298+0000","last_became_peered":"2024-09-17T03:21:53.550298+0000","last_unstale":"2024-09-17T03:21:55.530523+0000","last_undegraded":"2024-09-17T03:21:55.530523+0000","last_fullsized":"2024-09-17T03:21:55.530523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:36.269301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558824+0000","last_change":"2024-09-17T03:21:53.556064+0000","last_active":"2024-09-17T03:21:53.558824+0000","last_peered":"2024-09-17T03:21:53.558824+0000","last_clean":"2024-09-17T03:21:53.558824+0000","last_became_active":"2024-09-17T03:21:53.555733+0000","last_became_peered":"2024-09-17T03:21:53.555733+0000","last_unstale":"2024-09-17T03:21:53.558824+0000","last_undegraded":"2024-09-17T03:21:53.558824+0000","last_fullsized":"2024-09-17T03:21:53.558824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:04:44.401616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540048+0000","last_change":"2024-09-17T03:21:53.546109+0000","last_active":"2024-09-17T03:21:54.540048+0000","last_peered":"2024-09-17T03:21:54.540048+0000","last_clean":"2024-09-17T03:21:54.540048+0000","last_became_active":"2024-09-17T03:21:53.545503+0000","last_became_peered":"2024-09-17T03:21:53.545503+0000","last_unstale":"2024-09-17T03:21:54.540048+0000","last_undegraded":"2024-09-17T03:21:54.540048+0000","last_fullsized":"2024-09-17T03:21:54.540048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:02.167432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540154+0000","last_change":"2024-09-17T03:21:53.560492+0000","last_active":"2024-09-17T03:21:54.540154+0000","last_peered":"2024-09-17T03:21:54.540154+0000","last_clean":"2024-09-17T03:21:54.540154+0000","last_became_active":"2024-09-17T03:21:53.560169+0000","last_became_peered":"2024-09-17T03:21:53.560169+0000","last_unstale":"2024-09-17T03:21:54.540154+0000","last_undegraded":"2024-09-17T03:21:54.540154+0000","last_fullsized":"2024-09-17T03:21:54.540154+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:08.917749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540248+0000","last_change":"2024-09-17T03:21:53.544837+0000","last_active":"2024-09-17T03:21:54.540248+0000","last_peered":"2024-09-17T03:21:54.540248+0000","last_clean":"2024-09-17T03:21:54.540248+0000","last_became_active":"2024-09-17T03:21:53.542870+0000","last_became_peered":"2024-09-17T03:21:53.542870+0000","last_unstale":"2024-09-17T03:21:54.540248+0000","last_undegraded":"2024-09-17T03:21:54.540248+0000","last_fullsized":"2024-09-17T03:21:54.540248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:39.019276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558834+0000","last_change":"2024-09-17T03:21:53.555144+0000","last_active":"2024-09-17T03:21:53.558834+0000","last_peered":"2024-09-17T03:21:53.558834+0000","last_clean":"2024-09-17T03:21:53.558834+0000","last_became_active":"2024-09-17T03:21:53.554782+0000","last_became_peered":"2024-09-17T03:21:53.554782+0000","last_unstale":"2024-09-17T03:21:53.558834+0000","last_undegraded":"2024-09-17T03:21:53.558834+0000","last_fullsized":"2024-09-17T03:21:53.558834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:20.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541769+0000","last_change":"2024-09-17T03:21:53.539013+0000","last_active":"2024-09-17T03:21:54.541769+0000","last_peered":"2024-09-17T03:21:54.541769+0000","last_clean":"2024-09-17T03:21:54.541769+0000","last_became_active":"2024-09-17T03:21:53.538702+0000","last_became_peered":"2024-09-17T03:21:53.538702+0000","last_unstale":"2024-09-17T03:21:54.541769+0000","last_undegraded":"2024-09-17T03:21:54.541769+0000","last_fullsized":"2024-09-17T03:21:54.541769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:31.657085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.211707+0000","last_change":"2024-09-17T03:21:53.548816+0000","last_active":"2024-09-17T03:21:56.211707+0000","last_peered":"2024-09-17T03:21:56.211707+0000","last_clean":"2024-09-17T03:21:56.211707+0000","last_became_active":"2024-09-17T03:21:53.548450+0000","last_became_peered":"2024-09-17T03:21:53.548450+0000","last_unstale":"2024-09-17T03:21:56.211707+0000","last_undegraded":"2024-09-17T03:21:56.211707+0000","last_fullsized":"2024-09-17T03:21:56.211707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:48.840495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.212802+0000","last_change":"2024-09-17T03:21:53.542885+0000","last_active":"2024-09-17T03:21:56.212802+0000","last_peered":"2024-09-17T03:21:56.212802+0000","last_clean":"2024-09-17T03:21:56.212802+0000","last_became_active":"2024-09-17T03:21:53.542545+0000","last_became_peered":"2024-09-17T03:21:53.542545+0000","last_unstale":"2024-09-17T03:21:56.212802+0000","last_undegraded":"2024-09-17T03:21:56.212802+0000","last_fullsized":"2024-09-17T03:21:56.212802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:11.576032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213921+0000","last_change":"2024-09-17T03:21:53.551044+0000","last_active":"2024-09-17T03:21:56.213921+0000","last_peered":"2024-09-17T03:21:56.213921+0000","last_clean":"2024-09-17T03:21:56.213921+0000","last_became_active":"2024-09-17T03:21:53.549024+0000","last_became_peered":"2024-09-17T03:21:53.549024+0000","last_unstale":"2024-09-17T03:21:56.213921+0000","last_undegraded":"2024-09-17T03:21:56.213921+0000","last_fullsized":"2024-09-17T03:21:56.213921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:04.057797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.531009+0000","last_change":"2024-09-17T03:21:53.550799+0000","last_active":"2024-09-17T03:21:55.531009+0000","last_peered":"2024-09-17T03:21:55.531009+0000","last_clean":"2024-09-17T03:21:55.531009+0000","last_became_active":"2024-09-17T03:21:53.550476+0000","last_became_peered":"2024-09-17T03:21:53.550476+0000","last_unstale":"2024-09-17T03:21:55.531009+0000","last_undegraded":"2024-09-17T03:21:55.531009+0000","last_fullsized":"2024-09-17T03:21:55.531009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:44.459862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213607+0000","last_change":"2024-09-17T03:21:52.552331+0000","last_active":"2024-09-17T03:21:56.213607+0000","last_peered":"2024-09-17T03:21:56.213607+0000","last_clean":"2024-09-17T03:21:56.213607+0000","last_became_active":"2024-09-17T03:21:52.551956+0000","last_became_peered":"2024-09-17T03:21:52.551956+0000","last_unstale":"2024-09-17T03:21:56.213607+0000","last_undegraded":"2024-09-17T03:21:56.213607+0000","last_fullsized":"2024-09-17T03:21:56.213607+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:52.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213515+0000","last_change":"2024-09-17T03:21:53.547810+0000","last_active":"2024-09-17T03:21:56.213515+0000","last_peered":"2024-09-17T03:21:56.213515+0000","last_clean":"2024-09-17T03:21:56.213515+0000","last_became_active":"2024-09-17T03:21:53.547252+0000","last_became_peered":"2024-09-17T03:21:53.547252+0000","last_unstale":"2024-09-17T03:21:56.213515+0000","last_undegraded":"2024-09-17T03:21:56.213515+0000","last_fullsized":"2024-09-17T03:21:56.213515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:46.216277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530873+0000","last_change":"2024-09-17T03:21:52.534295+0000","last_active":"2024-09-17T03:21:55.530873+0000","last_peered":"2024-09-17T03:21:55.530873+0000","last_clean":"2024-09-17T03:21:55.530873+0000","last_became_active":"2024-09-17T03:21:52.533866+0000","last_became_peered":"2024-09-17T03:21:52.533866+0000","last_unstale":"2024-09-17T03:21:55.530873+0000","last_undegraded":"2024-09-17T03:21:55.530873+0000","last_fullsized":"2024-09-17T03:21:55.530873+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:45.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541946+0000","last_change":"2024-09-17T03:21:53.543410+0000","last_active":"2024-09-17T03:21:54.541946+0000","last_peered":"2024-09-17T03:21:54.541946+0000","last_clean":"2024-09-17T03:21:54.541946+0000","last_became_active":"2024-09-17T03:21:53.541175+0000","last_became_peered":"2024-09-17T03:21:53.541175+0000","last_unstale":"2024-09-17T03:21:54.541946+0000","last_undegraded":"2024-09-17T03:21:54.541946+0000","last_fullsized":"2024-09-17T03:21:54.541946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:48.080302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539638+0000","last_change":"2024-09-17T03:21:52.547470+0000","last_active":"2024-09-17T03:21:54.539638+0000","last_peered":"2024-09-17T03:21:54.539638+0000","last_clean":"2024-09-17T03:21:54.539638+0000","last_became_active":"2024-09-17T03:21:52.547238+0000","last_became_peered":"2024-09-17T03:21:52.547238+0000","last_unstale":"2024-09-17T03:21:54.539638+0000","last_undegraded":"2024-09-17T03:21:54.539638+0000","last_fullsized":"2024-09-17T03:21:54.539638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:35.644290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539437+0000","last_change":"2024-09-17T03:21:53.546443+0000","last_active":"2024-09-17T03:21:54.539437+0000","last_peered":"2024-09-17T03:21:54.539437+0000","last_clean":"2024-09-17T03:21:54.539437+0000","last_became_active":"2024-09-17T03:21:53.542497+0000","last_became_peered":"2024-09-17T03:21:53.542497+0000","last_unstale":"2024-09-17T03:21:54.539437+0000","last_undegraded":"2024-09-17T03:21:54.539437+0000","last_fullsized":"2024-09-17T03:21:54.539437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:07.512532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209223+0000","last_change":"2024-09-17T03:21:52.543539+0000","last_active":"2024-09-17T03:21:56.209223+0000","last_peered":"2024-09-17T03:21:56.209223+0000","last_clean":"2024-09-17T03:21:56.209223+0000","last_became_active":"2024-09-17T03:21:52.543201+0000","last_became_peered":"2024-09-17T03:21:52.543201+0000","last_unstale":"2024-09-17T03:21:56.209223+0000","last_undegraded":"2024-09-17T03:21:56.209223+0000","last_fullsized":"2024-09-17T03:21:56.209223+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:03.441922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213852+0000","last_change":"2024-09-17T03:21:53.549791+0000","last_active":"2024-09-17T03:21:56.213852+0000","last_peered":"2024-09-17T03:21:56.213852+0000","last_clean":"2024-09-17T03:21:56.213852+0000","last_became_active":"2024-09-17T03:21:53.549212+0000","last_became_peered":"2024-09-17T03:21:53.549212+0000","last_unstale":"2024-09-17T03:21:56.213852+0000","last_undegraded":"2024-09-17T03:21:56.213852+0000","last_fullsized":"2024-09-17T03:21:56.213852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:58.854209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541866+0000","last_change":"2024-09-17T03:21:52.539607+0000","last_active":"2024-09-17T03:21:54.541866+0000","last_peered":"2024-09-17T03:21:54.541866+0000","last_clean":"2024-09-17T03:21:54.541866+0000","last_became_active":"2024-09-17T03:21:52.539274+0000","last_became_peered":"2024-09-17T03:21:52.539274+0000","last_unstale":"2024-09-17T03:21:54.541866+0000","last_undegraded":"2024-09-17T03:21:54.541866+0000","last_fullsized":"2024-09-17T03:21:54.541866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:04.766587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558876+0000","last_change":"2024-09-17T03:21:53.555218+0000","last_active":"2024-09-17T03:21:53.558876+0000","last_peered":"2024-09-17T03:21:53.558876+0000","last_clean":"2024-09-17T03:21:53.558876+0000","last_became_active":"2024-09-17T03:21:53.554811+0000","last_became_peered":"2024-09-17T03:21:53.554811+0000","last_unstale":"2024-09-17T03:21:53.558876+0000","last_undegraded":"2024-09-17T03:21:53.558876+0000","last_fullsized":"2024-09-17T03:21:53.558876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:26.191852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541484+0000","last_change":"2024-09-17T03:21:52.539742+0000","last_active":"2024-09-17T03:21:54.541484+0000","last_peered":"2024-09-17T03:21:54.541484+0000","last_clean":"2024-09-17T03:21:54.541484+0000","last_became_active":"2024-09-17T03:21:52.539428+0000","last_became_peered":"2024-09-17T03:21:52.539428+0000","last_unstale":"2024-09-17T03:21:54.541484+0000","last_undegraded":"2024-09-17T03:21:54.541484+0000","last_fullsized":"2024-09-17T03:21:54.541484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:44.627845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541404+0000","last_change":"2024-09-17T03:21:53.544203+0000","last_active":"2024-09-17T03:21:54.541404+0000","last_peered":"2024-09-17T03:21:54.541404+0000","last_clean":"2024-09-17T03:21:54.541404+0000","last_became_active":"2024-09-17T03:21:53.542205+0000","last_became_peered":"2024-09-17T03:21:53.542205+0000","last_unstale":"2024-09-17T03:21:54.541404+0000","last_undegraded":"2024-09-17T03:21:54.541404+0000","last_fullsized":"2024-09-17T03:21:54.541404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:18.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538436+0000","last_change":"2024-09-17T03:21:52.531559+0000","last_active":"2024-09-17T03:21:54.538436+0000","last_peered":"2024-09-17T03:21:54.538436+0000","last_clean":"2024-09-17T03:21:54.538436+0000","last_became_active":"2024-09-17T03:21:52.531235+0000","last_became_peered":"2024-09-17T03:21:52.531235+0000","last_unstale":"2024-09-17T03:21:54.538436+0000","last_undegraded":"2024-09-17T03:21:54.538436+0000","last_fullsized":"2024-09-17T03:21:54.538436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:02.669724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.537876+0000","last_change":"2024-09-17T03:21:53.547516+0000","last_active":"2024-09-17T03:21:54.537876+0000","last_peered":"2024-09-17T03:21:54.537876+0000","last_clean":"2024-09-17T03:21:54.537876+0000","last_became_active":"2024-09-17T03:21:53.545852+0000","last_became_peered":"2024-09-17T03:21:53.545852+0000","last_unstale":"2024-09-17T03:21:54.537876+0000","last_undegraded":"2024-09-17T03:21:54.537876+0000","last_fullsized":"2024-09-17T03:21:54.537876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:50.047009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541227+0000","last_change":"2024-09-17T03:21:52.545742+0000","last_active":"2024-09-17T03:21:54.541227+0000","last_peered":"2024-09-17T03:21:54.541227+0000","last_clean":"2024-09-17T03:21:54.541227+0000","last_became_active":"2024-09-17T03:21:52.542634+0000","last_became_peered":"2024-09-17T03:21:52.542634+0000","last_unstale":"2024-09-17T03:21:54.541227+0000","last_undegraded":"2024-09-17T03:21:54.541227+0000","last_fullsized":"2024-09-17T03:21:54.541227+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:54.390671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.567928+0000","last_change":"2024-09-17T03:21:53.562382+0000","last_active":"2024-09-17T03:21:53.567928+0000","last_peered":"2024-09-17T03:21:53.567928+0000","last_clean":"2024-09-17T03:21:53.567928+0000","last_became_active":"2024-09-17T03:21:53.562005+0000","last_became_peered":"2024-09-17T03:21:53.562005+0000","last_unstale":"2024-09-17T03:21:53.567928+0000","last_undegraded":"2024-09-17T03:21:53.567928+0000","last_fullsized":"2024-09-17T03:21:53.567928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:51.526057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540516+0000","last_change":"2024-09-17T03:21:52.538465+0000","last_active":"2024-09-17T03:21:54.540516+0000","last_peered":"2024-09-17T03:21:54.540516+0000","last_clean":"2024-09-17T03:21:54.540516+0000","last_became_active":"2024-09-17T03:21:52.538159+0000","last_became_peered":"2024-09-17T03:21:52.538159+0000","last_unstale":"2024-09-17T03:21:54.540516+0000","last_undegraded":"2024-09-17T03:21:54.540516+0000","last_fullsized":"2024-09-17T03:21:54.540516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:26.922910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540448+0000","last_change":"2024-09-17T03:21:53.540179+0000","last_active":"2024-09-17T03:21:54.540448+0000","last_peered":"2024-09-17T03:21:54.540448+0000","last_clean":"2024-09-17T03:21:54.540448+0000","last_became_active":"2024-09-17T03:21:53.539884+0000","last_became_peered":"2024-09-17T03:21:53.539884+0000","last_unstale":"2024-09-17T03:21:54.540448+0000","last_undegraded":"2024-09-17T03:21:54.540448+0000","last_fullsized":"2024-09-17T03:21:54.540448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:32.162775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.531003+0000","last_change":"2024-09-17T03:21:52.537746+0000","last_active":"2024-09-17T03:21:55.531003+0000","last_peered":"2024-09-17T03:21:55.531003+0000","last_clean":"2024-09-17T03:21:55.531003+0000","last_became_active":"2024-09-17T03:21:52.537418+0000","last_became_peered":"2024-09-17T03:21:52.537418+0000","last_unstale":"2024-09-17T03:21:55.531003+0000","last_undegraded":"2024-09-17T03:21:55.531003+0000","last_fullsized":"2024-09-17T03:21:55.531003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:50.592582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539590+0000","last_change":"2024-09-17T03:21:53.561840+0000","last_active":"2024-09-17T03:21:54.539590+0000","last_peered":"2024-09-17T03:21:54.539590+0000","last_clean":"2024-09-17T03:21:54.539590+0000","last_became_active":"2024-09-17T03:21:53.559774+0000","last_became_peered":"2024-09-17T03:21:53.559774+0000","last_unstale":"2024-09-17T03:21:54.539590+0000","last_undegraded":"2024-09-17T03:21:54.539590+0000","last_fullsized":"2024-09-17T03:21:54.539590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:24.739520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558550+0000","last_change":"2024-09-17T03:21:52.542996+0000","last_active":"2024-09-17T03:21:53.558550+0000","last_peered":"2024-09-17T03:21:53.558550+0000","last_clean":"2024-09-17T03:21:53.558550+0000","last_became_active":"2024-09-17T03:21:52.542635+0000","last_became_peered":"2024-09-17T03:21:52.542635+0000","last_unstale":"2024-09-17T03:21:53.558550+0000","last_undegraded":"2024-09-17T03:21:53.558550+0000","last_fullsized":"2024-09-17T03:21:53.558550+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:33.173785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.556628+0000","last_change":"2024-09-17T03:21:53.547534+0000","last_active":"2024-09-17T03:21:53.556628+0000","last_peered":"2024-09-17T03:21:53.556628+0000","last_clean":"2024-09-17T03:21:53.556628+0000","last_became_active":"2024-09-17T03:21:53.545836+0000","last_became_peered":"2024-09-17T03:21:53.545836+0000","last_unstale":"2024-09-17T03:21:53.556628+0000","last_undegraded":"2024-09-17T03:21:53.556628+0000","last_fullsized":"2024-09-17T03:21:53.556628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:10.523386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539498+0000","last_change":"2024-09-17T03:21:52.546507+0000","last_active":"2024-09-17T03:21:54.539498+0000","last_peered":"2024-09-17T03:21:54.539498+0000","last_clean":"2024-09-17T03:21:54.539498+0000","last_became_active":"2024-09-17T03:21:52.546233+0000","last_became_peered":"2024-09-17T03:21:52.546233+0000","last_unstale":"2024-09-17T03:21:54.539498+0000","last_undegraded":"2024-09-17T03:21:54.539498+0000","last_fullsized":"2024-09-17T03:21:54.539498+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:57.348211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538085+0000","last_change":"2024-09-17T03:21:53.547736+0000","last_active":"2024-09-17T03:21:54.538085+0000","last_peered":"2024-09-17T03:21:54.538085+0000","last_clean":"2024-09-17T03:21:54.538085+0000","last_became_active":"2024-09-17T03:21:53.546510+0000","last_became_peered":"2024-09-17T03:21:53.546510+0000","last_unstale":"2024-09-17T03:21:54.538085+0000","last_undegraded":"2024-09-17T03:21:54.538085+0000","last_fullsized":"2024-09-17T03:21:54.538085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:35.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542035+0000","last_change":"2024-09-17T03:21:52.542313+0000","last_active":"2024-09-17T03:21:54.542035+0000","last_peered":"2024-09-17T03:21:54.542035+0000","last_clean":"2024-09-17T03:21:54.542035+0000","last_became_active":"2024-09-17T03:21:52.541963+0000","last_became_peered":"2024-09-17T03:21:52.541963+0000","last_unstale":"2024-09-17T03:21:54.542035+0000","last_undegraded":"2024-09-17T03:21:54.542035+0000","last_fullsized":"2024-09-17T03:21:54.542035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:52.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539241+0000","last_change":"2024-09-17T03:21:53.555940+0000","last_active":"2024-09-17T03:21:54.539241+0000","last_peered":"2024-09-17T03:21:54.539241+0000","last_clean":"2024-09-17T03:21:54.539241+0000","last_became_active":"2024-09-17T03:21:53.555423+0000","last_became_peered":"2024-09-17T03:21:53.555423+0000","last_unstale":"2024-09-17T03:21:54.539241+0000","last_undegraded":"2024-09-17T03:21:54.539241+0000","last_fullsized":"2024-09-17T03:21:54.539241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:07:20.943274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538261+0000","last_change":"2024-09-17T03:21:52.538771+0000","last_active":"2024-09-17T03:21:54.538261+0000","last_peered":"2024-09-17T03:21:54.538261+0000","last_clean":"2024-09-17T03:21:54.538261+0000","last_became_active":"2024-09-17T03:21:52.537006+0000","last_became_peered":"2024-09-17T03:21:52.537006+0000","last_unstale":"2024-09-17T03:21:54.538261+0000","last_undegraded":"2024-09-17T03:21:54.538261+0000","last_fullsized":"2024-09-17T03:21:54.538261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:27:07.695072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213894+0000","last_change":"2024-09-17T03:21:53.550422+0000","last_active":"2024-09-17T03:21:56.213894+0000","last_peered":"2024-09-17T03:21:56.213894+0000","last_clean":"2024-09-17T03:21:56.213894+0000","last_became_active":"2024-09-17T03:21:53.549519+0000","last_became_peered":"2024-09-17T03:21:53.549519+0000","last_unstale":"2024-09-17T03:21:56.213894+0000","last_undegraded":"2024-09-17T03:21:56.213894+0000","last_fullsized":"2024-09-17T03:21:56.213894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:13.254268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538183+0000","last_change":"2024-09-17T03:21:52.539288+0000","last_active":"2024-09-17T03:21:54.538183+0000","last_peered":"2024-09-17T03:21:54.538183+0000","last_clean":"2024-09-17T03:21:54.538183+0000","last_became_active":"2024-09-17T03:21:52.538955+0000","last_became_peered":"2024-09-17T03:21:52.538955+0000","last_unstale":"2024-09-17T03:21:54.538183+0000","last_undegraded":"2024-09-17T03:21:54.538183+0000","last_fullsized":"2024-09-17T03:21:54.538183+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:40.503141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213086+0000","last_change":"2024-09-17T03:21:53.543540+0000","last_active":"2024-09-17T03:21:56.213086+0000","last_peered":"2024-09-17T03:21:56.213086+0000","last_clean":"2024-09-17T03:21:56.213086+0000","last_became_active":"2024-09-17T03:21:53.543197+0000","last_became_peered":"2024-09-17T03:21:53.543197+0000","last_unstale":"2024-09-17T03:21:56.213086+0000","last_undegraded":"2024-09-17T03:21:56.213086+0000","last_fullsized":"2024-09-17T03:21:56.213086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:28:43.522796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538050+0000","last_change":"2024-09-17T03:21:52.547105+0000","last_active":"2024-09-17T03:21:54.538050+0000","last_peered":"2024-09-17T03:21:54.538050+0000","last_clean":"2024-09-17T03:21:54.538050+0000","last_became_active":"2024-09-17T03:21:52.546807+0000","last_became_peered":"2024-09-17T03:21:52.546807+0000","last_unstale":"2024-09-17T03:21:54.538050+0000","last_undegraded":"2024-09-17T03:21:54.538050+0000","last_fullsized":"2024-09-17T03:21:54.538050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:34.615668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542203+0000","last_change":"2024-09-17T03:21:53.554671+0000","last_active":"2024-09-17T03:21:54.542203+0000","last_peered":"2024-09-17T03:21:54.542203+0000","last_clean":"2024-09-17T03:21:54.542203+0000","last_became_active":"2024-09-17T03:21:53.554360+0000","last_became_peered":"2024-09-17T03:21:53.554360+0000","last_unstale":"2024-09-17T03:21:54.542203+0000","last_undegraded":"2024-09-17T03:21:54.542203+0000","last_fullsized":"2024-09-17T03:21:54.542203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:30.591964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540293+0000","last_change":"2024-09-17T03:21:52.545217+0000","last_active":"2024-09-17T03:21:54.540293+0000","last_peered":"2024-09-17T03:21:54.540293+0000","last_clean":"2024-09-17T03:21:54.540293+0000","last_became_active":"2024-09-17T03:21:52.544910+0000","last_became_peered":"2024-09-17T03:21:52.544910+0000","last_unstale":"2024-09-17T03:21:54.540293+0000","last_undegraded":"2024-09-17T03:21:54.540293+0000","last_fullsized":"2024-09-17T03:21:54.540293+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.581307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558486+0000","last_change":"2024-09-17T03:21:53.554146+0000","last_active":"2024-09-17T03:21:53.558486+0000","last_peered":"2024-09-17T03:21:53.558486+0000","last_clean":"2024-09-17T03:21:53.558486+0000","last_became_active":"2024-09-17T03:21:53.553848+0000","last_became_peered":"2024-09-17T03:21:53.553848+0000","last_unstale":"2024-09-17T03:21:53.558486+0000","last_undegraded":"2024-09-17T03:21:53.558486+0000","last_fullsized":"2024-09-17T03:21:53.558486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:19.698487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542070+0000","last_change":"2024-09-17T03:21:52.545331+0000","last_active":"2024-09-17T03:21:54.542070+0000","last_peered":"2024-09-17T03:21:54.542070+0000","last_clean":"2024-09-17T03:21:54.542070+0000","last_became_active":"2024-09-17T03:21:52.543054+0000","last_became_peered":"2024-09-17T03:21:52.543054+0000","last_unstale":"2024-09-17T03:21:54.542070+0000","last_undegraded":"2024-09-17T03:21:54.542070+0000","last_fullsized":"2024-09-17T03:21:54.542070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:23.472290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538817+0000","last_change":"2024-09-17T03:21:53.536062+0000","last_active":"2024-09-17T03:21:54.538817+0000","last_peered":"2024-09-17T03:21:54.538817+0000","last_clean":"2024-09-17T03:21:54.538817+0000","last_became_active":"2024-09-17T03:21:53.535655+0000","last_became_peered":"2024-09-17T03:21:53.535655+0000","last_unstale":"2024-09-17T03:21:54.538817+0000","last_undegraded":"2024-09-17T03:21:54.538817+0000","last_fullsized":"2024-09-17T03:21:54.538817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:42.491863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542230+0000","last_change":"2024-09-17T03:21:52.540408+0000","last_active":"2024-09-17T03:21:54.542230+0000","last_peered":"2024-09-17T03:21:54.542230+0000","last_clean":"2024-09-17T03:21:54.542230+0000","last_became_active":"2024-09-17T03:21:52.540062+0000","last_became_peered":"2024-09-17T03:21:52.540062+0000","last_unstale":"2024-09-17T03:21:54.542230+0000","last_undegraded":"2024-09-17T03:21:54.542230+0000","last_fullsized":"2024-09-17T03:21:54.542230+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:45.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209277+0000","last_change":"2024-09-17T03:21:53.554048+0000","last_active":"2024-09-17T03:21:56.209277+0000","last_peered":"2024-09-17T03:21:56.209277+0000","last_clean":"2024-09-17T03:21:56.209277+0000","last_became_active":"2024-09-17T03:21:53.553745+0000","last_became_peered":"2024-09-17T03:21:53.553745+0000","last_unstale":"2024-09-17T03:21:56.209277+0000","last_undegraded":"2024-09-17T03:21:56.209277+0000","last_fullsized":"2024-09-17T03:21:56.209277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:05.771794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541820+0000","last_change":"2024-09-17T03:21:52.541499+0000","last_active":"2024-09-17T03:21:54.541820+0000","last_peered":"2024-09-17T03:21:54.541820+0000","last_clean":"2024-09-17T03:21:54.541820+0000","last_became_active":"2024-09-17T03:21:52.539204+0000","last_became_peered":"2024-09-17T03:21:52.539204+0000","last_unstale":"2024-09-17T03:21:54.541820+0000","last_undegraded":"2024-09-17T03:21:54.541820+0000","last_fullsized":"2024-09-17T03:21:54.541820+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:40.905763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209508+0000","last_change":"2024-09-17T03:21:53.552284+0000","last_active":"2024-09-17T03:21:56.209508+0000","last_peered":"2024-09-17T03:21:56.209508+0000","last_clean":"2024-09-17T03:21:56.209508+0000","last_became_active":"2024-09-17T03:21:53.551890+0000","last_became_peered":"2024-09-17T03:21:53.551890+0000","last_unstale":"2024-09-17T03:21:56.209508+0000","last_undegraded":"2024-09-17T03:21:56.209508+0000","last_fullsized":"2024-09-17T03:21:56.209508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:05.203911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530677+0000","last_change":"2024-09-17T03:21:52.528899+0000","last_active":"2024-09-17T03:21:55.530677+0000","last_peered":"2024-09-17T03:21:55.530677+0000","last_clean":"2024-09-17T03:21:55.530677+0000","last_became_active":"2024-09-17T03:21:52.528518+0000","last_became_peered":"2024-09-17T03:21:52.528518+0000","last_unstale":"2024-09-17T03:21:55.530677+0000","last_undegraded":"2024-09-17T03:21:55.530677+0000","last_fullsized":"2024-09-17T03:21:55.530677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:18.545893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209503+0000","last_change":"2024-09-17T03:21:53.551589+0000","last_active":"2024-09-17T03:21:56.209503+0000","last_peered":"2024-09-17T03:21:56.209503+0000","last_clean":"2024-09-17T03:21:56.209503+0000","last_became_active":"2024-09-17T03:21:53.551206+0000","last_became_peered":"2024-09-17T03:21:53.551206+0000","last_unstale":"2024-09-17T03:21:56.209503+0000","last_undegraded":"2024-09-17T03:21:56.209503+0000","last_fullsized":"2024-09-17T03:21:56.209503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:19.551101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213562+0000","last_change":"2024-09-17T03:21:52.546295+0000","last_active":"2024-09-17T03:21:56.213562+0000","last_peered":"2024-09-17T03:21:56.213562+0000","last_clean":"2024-09-17T03:21:56.213562+0000","last_became_active":"2024-09-17T03:21:52.545969+0000","last_became_peered":"2024-09-17T03:21:52.545969+0000","last_unstale":"2024-09-17T03:21:56.213562+0000","last_undegraded":"2024-09-17T03:21:56.213562+0000","last_fullsized":"2024-09-17T03:21:56.213562+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:34.264773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209260+0000","last_change":"2024-09-17T03:21:53.547274+0000","last_active":"2024-09-17T03:21:56.209260+0000","last_peered":"2024-09-17T03:21:56.209260+0000","last_clean":"2024-09-17T03:21:56.209260+0000","last_became_active":"2024-09-17T03:21:53.546964+0000","last_became_peered":"2024-09-17T03:21:53.546964+0000","last_unstale":"2024-09-17T03:21:56.209260+0000","last_undegraded":"2024-09-17T03:21:56.209260+0000","last_fullsized":"2024-09-17T03:21:56.209260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.477497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541184+0000","last_change":"2024-09-17T03:21:52.531752+0000","last_active":"2024-09-17T03:21:54.541184+0000","last_peered":"2024-09-17T03:21:54.541184+0000","last_clean":"2024-09-17T03:21:54.541184+0000","last_became_active":"2024-09-17T03:21:52.530910+0000","last_became_peered":"2024-09-17T03:21:52.530910+0000","last_unstale":"2024-09-17T03:21:54.541184+0000","last_undegraded":"2024-09-17T03:21:54.541184+0000","last_fullsized":"2024-09-17T03:21:54.541184+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:34.264773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.212339+0000","last_change":"2024-09-17T03:21:53.549723+0000","last_active":"2024-09-17T03:21:56.212339+0000","last_peered":"2024-09-17T03:21:56.212339+0000","last_clean":"2024-09-17T03:21:56.212339+0000","last_became_active":"2024-09-17T03:21:53.549084+0000","last_became_peered":"2024-09-17T03:21:53.549084+0000","last_unstale":"2024-09-17T03:21:56.212339+0000","last_undegraded":"2024-09-17T03:21:56.212339+0000","last_fullsized":"2024-09-17T03:21:56.212339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:52.073779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.535249+0000","last_change":"2024-09-17T03:21:52.541036+0000","last_active":"2024-09-17T03:21:53.535249+0000","last_peered":"2024-09-17T03:21:53.535249+0000","last_clean":"2024-09-17T03:21:53.535249+0000","last_became_active":"2024-09-17T03:21:52.540723+0000","last_became_peered":"2024-09-17T03:21:52.540723+0000","last_unstale":"2024-09-17T03:21:53.535249+0000","last_undegraded":"2024-09-17T03:21:53.535249+0000","last_fullsized":"2024-09-17T03:21:53.535249+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:23.185290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209324+0000","last_change":"2024-09-17T03:21:53.552767+0000","last_active":"2024-09-17T03:21:56.209324+0000","last_peered":"2024-09-17T03:21:56.209324+0000","last_clean":"2024-09-17T03:21:56.209324+0000","last_became_active":"2024-09-17T03:21:53.552517+0000","last_became_peered":"2024-09-17T03:21:53.552517+0000","last_unstale":"2024-09-17T03:21:56.209324+0000","last_undegraded":"2024-09-17T03:21:56.209324+0000","last_fullsized":"2024-09-17T03:21:56.209324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:35.269981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540386+0000","last_change":"2024-09-17T03:21:52.539788+0000","last_active":"2024-09-17T03:21:54.540386+0000","last_peered":"2024-09-17T03:21:54.540386+0000","last_clean":"2024-09-17T03:21:54.540386+0000","last_became_active":"2024-09-17T03:21:52.539332+0000","last_became_peered":"2024-09-17T03:21:52.539332+0000","last_unstale":"2024-09-17T03:21:54.540386+0000","last_undegraded":"2024-09-17T03:21:54.540386+0000","last_fullsized":"2024-09-17T03:21:54.540386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:05.159629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558455+0000","last_change":"2024-09-17T03:21:53.551517+0000","last_active":"2024-09-17T03:21:53.558455+0000","last_peered":"2024-09-17T03:21:53.558455+0000","last_clean":"2024-09-17T03:21:53.558455+0000","last_became_active":"2024-09-17T03:21:53.551149+0000","last_became_peered":"2024-09-17T03:21:53.551149+0000","last_unstale":"2024-09-17T03:21:53.558455+0000","last_undegraded":"2024-09-17T03:21:53.558455+0000","last_fullsized":"2024-09-17T03:21:53.558455+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:55.646076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.531159+0000","last_change":"2024-09-17T03:21:52.530406+0000","last_active":"2024-09-17T03:21:55.531159+0000","last_peered":"2024-09-17T03:21:55.531159+0000","last_clean":"2024-09-17T03:21:55.531159+0000","last_became_active":"2024-09-17T03:21:52.530098+0000","last_became_peered":"2024-09-17T03:21:52.530098+0000","last_unstale":"2024-09-17T03:21:55.531159+0000","last_undegraded":"2024-09-17T03:21:55.531159+0000","last_fullsized":"2024-09-17T03:21:55.531159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:53.398862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530857+0000","last_change":"2024-09-17T03:21:53.547878+0000","last_active":"2024-09-17T03:21:55.530857+0000","last_peered":"2024-09-17T03:21:55.530857+0000","last_clean":"2024-09-17T03:21:55.530857+0000","last_became_active":"2024-09-17T03:21:53.541594+0000","last_became_peered":"2024-09-17T03:21:53.541594+0000","last_unstale":"2024-09-17T03:21:55.530857+0000","last_undegraded":"2024-09-17T03:21:55.530857+0000","last_fullsized":"2024-09-17T03:21:55.530857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:18.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538835+0000","last_change":"2024-09-17T03:21:52.544291+0000","last_active":"2024-09-17T03:21:54.538835+0000","last_peered":"2024-09-17T03:21:54.538835+0000","last_clean":"2024-09-17T03:21:54.538835+0000","last_became_active":"2024-09-17T03:21:52.543771+0000","last_became_peered":"2024-09-17T03:21:52.543771+0000","last_unstale":"2024-09-17T03:21:54.538835+0000","last_undegraded":"2024-09-17T03:21:54.538835+0000","last_fullsized":"2024-09-17T03:21:54.538835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:29.096589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213397+0000","last_change":"2024-09-17T03:21:53.545540+0000","last_active":"2024-09-17T03:21:56.213397+0000","last_peered":"2024-09-17T03:21:56.213397+0000","last_clean":"2024-09-17T03:21:56.213397+0000","last_became_active":"2024-09-17T03:21:53.545211+0000","last_became_peered":"2024-09-17T03:21:53.545211+0000","last_unstale":"2024-09-17T03:21:56.213397+0000","last_undegraded":"2024-09-17T03:21:56.213397+0000","last_fullsized":"2024-09-17T03:21:56.213397+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:36.269301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530867+0000","last_change":"2024-09-17T03:21:52.535843+0000","last_active":"2024-09-17T03:21:55.530867+0000","last_peered":"2024-09-17T03:21:55.530867+0000","last_clean":"2024-09-17T03:21:55.530867+0000","last_became_active":"2024-09-17T03:21:52.535549+0000","last_became_peered":"2024-09-17T03:21:52.535549+0000","last_unstale":"2024-09-17T03:21:55.530867+0000","last_undegraded":"2024-09-17T03:21:55.530867+0000","last_fullsized":"2024-09-17T03:21:55.530867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:43.831837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530863+0000","last_change":"2024-09-17T03:21:53.546468+0000","last_active":"2024-09-17T03:21:55.530863+0000","last_peered":"2024-09-17T03:21:55.530863+0000","last_clean":"2024-09-17T03:21:55.530863+0000","last_became_active":"2024-09-17T03:21:53.545809+0000","last_became_peered":"2024-09-17T03:21:53.545809+0000","last_unstale":"2024-09-17T03:21:55.530863+0000","last_undegraded":"2024-09-17T03:21:55.530863+0000","last_fullsized":"2024-09-17T03:21:55.530863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:46.216277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538341+0000","last_change":"2024-09-17T03:21:52.533439+0000","last_active":"2024-09-17T03:21:54.538341+0000","last_peered":"2024-09-17T03:21:54.538341+0000","last_clean":"2024-09-17T03:21:54.538341+0000","last_became_active":"2024-09-17T03:21:52.533039+0000","last_became_peered":"2024-09-17T03:21:52.533039+0000","last_unstale":"2024-09-17T03:21:54.538341+0000","last_undegraded":"2024-09-17T03:21:54.538341+0000","last_fullsized":"2024-09-17T03:21:54.538341+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:58.362820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538006+0000","last_change":"2024-09-17T03:21:53.540943+0000","last_active":"2024-09-17T03:21:54.538006+0000","last_peered":"2024-09-17T03:21:54.538006+0000","last_clean":"2024-09-17T03:21:54.538006+0000","last_became_active":"2024-09-17T03:21:53.540628+0000","last_became_peered":"2024-09-17T03:21:53.540628+0000","last_unstale":"2024-09-17T03:21:54.538006+0000","last_undegraded":"2024-09-17T03:21:54.538006+0000","last_fullsized":"2024-09-17T03:21:54.538006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:43.090203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541321+0000","last_change":"2024-09-17T03:21:52.535447+0000","last_active":"2024-09-17T03:21:54.541321+0000","last_peered":"2024-09-17T03:21:54.541321+0000","last_clean":"2024-09-17T03:21:54.541321+0000","last_became_active":"2024-09-17T03:21:52.535092+0000","last_became_peered":"2024-09-17T03:21:52.535092+0000","last_unstale":"2024-09-17T03:21:54.541321+0000","last_undegraded":"2024-09-17T03:21:54.541321+0000","last_fullsized":"2024-09-17T03:21:54.541321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:18.545893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541342+0000","last_change":"2024-09-17T03:21:53.547997+0000","last_active":"2024-09-17T03:21:54.541342+0000","last_peered":"2024-09-17T03:21:54.541342+0000","last_clean":"2024-09-17T03:21:54.541342+0000","last_became_active":"2024-09-17T03:21:53.546043+0000","last_became_peered":"2024-09-17T03:21:53.546043+0000","last_unstale":"2024-09-17T03:21:54.541342+0000","last_undegraded":"2024-09-17T03:21:54.541342+0000","last_fullsized":"2024-09-17T03:21:54.541342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:46.216277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.537964+0000","last_change":"2024-09-17T03:21:52.541349+0000","last_active":"2024-09-17T03:21:54.537964+0000","last_peered":"2024-09-17T03:21:54.537964+0000","last_clean":"2024-09-17T03:21:54.537964+0000","last_became_active":"2024-09-17T03:21:52.541013+0000","last_became_peered":"2024-09-17T03:21:52.541013+0000","last_unstale":"2024-09-17T03:21:54.537964+0000","last_undegraded":"2024-09-17T03:21:54.537964+0000","last_fullsized":"2024-09-17T03:21:54.537964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:07.235240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213550+0000","last_change":"2024-09-17T03:21:53.564667+0000","last_active":"2024-09-17T03:21:56.213550+0000","last_peered":"2024-09-17T03:21:56.213550+0000","last_clean":"2024-09-17T03:21:56.213550+0000","last_became_active":"2024-09-17T03:21:53.564367+0000","last_became_peered":"2024-09-17T03:21:53.564367+0000","last_unstale":"2024-09-17T03:21:56.213550+0000","last_undegraded":"2024-09-17T03:21:56.213550+0000","last_fullsized":"2024-09-17T03:21:56.213550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:27.233349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.212888+0000","last_change":"2024-09-17T03:21:52.548119+0000","last_active":"2024-09-17T03:21:56.212888+0000","last_peered":"2024-09-17T03:21:56.212888+0000","last_clean":"2024-09-17T03:21:56.212888+0000","last_became_active":"2024-09-17T03:21:52.547596+0000","last_became_peered":"2024-09-17T03:21:52.547596+0000","last_unstale":"2024-09-17T03:21:56.212888+0000","last_undegraded":"2024-09-17T03:21:56.212888+0000","last_fullsized":"2024-09-17T03:21:56.212888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:18.545893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.214031+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:56.214031+0000","last_peered":"2024-09-17T03:21:56.214031+0000","last_clean":"2024-09-17T03:21:56.214031+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:56.214031+0000","last_undegraded":"2024-09-17T03:21:56.214031+0000","last_fullsized":"2024-09-17T03:21:56.214031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":725837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":348160,"data_stored":129695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":135483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":368640,"data_stored":137013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":135449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":725747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":348160,"data_stored":129751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":137013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:57.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:21:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 -- 172.21.15.46:0/2860657581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 msgr2=0x7fba2806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 --2- 172.21.15.46:0/2860657581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba2806f3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba2806ae60 tx=0x7fba2001cb50 comp rx=0 tx=0).stop 2024-09-17T03:21:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 -- 172.21.15.46:0/2860657581 shutdown_connections 2024-09-17T03:21:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 --2- 172.21.15.46:0/2860657581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba2806f3b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 --2- 172.21.15.46:0/2860657581 >> v2:172.21.15.99:3300/0 conn(0x7fba280744d0 0x7fba2806ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 -- 172.21.15.46:0/2860657581 >> 172.21.15.46:0/2860657581 conn(0x7fba280b2c40 msgr2=0x7fba280b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 -- 172.21.15.46:0/2860657581 shutdown_connections 2024-09-17T03:21:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 -- 172.21.15.46:0/2860657581 wait complete. 2024-09-17T03:21:57.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 Processor -- start 2024-09-17T03:21:57.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.933+0000 7fba2f600640 1 -- start start 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba28141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba2815c5d0 0x7fba281f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba2814f510 0x7fba281fa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba28168d20 con 0x7fba2806efa0 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba28168ea0 con 0x7fba2815c5d0 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fba28169020 con 0x7fba2814f510 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba28141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba28141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46216/0 (socket says 172.21.15.46:46216) 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 -- 172.21.15.46:0/3966594562 learned_addr learned my addr 172.21.15.46:0/3966594562 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2d600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba2814f510 0x7fba281fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba1fe00640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba2815c5d0 0x7fba281f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 -- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba2814f510 msgr2=0x7fba281fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba2814f510 0x7fba281fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 -- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba2815c5d0 msgr2=0x7fba281f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:57.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba2815c5d0 0x7fba281f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:57.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 -- 172.21.15.46:0/3966594562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba281f7b10 con 0x7fba2806efa0 2024-09-17T03:21:57.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2cc00640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba28141480 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fba18001ec0 tx=0x7fba1800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18017070 con 0x7fba2806efa0 2024-09-17T03:21:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba18003600 con 0x7fba2806efa0 2024-09-17T03:21:57.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18003a50 con 0x7fba2806efa0 2024-09-17T03:21:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba281f7df0 con 0x7fba2806efa0 2024-09-17T03:21:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281f83b0 con 0x7fba2806efa0 2024-09-17T03:21:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.937+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2806f3b0 con 0x7fba2806efa0 2024-09-17T03:21:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.941+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba1801a070 con 0x7fba2806efa0 2024-09-17T03:21:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.941+0000 7fba25600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba04073200 0x7fba04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.941+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fba180935d0 con 0x7fba2806efa0 2024-09-17T03:21:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.941+0000 7fba1fe00640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba04073200 0x7fba04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.941+0000 7fba1fe00640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba04073200 0x7fba04075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba100058d0 tx=0x7fba10005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:57.945+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba1805e550 con 0x7fba2806efa0 2024-09-17T03:21:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.117+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba281f4560 con 0x7fba04073200 2024-09-17T03:21:58.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.121+0000 7fba25600640 1 -- 172.21.15.46:0/3966594562 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7fba281f4560 con 0x7fba04073200 2024-09-17T03:21:58.126 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:58.128 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba04073200 msgr2=0x7fba04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba04073200 0x7fba04075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba100058d0 tx=0x7fba10005860 comp rx=0 tx=0).stop 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 msgr2=0x7fba28141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba28141480 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fba18001ec0 tx=0x7fba1800c890 comp rx=0 tx=0).stop 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 shutdown_connections 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba04073200 0x7fba04075710 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba2814f510 0x7fba281fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba2815c5d0 0x7fba281f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 --2- 172.21.15.46:0/3966594562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2806efa0 0x7fba28141480 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 >> 172.21.15.46:0/3966594562 conn(0x7fba280b2c40 msgr2=0x7fba2815d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 shutdown_connections 2024-09-17T03:21:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.125+0000 7fba2f600640 1 -- 172.21.15.46:0/3966594562 wait complete. 2024-09-17T03:21:58.148 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T03:21:57.210583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241004,"kb_used_data":3952,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733716,"statfs":{"total":773094113280,"available":772847325184,"internally_reserved":0,"allocated":4046848,"data_stored":2255988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.704118"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213251+0000","last_change":"2024-09-17T03:21:53.554223+0000","last_active":"2024-09-17T03:21:56.213251+0000","last_peered":"2024-09-17T03:21:56.213251+0000","last_clean":"2024-09-17T03:21:56.213251+0000","last_became_active":"2024-09-17T03:21:53.553879+0000","last_became_peered":"2024-09-17T03:21:53.553879+0000","last_unstale":"2024-09-17T03:21:56.213251+0000","last_undegraded":"2024-09-17T03:21:56.213251+0000","last_fullsized":"2024-09-17T03:21:56.213251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:25.038591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539783+0000","last_change":"2024-09-17T03:21:53.546731+0000","last_active":"2024-09-17T03:21:54.539783+0000","last_peered":"2024-09-17T03:21:54.539783+0000","last_clean":"2024-09-17T03:21:54.539783+0000","last_became_active":"2024-09-17T03:21:53.546359+0000","last_became_peered":"2024-09-17T03:21:53.546359+0000","last_unstale":"2024-09-17T03:21:54.539783+0000","last_undegraded":"2024-09-17T03:21:54.539783+0000","last_fullsized":"2024-09-17T03:21:54.539783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:28.689089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.556598+0000","last_change":"2024-09-17T03:21:53.547245+0000","last_active":"2024-09-17T03:21:53.556598+0000","last_peered":"2024-09-17T03:21:53.556598+0000","last_clean":"2024-09-17T03:21:53.556598+0000","last_became_active":"2024-09-17T03:21:53.545526+0000","last_became_peered":"2024-09-17T03:21:53.545526+0000","last_unstale":"2024-09-17T03:21:53.556598+0000","last_undegraded":"2024-09-17T03:21:53.556598+0000","last_fullsized":"2024-09-17T03:21:53.556598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:53.045996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538707+0000","last_change":"2024-09-17T03:21:53.540990+0000","last_active":"2024-09-17T03:21:54.538707+0000","last_peered":"2024-09-17T03:21:54.538707+0000","last_clean":"2024-09-17T03:21:54.538707+0000","last_became_active":"2024-09-17T03:21:53.540663+0000","last_became_peered":"2024-09-17T03:21:53.540663+0000","last_unstale":"2024-09-17T03:21:54.538707+0000","last_undegraded":"2024-09-17T03:21:54.538707+0000","last_fullsized":"2024-09-17T03:21:54.538707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:30.953926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209242+0000","last_change":"2024-09-17T03:21:53.559972+0000","last_active":"2024-09-17T03:21:56.209242+0000","last_peered":"2024-09-17T03:21:56.209242+0000","last_clean":"2024-09-17T03:21:56.209242+0000","last_became_active":"2024-09-17T03:21:53.559648+0000","last_became_peered":"2024-09-17T03:21:53.559648+0000","last_unstale":"2024-09-17T03:21:56.209242+0000","last_undegraded":"2024-09-17T03:21:56.209242+0000","last_fullsized":"2024-09-17T03:21:56.209242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:55.395879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538733+0000","last_change":"2024-09-17T03:21:53.564719+0000","last_active":"2024-09-17T03:21:54.538733+0000","last_peered":"2024-09-17T03:21:54.538733+0000","last_clean":"2024-09-17T03:21:54.538733+0000","last_became_active":"2024-09-17T03:21:53.564405+0000","last_became_peered":"2024-09-17T03:21:53.564405+0000","last_unstale":"2024-09-17T03:21:54.538733+0000","last_undegraded":"2024-09-17T03:21:54.538733+0000","last_fullsized":"2024-09-17T03:21:54.538733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:23.432463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213924+0000","last_change":"2024-09-17T03:21:53.549854+0000","last_active":"2024-09-17T03:21:56.213924+0000","last_peered":"2024-09-17T03:21:56.213924+0000","last_clean":"2024-09-17T03:21:56.213924+0000","last_became_active":"2024-09-17T03:21:53.549430+0000","last_became_peered":"2024-09-17T03:21:53.549430+0000","last_unstale":"2024-09-17T03:21:56.213924+0000","last_undegraded":"2024-09-17T03:21:56.213924+0000","last_fullsized":"2024-09-17T03:21:56.213924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:03.890254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.211674+0000","last_change":"2024-09-17T03:21:53.562656+0000","last_active":"2024-09-17T03:21:56.211674+0000","last_peered":"2024-09-17T03:21:56.211674+0000","last_clean":"2024-09-17T03:21:56.211674+0000","last_became_active":"2024-09-17T03:21:53.562136+0000","last_became_peered":"2024-09-17T03:21:53.562136+0000","last_unstale":"2024-09-17T03:21:56.211674+0000","last_undegraded":"2024-09-17T03:21:56.211674+0000","last_fullsized":"2024-09-17T03:21:56.211674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:13.791865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542282+0000","last_change":"2024-09-17T03:21:53.537342+0000","last_active":"2024-09-17T03:21:54.542282+0000","last_peered":"2024-09-17T03:21:54.542282+0000","last_clean":"2024-09-17T03:21:54.542282+0000","last_became_active":"2024-09-17T03:21:53.536959+0000","last_became_peered":"2024-09-17T03:21:53.536959+0000","last_unstale":"2024-09-17T03:21:54.542282+0000","last_undegraded":"2024-09-17T03:21:54.542282+0000","last_fullsized":"2024-09-17T03:21:54.542282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:42.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539789+0000","last_change":"2024-09-17T03:21:53.544212+0000","last_active":"2024-09-17T03:21:54.539789+0000","last_peered":"2024-09-17T03:21:54.539789+0000","last_clean":"2024-09-17T03:21:54.539789+0000","last_became_active":"2024-09-17T03:21:53.543735+0000","last_became_peered":"2024-09-17T03:21:53.543735+0000","last_unstale":"2024-09-17T03:21:54.539789+0000","last_undegraded":"2024-09-17T03:21:54.539789+0000","last_fullsized":"2024-09-17T03:21:54.539789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:41.198957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213404+0000","last_change":"2024-09-17T03:21:53.556648+0000","last_active":"2024-09-17T03:21:56.213404+0000","last_peered":"2024-09-17T03:21:56.213404+0000","last_clean":"2024-09-17T03:21:56.213404+0000","last_became_active":"2024-09-17T03:21:53.556328+0000","last_became_peered":"2024-09-17T03:21:53.556328+0000","last_unstale":"2024-09-17T03:21:56.213404+0000","last_undegraded":"2024-09-17T03:21:56.213404+0000","last_fullsized":"2024-09-17T03:21:56.213404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:50.900224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.567953+0000","last_change":"2024-09-17T03:21:53.562375+0000","last_active":"2024-09-17T03:21:53.567953+0000","last_peered":"2024-09-17T03:21:53.567953+0000","last_clean":"2024-09-17T03:21:53.567953+0000","last_became_active":"2024-09-17T03:21:53.562031+0000","last_became_peered":"2024-09-17T03:21:53.562031+0000","last_unstale":"2024-09-17T03:21:53.567953+0000","last_undegraded":"2024-09-17T03:21:53.567953+0000","last_fullsized":"2024-09-17T03:21:53.567953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:37.815602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209245+0000","last_change":"2024-09-17T03:21:53.549799+0000","last_active":"2024-09-17T03:21:56.209245+0000","last_peered":"2024-09-17T03:21:56.209245+0000","last_clean":"2024-09-17T03:21:56.209245+0000","last_became_active":"2024-09-17T03:21:53.549448+0000","last_became_peered":"2024-09-17T03:21:53.549448+0000","last_unstale":"2024-09-17T03:21:56.209245+0000","last_undegraded":"2024-09-17T03:21:56.209245+0000","last_fullsized":"2024-09-17T03:21:56.209245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:02.073151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538955+0000","last_change":"2024-09-17T03:21:53.546744+0000","last_active":"2024-09-17T03:21:54.538955+0000","last_peered":"2024-09-17T03:21:54.538955+0000","last_clean":"2024-09-17T03:21:54.538955+0000","last_became_active":"2024-09-17T03:21:53.541722+0000","last_became_peered":"2024-09-17T03:21:53.541722+0000","last_unstale":"2024-09-17T03:21:54.538955+0000","last_undegraded":"2024-09-17T03:21:54.538955+0000","last_fullsized":"2024-09-17T03:21:54.538955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:57.520193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.556674+0000","last_change":"2024-09-17T03:21:53.548206+0000","last_active":"2024-09-17T03:21:53.556674+0000","last_peered":"2024-09-17T03:21:53.556674+0000","last_clean":"2024-09-17T03:21:53.556674+0000","last_became_active":"2024-09-17T03:21:53.547914+0000","last_became_peered":"2024-09-17T03:21:53.547914+0000","last_unstale":"2024-09-17T03:21:53.556674+0000","last_undegraded":"2024-09-17T03:21:53.556674+0000","last_fullsized":"2024-09-17T03:21:53.556674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:58.325382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.214075+0000","last_change":"2024-09-17T03:21:53.570916+0000","last_active":"2024-09-17T03:21:56.214075+0000","last_peered":"2024-09-17T03:21:56.214075+0000","last_clean":"2024-09-17T03:21:56.214075+0000","last_became_active":"2024-09-17T03:21:53.570608+0000","last_became_peered":"2024-09-17T03:21:53.570608+0000","last_unstale":"2024-09-17T03:21:56.214075+0000","last_undegraded":"2024-09-17T03:21:56.214075+0000","last_fullsized":"2024-09-17T03:21:56.214075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:39.924918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213467+0000","last_change":"2024-09-17T03:21:53.551557+0000","last_active":"2024-09-17T03:21:56.213467+0000","last_peered":"2024-09-17T03:21:56.213467+0000","last_clean":"2024-09-17T03:21:56.213467+0000","last_became_active":"2024-09-17T03:21:53.548597+0000","last_became_peered":"2024-09-17T03:21:53.548597+0000","last_unstale":"2024-09-17T03:21:56.213467+0000","last_undegraded":"2024-09-17T03:21:56.213467+0000","last_fullsized":"2024-09-17T03:21:56.213467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:13.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539992+0000","last_change":"2024-09-17T03:21:53.538855+0000","last_active":"2024-09-17T03:21:54.539992+0000","last_peered":"2024-09-17T03:21:54.539992+0000","last_clean":"2024-09-17T03:21:54.539992+0000","last_became_active":"2024-09-17T03:21:53.538535+0000","last_became_peered":"2024-09-17T03:21:53.538535+0000","last_unstale":"2024-09-17T03:21:54.539992+0000","last_undegraded":"2024-09-17T03:21:54.539992+0000","last_fullsized":"2024-09-17T03:21:54.539992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:59.464205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539616+0000","last_change":"2024-09-17T03:21:53.538613+0000","last_active":"2024-09-17T03:21:54.539616+0000","last_peered":"2024-09-17T03:21:54.539616+0000","last_clean":"2024-09-17T03:21:54.539616+0000","last_became_active":"2024-09-17T03:21:53.538257+0000","last_became_peered":"2024-09-17T03:21:53.538257+0000","last_unstale":"2024-09-17T03:21:54.539616+0000","last_undegraded":"2024-09-17T03:21:54.539616+0000","last_fullsized":"2024-09-17T03:21:54.539616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:05.207778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209401+0000","last_change":"2024-09-17T03:21:53.559462+0000","last_active":"2024-09-17T03:21:56.209401+0000","last_peered":"2024-09-17T03:21:56.209401+0000","last_clean":"2024-09-17T03:21:56.209401+0000","last_became_active":"2024-09-17T03:21:53.559104+0000","last_became_peered":"2024-09-17T03:21:53.559104+0000","last_unstale":"2024-09-17T03:21:56.209401+0000","last_undegraded":"2024-09-17T03:21:56.209401+0000","last_fullsized":"2024-09-17T03:21:56.209401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:53.263133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541784+0000","last_change":"2024-09-17T03:21:53.554251+0000","last_active":"2024-09-17T03:21:54.541784+0000","last_peered":"2024-09-17T03:21:54.541784+0000","last_clean":"2024-09-17T03:21:54.541784+0000","last_became_active":"2024-09-17T03:21:53.553896+0000","last_became_peered":"2024-09-17T03:21:53.553896+0000","last_unstale":"2024-09-17T03:21:54.541784+0000","last_undegraded":"2024-09-17T03:21:54.541784+0000","last_fullsized":"2024-09-17T03:21:54.541784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:44.459862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541967+0000","last_change":"2024-09-17T03:21:53.541374+0000","last_active":"2024-09-17T03:21:54.541967+0000","last_peered":"2024-09-17T03:21:54.541967+0000","last_clean":"2024-09-17T03:21:54.541967+0000","last_became_active":"2024-09-17T03:21:53.541026+0000","last_became_peered":"2024-09-17T03:21:53.541026+0000","last_unstale":"2024-09-17T03:21:54.541967+0000","last_undegraded":"2024-09-17T03:21:54.541967+0000","last_fullsized":"2024-09-17T03:21:54.541967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:21.499182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530523+0000","last_change":"2024-09-17T03:21:53.550640+0000","last_active":"2024-09-17T03:21:55.530523+0000","last_peered":"2024-09-17T03:21:55.530523+0000","last_clean":"2024-09-17T03:21:55.530523+0000","last_became_active":"2024-09-17T03:21:53.550298+0000","last_became_peered":"2024-09-17T03:21:53.550298+0000","last_unstale":"2024-09-17T03:21:55.530523+0000","last_undegraded":"2024-09-17T03:21:55.530523+0000","last_fullsized":"2024-09-17T03:21:55.530523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:36.269301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558824+0000","last_change":"2024-09-17T03:21:53.556064+0000","last_active":"2024-09-17T03:21:53.558824+0000","last_peered":"2024-09-17T03:21:53.558824+0000","last_clean":"2024-09-17T03:21:53.558824+0000","last_became_active":"2024-09-17T03:21:53.555733+0000","last_became_peered":"2024-09-17T03:21:53.555733+0000","last_unstale":"2024-09-17T03:21:53.558824+0000","last_undegraded":"2024-09-17T03:21:53.558824+0000","last_fullsized":"2024-09-17T03:21:53.558824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:04:44.401616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540048+0000","last_change":"2024-09-17T03:21:53.546109+0000","last_active":"2024-09-17T03:21:54.540048+0000","last_peered":"2024-09-17T03:21:54.540048+0000","last_clean":"2024-09-17T03:21:54.540048+0000","last_became_active":"2024-09-17T03:21:53.545503+0000","last_became_peered":"2024-09-17T03:21:53.545503+0000","last_unstale":"2024-09-17T03:21:54.540048+0000","last_undegraded":"2024-09-17T03:21:54.540048+0000","last_fullsized":"2024-09-17T03:21:54.540048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:02.167432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540154+0000","last_change":"2024-09-17T03:21:53.560492+0000","last_active":"2024-09-17T03:21:54.540154+0000","last_peered":"2024-09-17T03:21:54.540154+0000","last_clean":"2024-09-17T03:21:54.540154+0000","last_became_active":"2024-09-17T03:21:53.560169+0000","last_became_peered":"2024-09-17T03:21:53.560169+0000","last_unstale":"2024-09-17T03:21:54.540154+0000","last_undegraded":"2024-09-17T03:21:54.540154+0000","last_fullsized":"2024-09-17T03:21:54.540154+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:08.917749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540248+0000","last_change":"2024-09-17T03:21:53.544837+0000","last_active":"2024-09-17T03:21:54.540248+0000","last_peered":"2024-09-17T03:21:54.540248+0000","last_clean":"2024-09-17T03:21:54.540248+0000","last_became_active":"2024-09-17T03:21:53.542870+0000","last_became_peered":"2024-09-17T03:21:53.542870+0000","last_unstale":"2024-09-17T03:21:54.540248+0000","last_undegraded":"2024-09-17T03:21:54.540248+0000","last_fullsized":"2024-09-17T03:21:54.540248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:39.019276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558834+0000","last_change":"2024-09-17T03:21:53.555144+0000","last_active":"2024-09-17T03:21:53.558834+0000","last_peered":"2024-09-17T03:21:53.558834+0000","last_clean":"2024-09-17T03:21:53.558834+0000","last_became_active":"2024-09-17T03:21:53.554782+0000","last_became_peered":"2024-09-17T03:21:53.554782+0000","last_unstale":"2024-09-17T03:21:53.558834+0000","last_undegraded":"2024-09-17T03:21:53.558834+0000","last_fullsized":"2024-09-17T03:21:53.558834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:20.186745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541769+0000","last_change":"2024-09-17T03:21:53.539013+0000","last_active":"2024-09-17T03:21:54.541769+0000","last_peered":"2024-09-17T03:21:54.541769+0000","last_clean":"2024-09-17T03:21:54.541769+0000","last_became_active":"2024-09-17T03:21:53.538702+0000","last_became_peered":"2024-09-17T03:21:53.538702+0000","last_unstale":"2024-09-17T03:21:54.541769+0000","last_undegraded":"2024-09-17T03:21:54.541769+0000","last_fullsized":"2024-09-17T03:21:54.541769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:31.657085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.211707+0000","last_change":"2024-09-17T03:21:53.548816+0000","last_active":"2024-09-17T03:21:56.211707+0000","last_peered":"2024-09-17T03:21:56.211707+0000","last_clean":"2024-09-17T03:21:56.211707+0000","last_became_active":"2024-09-17T03:21:53.548450+0000","last_became_peered":"2024-09-17T03:21:53.548450+0000","last_unstale":"2024-09-17T03:21:56.211707+0000","last_undegraded":"2024-09-17T03:21:56.211707+0000","last_fullsized":"2024-09-17T03:21:56.211707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:48.840495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.212802+0000","last_change":"2024-09-17T03:21:53.542885+0000","last_active":"2024-09-17T03:21:56.212802+0000","last_peered":"2024-09-17T03:21:56.212802+0000","last_clean":"2024-09-17T03:21:56.212802+0000","last_became_active":"2024-09-17T03:21:53.542545+0000","last_became_peered":"2024-09-17T03:21:53.542545+0000","last_unstale":"2024-09-17T03:21:56.212802+0000","last_undegraded":"2024-09-17T03:21:56.212802+0000","last_fullsized":"2024-09-17T03:21:56.212802+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:11.576032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213921+0000","last_change":"2024-09-17T03:21:53.551044+0000","last_active":"2024-09-17T03:21:56.213921+0000","last_peered":"2024-09-17T03:21:56.213921+0000","last_clean":"2024-09-17T03:21:56.213921+0000","last_became_active":"2024-09-17T03:21:53.549024+0000","last_became_peered":"2024-09-17T03:21:53.549024+0000","last_unstale":"2024-09-17T03:21:56.213921+0000","last_undegraded":"2024-09-17T03:21:56.213921+0000","last_fullsized":"2024-09-17T03:21:56.213921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:04.057797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.531009+0000","last_change":"2024-09-17T03:21:53.550799+0000","last_active":"2024-09-17T03:21:55.531009+0000","last_peered":"2024-09-17T03:21:55.531009+0000","last_clean":"2024-09-17T03:21:55.531009+0000","last_became_active":"2024-09-17T03:21:53.550476+0000","last_became_peered":"2024-09-17T03:21:53.550476+0000","last_unstale":"2024-09-17T03:21:55.531009+0000","last_undegraded":"2024-09-17T03:21:55.531009+0000","last_fullsized":"2024-09-17T03:21:55.531009+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:44.459862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213607+0000","last_change":"2024-09-17T03:21:52.552331+0000","last_active":"2024-09-17T03:21:56.213607+0000","last_peered":"2024-09-17T03:21:56.213607+0000","last_clean":"2024-09-17T03:21:56.213607+0000","last_became_active":"2024-09-17T03:21:52.551956+0000","last_became_peered":"2024-09-17T03:21:52.551956+0000","last_unstale":"2024-09-17T03:21:56.213607+0000","last_undegraded":"2024-09-17T03:21:56.213607+0000","last_fullsized":"2024-09-17T03:21:56.213607+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:52.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213515+0000","last_change":"2024-09-17T03:21:53.547810+0000","last_active":"2024-09-17T03:21:56.213515+0000","last_peered":"2024-09-17T03:21:56.213515+0000","last_clean":"2024-09-17T03:21:56.213515+0000","last_became_active":"2024-09-17T03:21:53.547252+0000","last_became_peered":"2024-09-17T03:21:53.547252+0000","last_unstale":"2024-09-17T03:21:56.213515+0000","last_undegraded":"2024-09-17T03:21:56.213515+0000","last_fullsized":"2024-09-17T03:21:56.213515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:46.216277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530873+0000","last_change":"2024-09-17T03:21:52.534295+0000","last_active":"2024-09-17T03:21:55.530873+0000","last_peered":"2024-09-17T03:21:55.530873+0000","last_clean":"2024-09-17T03:21:55.530873+0000","last_became_active":"2024-09-17T03:21:52.533866+0000","last_became_peered":"2024-09-17T03:21:52.533866+0000","last_unstale":"2024-09-17T03:21:55.530873+0000","last_undegraded":"2024-09-17T03:21:55.530873+0000","last_fullsized":"2024-09-17T03:21:55.530873+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:45.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541946+0000","last_change":"2024-09-17T03:21:53.543410+0000","last_active":"2024-09-17T03:21:54.541946+0000","last_peered":"2024-09-17T03:21:54.541946+0000","last_clean":"2024-09-17T03:21:54.541946+0000","last_became_active":"2024-09-17T03:21:53.541175+0000","last_became_peered":"2024-09-17T03:21:53.541175+0000","last_unstale":"2024-09-17T03:21:54.541946+0000","last_undegraded":"2024-09-17T03:21:54.541946+0000","last_fullsized":"2024-09-17T03:21:54.541946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:48.080302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539638+0000","last_change":"2024-09-17T03:21:52.547470+0000","last_active":"2024-09-17T03:21:54.539638+0000","last_peered":"2024-09-17T03:21:54.539638+0000","last_clean":"2024-09-17T03:21:54.539638+0000","last_became_active":"2024-09-17T03:21:52.547238+0000","last_became_peered":"2024-09-17T03:21:52.547238+0000","last_unstale":"2024-09-17T03:21:54.539638+0000","last_undegraded":"2024-09-17T03:21:54.539638+0000","last_fullsized":"2024-09-17T03:21:54.539638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:35.644290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539437+0000","last_change":"2024-09-17T03:21:53.546443+0000","last_active":"2024-09-17T03:21:54.539437+0000","last_peered":"2024-09-17T03:21:54.539437+0000","last_clean":"2024-09-17T03:21:54.539437+0000","last_became_active":"2024-09-17T03:21:53.542497+0000","last_became_peered":"2024-09-17T03:21:53.542497+0000","last_unstale":"2024-09-17T03:21:54.539437+0000","last_undegraded":"2024-09-17T03:21:54.539437+0000","last_fullsized":"2024-09-17T03:21:54.539437+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:07.512532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209223+0000","last_change":"2024-09-17T03:21:52.543539+0000","last_active":"2024-09-17T03:21:56.209223+0000","last_peered":"2024-09-17T03:21:56.209223+0000","last_clean":"2024-09-17T03:21:56.209223+0000","last_became_active":"2024-09-17T03:21:52.543201+0000","last_became_peered":"2024-09-17T03:21:52.543201+0000","last_unstale":"2024-09-17T03:21:56.209223+0000","last_undegraded":"2024-09-17T03:21:56.209223+0000","last_fullsized":"2024-09-17T03:21:56.209223+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:03.441922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213852+0000","last_change":"2024-09-17T03:21:53.549791+0000","last_active":"2024-09-17T03:21:56.213852+0000","last_peered":"2024-09-17T03:21:56.213852+0000","last_clean":"2024-09-17T03:21:56.213852+0000","last_became_active":"2024-09-17T03:21:53.549212+0000","last_became_peered":"2024-09-17T03:21:53.549212+0000","last_unstale":"2024-09-17T03:21:56.213852+0000","last_undegraded":"2024-09-17T03:21:56.213852+0000","last_fullsized":"2024-09-17T03:21:56.213852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:58.854209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541866+0000","last_change":"2024-09-17T03:21:52.539607+0000","last_active":"2024-09-17T03:21:54.541866+0000","last_peered":"2024-09-17T03:21:54.541866+0000","last_clean":"2024-09-17T03:21:54.541866+0000","last_became_active":"2024-09-17T03:21:52.539274+0000","last_became_peered":"2024-09-17T03:21:52.539274+0000","last_unstale":"2024-09-17T03:21:54.541866+0000","last_undegraded":"2024-09-17T03:21:54.541866+0000","last_fullsized":"2024-09-17T03:21:54.541866+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:04.766587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558876+0000","last_change":"2024-09-17T03:21:53.555218+0000","last_active":"2024-09-17T03:21:53.558876+0000","last_peered":"2024-09-17T03:21:53.558876+0000","last_clean":"2024-09-17T03:21:53.558876+0000","last_became_active":"2024-09-17T03:21:53.554811+0000","last_became_peered":"2024-09-17T03:21:53.554811+0000","last_unstale":"2024-09-17T03:21:53.558876+0000","last_undegraded":"2024-09-17T03:21:53.558876+0000","last_fullsized":"2024-09-17T03:21:53.558876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:26.191852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541484+0000","last_change":"2024-09-17T03:21:52.539742+0000","last_active":"2024-09-17T03:21:54.541484+0000","last_peered":"2024-09-17T03:21:54.541484+0000","last_clean":"2024-09-17T03:21:54.541484+0000","last_became_active":"2024-09-17T03:21:52.539428+0000","last_became_peered":"2024-09-17T03:21:52.539428+0000","last_unstale":"2024-09-17T03:21:54.541484+0000","last_undegraded":"2024-09-17T03:21:54.541484+0000","last_fullsized":"2024-09-17T03:21:54.541484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:44.627845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541404+0000","last_change":"2024-09-17T03:21:53.544203+0000","last_active":"2024-09-17T03:21:54.541404+0000","last_peered":"2024-09-17T03:21:54.541404+0000","last_clean":"2024-09-17T03:21:54.541404+0000","last_became_active":"2024-09-17T03:21:53.542205+0000","last_became_peered":"2024-09-17T03:21:53.542205+0000","last_unstale":"2024-09-17T03:21:54.541404+0000","last_undegraded":"2024-09-17T03:21:54.541404+0000","last_fullsized":"2024-09-17T03:21:54.541404+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:18.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538436+0000","last_change":"2024-09-17T03:21:52.531559+0000","last_active":"2024-09-17T03:21:54.538436+0000","last_peered":"2024-09-17T03:21:54.538436+0000","last_clean":"2024-09-17T03:21:54.538436+0000","last_became_active":"2024-09-17T03:21:52.531235+0000","last_became_peered":"2024-09-17T03:21:52.531235+0000","last_unstale":"2024-09-17T03:21:54.538436+0000","last_undegraded":"2024-09-17T03:21:54.538436+0000","last_fullsized":"2024-09-17T03:21:54.538436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:02.669724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.537876+0000","last_change":"2024-09-17T03:21:53.547516+0000","last_active":"2024-09-17T03:21:54.537876+0000","last_peered":"2024-09-17T03:21:54.537876+0000","last_clean":"2024-09-17T03:21:54.537876+0000","last_became_active":"2024-09-17T03:21:53.545852+0000","last_became_peered":"2024-09-17T03:21:53.545852+0000","last_unstale":"2024-09-17T03:21:54.537876+0000","last_undegraded":"2024-09-17T03:21:54.537876+0000","last_fullsized":"2024-09-17T03:21:54.537876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:50.047009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541227+0000","last_change":"2024-09-17T03:21:52.545742+0000","last_active":"2024-09-17T03:21:54.541227+0000","last_peered":"2024-09-17T03:21:54.541227+0000","last_clean":"2024-09-17T03:21:54.541227+0000","last_became_active":"2024-09-17T03:21:52.542634+0000","last_became_peered":"2024-09-17T03:21:52.542634+0000","last_unstale":"2024-09-17T03:21:54.541227+0000","last_undegraded":"2024-09-17T03:21:54.541227+0000","last_fullsized":"2024-09-17T03:21:54.541227+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:54.390671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.567928+0000","last_change":"2024-09-17T03:21:53.562382+0000","last_active":"2024-09-17T03:21:53.567928+0000","last_peered":"2024-09-17T03:21:53.567928+0000","last_clean":"2024-09-17T03:21:53.567928+0000","last_became_active":"2024-09-17T03:21:53.562005+0000","last_became_peered":"2024-09-17T03:21:53.562005+0000","last_unstale":"2024-09-17T03:21:53.567928+0000","last_undegraded":"2024-09-17T03:21:53.567928+0000","last_fullsized":"2024-09-17T03:21:53.567928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:51.526057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540516+0000","last_change":"2024-09-17T03:21:52.538465+0000","last_active":"2024-09-17T03:21:54.540516+0000","last_peered":"2024-09-17T03:21:54.540516+0000","last_clean":"2024-09-17T03:21:54.540516+0000","last_became_active":"2024-09-17T03:21:52.538159+0000","last_became_peered":"2024-09-17T03:21:52.538159+0000","last_unstale":"2024-09-17T03:21:54.540516+0000","last_undegraded":"2024-09-17T03:21:54.540516+0000","last_fullsized":"2024-09-17T03:21:54.540516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:26.922910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540448+0000","last_change":"2024-09-17T03:21:53.540179+0000","last_active":"2024-09-17T03:21:54.540448+0000","last_peered":"2024-09-17T03:21:54.540448+0000","last_clean":"2024-09-17T03:21:54.540448+0000","last_became_active":"2024-09-17T03:21:53.539884+0000","last_became_peered":"2024-09-17T03:21:53.539884+0000","last_unstale":"2024-09-17T03:21:54.540448+0000","last_undegraded":"2024-09-17T03:21:54.540448+0000","last_fullsized":"2024-09-17T03:21:54.540448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:32.162775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.531003+0000","last_change":"2024-09-17T03:21:52.537746+0000","last_active":"2024-09-17T03:21:55.531003+0000","last_peered":"2024-09-17T03:21:55.531003+0000","last_clean":"2024-09-17T03:21:55.531003+0000","last_became_active":"2024-09-17T03:21:52.537418+0000","last_became_peered":"2024-09-17T03:21:52.537418+0000","last_unstale":"2024-09-17T03:21:55.531003+0000","last_undegraded":"2024-09-17T03:21:55.531003+0000","last_fullsized":"2024-09-17T03:21:55.531003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:50.592582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539590+0000","last_change":"2024-09-17T03:21:53.561840+0000","last_active":"2024-09-17T03:21:54.539590+0000","last_peered":"2024-09-17T03:21:54.539590+0000","last_clean":"2024-09-17T03:21:54.539590+0000","last_became_active":"2024-09-17T03:21:53.559774+0000","last_became_peered":"2024-09-17T03:21:53.559774+0000","last_unstale":"2024-09-17T03:21:54.539590+0000","last_undegraded":"2024-09-17T03:21:54.539590+0000","last_fullsized":"2024-09-17T03:21:54.539590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:24.739520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558550+0000","last_change":"2024-09-17T03:21:52.542996+0000","last_active":"2024-09-17T03:21:53.558550+0000","last_peered":"2024-09-17T03:21:53.558550+0000","last_clean":"2024-09-17T03:21:53.558550+0000","last_became_active":"2024-09-17T03:21:52.542635+0000","last_became_peered":"2024-09-17T03:21:52.542635+0000","last_unstale":"2024-09-17T03:21:53.558550+0000","last_undegraded":"2024-09-17T03:21:53.558550+0000","last_fullsized":"2024-09-17T03:21:53.558550+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:33.173785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.556628+0000","last_change":"2024-09-17T03:21:53.547534+0000","last_active":"2024-09-17T03:21:53.556628+0000","last_peered":"2024-09-17T03:21:53.556628+0000","last_clean":"2024-09-17T03:21:53.556628+0000","last_became_active":"2024-09-17T03:21:53.545836+0000","last_became_peered":"2024-09-17T03:21:53.545836+0000","last_unstale":"2024-09-17T03:21:53.556628+0000","last_undegraded":"2024-09-17T03:21:53.556628+0000","last_fullsized":"2024-09-17T03:21:53.556628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:10.523386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539498+0000","last_change":"2024-09-17T03:21:52.546507+0000","last_active":"2024-09-17T03:21:54.539498+0000","last_peered":"2024-09-17T03:21:54.539498+0000","last_clean":"2024-09-17T03:21:54.539498+0000","last_became_active":"2024-09-17T03:21:52.546233+0000","last_became_peered":"2024-09-17T03:21:52.546233+0000","last_unstale":"2024-09-17T03:21:54.539498+0000","last_undegraded":"2024-09-17T03:21:54.539498+0000","last_fullsized":"2024-09-17T03:21:54.539498+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:57.348211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538085+0000","last_change":"2024-09-17T03:21:53.547736+0000","last_active":"2024-09-17T03:21:54.538085+0000","last_peered":"2024-09-17T03:21:54.538085+0000","last_clean":"2024-09-17T03:21:54.538085+0000","last_became_active":"2024-09-17T03:21:53.546510+0000","last_became_peered":"2024-09-17T03:21:53.546510+0000","last_unstale":"2024-09-17T03:21:54.538085+0000","last_undegraded":"2024-09-17T03:21:54.538085+0000","last_fullsized":"2024-09-17T03:21:54.538085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:35.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542035+0000","last_change":"2024-09-17T03:21:52.542313+0000","last_active":"2024-09-17T03:21:54.542035+0000","last_peered":"2024-09-17T03:21:54.542035+0000","last_clean":"2024-09-17T03:21:54.542035+0000","last_became_active":"2024-09-17T03:21:52.541963+0000","last_became_peered":"2024-09-17T03:21:52.541963+0000","last_unstale":"2024-09-17T03:21:54.542035+0000","last_undegraded":"2024-09-17T03:21:54.542035+0000","last_fullsized":"2024-09-17T03:21:54.542035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:52.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.539241+0000","last_change":"2024-09-17T03:21:53.555940+0000","last_active":"2024-09-17T03:21:54.539241+0000","last_peered":"2024-09-17T03:21:54.539241+0000","last_clean":"2024-09-17T03:21:54.539241+0000","last_became_active":"2024-09-17T03:21:53.555423+0000","last_became_peered":"2024-09-17T03:21:53.555423+0000","last_unstale":"2024-09-17T03:21:54.539241+0000","last_undegraded":"2024-09-17T03:21:54.539241+0000","last_fullsized":"2024-09-17T03:21:54.539241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:07:20.943274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538261+0000","last_change":"2024-09-17T03:21:52.538771+0000","last_active":"2024-09-17T03:21:54.538261+0000","last_peered":"2024-09-17T03:21:54.538261+0000","last_clean":"2024-09-17T03:21:54.538261+0000","last_became_active":"2024-09-17T03:21:52.537006+0000","last_became_peered":"2024-09-17T03:21:52.537006+0000","last_unstale":"2024-09-17T03:21:54.538261+0000","last_undegraded":"2024-09-17T03:21:54.538261+0000","last_fullsized":"2024-09-17T03:21:54.538261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:27:07.695072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213894+0000","last_change":"2024-09-17T03:21:53.550422+0000","last_active":"2024-09-17T03:21:56.213894+0000","last_peered":"2024-09-17T03:21:56.213894+0000","last_clean":"2024-09-17T03:21:56.213894+0000","last_became_active":"2024-09-17T03:21:53.549519+0000","last_became_peered":"2024-09-17T03:21:53.549519+0000","last_unstale":"2024-09-17T03:21:56.213894+0000","last_undegraded":"2024-09-17T03:21:56.213894+0000","last_fullsized":"2024-09-17T03:21:56.213894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:13.254268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538183+0000","last_change":"2024-09-17T03:21:52.539288+0000","last_active":"2024-09-17T03:21:54.538183+0000","last_peered":"2024-09-17T03:21:54.538183+0000","last_clean":"2024-09-17T03:21:54.538183+0000","last_became_active":"2024-09-17T03:21:52.538955+0000","last_became_peered":"2024-09-17T03:21:52.538955+0000","last_unstale":"2024-09-17T03:21:54.538183+0000","last_undegraded":"2024-09-17T03:21:54.538183+0000","last_fullsized":"2024-09-17T03:21:54.538183+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:40.503141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213086+0000","last_change":"2024-09-17T03:21:53.543540+0000","last_active":"2024-09-17T03:21:56.213086+0000","last_peered":"2024-09-17T03:21:56.213086+0000","last_clean":"2024-09-17T03:21:56.213086+0000","last_became_active":"2024-09-17T03:21:53.543197+0000","last_became_peered":"2024-09-17T03:21:53.543197+0000","last_unstale":"2024-09-17T03:21:56.213086+0000","last_undegraded":"2024-09-17T03:21:56.213086+0000","last_fullsized":"2024-09-17T03:21:56.213086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:28:43.522796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538050+0000","last_change":"2024-09-17T03:21:52.547105+0000","last_active":"2024-09-17T03:21:54.538050+0000","last_peered":"2024-09-17T03:21:54.538050+0000","last_clean":"2024-09-17T03:21:54.538050+0000","last_became_active":"2024-09-17T03:21:52.546807+0000","last_became_peered":"2024-09-17T03:21:52.546807+0000","last_unstale":"2024-09-17T03:21:54.538050+0000","last_undegraded":"2024-09-17T03:21:54.538050+0000","last_fullsized":"2024-09-17T03:21:54.538050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:34.615668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542203+0000","last_change":"2024-09-17T03:21:53.554671+0000","last_active":"2024-09-17T03:21:54.542203+0000","last_peered":"2024-09-17T03:21:54.542203+0000","last_clean":"2024-09-17T03:21:54.542203+0000","last_became_active":"2024-09-17T03:21:53.554360+0000","last_became_peered":"2024-09-17T03:21:53.554360+0000","last_unstale":"2024-09-17T03:21:54.542203+0000","last_undegraded":"2024-09-17T03:21:54.542203+0000","last_fullsized":"2024-09-17T03:21:54.542203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:30.591964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540293+0000","last_change":"2024-09-17T03:21:52.545217+0000","last_active":"2024-09-17T03:21:54.540293+0000","last_peered":"2024-09-17T03:21:54.540293+0000","last_clean":"2024-09-17T03:21:54.540293+0000","last_became_active":"2024-09-17T03:21:52.544910+0000","last_became_peered":"2024-09-17T03:21:52.544910+0000","last_unstale":"2024-09-17T03:21:54.540293+0000","last_undegraded":"2024-09-17T03:21:54.540293+0000","last_fullsized":"2024-09-17T03:21:54.540293+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.581307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558486+0000","last_change":"2024-09-17T03:21:53.554146+0000","last_active":"2024-09-17T03:21:53.558486+0000","last_peered":"2024-09-17T03:21:53.558486+0000","last_clean":"2024-09-17T03:21:53.558486+0000","last_became_active":"2024-09-17T03:21:53.553848+0000","last_became_peered":"2024-09-17T03:21:53.553848+0000","last_unstale":"2024-09-17T03:21:53.558486+0000","last_undegraded":"2024-09-17T03:21:53.558486+0000","last_fullsized":"2024-09-17T03:21:53.558486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:19.698487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542070+0000","last_change":"2024-09-17T03:21:52.545331+0000","last_active":"2024-09-17T03:21:54.542070+0000","last_peered":"2024-09-17T03:21:54.542070+0000","last_clean":"2024-09-17T03:21:54.542070+0000","last_became_active":"2024-09-17T03:21:52.543054+0000","last_became_peered":"2024-09-17T03:21:52.543054+0000","last_unstale":"2024-09-17T03:21:54.542070+0000","last_undegraded":"2024-09-17T03:21:54.542070+0000","last_fullsized":"2024-09-17T03:21:54.542070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:23.472290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538817+0000","last_change":"2024-09-17T03:21:53.536062+0000","last_active":"2024-09-17T03:21:54.538817+0000","last_peered":"2024-09-17T03:21:54.538817+0000","last_clean":"2024-09-17T03:21:54.538817+0000","last_became_active":"2024-09-17T03:21:53.535655+0000","last_became_peered":"2024-09-17T03:21:53.535655+0000","last_unstale":"2024-09-17T03:21:54.538817+0000","last_undegraded":"2024-09-17T03:21:54.538817+0000","last_fullsized":"2024-09-17T03:21:54.538817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:42.491863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.542230+0000","last_change":"2024-09-17T03:21:52.540408+0000","last_active":"2024-09-17T03:21:54.542230+0000","last_peered":"2024-09-17T03:21:54.542230+0000","last_clean":"2024-09-17T03:21:54.542230+0000","last_became_active":"2024-09-17T03:21:52.540062+0000","last_became_peered":"2024-09-17T03:21:52.540062+0000","last_unstale":"2024-09-17T03:21:54.542230+0000","last_undegraded":"2024-09-17T03:21:54.542230+0000","last_fullsized":"2024-09-17T03:21:54.542230+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:45.160058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209277+0000","last_change":"2024-09-17T03:21:53.554048+0000","last_active":"2024-09-17T03:21:56.209277+0000","last_peered":"2024-09-17T03:21:56.209277+0000","last_clean":"2024-09-17T03:21:56.209277+0000","last_became_active":"2024-09-17T03:21:53.553745+0000","last_became_peered":"2024-09-17T03:21:53.553745+0000","last_unstale":"2024-09-17T03:21:56.209277+0000","last_undegraded":"2024-09-17T03:21:56.209277+0000","last_fullsized":"2024-09-17T03:21:56.209277+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:05.771794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541820+0000","last_change":"2024-09-17T03:21:52.541499+0000","last_active":"2024-09-17T03:21:54.541820+0000","last_peered":"2024-09-17T03:21:54.541820+0000","last_clean":"2024-09-17T03:21:54.541820+0000","last_became_active":"2024-09-17T03:21:52.539204+0000","last_became_peered":"2024-09-17T03:21:52.539204+0000","last_unstale":"2024-09-17T03:21:54.541820+0000","last_undegraded":"2024-09-17T03:21:54.541820+0000","last_fullsized":"2024-09-17T03:21:54.541820+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:40.905763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209508+0000","last_change":"2024-09-17T03:21:53.552284+0000","last_active":"2024-09-17T03:21:56.209508+0000","last_peered":"2024-09-17T03:21:56.209508+0000","last_clean":"2024-09-17T03:21:56.209508+0000","last_became_active":"2024-09-17T03:21:53.551890+0000","last_became_peered":"2024-09-17T03:21:53.551890+0000","last_unstale":"2024-09-17T03:21:56.209508+0000","last_undegraded":"2024-09-17T03:21:56.209508+0000","last_fullsized":"2024-09-17T03:21:56.209508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:05.203911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530677+0000","last_change":"2024-09-17T03:21:52.528899+0000","last_active":"2024-09-17T03:21:55.530677+0000","last_peered":"2024-09-17T03:21:55.530677+0000","last_clean":"2024-09-17T03:21:55.530677+0000","last_became_active":"2024-09-17T03:21:52.528518+0000","last_became_peered":"2024-09-17T03:21:52.528518+0000","last_unstale":"2024-09-17T03:21:55.530677+0000","last_undegraded":"2024-09-17T03:21:55.530677+0000","last_fullsized":"2024-09-17T03:21:55.530677+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:18.545893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209503+0000","last_change":"2024-09-17T03:21:53.551589+0000","last_active":"2024-09-17T03:21:56.209503+0000","last_peered":"2024-09-17T03:21:56.209503+0000","last_clean":"2024-09-17T03:21:56.209503+0000","last_became_active":"2024-09-17T03:21:53.551206+0000","last_became_peered":"2024-09-17T03:21:53.551206+0000","last_unstale":"2024-09-17T03:21:56.209503+0000","last_undegraded":"2024-09-17T03:21:56.209503+0000","last_fullsized":"2024-09-17T03:21:56.209503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:19.551101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213562+0000","last_change":"2024-09-17T03:21:52.546295+0000","last_active":"2024-09-17T03:21:56.213562+0000","last_peered":"2024-09-17T03:21:56.213562+0000","last_clean":"2024-09-17T03:21:56.213562+0000","last_became_active":"2024-09-17T03:21:52.545969+0000","last_became_peered":"2024-09-17T03:21:52.545969+0000","last_unstale":"2024-09-17T03:21:56.213562+0000","last_undegraded":"2024-09-17T03:21:56.213562+0000","last_fullsized":"2024-09-17T03:21:56.213562+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:34.264773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209260+0000","last_change":"2024-09-17T03:21:53.547274+0000","last_active":"2024-09-17T03:21:56.209260+0000","last_peered":"2024-09-17T03:21:56.209260+0000","last_clean":"2024-09-17T03:21:56.209260+0000","last_became_active":"2024-09-17T03:21:53.546964+0000","last_became_peered":"2024-09-17T03:21:53.546964+0000","last_unstale":"2024-09-17T03:21:56.209260+0000","last_undegraded":"2024-09-17T03:21:56.209260+0000","last_fullsized":"2024-09-17T03:21:56.209260+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.477497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541184+0000","last_change":"2024-09-17T03:21:52.531752+0000","last_active":"2024-09-17T03:21:54.541184+0000","last_peered":"2024-09-17T03:21:54.541184+0000","last_clean":"2024-09-17T03:21:54.541184+0000","last_became_active":"2024-09-17T03:21:52.530910+0000","last_became_peered":"2024-09-17T03:21:52.530910+0000","last_unstale":"2024-09-17T03:21:54.541184+0000","last_undegraded":"2024-09-17T03:21:54.541184+0000","last_fullsized":"2024-09-17T03:21:54.541184+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:34.264773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.212339+0000","last_change":"2024-09-17T03:21:53.549723+0000","last_active":"2024-09-17T03:21:56.212339+0000","last_peered":"2024-09-17T03:21:56.212339+0000","last_clean":"2024-09-17T03:21:56.212339+0000","last_became_active":"2024-09-17T03:21:53.549084+0000","last_became_peered":"2024-09-17T03:21:53.549084+0000","last_unstale":"2024-09-17T03:21:56.212339+0000","last_undegraded":"2024-09-17T03:21:56.212339+0000","last_fullsized":"2024-09-17T03:21:56.212339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:52.073779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.535249+0000","last_change":"2024-09-17T03:21:52.541036+0000","last_active":"2024-09-17T03:21:53.535249+0000","last_peered":"2024-09-17T03:21:53.535249+0000","last_clean":"2024-09-17T03:21:53.535249+0000","last_became_active":"2024-09-17T03:21:52.540723+0000","last_became_peered":"2024-09-17T03:21:52.540723+0000","last_unstale":"2024-09-17T03:21:53.535249+0000","last_undegraded":"2024-09-17T03:21:53.535249+0000","last_fullsized":"2024-09-17T03:21:53.535249+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:23.185290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.209324+0000","last_change":"2024-09-17T03:21:53.552767+0000","last_active":"2024-09-17T03:21:56.209324+0000","last_peered":"2024-09-17T03:21:56.209324+0000","last_clean":"2024-09-17T03:21:56.209324+0000","last_became_active":"2024-09-17T03:21:53.552517+0000","last_became_peered":"2024-09-17T03:21:53.552517+0000","last_unstale":"2024-09-17T03:21:56.209324+0000","last_undegraded":"2024-09-17T03:21:56.209324+0000","last_fullsized":"2024-09-17T03:21:56.209324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:35.269981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.540386+0000","last_change":"2024-09-17T03:21:52.539788+0000","last_active":"2024-09-17T03:21:54.540386+0000","last_peered":"2024-09-17T03:21:54.540386+0000","last_clean":"2024-09-17T03:21:54.540386+0000","last_became_active":"2024-09-17T03:21:52.539332+0000","last_became_peered":"2024-09-17T03:21:52.539332+0000","last_unstale":"2024-09-17T03:21:54.540386+0000","last_undegraded":"2024-09-17T03:21:54.540386+0000","last_fullsized":"2024-09-17T03:21:54.540386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:05.159629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T03:21:53.558455+0000","last_change":"2024-09-17T03:21:53.551517+0000","last_active":"2024-09-17T03:21:53.558455+0000","last_peered":"2024-09-17T03:21:53.558455+0000","last_clean":"2024-09-17T03:21:53.558455+0000","last_became_active":"2024-09-17T03:21:53.551149+0000","last_became_peered":"2024-09-17T03:21:53.551149+0000","last_unstale":"2024-09-17T03:21:53.558455+0000","last_undegraded":"2024-09-17T03:21:53.558455+0000","last_fullsized":"2024-09-17T03:21:53.558455+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:55.646076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.531159+0000","last_change":"2024-09-17T03:21:52.530406+0000","last_active":"2024-09-17T03:21:55.531159+0000","last_peered":"2024-09-17T03:21:55.531159+0000","last_clean":"2024-09-17T03:21:55.531159+0000","last_became_active":"2024-09-17T03:21:52.530098+0000","last_became_peered":"2024-09-17T03:21:52.530098+0000","last_unstale":"2024-09-17T03:21:55.531159+0000","last_undegraded":"2024-09-17T03:21:55.531159+0000","last_fullsized":"2024-09-17T03:21:55.531159+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:53.398862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530857+0000","last_change":"2024-09-17T03:21:53.547878+0000","last_active":"2024-09-17T03:21:55.530857+0000","last_peered":"2024-09-17T03:21:55.530857+0000","last_clean":"2024-09-17T03:21:55.530857+0000","last_became_active":"2024-09-17T03:21:53.541594+0000","last_became_peered":"2024-09-17T03:21:53.541594+0000","last_unstale":"2024-09-17T03:21:55.530857+0000","last_undegraded":"2024-09-17T03:21:55.530857+0000","last_fullsized":"2024-09-17T03:21:55.530857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:18.391663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538835+0000","last_change":"2024-09-17T03:21:52.544291+0000","last_active":"2024-09-17T03:21:54.538835+0000","last_peered":"2024-09-17T03:21:54.538835+0000","last_clean":"2024-09-17T03:21:54.538835+0000","last_became_active":"2024-09-17T03:21:52.543771+0000","last_became_peered":"2024-09-17T03:21:52.543771+0000","last_unstale":"2024-09-17T03:21:54.538835+0000","last_undegraded":"2024-09-17T03:21:54.538835+0000","last_fullsized":"2024-09-17T03:21:54.538835+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:29.096589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213397+0000","last_change":"2024-09-17T03:21:53.545540+0000","last_active":"2024-09-17T03:21:56.213397+0000","last_peered":"2024-09-17T03:21:56.213397+0000","last_clean":"2024-09-17T03:21:56.213397+0000","last_became_active":"2024-09-17T03:21:53.545211+0000","last_became_peered":"2024-09-17T03:21:53.545211+0000","last_unstale":"2024-09-17T03:21:56.213397+0000","last_undegraded":"2024-09-17T03:21:56.213397+0000","last_fullsized":"2024-09-17T03:21:56.213397+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:36.269301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530867+0000","last_change":"2024-09-17T03:21:52.535843+0000","last_active":"2024-09-17T03:21:55.530867+0000","last_peered":"2024-09-17T03:21:55.530867+0000","last_clean":"2024-09-17T03:21:55.530867+0000","last_became_active":"2024-09-17T03:21:52.535549+0000","last_became_peered":"2024-09-17T03:21:52.535549+0000","last_unstale":"2024-09-17T03:21:55.530867+0000","last_undegraded":"2024-09-17T03:21:55.530867+0000","last_fullsized":"2024-09-17T03:21:55.530867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:43.831837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:55.530863+0000","last_change":"2024-09-17T03:21:53.546468+0000","last_active":"2024-09-17T03:21:55.530863+0000","last_peered":"2024-09-17T03:21:55.530863+0000","last_clean":"2024-09-17T03:21:55.530863+0000","last_became_active":"2024-09-17T03:21:53.545809+0000","last_became_peered":"2024-09-17T03:21:53.545809+0000","last_unstale":"2024-09-17T03:21:55.530863+0000","last_undegraded":"2024-09-17T03:21:55.530863+0000","last_fullsized":"2024-09-17T03:21:55.530863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:46.216277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538341+0000","last_change":"2024-09-17T03:21:52.533439+0000","last_active":"2024-09-17T03:21:54.538341+0000","last_peered":"2024-09-17T03:21:54.538341+0000","last_clean":"2024-09-17T03:21:54.538341+0000","last_became_active":"2024-09-17T03:21:52.533039+0000","last_became_peered":"2024-09-17T03:21:52.533039+0000","last_unstale":"2024-09-17T03:21:54.538341+0000","last_undegraded":"2024-09-17T03:21:54.538341+0000","last_fullsized":"2024-09-17T03:21:54.538341+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:58.362820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.538006+0000","last_change":"2024-09-17T03:21:53.540943+0000","last_active":"2024-09-17T03:21:54.538006+0000","last_peered":"2024-09-17T03:21:54.538006+0000","last_clean":"2024-09-17T03:21:54.538006+0000","last_became_active":"2024-09-17T03:21:53.540628+0000","last_became_peered":"2024-09-17T03:21:53.540628+0000","last_unstale":"2024-09-17T03:21:54.538006+0000","last_undegraded":"2024-09-17T03:21:54.538006+0000","last_fullsized":"2024-09-17T03:21:54.538006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:43.090203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541321+0000","last_change":"2024-09-17T03:21:52.535447+0000","last_active":"2024-09-17T03:21:54.541321+0000","last_peered":"2024-09-17T03:21:54.541321+0000","last_clean":"2024-09-17T03:21:54.541321+0000","last_became_active":"2024-09-17T03:21:52.535092+0000","last_became_peered":"2024-09-17T03:21:52.535092+0000","last_unstale":"2024-09-17T03:21:54.541321+0000","last_undegraded":"2024-09-17T03:21:54.541321+0000","last_fullsized":"2024-09-17T03:21:54.541321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:18.545893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.541342+0000","last_change":"2024-09-17T03:21:53.547997+0000","last_active":"2024-09-17T03:21:54.541342+0000","last_peered":"2024-09-17T03:21:54.541342+0000","last_clean":"2024-09-17T03:21:54.541342+0000","last_became_active":"2024-09-17T03:21:53.546043+0000","last_became_peered":"2024-09-17T03:21:53.546043+0000","last_unstale":"2024-09-17T03:21:54.541342+0000","last_undegraded":"2024-09-17T03:21:54.541342+0000","last_fullsized":"2024-09-17T03:21:54.541342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:46.216277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:54.537964+0000","last_change":"2024-09-17T03:21:52.541349+0000","last_active":"2024-09-17T03:21:54.537964+0000","last_peered":"2024-09-17T03:21:54.537964+0000","last_clean":"2024-09-17T03:21:54.537964+0000","last_became_active":"2024-09-17T03:21:52.541013+0000","last_became_peered":"2024-09-17T03:21:52.541013+0000","last_unstale":"2024-09-17T03:21:54.537964+0000","last_undegraded":"2024-09-17T03:21:54.537964+0000","last_fullsized":"2024-09-17T03:21:54.537964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:07.235240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.213550+0000","last_change":"2024-09-17T03:21:53.564667+0000","last_active":"2024-09-17T03:21:56.213550+0000","last_peered":"2024-09-17T03:21:56.213550+0000","last_clean":"2024-09-17T03:21:56.213550+0000","last_became_active":"2024-09-17T03:21:53.564367+0000","last_became_peered":"2024-09-17T03:21:53.564367+0000","last_unstale":"2024-09-17T03:21:56.213550+0000","last_undegraded":"2024-09-17T03:21:56.213550+0000","last_fullsized":"2024-09-17T03:21:56.213550+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:52.506352+0000","last_clean_scrub_stamp":"2024-09-17T03:21:52.506352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:27.233349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.212888+0000","last_change":"2024-09-17T03:21:52.548119+0000","last_active":"2024-09-17T03:21:56.212888+0000","last_peered":"2024-09-17T03:21:56.212888+0000","last_clean":"2024-09-17T03:21:56.212888+0000","last_became_active":"2024-09-17T03:21:52.547596+0000","last_became_peered":"2024-09-17T03:21:52.547596+0000","last_unstale":"2024-09-17T03:21:56.212888+0000","last_undegraded":"2024-09-17T03:21:56.212888+0000","last_fullsized":"2024-09-17T03:21:56.212888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:21:51.501144+0000","last_clean_scrub_stamp":"2024-09-17T03:21:51.501144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:18.545893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T03:21:56.214031+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:21:56.214031+0000","last_peered":"2024-09-17T03:21:56.214031+0000","last_clean":"2024-09-17T03:21:56.214031+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:21:56.214031+0000","last_undegraded":"2024-09-17T03:21:56.214031+0000","last_fullsized":"2024-09-17T03:21:56.214031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":725837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":348160,"data_stored":129695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":135483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":368640,"data_stored":137013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":135449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":725747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":348160,"data_stored":129751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":137013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:21:58.152 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:21:58.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 -- 172.21.15.46:0/1047432404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 msgr2=0x7f5e940b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 --2- 172.21.15.46:0/1047432404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940b51f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e94070710 tx=0x7f5e8c01cac0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 -- 172.21.15.46:0/1047432404 shutdown_connections 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 --2- 172.21.15.46:0/1047432404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940b51f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 --2- 172.21.15.46:0/1047432404 >> v2:172.21.15.99:3300/0 conn(0x7f5e941527b0 0x7f5e9415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 -- 172.21.15.46:0/1047432404 >> 172.21.15.46:0/1047432404 conn(0x7f5e940b2040 msgr2=0x7f5e940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 -- 172.21.15.46:0/1047432404 shutdown_connections 2024-09-17T03:21:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.353+0000 7f5e99c00640 1 -- 172.21.15.46:0/1047432404 wait complete. 2024-09-17T03:21:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 Processor -- start 2024-09-17T03:21:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 -- start start 2024-09-17T03:21:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e940b5c70 0x7f5e940cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e941527b0 0x7f5e941f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e94168920 con 0x7f5e940b4de0 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941687a0 con 0x7f5e940b5c70 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5e94168aa0 con 0x7f5e941527b0 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46226/0 (socket says 172.21.15.46:46226) 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 -- 172.21.15.46:0/2843078524 learned_addr learned my addr 172.21.15.46:0/2843078524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 -- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e941527b0 msgr2=0x7f5e941f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e941527b0 0x7f5e941f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 -- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e940b5c70 msgr2=0x7f5e940cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e8be00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e940b5c70 0x7f5e940cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e940b5c70 0x7f5e940cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 -- 172.21.15.46:0/2843078524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94157460 con 0x7f5e940b4de0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e8be00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e940b5c70 0x7f5e940cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e92a00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940ceda0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c760 tx=0x7f5e8400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84016020 con 0x7f5e940b4de0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e8400a950 con 0x7f5e940b4de0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400ada0 con 0x7f5e940b4de0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e94157740 con 0x7f5e940b4de0 2024-09-17T03:21:58.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.357+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e941f6df0 con 0x7f5e940b4de0 2024-09-17T03:21:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.361+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e940b51f0 con 0x7f5e940b4de0 2024-09-17T03:21:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.361+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e84019020 con 0x7f5e940b4de0 2024-09-17T03:21:58.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.361+0000 7f5e8aa00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e70073200 0x7f5e70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.361+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5e840941e0 con 0x7f5e940b4de0 2024-09-17T03:21:58.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.361+0000 7f5e8be00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e70073200 0x7f5e70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.361+0000 7f5e8be00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e70073200 0x7f5e70075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c005fd0 tx=0x7f5e7c005de0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:58.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.365+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e84020020 con 0x7f5e940b4de0 2024-09-17T03:21:58.458 INFO:tasks.ceph.mds.d.smithi099.stderr:2024-09-17T03:21:58.455+0000 7fbc39000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:21:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.605+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5e940cc710 con 0x7f5e940b4de0 2024-09-17T03:21:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e8aa00640 1 -- 172.21.15.46:0/2843078524 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) ==== 190+0+147 (secure 0 0 0) 0x7f5e940b5730 con 0x7f5e940b4de0 2024-09-17T03:21:58.610 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:21:58.610 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:21:58.610 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:21:58.610 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:21:58.610 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:21:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e70073200 msgr2=0x7f5e70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e70073200 0x7f5e70075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c005fd0 tx=0x7f5e7c005de0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 msgr2=0x7f5e940ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940ceda0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c760 tx=0x7f5e8400cc20 comp rx=0 tx=0).stop 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 shutdown_connections 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e70073200 0x7f5e70075710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e941527b0 0x7f5e941f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e940b5c70 0x7f5e940cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 --2- 172.21.15.46:0/2843078524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e940b4de0 0x7f5e940ceda0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 >> 172.21.15.46:0/2843078524 conn(0x7f5e940b2040 msgr2=0x7f5e941511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 shutdown_connections 2024-09-17T03:21:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.609+0000 7f5e99c00640 1 -- 172.21.15.46:0/2843078524 wait complete. 2024-09-17T03:21:58.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 -- 172.21.15.46:0/1800499152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 msgr2=0x7f5cdc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 --2- 172.21.15.46:0/1800499152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc15bb60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc070710 tx=0x7f5ccc01c740 comp rx=0 tx=0).stop 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 -- 172.21.15.46:0/1800499152 shutdown_connections 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 --2- 172.21.15.46:0/1800499152 >> v2:172.21.15.99:3300/0 conn(0x7f5cdc1527b0 0x7f5cdc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 --2- 172.21.15.46:0/1800499152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc15bb60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 -- 172.21.15.46:0/1800499152 >> 172.21.15.46:0/1800499152 conn(0x7f5cdc0b2040 msgr2=0x7f5cdc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 -- 172.21.15.46:0/1800499152 shutdown_connections 2024-09-17T03:21:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.789+0000 7f5ce2800640 1 -- 172.21.15.46:0/1800499152 wait complete. 2024-09-17T03:21:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 Processor -- start 2024-09-17T03:21:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- start start 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cdc0b4de0 0x7f5cdc14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cdc1527b0 0x7f5cdc15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc168b20 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1689a0 con 0x7f5cdc0b4de0 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5cdc168ca0 con 0x7f5cdc1527b0 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46250/0 (socket says 172.21.15.46:46250) 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 -- 172.21.15.46:0/650238473 learned_addr learned my addr 172.21.15.46:0/650238473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cdb400640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cdc0b4de0 0x7f5cdc14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cdbe00640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cdc1527b0 0x7f5cdc15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 -- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cdc1527b0 msgr2=0x7f5cdc15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cdc1527b0 0x7f5cdc15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 -- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cdc0b4de0 msgr2=0x7f5cdc14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cdc0b4de0 0x7f5cdc14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 -- 172.21.15.46:0/650238473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc14e7f0 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cdbe00640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cdc1527b0 0x7f5cdc15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3e00640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc149920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d750 tx=0x7f5cc400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4017070 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cc4004310 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc400de60 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc149e60 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc14ef50 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.793+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cdc14e9c0 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.797+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5cc4009c30 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.797+0000 7f5cd3400640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cb8073200 0x7f5cb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.797+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5cc4094660 con 0x7f5cdc0b5c70 2024-09-17T03:21:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.797+0000 7f5cdb400640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cb8073200 0x7f5cb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.797+0000 7f5cdb400640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cb8073200 0x7f5cb8075710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0af000 tx=0x7f5ccc01af80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:58.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:21:58.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:21:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:58.801+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5cc4020020 con 0x7f5cdc0b5c70 2024-09-17T03:21:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cdc15bb60 con 0x7f5cdc0b5c70 2024-09-17T03:21:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5cd3400640 1 -- 172.21.15.46:0/650238473 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5236 (secure 0 0 0) 0x7f5cc405f5e0 con 0x7f5cdc0b5c70 2024-09-17T03:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:21:59.014 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-17T03:21:56:541550+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:52.949184+0000","modified":"2024-09-17T03:21:55.955106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244821":{"gid":244821,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3646914597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3646914597},{"type":"v1","addr":"172.21.15.99:6835","nonce":3646914597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244821,"qdb_cluster":[244821]},"id":2}]} 2024-09-17T03:21:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cb8073200 msgr2=0x7f5cb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cb8073200 0x7f5cb8075710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0af000 tx=0x7f5ccc01af80 comp rx=0 tx=0).stop 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 msgr2=0x7f5cdc149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc149920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d750 tx=0x7f5cc400dc10 comp rx=0 tx=0).stop 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 shutdown_connections 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cb8073200 0x7f5cb8075710 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cdc1527b0 0x7f5cdc15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0b5c70 0x7f5cdc149920 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 --2- 172.21.15.46:0/650238473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cdc0b4de0 0x7f5cdc14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 >> 172.21.15.46:0/650238473 conn(0x7f5cdc0b2040 msgr2=0x7f5cdc0b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 shutdown_connections 2024-09-17T03:21:59.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.013+0000 7f5ce2800640 1 -- 172.21.15.46:0/650238473 wait complete. 2024-09-17T03:21:59.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:21:52.949184+0000', 'modified': '2024-09-17T03:21:55.955106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 244821}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_244821': {'gid': 244821, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/3646914597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6833', 'nonce': 3646914597}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 3646914597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 244821, 'qdb_cluster': [244821]} 2024-09-17T03:21:59.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:21:59.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T03:21:59.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 -- 172.21.15.46:0/2894106058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc1527b0 msgr2=0x7f46fc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 --2- 172.21.15.46:0/2894106058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc1527b0 0x7f46fc074e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46fc072e30 tx=0x7f46e801c800 comp rx=0 tx=0).stop 2024-09-17T03:21:59.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 -- 172.21.15.46:0/2894106058 shutdown_connections 2024-09-17T03:21:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 --2- 172.21.15.46:0/2894106058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc1527b0 0x7f46fc074e30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 --2- 172.21.15.46:0/2894106058 >> v2:172.21.15.99:3300/0 conn(0x7f46fc0b4de0 0x7f46fc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 --2- 172.21.15.46:0/2894106058 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46fc0b5c70 0x7f46fc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 -- 172.21.15.46:0/2894106058 >> 172.21.15.46:0/2894106058 conn(0x7f46fc0b2040 msgr2=0x7f46fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 -- 172.21.15.46:0/2894106058 shutdown_connections 2024-09-17T03:21:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.605+0000 7f4702a00640 1 -- 172.21.15.46:0/2894106058 wait complete. 2024-09-17T03:21:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 Processor -- start 2024-09-17T03:21:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 -- start start 2024-09-17T03:21:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46fc0b4de0 0x7f46fc1fc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 0x7f46fc153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46fc1527b0 0x7f46fc149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc15f700 con 0x7f46fc0b5c70 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc15f580 con 0x7f46fc0b4de0 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4702a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc15f880 con 0x7f46fc1527b0 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 0x7f46fc153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 0x7f46fc153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46254/0 (socket says 172.21.15.46:46254) 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 -- 172.21.15.46:0/284434788 learned_addr learned my addr 172.21.15.46:0/284434788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fbe00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46fc0b4de0 0x7f46fc1fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f4700a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46fc1527b0 0x7f46fc149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 -- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46fc1527b0 msgr2=0x7f46fc149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46fc1527b0 0x7f46fc149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 -- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46fc0b4de0 msgr2=0x7f46fc1fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46fc0b4de0 0x7f46fc1fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.609+0000 7f46fb400640 1 -- 172.21.15.46:0/284434788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc152cc0 con 0x7f46fc0b5c70 2024-09-17T03:21:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.613+0000 7f46fb400640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 0x7f46fc153730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f46e801aeb0 tx=0x7f46e80a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.613+0000 7f46f3e00640 1 -- 172.21.15.46:0/284434788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80ba020 con 0x7f46fc0b5c70 2024-09-17T03:21:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.613+0000 7f46f3e00640 1 -- 172.21.15.46:0/284434788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e8004430 con 0x7f46fc0b5c70 2024-09-17T03:21:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.613+0000 7f46f3e00640 1 -- 172.21.15.46:0/284434788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80b2040 con 0x7f46fc0b5c70 2024-09-17T03:21:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.613+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc149c60 con 0x7f46fc0b5c70 2024-09-17T03:21:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.613+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc152f30 con 0x7f46fc0b5c70 2024-09-17T03:21:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46f3e00640 1 -- 172.21.15.46:0/284434788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46e8005290 con 0x7f46fc0b5c70 2024-09-17T03:21:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 Processor -- start 2024-09-17T03:21:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46f3e00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46dc073200 0x7f46dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 -- start start 2024-09-17T03:21:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46f3e00640 1 -- 172.21.15.46:0/284434788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f46e812e0e0 con 0x7f46fc0b5c70 2024-09-17T03:21:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f4700a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46dc073200 0x7f46dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f4700a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46dc073200 0x7f46dc075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46f4007d30 tx=0x7f46f4007460 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98a927460 0x55c98ae686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad17950 0x55c98ae68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 0x55c98ae69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c98ad3d410 con 0x55c98ad00ca0 2024-09-17T03:21:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c98ac7ab40 con 0x55c98ad17950 2024-09-17T03:21:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55c98ac69f70 con 0x55c98a927460 2024-09-17T03:21:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f4700a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 0x55c98ae69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98a927460 0x55c98ae686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fb400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad17950 0x55c98ae68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f4700a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 0x55c98ae69120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46262/0 (socket says 172.21.15.46:46262) 2024-09-17T03:21:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f4700a00640 1 -- 172.21.15.46:0/3488004245 learned_addr learned my addr 172.21.15.46:0/3488004245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fb400640 1 -- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98a927460 msgr2=0x55c98ae686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fb400640 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98a927460 0x55c98ae686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fb400640 1 -- 172.21.15.46:0/3488004245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 msgr2=0x55c98ae69120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fb400640 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 0x55c98ae69120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f46fb400640 1 -- 172.21.15.46:0/3488004245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98ac315b0 con 0x55c98ad17950 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.617+0000 7f4700a00640 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 0x55c98ae69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f46fb400640 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad17950 0x55c98ae68be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f46e812c6f0 tx=0x7f46e80f53d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f383bbd0cd50805 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f46e7e00640 1 -- 172.21.15.46:0/3488004245 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e80f8c50 con 0x55c98ad17950 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f46e7e00640 1 -- 172.21.15.46:0/3488004245 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e80fd600 con 0x55c98ad17950 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f46e7e00640 1 -- 172.21.15.46:0/3488004245 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e81024b0 con 0x55c98ad17950 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad17950 msgr2=0x55c98ae68be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad17950 0x55c98ae68be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f46e812c6f0 tx=0x7f46e80f53d0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- 172.21.15.46:0/3488004245 shutdown_connections 2024-09-17T03:21:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad00ca0 0x55c98ae69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad17950 0x55c98ae68be0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- 172.21.15.46:0/3488004245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98a927460 0x55c98ae686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- 172.21.15.46:0/3488004245 >> 172.21.15.46:0/3488004245 conn(0x55c98adedd90 msgr2=0x55c98ae1d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- 172.21.15.46:0/3488004245 shutdown_connections 2024-09-17T03:21:59.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- 172.21.15.46:0/3488004245 wait complete. 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- start start 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 Processor -- start 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98ae6f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98ae6fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98ae703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c98ac045f0 con 0x55c98a927460 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c98ac6dd80 con 0x55c98ad00ca0 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.621+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55c98ac10520 con 0x55c98ad17950 2024-09-17T03:21:59.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 10 client.0 _ll_register_callbacks cb 0x55c98ac14830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98ae6f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98ae6f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46272/0 (socket says 172.21.15.46:46272) 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 -- 172.21.15.46:0/1824626611 learned_addr learned my addr 172.21.15.46:0/1824626611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f4700a00640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98ae703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fb400640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98ae6fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 -- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 msgr2=0x55c98ae703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98ae703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 -- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 msgr2=0x55c98ae6fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98ae6fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 -- 172.21.15.46:0/1824626611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98ac08190 con 0x55c98a927460 2024-09-17T03:21:59.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fbe00640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98ae6f920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b9f0 tx=0x7f46ec00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628bd6be8b513be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:21:59.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 1 -- 172.21.15.46:0/1824626611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec003e90 con 0x55c98a927460 2024-09-17T03:21:59.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 1 -- 172.21.15.46:0/1824626611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f46ec01cad0 con 0x55c98a927460 2024-09-17T03:21:59.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c98ac02110 con 0x55c98a927460 2024-09-17T03:21:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 1 -- 172.21.15.46:0/1824626611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f46ec01ccd0 con 0x55c98a927460 2024-09-17T03:21:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 10 client.244995 fetch_fsmap learned FSMap version 17 2024-09-17T03:21:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c98aceef10 con 0x55c98a927460 2024-09-17T03:21:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 1 -- 172.21.15.46:0/1824626611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c98ac02110 con 0x55c98a927460 2024-09-17T03:21:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 1 -- 172.21.15.46:0/1824626611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f46ec023050 con 0x55c98a927460 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 10 client.244995 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 10 client.244995 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 20 client.244995 populate_metadata read hostname 'smithi046' 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a90c7d0 0x55c98ae79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 4 client.244995 mds_command: new command op to 244821 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 --> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c98ac7ca90 con 0x55c98a90c7d0 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fb400640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a90c7d0 0x55c98ae79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46fb400640 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a90c7d0 0x55c98ae79d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b4b274df8f954 server_cookie=9787f0ad5d76c1a1 in_seq=0 out_seq=0 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.625+0000 7f46e4c00640 10 client.244995 ms_handle_connect on v2:172.21.15.99:6833/3646914597 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f46e4c00640 1 -- 172.21.15.46:0/1824626611 <== mds.0 v2:172.21.15.99:6833/3646914597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c98ac7ca90 con 0x55c98a90c7d0 2024-09-17T03:21:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f46e4c00640 10 client.244995 handle_command_reply: tid=0 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 client.244995 shutdown 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 msgr2=0x55c98ae6f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98ae6f920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b9f0 tx=0x7f46ec00beb0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 shutdown_connections 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a90c7d0 0x55c98ae79d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98ae703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98ae6fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1824626611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98ae6f920 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f46e4c00640 0 client.244995 ms_handle_reset on v2:172.21.15.99:6833/3646914597 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 >> 172.21.15.46:0/1824626611 conn(0x55c98adedd90 msgr2=0x55c98ae24440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 shutdown_connections 2024-09-17T03:21:59.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 1 -- 172.21.15.46:0/1824626611 wait complete. 2024-09-17T03:21:59.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.629+0000 7f470a0cf000 20 client.244995 trim_cache size 0 max 16384 2024-09-17T03:21:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 Processor -- start 2024-09-17T03:21:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 -- start start 2024-09-17T03:21:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98aeda620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98aedab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98aedb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc205160 con 0x55c98a927460 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc204140 con 0x55c98ad00ca0 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc1f8b90 con 0x55c98ad17950 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98aeda620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98aeda620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46286/0 (socket says 172.21.15.46:46286) 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 -- 172.21.15.46:0/4200921964 learned_addr learned my addr 172.21.15.46:0/4200921964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 -- 172.21.15.46:0/4200921964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 msgr2=0x55c98aedb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98aedb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 -- 172.21.15.46:0/4200921964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 msgr2=0x55c98aedab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98aedab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 -- 172.21.15.46:0/4200921964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98acc5db0 con 0x55c98a927460 2024-09-17T03:21:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46fbe00640 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98aeda620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46ec001ec0 tx=0x7f46ec003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c39febe362031b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46e7400640 1 -- 172.21.15.46:0/4200921964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec005020 con 0x55c98a927460 2024-09-17T03:21:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46e7400640 1 -- 172.21.15.46:0/4200921964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec0051c0 con 0x55c98a927460 2024-09-17T03:21:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.649+0000 7f46e7400640 1 -- 172.21.15.46:0/4200921964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0055d0 con 0x55c98a927460 2024-09-17T03:21:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- 172.21.15.46:0/4200921964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 msgr2=0x55c98aeda620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98aeda620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46ec001ec0 tx=0x7f46ec003a90 comp rx=0 tx=0).stop 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- 172.21.15.46:0/4200921964 shutdown_connections 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad17950 0x55c98aedb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98ad00ca0 0x55c98aedab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- 172.21.15.46:0/4200921964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98a927460 0x55c98aeda620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- 172.21.15.46:0/4200921964 >> 172.21.15.46:0/4200921964 conn(0x55c98adedd90 msgr2=0x55c98ae66620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- 172.21.15.46:0/4200921964 shutdown_connections 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- 172.21.15.46:0/4200921964 wait complete. 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- start start 2024-09-17T03:21:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 Processor -- start 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98a927460 0x55c98aede390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad00ca0 0x55c98aede8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 0x55c98aedee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c98ac6d090 con 0x55c98ad17950 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c98ac79910 con 0x55c98a927460 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55c98ac7ab40 con 0x55c98ad00ca0 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f470a0cf000 10 client.0 _ll_register_callbacks cb 0x7f46ec004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 0x55c98aedee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 0x55c98aedee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46296/0 (socket says 172.21.15.46:46296) 2024-09-17T03:21:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 -- 172.21.15.46:0/1745374288 learned_addr learned my addr 172.21.15.46:0/1745374288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f46fbe00640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98a927460 0x55c98aede390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f46fb400640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad00ca0 0x55c98aede8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 -- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad00ca0 msgr2=0x55c98aede8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad00ca0 0x55c98aede8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 -- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98a927460 msgr2=0x55c98aede390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98a927460 0x55c98aede390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 -- 172.21.15.46:0/1745374288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98ac30ab0 con 0x55c98ad17950 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f4700a00640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 0x55c98aedee10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f46f4004080 tx=0x7f46f4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30e9b979bda34bd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.653+0000 7f46e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 1 -- 172.21.15.46:0/1745374288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4011440 con 0x55c98ad17950 2024-09-17T03:21:59.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 1 -- 172.21.15.46:0/1745374288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f46f40115e0 con 0x55c98ad17950 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ec0051c0 con 0x55c98ad17950 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 1 -- 172.21.15.46:0/1745374288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f46f4032050 con 0x55c98ad17950 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 10 client.245001 fetch_fsmap learned FSMap version 17 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c98ac0f070 con 0x55c98ad17950 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 1 -- 172.21.15.46:0/1745374288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec0051c0 con 0x55c98ad17950 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 1 -- 172.21.15.46:0/1745374288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f46f4035020 con 0x55c98ad17950 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 10 client.245001 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 10 client.245001 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 20 client.245001 populate_metadata read hostname 'smithi046' 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a8fa830 0x55c98ae9b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 4 client.245001 mds_command: new command op to 244821 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:21:59.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 --> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] -- command(tid 0: {"prefix": "status"}) -- 0x55c98ac78540 con 0x55c98a8fa830 2024-09-17T03:21:59.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46fbe00640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a8fa830 0x55c98ae9b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46fbe00640 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a8fa830 0x55c98ae9b590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796f4ab88d69c7a3 server_cookie=28f85786ec940f59 in_seq=0 out_seq=0 2024-09-17T03:21:59.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 10 client.245001 ms_handle_connect on v2:172.21.15.99:6833/3646914597 2024-09-17T03:21:59.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 1 -- 172.21.15.46:0/1745374288 <== mds.0 v2:172.21.15.99:6833/3646914597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55c98ac78540 con 0x55c98a8fa830 2024-09-17T03:21:59.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 10 client.245001 handle_command_reply: tid=0 2024-09-17T03:21:59.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 client.245001 shutdown 2024-09-17T03:21:59.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 msgr2=0x55c98aedee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 0x55c98aedee10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f46f4004080 tx=0x7f46f4004a50 comp rx=0 tx=0).stop 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 shutdown_connections 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:6833/3646914597,v1:172.21.15.99:6835/3646914597] conn(0x55c98a8fa830 0x55c98ae9b590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c98ad17950 0x55c98aedee10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55c98ad00ca0 0x55c98aede8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f46e4c00640 0 client.245001 ms_handle_reset on v2:172.21.15.99:6833/3646914597 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 --2- 172.21.15.46:0/1745374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c98a927460 0x55c98aede390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 >> 172.21.15.46:0/1745374288 conn(0x55c98adedd90 msgr2=0x55c98ae66fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 shutdown_connections 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 1 -- 172.21.15.46:0/1745374288 wait complete. 2024-09-17T03:21:59.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f470a0cf000 20 client.245001 trim_cache size 0 max 16384 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 244821, 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.142275379, 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 17, 2024-09-17T03:21:59.664 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 32, 2024-09-17T03:21:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 32, 2024-09-17T03:21:59.665 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 13.000408953999999 2024-09-17T03:21:59.665 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46dc073200 msgr2=0x7f46dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46dc073200 0x7f46dc075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46f4007d30 tx=0x7f46f4007460 comp rx=0 tx=0).stop 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 msgr2=0x7f46fc153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 0x7f46fc153730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f46e801aeb0 tx=0x7f46e80a5790 comp rx=0 tx=0).stop 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 shutdown_connections 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f46dc073200 0x7f46dc075710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46fc1527b0 0x7f46fc149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46fc0b5c70 0x7f46fc153730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 --2- 172.21.15.46:0/284434788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46fc0b4de0 0x7f46fc1fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 >> 172.21.15.46:0/284434788 conn(0x7f46fc0b2040 msgr2=0x7f46fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 shutdown_connections 2024-09-17T03:21:59.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.657+0000 7f4702a00640 1 -- 172.21.15.46:0/284434788 wait complete. 2024-09-17T03:21:59.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:21:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 -- 172.21.15.46:0/2872121343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 msgr2=0x7f36bc073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 --2- 172.21.15.46:0/2872121343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc073dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36bc15f0b0 tx=0x7f36ac01c810 comp rx=0 tx=0).stop 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 -- 172.21.15.46:0/2872121343 shutdown_connections 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 --2- 172.21.15.46:0/2872121343 >> v2:172.21.15.99:3300/0 conn(0x7f36bc15f6b0 0x7f36bc15fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 --2- 172.21.15.46:0/2872121343 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f36bc074300 0x7f36bc15f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 --2- 172.21.15.46:0/2872121343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc073dc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 -- 172.21.15.46:0/2872121343 >> 172.21.15.46:0/2872121343 conn(0x7f36bc0b2c40 msgr2=0x7f36bc0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 -- 172.21.15.46:0/2872121343 shutdown_connections 2024-09-17T03:21:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 -- 172.21.15.46:0/2872121343 wait complete. 2024-09-17T03:21:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 Processor -- start 2024-09-17T03:21:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.837+0000 7f36c3000640 1 -- start start 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f36bc074300 0x7f36bc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36bc15f6b0 0x7f36bc152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc073640 con 0x7f36bc15c5d0 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc0734c0 con 0x7f36bc15f6b0 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f36bc0737c0 con 0x7f36bc074300 2024-09-17T03:21:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc0af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46320/0 (socket says 172.21.15.46:46320) 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 -- 172.21.15.46:0/3160042141 learned_addr learned my addr 172.21.15.46:0/3160042141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c1000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36bc15f6b0 0x7f36bc152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36bbe00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f36bc074300 0x7f36bc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 -- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f36bc074300 msgr2=0x7f36bc15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f36bc074300 0x7f36bc15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 -- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36bc15f6b0 msgr2=0x7f36bc152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36bc15f6b0 0x7f36bc152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 -- 172.21.15.46:0/3160042141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc14ea50 con 0x7f36bc15c5d0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b3e00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc0af720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c970 tx=0x7f36a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4007bb0 con 0x7f36bc15c5d0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a4007e40 con 0x7f36bc15c5d0 2024-09-17T03:21:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a400f6b0 con 0x7f36bc15c5d0 2024-09-17T03:21:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc0aa600 con 0x7f36bc15c5d0 2024-09-17T03:21:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1fc820 con 0x7f36bc15c5d0 2024-09-17T03:21:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.841+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc1fcb00 con 0x7f36bc15c5d0 2024-09-17T03:21:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.845+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36a400f850 con 0x7f36bc15c5d0 2024-09-17T03:21:59.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.845+0000 7f36b8c00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3698073200 0x7f3698075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:21:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.845+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f36a4094890 con 0x7f36bc15c5d0 2024-09-17T03:21:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.845+0000 7f36bbe00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3698073200 0x7f3698075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:21:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.845+0000 7f36bbe00640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3698073200 0x7f3698075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0af000 tx=0x7f36ac01b0b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:21:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:21:59.849+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36a4014030 con 0x7f36bc15c5d0 2024-09-17T03:22:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.061+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36bc1538d0 con 0x7f36bc15c5d0 2024-09-17T03:22:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.061+0000 7f36b8c00640 1 -- 172.21.15.46:0/3160042141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f36a405f2e0 con 0x7f36bc15c5d0 2024-09-17T03:22:00.063 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:22:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3698073200 msgr2=0x7f3698075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:00.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3698073200 0x7f3698075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0af000 tx=0x7f36ac01b0b0 comp rx=0 tx=0).stop 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 msgr2=0x7f36bc0af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc0af720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c970 tx=0x7f36a400ce30 comp rx=0 tx=0).stop 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 shutdown_connections 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3698073200 0x7f3698075710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36bc15f6b0 0x7f36bc152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc15c5d0 0x7f36bc0af720 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 --2- 172.21.15.46:0/3160042141 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f36bc074300 0x7f36bc15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 >> 172.21.15.46:0/3160042141 conn(0x7f36bc0b2c40 msgr2=0x7f36bc06fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 shutdown_connections 2024-09-17T03:22:00.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00.065+0000 7f36c3000640 1 -- 172.21.15.46:0/3160042141 wait complete. 2024-09-17T03:22:00.085 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:22:00.085 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:22:00.085 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:22:00.085 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:22:00.085 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:22:00.085 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:22:00.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:22:00.086 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:22:00.086 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:22:00.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.086 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:22:00.095 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:22:00.096 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42369sec preferred_lft 42369sec 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:22:00.097 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:22:00.098 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:22:00.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.099 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:22:00.158 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:22:00.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.160 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:22:00.218 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:22:00.218 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:22:00.218 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:22:00.218 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:22:00.218 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:22:00.218 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:22:00.219 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:22:00.219 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:22:00.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:22:00.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:22:00.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:22:00.219 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:22:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:22:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:22:00.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.358 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:22:00.367 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:22:00.368 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:22:00.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:22:00.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:22:00.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:22:00.436 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:22:00.485 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:22:00.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:22:00.485 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:22:00.536 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:22:00.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:22:00.643 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:22:00.643 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:22:00.657 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:22:00.657 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:22:00.658 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:22:00.658 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:22:00.658 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:22:00.661 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:22:00.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:00.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:22:00.682 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:22:00.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:22:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:22:00.791 INFO:teuthology.orchestra.run.smithi046.stdout:client244712 2024-09-17T03:22:00.792 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:00.792 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:00.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:22:01.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 -- 172.21.15.46:0/738830874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114157030 msgr2=0x7ff1140b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 --2- 172.21.15.46:0/738830874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114157030 0x7ff1140b5a80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff1140b6a50 tx=0x7ff10401c830 comp rx=0 tx=0).stop 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 -- 172.21.15.46:0/738830874 shutdown_connections 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 --2- 172.21.15.46:0/738830874 >> v2:172.21.15.99:3300/0 conn(0x7ff114074b20 0x7ff11415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 --2- 172.21.15.46:0/738830874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff114157030 0x7ff1140b5a80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 -- 172.21.15.46:0/738830874 >> 172.21.15.46:0/738830874 conn(0x7ff1140708c0 msgr2=0x7ff114070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 -- 172.21.15.46:0/738830874 shutdown_connections 2024-09-17T03:22:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.005+0000 7ff11ae00640 1 -- 172.21.15.46:0/738830874 wait complete. 2024-09-17T03:22:01.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 Processor -- start 2024-09-17T03:22:01.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 -- start start 2024-09-17T03:22:01.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff114157030 0x7ff11413dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff114160d50 0x7ff114155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 0x7ff114155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff11416a740 con 0x7ff11414acf0 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff11416a5c0 con 0x7ff114160d50 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff11416a8c0 con 0x7ff114157030 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 0x7ff114155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 0x7ff114155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46346/0 (socket says 172.21.15.46:46346) 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 -- 172.21.15.46:0/2330443584 learned_addr learned my addr 172.21.15.46:0/2330443584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff10be00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff114160d50 0x7ff114155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff113e00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff114157030 0x7ff11413dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 -- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff114157030 msgr2=0x7ff11413dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff114157030 0x7ff11413dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 -- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff114160d50 msgr2=0x7ff114155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff114160d50 0x7ff114155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 -- 172.21.15.46:0/2330443584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11414b4f0 con 0x7ff11414acf0 2024-09-17T03:22:01.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff10be00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff114160d50 0x7ff114155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff118e00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 0x7ff114155a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00e970 tx=0x7ff10c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c004d30 con 0x7ff11414acf0 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff10c00cb20 con 0x7ff11414acf0 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1141fcbf0 con 0x7ff11414acf0 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c005020 con 0x7ff11414acf0 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.009+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141fd1b0 con 0x7ff11414acf0 2024-09-17T03:22:01.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.013+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1141fc960 con 0x7ff11414acf0 2024-09-17T03:22:01.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.013+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff10c0051c0 con 0x7ff11414acf0 2024-09-17T03:22:01.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.013+0000 7ff110c00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f0073200 0x7ff0f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.013+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff10c094a60 con 0x7ff11414acf0 2024-09-17T03:22:01.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.013+0000 7ff113e00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f0073200 0x7ff0f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.017+0000 7ff113e00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f0073200 0x7ff0f0075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1040af000 tx=0x7ff10401b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.017+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff10c016030 con 0x7ff11414acf0 2024-09-17T03:22:01.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.233+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff11415cf90 con 0x7ff11414acf0 2024-09-17T03:22:01.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff110c00640 1 -- 172.21.15.46:0/2330443584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7ff10c05f4c0 con 0x7ff11414acf0 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f0073200 msgr2=0x7ff0f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f0073200 0x7ff0f0075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1040af000 tx=0x7ff10401b070 comp rx=0 tx=0).stop 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 msgr2=0x7ff114155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 0x7ff114155a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00e970 tx=0x7ff10c00ee30 comp rx=0 tx=0).stop 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 shutdown_connections 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f0073200 0x7ff0f0075710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11414acf0 0x7ff114155a60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff114160d50 0x7ff114155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 --2- 172.21.15.46:0/2330443584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff114157030 0x7ff11413dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.616+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 >> 172.21.15.46:0/2330443584 conn(0x7ff1140708c0 msgr2=0x7ff11415f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.620+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 shutdown_connections 2024-09-17T03:22:01.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.620+0000 7ff11ae00640 1 -- 172.21.15.46:0/2330443584 wait complete. 2024-09-17T03:22:01.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:22:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 -- 172.21.15.46:0/217577609 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0b54f0 msgr2=0x7fe69c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 --2- 172.21.15.46:0/217577609 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0b54f0 0x7fe69c15bb60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe694009bf0 tx=0x7fe69401cbb0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 -- 172.21.15.46:0/217577609 shutdown_connections 2024-09-17T03:22:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 --2- 172.21.15.46:0/217577609 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0b54f0 0x7fe69c15bb60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 -- 172.21.15.46:0/217577609 >> 172.21.15.46:0/217577609 conn(0x7fe69c0b2040 msgr2=0x7fe69c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 -- 172.21.15.46:0/217577609 shutdown_connections 2024-09-17T03:22:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.800+0000 7fe6a4000640 1 -- 172.21.15.46:0/217577609 wait complete. 2024-09-17T03:22:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 Processor -- start 2024-09-17T03:22:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 -- start start 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 0x7fe69c151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69c1527b0 0x7fe69c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0c57f0 0x7fe69c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c169bd0 con 0x7fe69c0b5c70 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c169a50 con 0x7fe69c1527b0 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c169d50 con 0x7fe69c0c57f0 2024-09-17T03:22:01.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 0x7fe69c151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 0x7fe69c151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46356/0 (socket says 172.21.15.46:46356) 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 -- 172.21.15.46:0/3425804728 learned_addr learned my addr 172.21.15.46:0/3425804728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a0c00640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69c1527b0 0x7fe69c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a2000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0c57f0 0x7fe69c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 -- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0c57f0 msgr2=0x7fe69c0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0c57f0 0x7fe69c0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 -- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69c1527b0 msgr2=0x7fe69c0c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69c1527b0 0x7fe69c0c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 -- 172.21.15.46:0/3425804728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c158670 con 0x7fe69c0b5c70 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a1600640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 0x7fe69c151e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c430 tx=0x7fe69000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690017070 con 0x7fe69c0b5c70 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6900053c0 con 0x7fe69c0b5c70 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c158950 con 0x7fe69c0b5c70 2024-09-17T03:22:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690003860 con 0x7fe69c0b5c70 2024-09-17T03:22:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.804+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c158f10 con 0x7fe69c0b5c70 2024-09-17T03:22:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.808+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c15bb60 con 0x7fe69c0b5c70 2024-09-17T03:22:01.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.808+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe69001a070 con 0x7fe69c0b5c70 2024-09-17T03:22:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.808+0000 7fe699600640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe67c073230 0x7fe67c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.808+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe6900935c0 con 0x7fe69c0b5c70 2024-09-17T03:22:01.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.808+0000 7fe6a0c00640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe67c073230 0x7fe67c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:01.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.808+0000 7fe6a0c00640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe67c073230 0x7fe67c075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe688007f30 tx=0x7fe688007ec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:01.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:01.812+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe69005df00 con 0x7fe69c0b5c70 2024-09-17T03:22:02.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.008+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe69c06b540 con 0x7fe67c073230 2024-09-17T03:22:02.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe699600640 1 -- 172.21.15.46:0/3425804728 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe6880014a0 con 0x7fe67c073230 2024-09-17T03:22:02.013 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:22:02.013 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:22:02.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:22:02.014 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:22:02.014 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:22:02.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe67c073230 msgr2=0x7fe67c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe67c073230 0x7fe67c075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe688007f30 tx=0x7fe688007ec0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 msgr2=0x7fe69c151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 0x7fe69c151e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c430 tx=0x7fe69000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 shutdown_connections 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe67c073230 0x7fe67c075740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe69c0c57f0 0x7fe69c0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69c1527b0 0x7fe69c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 --2- 172.21.15.46:0/3425804728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe69c0b5c70 0x7fe69c151e10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 >> 172.21.15.46:0/3425804728 conn(0x7fe69c0b2040 msgr2=0x7fe69c159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 shutdown_connections 2024-09-17T03:22:02.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.012+0000 7fe6a4000640 1 -- 172.21.15.46:0/3425804728 wait complete. 2024-09-17T03:22:02.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:22:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 -- 172.21.15.46:0/2994063105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000160d50 msgr2=0x7f8000074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 --2- 172.21.15.46:0/2994063105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000160d50 0x7f8000074410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009840 tx=0x7f7fe801c820 comp rx=0 tx=0).stop 2024-09-17T03:22:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 -- 172.21.15.46:0/2994063105 shutdown_connections 2024-09-17T03:22:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 --2- 172.21.15.46:0/2994063105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000160d50 0x7f8000074410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 --2- 172.21.15.46:0/2994063105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8000157030 0x7f80000b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 -- 172.21.15.46:0/2994063105 >> 172.21.15.46:0/2994063105 conn(0x7f80000708c0 msgr2=0x7f8000070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:02.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 -- 172.21.15.46:0/2994063105 shutdown_connections 2024-09-17T03:22:02.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 -- 172.21.15.46:0/2994063105 wait complete. 2024-09-17T03:22:02.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 Processor -- start 2024-09-17T03:22:02.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.196+0000 7f8004a00640 1 -- start start 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000157030 0x7f80001564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8000160d50 0x7f80001f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 0x7f80001fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f800016a7b0 con 0x7f8000161160 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f800016a630 con 0x7f8000157030 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f800016a930 con 0x7f8000160d50 2024-09-17T03:22:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 0x7f80001fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 0x7f80001fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46366/0 (socket says 172.21.15.46:46366) 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 -- 172.21.15.46:0/3322827414 learned_addr learned my addr 172.21.15.46:0/3322827414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffde00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000157030 0x7f80001564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ff7e00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8000160d50 0x7f80001f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 -- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8000160d50 msgr2=0x7f80001f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8000160d50 0x7f80001f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 -- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000157030 msgr2=0x7f80001564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000157030 0x7f80001564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 -- 172.21.15.46:0/3322827414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80001fa100 con 0x7f8000161160 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ffe800640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 0x7f80001fd690 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c760 tx=0x7f7ff800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8003b80 con 0x7f8000161160 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff8003d40 con 0x7f8000161160 2024-09-17T03:22:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800aa50 con 0x7f8000161160 2024-09-17T03:22:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80001fa3e0 con 0x7f8000161160 2024-09-17T03:22:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800006ddf0 con 0x7f8000161160 2024-09-17T03:22:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.200+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80000d2260 con 0x7f8000161160 2024-09-17T03:22:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.204+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7ff800abf0 con 0x7f8000161160 2024-09-17T03:22:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.204+0000 7f7ff6000640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fdc073200 0x7f7fdc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.204+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff8094060 con 0x7f8000161160 2024-09-17T03:22:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.204+0000 7f7ffde00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fdc073200 0x7f7fdc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.204+0000 7f7ffde00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fdc073200 0x7f7fdc075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00096c0 tx=0x7f7ff0009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.208+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7ff805e910 con 0x7f8000161160 2024-09-17T03:22:02.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f80001fa5b0 con 0x7f8000161160 2024-09-17T03:22:02.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f7ff6000640 1 -- 172.21.15.46:0/3322827414 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f7ff8063340 con 0x7f8000161160 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fdc073200 msgr2=0x7f7fdc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fdc073200 0x7f7fdc075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00096c0 tx=0x7f7ff0009260 comp rx=0 tx=0).stop 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 msgr2=0x7f80001fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 0x7f80001fd690 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c760 tx=0x7f7ff800cc20 comp rx=0 tx=0).stop 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 shutdown_connections 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fdc073200 0x7f7fdc075710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8000161160 0x7f80001fd690 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8000160d50 0x7f80001f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 --2- 172.21.15.46:0/3322827414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8000157030 0x7f80001564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 >> 172.21.15.46:0/3322827414 conn(0x7f80000708c0 msgr2=0x7f800015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:02.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 shutdown_connections 2024-09-17T03:22:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.384+0000 7f8004a00640 1 -- 172.21.15.46:0/3322827414 wait complete. 2024-09-17T03:22:02.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 -- 172.21.15.46:0/155583518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc004810 msgr2=0x7f48fc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 --2- 172.21.15.46:0/155583518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48fc004810 0x7f48fc004c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4900001860 tx=0x7f490001f050 comp rx=0 tx=0).stop 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491d000640 1 -- 172.21.15.46:0/155583518 reap_dead start 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 -- 172.21.15.46:0/155583518 shutdown_connections 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 -- 172.21.15.46:0/155583518 >> 172.21.15.46:0/155583518 conn(0x7f49180b2040 msgr2=0x7f49180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 -- 172.21.15.46:0/155583518 shutdown_connections 2024-09-17T03:22:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 -- 172.21.15.46:0/155583518 wait complete. 2024-09-17T03:22:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 Processor -- start 2024-09-17T03:22:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.568+0000 7f491f000640 1 -- start start 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48fc004810 0x7f49181e4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49181546b0 0x7f49180cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 0x7f49181f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49180b3910 con 0x7f4918154ac0 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49180b3790 con 0x7f49181546b0 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f49180b3a90 con 0x7f48fc004810 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 0x7f49181f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 0x7f49181f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58582/0 (socket says 172.21.15.46:58582) 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 -- 172.21.15.46:0/1959679599 learned_addr learned my addr 172.21.15.46:0/1959679599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f490fe00640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49181546b0 0x7f49180cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f4917e00640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48fc004810 0x7f49181e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 -- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48fc004810 msgr2=0x7f49181e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48fc004810 0x7f49181e4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 -- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49181546b0 msgr2=0x7f49180cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49181546b0 0x7f49180cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 -- 172.21.15.46:0/1959679599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49181e4c40 con 0x7f4918154ac0 2024-09-17T03:22:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491d000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 0x7f49181f5a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49180719a0 tx=0x7f491000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910003980 con 0x7f4918154ac0 2024-09-17T03:22:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4910003b20 con 0x7f4918154ac0 2024-09-17T03:22:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910004030 con 0x7f4918154ac0 2024-09-17T03:22:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49180ce3a0 con 0x7f4918154ac0 2024-09-17T03:22:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49180ce660 con 0x7f4918154ac0 2024-09-17T03:22:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.572+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49181f6560 con 0x7f4918154ac0 2024-09-17T03:22:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.576+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f49100041d0 con 0x7f4918154ac0 2024-09-17T03:22:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.576+0000 7f4914c00640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48f4073200 0x7f48f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.576+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4910094a60 con 0x7f4918154ac0 2024-09-17T03:22:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.576+0000 7f4917e00640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48f4073200 0x7f48f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.576+0000 7f4917e00640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48f4073200 0x7f48f4075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4908001230 tx=0x7f4908001120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:02.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.580+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4910020020 con 0x7f4918154ac0 2024-09-17T03:22:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.756+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f491815d180 con 0x7f4918154ac0 2024-09-17T03:22:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.756+0000 7f4914c00640 1 -- 172.21.15.46:0/1959679599 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v32) ==== 116+0+2 (secure 0 0 0) 0x7f491005f420 con 0x7f4918154ac0 2024-09-17T03:22:02.760 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48f4073200 msgr2=0x7f48f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48f4073200 0x7f48f4075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4908001230 tx=0x7f4908001120 comp rx=0 tx=0).stop 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 msgr2=0x7f49181f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 0x7f49181f5a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49180719a0 tx=0x7f491000d7b0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 shutdown_connections 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48f4073200 0x7f48f4075710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4918154ac0 0x7f49181f5a50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49181546b0 0x7f49180cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 --2- 172.21.15.46:0/1959679599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48fc004810 0x7f49181e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 >> 172.21.15.46:0/1959679599 conn(0x7f49180b2040 msgr2=0x7f49180af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 shutdown_connections 2024-09-17T03:22:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.760+0000 7f491f000640 1 -- 172.21.15.46:0/1959679599 wait complete. 2024-09-17T03:22:02.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:22:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 -- 172.21.15.46:0/2501017089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c5d0 msgr2=0x7fefb0073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 --2- 172.21.15.46:0/2501017089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c5d0 0x7fefb0073dc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fefb006ae60 tx=0x7fefa001c810 comp rx=0 tx=0).stop 2024-09-17T03:22:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 -- 172.21.15.46:0/2501017089 shutdown_connections 2024-09-17T03:22:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 --2- 172.21.15.46:0/2501017089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefb015f6b0 0x7fefb015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 --2- 172.21.15.46:0/2501017089 >> v2:172.21.15.99:3300/0 conn(0x7fefb0074300 0x7fefb015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 --2- 172.21.15.46:0/2501017089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c5d0 0x7fefb0073dc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 -- 172.21.15.46:0/2501017089 >> 172.21.15.46:0/2501017089 conn(0x7fefb00b2c40 msgr2=0x7fefb00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 -- 172.21.15.46:0/2501017089 shutdown_connections 2024-09-17T03:22:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.940+0000 7fefb6a00640 1 -- 172.21.15.46:0/2501017089 wait complete. 2024-09-17T03:22:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 Processor -- start 2024-09-17T03:22:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 -- start start 2024-09-17T03:22:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 0x7fefb015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefb015c5d0 0x7fefb015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefb015f6b0 0x7fefb015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0073640 con 0x7fefb0074300 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fefb00734c0 con 0x7fefb015f6b0 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fefb00737c0 con 0x7fefb015c5d0 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 0x7fefb015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 0x7fefb015d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58602/0 (socket says 172.21.15.46:58602) 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 -- 172.21.15.46:0/3946899815 learned_addr learned my addr 172.21.15.46:0/3946899815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefa7e00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefb015c5d0 0x7fefb015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb4a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefb015f6b0 0x7fefb015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 -- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefb015c5d0 msgr2=0x7fefb015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefb015c5d0 0x7fefb015d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 -- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefb015f6b0 msgr2=0x7fefb015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefb015f6b0 0x7fefb015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 -- 172.21.15.46:0/3946899815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb015eb00 con 0x7fefb0074300 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefafe00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 0x7fefb015d1b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009850 tx=0x7fefa00a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00b2040 con 0x7fefb0074300 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefa00044b0 con 0x7fefb0074300 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb00ce1a0 con 0x7fefb0074300 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003860 con 0x7fefb0074300 2024-09-17T03:22:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.944+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb015e540 con 0x7fefb0074300 2024-09-17T03:22:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.948+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb0073dc0 con 0x7fefb0074300 2024-09-17T03:22:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.948+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefa00bd060 con 0x7fefb0074300 2024-09-17T03:22:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.948+0000 7fefacc00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef8c073200 0x7fef8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.948+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fefa012dc30 con 0x7fefb0074300 2024-09-17T03:22:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.948+0000 7fefa7e00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef8c073200 0x7fef8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.948+0000 7fefa7e00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef8c073200 0x7fef8c075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef98005fd0 tx=0x7fef98005d10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:02.952+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fefa0016490 con 0x7fefb0074300 2024-09-17T03:22:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.124+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fefb00cd370 con 0x7fefb0074300 2024-09-17T03:22:03.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.128+0000 7fefacc00640 1 -- 172.21.15.46:0/3946899815 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v33)=0 v33) ==== 131+0+0 (secure 0 0 0) 0x7fefa00f8d00 con 0x7fefb0074300 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef8c073200 msgr2=0x7fef8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef8c073200 0x7fef8c075710 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef98005fd0 tx=0x7fef98005d10 comp rx=0 tx=0).stop 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 msgr2=0x7fefb015d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 0x7fefb015d1b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009850 tx=0x7fefa00a5730 comp rx=0 tx=0).stop 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 shutdown_connections 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fef8c073200 0x7fef8c075710 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefb015f6b0 0x7fefb015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fefb015c5d0 0x7fefb015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 --2- 172.21.15.46:0/3946899815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb0074300 0x7fefb015d1b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 >> 172.21.15.46:0/3946899815 conn(0x7fefb00b2c40 msgr2=0x7fefb00b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 shutdown_connections 2024-09-17T03:22:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.132+0000 7fefb6a00640 1 -- 172.21.15.46:0/3946899815 wait complete. 2024-09-17T03:22:03.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 -- 172.21.15.46:0/2000274299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b815efd0 msgr2=0x7fa9b815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 --2- 172.21.15.46:0/2000274299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b815efd0 0x7fa9b815d280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000a950 tx=0x7fa9a001dae0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 -- 172.21.15.46:0/2000274299 shutdown_connections 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 --2- 172.21.15.46:0/2000274299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b815efd0 0x7fa9b815d280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 -- 172.21.15.46:0/2000274299 >> 172.21.15.46:0/2000274299 conn(0x7fa9b80b2040 msgr2=0x7fa9b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 -- 172.21.15.46:0/2000274299 shutdown_connections 2024-09-17T03:22:03.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 -- 172.21.15.46:0/2000274299 wait complete. 2024-09-17T03:22:03.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 Processor -- start 2024-09-17T03:22:03.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.316+0000 7fa9be600640 1 -- start start 2024-09-17T03:22:03.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 0x7fa9b8158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b815d7c0 0x7fa9b813ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b813d120 0x7fa9b80b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b8168c70 con 0x7fa9b80cf900 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b80b5510 con 0x7fa9b813d120 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b8168df0 con 0x7fa9b815d7c0 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 0x7fa9b8158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 0x7fa9b8158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58616/0 (socket says 172.21.15.46:58616) 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 -- 172.21.15.46:0/3390922694 learned_addr learned my addr 172.21.15.46:0/3390922694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9afe00640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b815d7c0 0x7fa9b813ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b813d120 0x7fa9b80b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 -- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b815d7c0 msgr2=0x7fa9b813ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b815d7c0 0x7fa9b813ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 -- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b813d120 msgr2=0x7fa9b80b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b813d120 0x7fa9b80b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 -- 172.21.15.46:0/3390922694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b80772a0 con 0x7fa9b80cf900 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7e00640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b813d120 0x7fa9b80b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9b7400640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 0x7fa9b8158600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800c450 tx=0x7fa9a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8017070 con 0x7fa9b80cf900 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9a80053c0 con 0x7fa9b80cf900 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8003860 con 0x7fa9b80cf900 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b8156c00 con 0x7fa9b80cf900 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81571c0 con 0x7fa9b80cf900 2024-09-17T03:22:03.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.320+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b815d280 con 0x7fa9b80cf900 2024-09-17T03:22:03.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.324+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa9a801a070 con 0x7fa9b80cf900 2024-09-17T03:22:03.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.324+0000 7fa9af400640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa994073200 0x7fa994075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.324+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a8093680 con 0x7fa9b80cf900 2024-09-17T03:22:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.324+0000 7fa9afe00640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa994073200 0x7fa994075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.324+0000 7fa9afe00640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa994073200 0x7fa994075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b0000 tx=0x7fa9a001c320 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.328+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa9a805dfb0 con 0x7fa9b80cf900 2024-09-17T03:22:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.524+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fa9b806b540 con 0x7fa994073200 2024-09-17T03:22:03.534 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:22:03.532+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:22:03.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:22:03.532+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:22:03.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:22:03.532+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:22:03.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:22:03.532+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:22:03.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:22:03.532+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:22:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9af400640 1 -- 172.21.15.46:0/3390922694 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9a00040c0 con 0x7fa994073200 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa994073200 msgr2=0x7fa994075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa994073200 0x7fa994075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b0000 tx=0x7fa9a001c320 comp rx=0 tx=0).stop 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 msgr2=0x7fa9b8158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 0x7fa9b8158600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800c450 tx=0x7fa9a800c910 comp rx=0 tx=0).stop 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 shutdown_connections 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa994073200 0x7fa994075710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b813d120 0x7fa9b80b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9b815d7c0 0x7fa9b813ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 --2- 172.21.15.46:0/3390922694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b80cf900 0x7fa9b8158600 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 >> 172.21.15.46:0/3390922694 conn(0x7fa9b80b2040 msgr2=0x7fa9b806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 shutdown_connections 2024-09-17T03:22:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.552+0000 7fa9be600640 1 -- 172.21.15.46:0/3390922694 wait complete. 2024-09-17T03:22:03.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-17T03:22:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 -- 172.21.15.46:0/3628004860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85541527b0 msgr2=0x7f855415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 --2- 172.21.15.46:0/3628004860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85541527b0 0x7f855415bb60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8540009900 tx=0x7f854001c6d0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 -- 172.21.15.46:0/3628004860 shutdown_connections 2024-09-17T03:22:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 --2- 172.21.15.46:0/3628004860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85540b4de0 0x7f85540b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 --2- 172.21.15.46:0/3628004860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85541527b0 0x7f855415bb60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 -- 172.21.15.46:0/3628004860 >> 172.21.15.46:0/3628004860 conn(0x7f85540b2040 msgr2=0x7f85540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 -- 172.21.15.46:0/3628004860 shutdown_connections 2024-09-17T03:22:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.740+0000 7f855bc00640 1 -- 172.21.15.46:0/3628004860 wait complete. 2024-09-17T03:22:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 Processor -- start 2024-09-17T03:22:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 -- start start 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 0x7f8554145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85540b5c70 0x7f855413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85541527b0 0x7f85541fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8554168a50 con 0x7f85540b4de0 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85541688d0 con 0x7f85541527b0 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8554168bd0 con 0x7f85540b5c70 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 0x7f8554145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 0x7f8554145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58652/0 (socket says 172.21.15.46:58652) 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 -- 172.21.15.46:0/1785346155 learned_addr learned my addr 172.21.15.46:0/1785346155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8553e00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85540b5c70 0x7f855413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559c00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85541527b0 0x7f85541fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 -- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85540b5c70 msgr2=0x7f855413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85540b5c70 0x7f855413eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 -- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85541527b0 msgr2=0x7f85541fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85541527b0 0x7f85541fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 -- 172.21.15.46:0/1785346155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855413f070 con 0x7f85540b4de0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8559200640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 0x7f8554145300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f854800b410 tx=0x7f854800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548015030 con 0x7f85540b4de0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8548004020 con 0x7f85540b4de0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480044b0 con 0x7f85540b4de0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.744+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8554142b50 con 0x7f85540b4de0 2024-09-17T03:22:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.748+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8554143110 con 0x7f85540b4de0 2024-09-17T03:22:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.748+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85540b51f0 con 0x7f85540b4de0 2024-09-17T03:22:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.748+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8548018070 con 0x7f85540b4de0 2024-09-17T03:22:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.748+0000 7f8550c00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8534073230 0x7f8534075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.748+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8548093c80 con 0x7f85540b4de0 2024-09-17T03:22:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.748+0000 7f8553e00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8534073230 0x7f8534075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.752+0000 7f8553e00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8534073230 0x7f8534075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f85400af000 tx=0x7f854001af10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.752+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f854805e640 con 0x7f85540b4de0 2024-09-17T03:22:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:03.956+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f855406b540 con 0x7f8534073230 2024-09-17T03:22:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.088+0000 7f8550c00640 1 -- 172.21.15.46:0/1785346155 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85400040c0 con 0x7f8534073230 2024-09-17T03:22:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8534073230 msgr2=0x7f8534075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8534073230 0x7f8534075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f85400af000 tx=0x7f854001af10 comp rx=0 tx=0).stop 2024-09-17T03:22:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 msgr2=0x7f8554145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 0x7f8554145300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f854800b410 tx=0x7f854800b8d0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 shutdown_connections 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8534073230 0x7f8534075740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85541527b0 0x7f85541fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85540b5c70 0x7f855413eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 --2- 172.21.15.46:0/1785346155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85540b4de0 0x7f8554145300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 >> 172.21.15.46:0/1785346155 conn(0x7f85540b2040 msgr2=0x7f85540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 shutdown_connections 2024-09-17T03:22:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.092+0000 7f855bc00640 1 -- 172.21.15.46:0/1785346155 wait complete. 2024-09-17T03:22:04.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-17T03:22:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 -- 172.21.15.46:0/3449594815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 msgr2=0x7fe3e80b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 --2- 172.21.15.46:0/3449594815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e80b5a80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009840 tx=0x7fe3d801c820 comp rx=0 tx=0).stop 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 -- 172.21.15.46:0/3449594815 shutdown_connections 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 --2- 172.21.15.46:0/3449594815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e8074b20 0x7fe3e815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 --2- 172.21.15.46:0/3449594815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e80b5a80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 -- 172.21.15.46:0/3449594815 >> 172.21.15.46:0/3449594815 conn(0x7fe3e80708c0 msgr2=0x7fe3e8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 -- 172.21.15.46:0/3449594815 shutdown_connections 2024-09-17T03:22:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 -- 172.21.15.46:0/3449594815 wait complete. 2024-09-17T03:22:04.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 Processor -- start 2024-09-17T03:22:04.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.276+0000 7fe3eea00640 1 -- start start 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e8160d50 0x7fe3e81fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e814f110 0x7fe3e81fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e816a740 con 0x7fe3e8157030 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e816a5c0 con 0x7fe3e8160d50 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e816a8c0 con 0x7fe3e814f110 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e814e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58680/0 (socket says 172.21.15.46:58680) 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 -- 172.21.15.46:0/684472823 learned_addr learned my addr 172.21.15.46:0/684472823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eca00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e814f110 0x7fe3e81fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3df400640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e8160d50 0x7fe3e81fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 -- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e814f110 msgr2=0x7fe3e81fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e814f110 0x7fe3e81fc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 -- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e8160d50 msgr2=0x7fe3e81fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e8160d50 0x7fe3e81fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 -- 172.21.15.46:0/684472823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81fd270 con 0x7fe3e8157030 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eca00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e814f110 0x7fe3e81fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e7e00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e814e460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe3d801cd00 tx=0x7fe3d80a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a66e0 con 0x7fe3e8157030 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3d8004130 con 0x7fe3e8157030 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e80b2bd0 con 0x7fe3e8157030 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8003600 con 0x7fe3e8157030 2024-09-17T03:22:04.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e80b3190 con 0x7fe3e8157030 2024-09-17T03:22:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e815cf90 con 0x7fe3e8157030 2024-09-17T03:22:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.280+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3d80c1020 con 0x7fe3e8157030 2024-09-17T03:22:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.284+0000 7fe3e4c00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe3c40733d0 0x7fe3c40758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:22:04.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.284+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe3d812eeb0 con 0x7fe3e8157030 2024-09-17T03:22:04.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.284+0000 7fe3df400640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe3c40733d0 0x7fe3c40758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:22:04.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.284+0000 7fe3df400640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe3c40733d0 0x7fe3c40758e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe3e814f5a0 tx=0x7fe3d0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:22:04.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.288+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3d80c8020 con 0x7fe3e8157030 2024-09-17T03:22:04.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.488+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fe3e806b540 con 0x7fe3c40733d0 2024-09-17T03:22:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3e4c00640 1 -- 172.21.15.46:0/684472823 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7fe3d0001c90 con 0x7fe3c40733d0 2024-09-17T03:22:04.495 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe3c40733d0 msgr2=0x7fe3c40758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3eea00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe3c40733d0 0x7fe3c40758e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe3e814f5a0 tx=0x7fe3d0009040 comp rx=0 tx=0).stop 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 msgr2=0x7fe3e814e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3eea00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e814e460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe3d801cd00 tx=0x7fe3d80a64f0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 shutdown_connections 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.492+0000 7fe3eea00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe3c40733d0 0x7fe3c40758e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.496+0000 7fe3eea00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe3e814f110 0x7fe3e81fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.496+0000 7fe3eea00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e8160d50 0x7fe3e81fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.496+0000 7fe3eea00640 1 --2- 172.21.15.46:0/684472823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e8157030 0x7fe3e814e460 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:22:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.496+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 >> 172.21.15.46:0/684472823 conn(0x7fe3e80708c0 msgr2=0x7fe3e8075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:22:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.496+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 shutdown_connections 2024-09-17T03:22:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:22:04.496+0000 7fe3eea00640 1 -- 172.21.15.46:0/684472823 wait complete. 2024-09-17T03:22:04.517 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:22:04.517 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:04.517 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:04.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:04.526 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:22:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:22:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.23214 s, 130 MB/s 2024-09-17T03:22:12.822 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:12.823 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:12.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:12.832 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:22:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.37236 s, 128 MB/s 2024-09-17T03:22:21.260 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:21.260 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:21.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:21.269 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:28.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:22:28.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:22:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:22:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:22:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.56386 s, 125 MB/s 2024-09-17T03:22:29.888 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:29.888 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:29.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:29.897 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:22:38.399 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:22:38.399 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.44768 s, 127 MB/s 2024-09-17T03:22:38.401 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:38.401 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:38.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:38.411 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:22:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:22:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.48881 s, 126 MB/s 2024-09-17T03:22:46.953 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:46.953 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:46.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:46.963 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:22:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:22:55.607 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.58878 s, 125 MB/s 2024-09-17T03:22:55.608 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:22:55.608 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:22:55.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:22:55.617 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:22:58.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:22:58.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:23:04.108 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:23:04.108 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:23:04.108 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.43723 s, 127 MB/s 2024-09-17T03:23:04.109 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:04.109 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:04.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:04.119 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:23:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:23:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:23:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.3766 s, 128 MB/s 2024-09-17T03:23:12.547 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:12.547 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:12.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:12.556 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:23:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:23:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:23:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.5442 s, 126 MB/s 2024-09-17T03:23:21.159 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:21.159 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:21.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:21.168 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:23:28.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:23:28.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:23:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:23:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:23:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.21855 s, 131 MB/s 2024-09-17T03:23:29.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 -- 172.21.15.46:0/4073816281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c06f170 msgr2=0x7f0e0c06f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 --2- 172.21.15.46:0/4073816281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c06f170 0x7f0e0c06f580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009b40 tx=0x7f0e0401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 -- 172.21.15.46:0/4073816281 shutdown_connections 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 --2- 172.21.15.46:0/4073816281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c06f170 0x7f0e0c06f580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 -- 172.21.15.46:0/4073816281 >> 172.21.15.46:0/4073816281 conn(0x7f0e0c0b2c40 msgr2=0x7f0e0c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 -- 172.21.15.46:0/4073816281 shutdown_connections 2024-09-17T03:23:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.608+0000 7f0e14400640 1 -- 172.21.15.46:0/4073816281 wait complete. 2024-09-17T03:23:29.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 Processor -- start 2024-09-17T03:23:29.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 -- start start 2024-09-17T03:23:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e0c06f170 0x7f0e0c1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c15c5d0 0x7f0e0c148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 0x7f0e0c148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c169f60 con 0x7f0e0c154390 2024-09-17T03:23:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c169de0 con 0x7f0e0c15c5d0 2024-09-17T03:23:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c16a0e0 con 0x7f0e0c06f170 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 0x7f0e0c148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 0x7f0e0c148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46244/0 (socket says 172.21.15.46:46244) 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 -- 172.21.15.46:0/3271765311 learned_addr learned my addr 172.21.15.46:0/3271765311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e11a00640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e0c06f170 0x7f0e0c1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e11000640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c15c5d0 0x7f0e0c148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 -- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e0c06f170 msgr2=0x7f0e0c1482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e0c06f170 0x7f0e0c1482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 -- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c15c5d0 msgr2=0x7f0e0c148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c15c5d0 0x7f0e0c148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 -- 172.21.15.46:0/3271765311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c15ab90 con 0x7f0e0c154390 2024-09-17T03:23:29.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e11000640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c15c5d0 0x7f0e0c148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:29.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e12400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 0x7f0e0c148ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401af90 tx=0x7f0e04009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:29.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040bb070 con 0x7f0e0c154390 2024-09-17T03:23:29.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e04003d70 con 0x7f0e0c154390 2024-09-17T03:23:29.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b2040 con 0x7f0e0c154390 2024-09-17T03:23:29.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c15ae70 con 0x7f0e0c154390 2024-09-17T03:23:29.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.612+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c153e70 con 0x7f0e0c154390 2024-09-17T03:23:29.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.616+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c14c330 con 0x7f0e0c154390 2024-09-17T03:23:29.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.616+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 7) ==== 97234+0+0 (secure 0 0 0) 0x7f0e040be070 con 0x7f0e0c154390 2024-09-17T03:23:29.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.616+0000 7f0e09600640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0dec073200 0x7f0dec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:29.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.616+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0e0412dda0 con 0x7f0e0c154390 2024-09-17T03:23:29.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.616+0000 7f0e11a00640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0dec073200 0x7f0dec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:29.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.620+0000 7f0e11a00640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0dec073200 0x7f0dec075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001230 tx=0x7f0e00009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:29.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.620+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0e04016650 con 0x7f0e0c154390 2024-09-17T03:23:29.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.820+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f0e0c06b540 con 0x7f0dec073200 2024-09-17T03:23:29.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:29.828+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:29.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:29.828+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:29.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:29.828+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:29.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:29.828+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:29.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:29.828+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.852+0000 7f0e09600640 1 -- 172.21.15.46:0/3271765311 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e00007e20 con 0x7f0dec073200 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0dec073200 msgr2=0x7f0dec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0dec073200 0x7f0dec075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001230 tx=0x7f0e00009040 comp rx=0 tx=0).stop 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 msgr2=0x7f0e0c148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 0x7f0e0c148ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401af90 tx=0x7f0e04009280 comp rx=0 tx=0).stop 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 shutdown_connections 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0dec073200 0x7f0dec075710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e0c154390 0x7f0e0c148ea0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e0c15c5d0 0x7f0e0c148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 --2- 172.21.15.46:0/3271765311 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0e0c06f170 0x7f0e0c1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 >> 172.21.15.46:0/3271765311 conn(0x7f0e0c0b2c40 msgr2=0x7f0e0c0b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 shutdown_connections 2024-09-17T03:23:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:29.856+0000 7f0e14400640 1 -- 172.21.15.46:0/3271765311 wait complete. 2024-09-17T03:23:29.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-17T03:23:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 -- 172.21.15.46:0/2256593525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b3840 msgr2=0x7f07e0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 --2- 172.21.15.46:0/2256593525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b3840 0x7f07e0072030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f07d800acd0 tx=0x7f07d801dec0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 -- 172.21.15.46:0/2256593525 shutdown_connections 2024-09-17T03:23:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 --2- 172.21.15.46:0/2256593525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b3840 0x7f07e0072030 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 --2- 172.21.15.46:0/2256593525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b5040 0x7f07e00cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 -- 172.21.15.46:0/2256593525 >> 172.21.15.46:0/2256593525 conn(0x7f07e006cdf0 msgr2=0x7f07e006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:30.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 -- 172.21.15.46:0/2256593525 shutdown_connections 2024-09-17T03:23:30.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.036+0000 7f07e5200640 1 -- 172.21.15.46:0/2256593525 wait complete. 2024-09-17T03:23:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 Processor -- start 2024-09-17T03:23:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 -- start start 2024-09-17T03:23:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 0x7f07e01f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b5040 0x7f07e01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00d0380 0x7f07e01527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0167d50 con 0x7f07e00b3840 2024-09-17T03:23:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0167bd0 con 0x7f07e00b5040 2024-09-17T03:23:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07e5200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f07e0167ed0 con 0x7f07e00d0380 2024-09-17T03:23:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 0x7f07e01f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 0x7f07e01f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46262/0 (socket says 172.21.15.46:46262) 2024-09-17T03:23:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 -- 172.21.15.46:0/30710998 learned_addr learned my addr 172.21.15.46:0/30710998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07d7e00640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b5040 0x7f07e01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07dfe00640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00d0380 0x7f07e01527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 -- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00d0380 msgr2=0x7f07e01527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00d0380 0x7f07e01527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 -- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b5040 msgr2=0x7f07e01522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b5040 0x7f07e01522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.040+0000 7f07df400640 1 -- 172.21.15.46:0/30710998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e013bfe0 con 0x7f07e00b3840 2024-09-17T03:23:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07df400640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 0x7f07e01f6600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07d000e9f0 tx=0x7f07d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d000cd80 con 0x7f07e00b3840 2024-09-17T03:23:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d0004020 con 0x7f07e00b3840 2024-09-17T03:23:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e0153eb0 con 0x7f07e00b3840 2024-09-17T03:23:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004470 con 0x7f07e00b3840 2024-09-17T03:23:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e0154470 con 0x7f07e00b3840 2024-09-17T03:23:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e0072030 con 0x7f07e00b3840 2024-09-17T03:23:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.044+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 7) ==== 97234+0+0 (secure 0 0 0) 0x7f07d0003600 con 0x7f07e00b3840 2024-09-17T03:23:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.048+0000 7f07d7400640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc073230 0x7f07bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.048+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f07d0094a20 con 0x7f07e00b3840 2024-09-17T03:23:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.048+0000 7f07d7e00640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc073230 0x7f07bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.048+0000 7f07d7e00640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc073230 0x7f07bc075740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07e01fa2f0 tx=0x7f07c8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.052+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07d005f3e0 con 0x7f07e00b3840 2024-09-17T03:23:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.256+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f07e00cd9e0 con 0x7f07bc073230 2024-09-17T03:23:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.260+0000 7f07d7400640 1 -- 172.21.15.46:0/30710998 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f07e00cd9e0 con 0x7f07bc073230 2024-09-17T03:23:30.263 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc073230 msgr2=0x7f07bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc073230 0x7f07bc075740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07e01fa2f0 tx=0x7f07c8009040 comp rx=0 tx=0).stop 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 msgr2=0x7f07e01f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 0x7f07e01f6600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07d000e9f0 tx=0x7f07d000edf0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 shutdown_connections 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc073230 0x7f07bc075740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00d0380 0x7f07e01527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b5040 0x7f07e01522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 --2- 172.21.15.46:0/30710998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e00b3840 0x7f07e01f6600 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 >> 172.21.15.46:0/30710998 conn(0x7f07e006cdf0 msgr2=0x7f07e00b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 shutdown_connections 2024-09-17T03:23:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:30.264+0000 7f07e5200640 1 -- 172.21.15.46:0/30710998 wait complete. 2024-09-17T03:23:30.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:30.286 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:23:30.286 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:23:30.297 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:23:30.297 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10488365056" 2024-09-17T03:23:30.297 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:30.298 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-17T03:23:32.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:32.299 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:23:32.299 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:23:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/group1/sv1/aaf467ef-a6ad-4028-bd09-df46cea16e11 2024-09-17T03:23:32.310 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10737418240" 2024-09-17T03:23:32.310 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:32.311 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-17T03:23:32.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-17T03:23:32.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a9de00640 1 -- 172.21.15.46:0/3587111212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7c004810 msgr2=0x7f3a7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a9de00640 1 --2- 172.21.15.46:0/3587111212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7c004810 0x7f3a7c004c80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3a98071760 tx=0x7f3a8001d210 comp rx=0 tx=0).stop 2024-09-17T03:23:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a97400640 1 -- 172.21.15.46:0/3587111212 reap_dead start 2024-09-17T03:23:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a9de00640 1 -- 172.21.15.46:0/3587111212 shutdown_connections 2024-09-17T03:23:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a9de00640 1 -- 172.21.15.46:0/3587111212 >> 172.21.15.46:0/3587111212 conn(0x7f3a980b1ff0 msgr2=0x7f3a980b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a9de00640 1 -- 172.21.15.46:0/3587111212 shutdown_connections 2024-09-17T03:23:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.528+0000 7f3a9de00640 1 -- 172.21.15.46:0/3587111212 wait complete. 2024-09-17T03:23:32.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 Processor -- start 2024-09-17T03:23:32.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 -- start start 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c004810 0x7f3a980739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 0x7f3a9814e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a9806e670 0x7f3a9806ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9815fd10 con 0x7f3a9814edc0 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9815fb90 con 0x7f3a7c004810 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a9de00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9815fe90 con 0x7f3a9806e670 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 0x7f3a9814e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 0x7f3a9814e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51376/0 (socket says 172.21.15.46:51376) 2024-09-17T03:23:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 -- 172.21.15.46:0/6423093 learned_addr learned my addr 172.21.15.46:0/6423093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a96a00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c004810 0x7f3a980739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a97400640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a9806e670 0x7f3a9806ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 -- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a9806e670 msgr2=0x7f3a9806ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a9806e670 0x7f3a9806ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 -- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c004810 msgr2=0x7f3a980739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c004810 0x7f3a980739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 -- 172.21.15.46:0/6423093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a98155bd0 con 0x7f3a9814edc0 2024-09-17T03:23:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.532+0000 7f3a8e000640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 0x7f3a9814e5f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a8001d040 tx=0x7f3a80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800bb030 con 0x7f3a9814edc0 2024-09-17T03:23:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a80004880 con 0x7f3a9814edc0 2024-09-17T03:23:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004cb0 con 0x7f3a9814edc0 2024-09-17T03:23:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9806efc0 con 0x7f3a9814edc0 2024-09-17T03:23:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a98072ee0 con 0x7f3a9814edc0 2024-09-17T03:23:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a9815f480 con 0x7f3a9814edc0 2024-09-17T03:23:32.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.536+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 97285+0+0 (secure 0 0 0) 0x7f3a800be030 con 0x7f3a9814edc0 2024-09-17T03:23:32.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.540+0000 7f3a8fe00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a74073290 0x7f3a740757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:32.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.540+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3a8012f860 con 0x7f3a9814edc0 2024-09-17T03:23:32.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.540+0000 7f3a96a00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a74073290 0x7f3a740757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.540+0000 7f3a96a00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a74073290 0x7f3a740757a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a880064b0 tx=0x7f3a88006c80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:32.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.544+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a9806efc0 con 0x7f3a9814edc0 2024-09-17T03:23:32.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.748+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f3a9806b540 con 0x7f3a74073290 2024-09-17T03:23:32.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.820+0000 7f3a8fe00640 1 -- 172.21.15.46:0/6423093 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a88001570 con 0x7f3a74073290 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a74073290 msgr2=0x7f3a740757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a74073290 0x7f3a740757a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a880064b0 tx=0x7f3a88006c80 comp rx=0 tx=0).stop 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 msgr2=0x7f3a9814e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 0x7f3a9814e5f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a8001d040 tx=0x7f3a80009840 comp rx=0 tx=0).stop 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 shutdown_connections 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a74073290 0x7f3a740757a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a9806e670 0x7f3a9806ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a9814edc0 0x7f3a9814e5f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 --2- 172.21.15.46:0/6423093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a7c004810 0x7f3a980739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:32.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 >> 172.21.15.46:0/6423093 conn(0x7f3a980b1ff0 msgr2=0x7f3a98166220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:32.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 shutdown_connections 2024-09-17T03:23:32.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:32.824+0000 7f3a9de00640 1 -- 172.21.15.46:0/6423093 wait complete. 2024-09-17T03:23:32.827 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.824+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.827 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.824+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.827 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.824+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.828 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.824+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.828 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.824+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:23:32.849 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.844+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.849 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.844+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.849 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.844+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.850 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.844+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:32.850 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:32.844+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 -- 172.21.15.46:0/1976460707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 msgr2=0x7f940815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 --2- 172.21.15.46:0/1976460707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f940815bb60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9408070710 tx=0x7f93f801c800 comp rx=0 tx=0).stop 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 -- 172.21.15.46:0/1976460707 shutdown_connections 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 --2- 172.21.15.46:0/1976460707 >> v2:172.21.15.99:3300/0 conn(0x7f94081527b0 0x7f9408074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 --2- 172.21.15.46:0/1976460707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f940815bb60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 -- 172.21.15.46:0/1976460707 >> 172.21.15.46:0/1976460707 conn(0x7f94080b2040 msgr2=0x7f94080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 -- 172.21.15.46:0/1976460707 shutdown_connections 2024-09-17T03:23:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 -- 172.21.15.46:0/1976460707 wait complete. 2024-09-17T03:23:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 Processor -- start 2024-09-17T03:23:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.008+0000 7f940ca00640 1 -- start start 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94080b4de0 0x7f94080c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f94080b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94081527b0 0x7f94080b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9408168b20 con 0x7f94080b5c70 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94081689a0 con 0x7f94080b4de0 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9408168ca0 con 0x7f94081527b0 2024-09-17T03:23:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f94080b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f94080b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51426/0 (socket says 172.21.15.46:51426) 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 -- 172.21.15.46:0/3784950557 learned_addr learned my addr 172.21.15.46:0/3784950557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f9405e00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94080b4de0 0x7f94080c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f9406800640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94081527b0 0x7f94080b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 -- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94081527b0 msgr2=0x7f94080b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94081527b0 0x7f94080b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 -- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94080b4de0 msgr2=0x7f94080c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94080b4de0 0x7f94080c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 -- 172.21.15.46:0/3784950557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9408141900 con 0x7f94080b5c70 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f9405e00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94080b4de0 0x7f94080c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93ffe00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f94080b3c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f93f000e7f0 tx=0x7f93f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000ccc0 con 0x7f94080b5c70 2024-09-17T03:23:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93f000ce80 con 0x7f94080b5c70 2024-09-17T03:23:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0005280 con 0x7f94080b5c70 2024-09-17T03:23:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9408141be0 con 0x7f94080b5c70 2024-09-17T03:23:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94081421a0 con 0x7f94080b5c70 2024-09-17T03:23:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.012+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940813f3f0 con 0x7f94080b5c70 2024-09-17T03:23:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.016+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 97285+0+0 (secure 0 0 0) 0x7f93f0005420 con 0x7f94080b5c70 2024-09-17T03:23:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.016+0000 7f93fe000640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93e4073290 0x7f93e40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.016+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f93f0094c60 con 0x7f94080b5c70 2024-09-17T03:23:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.016+0000 7f9405e00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93e4073290 0x7f93e40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.016+0000 7f9405e00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93e4073290 0x7f93e40757a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f93f80af000 tx=0x7f93f801b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.020+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f940813f3f0 con 0x7f94080b5c70 2024-09-17T03:23:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.256+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f940815bb60 con 0x7f94080b5c70 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.256+0000 7f93fe000640 1 -- 172.21.15.46:0/3784950557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1884 (secure 0 0 0) 0x7f93f005f4c0 con 0x7f94080b5c70 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.259 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 171, 2024-09-17T03:23:33.260 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 32, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:23:33.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2422, 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10054865231, 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 20322082816, 2024-09-17T03:23:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 752772030464, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 121961621, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 0, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 30 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 18, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:22:01:614270+0000", 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:23:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 2, 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "d", 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 244821 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:23:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 5, 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:22:59.231800+0000", 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:23:33.265 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:23:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93e4073290 msgr2=0x7f93e40757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93e4073290 0x7f93e40757a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f93f80af000 tx=0x7f93f801b040 comp rx=0 tx=0).stop 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 msgr2=0x7f94080b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f94080b3c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f93f000e7f0 tx=0x7f93f000ecb0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 shutdown_connections 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93e4073290 0x7f93e40757a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94081527b0 0x7f94080b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94080b5c70 0x7f94080b3c80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 --2- 172.21.15.46:0/3784950557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94080b4de0 0x7f94080c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 >> 172.21.15.46:0/3784950557 conn(0x7f94080b2040 msgr2=0x7f9408151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 shutdown_connections 2024-09-17T03:23:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:33.260+0000 7f940ca00640 1 -- 172.21.15.46:0/3784950557 wait complete. 2024-09-17T03:23:33.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:33.832+0000 7fbe41400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:33.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:33.832+0000 7fbe41400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:33.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:33.832+0000 7fbe41400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:33.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:33.832+0000 7fbe41400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:33.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:33.832+0000 7fbe41400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:34.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:23:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.572+0000 7fe7a7e00640 1 -- 172.21.15.46:0/3382978814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00cd820 msgr2=0x7fe7a00b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.572+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/3382978814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00cd820 0x7fe7a00b5060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7a006fe00 tx=0x7fe79401c520 comp rx=0 tx=0).stop 2024-09-17T03:23:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.576+0000 7fe7a7e00640 1 -- 172.21.15.46:0/3382978814 shutdown_connections 2024-09-17T03:23:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.576+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/3382978814 >> v2:172.21.15.99:3300/0 conn(0x7fe7a00b55a0 0x7fe7a00ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.576+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/3382978814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00cd820 0x7fe7a00b5060 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.576+0000 7fe7a7e00640 1 -- 172.21.15.46:0/3382978814 >> 172.21.15.46:0/3382978814 conn(0x7fe7a00d0460 msgr2=0x7fe7a00d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.576+0000 7fe7a7e00640 1 -- 172.21.15.46:0/3382978814 shutdown_connections 2024-09-17T03:23:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.576+0000 7fe7a7e00640 1 -- 172.21.15.46:0/3382978814 wait complete. 2024-09-17T03:23:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 Processor -- start 2024-09-17T03:23:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 -- start start 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 0x7fe7a014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7a00cd820 0x7fe7a0145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7a00ce6d0 0x7fe7a0149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a016ffb0 con 0x7fe7a00b55a0 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a015bd70 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a7e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a0170130 con 0x7fe7a00cd820 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 0x7fe7a014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 0x7fe7a014a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51458/0 (socket says 172.21.15.46:51458) 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a5400640 1 -- 172.21.15.46:0/424792573 learned_addr learned my addr 172.21.15.46:0/424792573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a5e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7a00ce6d0 0x7fe7a0149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.580+0000 7fe7a4a00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7a00cd820 0x7fe7a0145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5e00640 1 -- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7a00cd820 msgr2=0x7fe7a0145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7a00cd820 0x7fe7a0145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5e00640 1 -- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 msgr2=0x7fe7a014a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 0x7fe7a014a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5e00640 1 -- 172.21.15.46:0/424792573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a0072100 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5400640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 0x7fe7a014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a4a00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7a00cd820 0x7fe7a0145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.584+0000 7fe7a5e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7a00ce6d0 0x7fe7a0149190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe79800ca70 tx=0x7fe79800cb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.588+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798016020 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.588+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe79800a8d0 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.588+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79800ad20 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.588+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a0151800 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.588+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a00723a0 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.588+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a01466c0 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.592+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fe798019020 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.592+0000 7fe79d600640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe780073400 0x7fe780075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.592+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe798097400 con 0x7fe7a00ce6d0 2024-09-17T03:23:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.596+0000 7fe7a5400640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe780073400 0x7fe780075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.596+0000 7fe7a5400640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe780073400 0x7fe780075910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c10 tx=0x7fe79401adc0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:34.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:34.620+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe79809d080 con 0x7fe7a00ce6d0 2024-09-17T03:23:35.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.112+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe7a014f6a0 con 0x7fe7a00ce6d0 2024-09-17T03:23:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.120+0000 7fe79d600640 1 -- 172.21.15.46:0/424792573 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1914 (secure 0 0 0) 0x7fe7a00ceae0 con 0x7fe7a00ce6d0 2024-09-17T03:23:35.124 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:35.124 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:23:35.124 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:23:35.124 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:23:35.125 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 172, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 32, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:23:35.126 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2585, 2024-09-17T03:23:35.127 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10738568796, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 21743984640, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 751350128640, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 85, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 154463739, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 0, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 39 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 18, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:22:01:614270+0000", 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:23:35.128 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 2, 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "d", 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 244821 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:23:35.129 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 5, 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:22:59.231800+0000", 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:35.130 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:23:35.131 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:23:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe780073400 msgr2=0x7fe780075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe780073400 0x7fe780075910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c10 tx=0x7fe79401adc0 comp rx=0 tx=0).stop 2024-09-17T03:23:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7a00ce6d0 msgr2=0x7fe7a0149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:35.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7a00ce6d0 0x7fe7a0149190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe79800ca70 tx=0x7fe79800cb60 comp rx=0 tx=0).stop 2024-09-17T03:23:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 shutdown_connections 2024-09-17T03:23:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe780073400 0x7fe780075910 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c10 tx=0x7fe79401adc0 comp rx=0 tx=0).stop 2024-09-17T03:23:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7a00ce6d0 0x7fe7a0149190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7a00cd820 0x7fe7a0145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 --2- 172.21.15.46:0/424792573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a00b55a0 0x7fe7a014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:35.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.128+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 >> 172.21.15.46:0/424792573 conn(0x7fe7a00d0460 msgr2=0x7fe7a015eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:35.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.132+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 shutdown_connections 2024-09-17T03:23:35.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:35.132+0000 7fe7a7e00640 1 -- 172.21.15.46:0/424792573 wait complete. 2024-09-17T03:23:36.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 -- 172.21.15.46:0/3665583074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f380703a0 msgr2=0x7f5f3806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/3665583074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f380703a0 0x7f5f3806d200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009c80 tx=0x7f5f3001cb30 comp rx=0 tx=0).stop 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 -- 172.21.15.46:0/3665583074 shutdown_connections 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/3665583074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f380703a0 0x7f5f3806d200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/3665583074 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f38159130 0x7f5f3815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 -- 172.21.15.46:0/3665583074 >> 172.21.15.46:0/3665583074 conn(0x7f5f380b35f0 msgr2=0x7f5f380b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 -- 172.21.15.46:0/3665583074 shutdown_connections 2024-09-17T03:23:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.520+0000 7f5f3ec00640 1 -- 172.21.15.46:0/3665583074 wait complete. 2024-09-17T03:23:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 Processor -- start 2024-09-17T03:23:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 -- start start 2024-09-17T03:23:36.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f3806cdf0 0x7f5f38150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f380703a0 0x7f5f38167310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f38159130 0x7f5f38167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38166eb0 con 0x7f5f38159130 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38166d30 con 0x7f5f380703a0 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.524+0000 7f5f3ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5f38167030 con 0x7f5f3806cdf0 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f3cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f38159130 0x7f5f38167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f3cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f38159130 0x7f5f38167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51468/0 (socket says 172.21.15.46:51468) 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f3cc00640 1 -- 172.21.15.46:0/2997059022 learned_addr learned my addr 172.21.15.46:0/2997059022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f3806cdf0 0x7f5f38150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37400640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f380703a0 0x7f5f38167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 -- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f380703a0 msgr2=0x7f5f38167310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f380703a0 0x7f5f38167310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 -- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f38159130 msgr2=0x7f5f38167850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f38159130 0x7f5f38167850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 -- 172.21.15.46:0/2997059022 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3815b310 con 0x7f5f3806cdf0 2024-09-17T03:23:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37400640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f380703a0 0x7f5f38167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:23:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.528+0000 7f5f37e00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f3806cdf0 0x7f5f38150110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e6f0 tx=0x7f5f2800ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.532+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28004220 con 0x7f5f3806cdf0 2024-09-17T03:23:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.532+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f2800c8b0 con 0x7f5f3806cdf0 2024-09-17T03:23:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.532+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800cd00 con 0x7f5f3806cdf0 2024-09-17T03:23:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.532+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f38159a00 con 0x7f5f3806cdf0 2024-09-17T03:23:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.532+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f38159fc0 con 0x7f5f3806cdf0 2024-09-17T03:23:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.532+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3806d200 con 0x7f5f3806cdf0 2024-09-17T03:23:36.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.536+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f5f28038d10 con 0x7f5f3806cdf0 2024-09-17T03:23:36.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.540+0000 7f5f2fe00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f18073440 0x7f5f18075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:36.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.540+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5f28094cd0 con 0x7f5f3806cdf0 2024-09-17T03:23:36.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.540+0000 7f5f37400640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f18073440 0x7f5f18075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:36.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.540+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5f2805f8f0 con 0x7f5f3806cdf0 2024-09-17T03:23:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:36.560+0000 7f5f37400640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f18073440 0x7f5f18075950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5f38151870 tx=0x7f5f24008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.008+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5f38164e60 con 0x7f5f3806cdf0 2024-09-17T03:23:37.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.012+0000 7f5f2fe00640 1 -- 172.21.15.46:0/2997059022 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2163 (secure 0 0 0) 0x7f5f2805f1f0 con 0x7f5f3806cdf0 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:23:37.020 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 174, 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:23:37.021 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 32, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.022 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2585, 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10738568796, 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 21743984640, 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 751350128640, 2024-09-17T03:23:37.023 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 85, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 92958417, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 0, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 23 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 18, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:22:01:614270+0000", 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:23:37.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 2, 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "d", 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 244821 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:23:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 5, 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:22:59.231800+0000", 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "1 ongoing clones - average progress is 3.125% (1s)\n [............................] (remaining: 34s)", 2024-09-17T03:23:37.026 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.03125, 2024-09-17T03:23:37.027 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:23:37.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:37.027 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:37.027 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:23:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.020+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f18073440 msgr2=0x7f5f18075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.020+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f18073440 0x7f5f18075950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5f38151870 tx=0x7f5f24008040 comp rx=0 tx=0).stop 2024-09-17T03:23:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.020+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f3806cdf0 msgr2=0x7f5f38150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.020+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f3806cdf0 0x7f5f38150110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e6f0 tx=0x7f5f2800ebb0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 shutdown_connections 2024-09-17T03:23:37.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f18073440 0x7f5f18075950 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f38159130 0x7f5f38167850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f380703a0 0x7f5f38167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 --2- 172.21.15.46:0/2997059022 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f3806cdf0 0x7f5f38150110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 >> 172.21.15.46:0/2997059022 conn(0x7f5f380b35f0 msgr2=0x7f5f38154090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 shutdown_connections 2024-09-17T03:23:37.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.024+0000 7f5f3ec00640 1 -- 172.21.15.46:0/2997059022 wait complete. 2024-09-17T03:23:37.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 -- 172.21.15.46:0/27619395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a00cf900 msgr2=0x7f58a015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 --2- 172.21.15.46:0/27619395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a00cf900 0x7f58a015e8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5890009840 tx=0x7f589001c7f0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 -- 172.21.15.46:0/27619395 shutdown_connections 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 --2- 172.21.15.46:0/27619395 >> v2:172.21.15.99:3300/0 conn(0x7f58a015ee00 0x7f58a015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 --2- 172.21.15.46:0/27619395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a00cf900 0x7f58a015e8c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 -- 172.21.15.46:0/27619395 >> 172.21.15.46:0/27619395 conn(0x7f58a00b2040 msgr2=0x7f58a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 -- 172.21.15.46:0/27619395 shutdown_connections 2024-09-17T03:23:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.404+0000 7f58a5800640 1 -- 172.21.15.46:0/27619395 wait complete. 2024-09-17T03:23:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.408+0000 7f58a5800640 1 Processor -- start 2024-09-17T03:23:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.408+0000 7f58a5800640 1 -- start start 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f58a5800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a00cf900 0x7f58a0154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f58a5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 0x7f58a0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f58a5800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a015ee00 0x7f58a0159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f58a5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58a015d610 con 0x7f58a015b7b0 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f58a5800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58a015d490 con 0x7f58a00cf900 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f58a5800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f58a015d790 con 0x7f58a015ee00 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 0x7f58a0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 0x7f58a0158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51482/0 (socket says 172.21.15.46:51482) 2024-09-17T03:23:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 -- 172.21.15.46:0/3104129860 learned_addr learned my addr 172.21.15.46:0/3104129860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 -- 172.21.15.46:0/3104129860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a015ee00 msgr2=0x7f58a0159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a015ee00 0x7f58a0159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 -- 172.21.15.46:0/3104129860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a00cf900 msgr2=0x7f58a0154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a00cf900 0x7f58a0154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.412+0000 7f589e000640 1 -- 172.21.15.46:0/3104129860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a013dd00 con 0x7f58a015b7b0 2024-09-17T03:23:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f589e000640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 0x7f58a0158ec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f588c00e7a0 tx=0x7f588c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c00ccc0 con 0x7f58a015b7b0 2024-09-17T03:23:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f588c010430 con 0x7f58a015b7b0 2024-09-17T03:23:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c010860 con 0x7f58a015b7b0 2024-09-17T03:23:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a0157940 con 0x7f58a015b7b0 2024-09-17T03:23:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a0157f00 con 0x7f58a015b7b0 2024-09-17T03:23:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.416+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a015e8c0 con 0x7f58a015b7b0 2024-09-17T03:23:37.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.420+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f588c010a00 con 0x7f58a015b7b0 2024-09-17T03:23:37.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.420+0000 7f5896a00640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5880073440 0x7f5880075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.420+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f588c0959e0 con 0x7f58a015b7b0 2024-09-17T03:23:37.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.420+0000 7f589ea00640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5880073440 0x7f5880075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:37.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.424+0000 7f589ea00640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5880073440 0x7f5880075950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58900af000 tx=0x7f58900195b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.428+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f588c060570 con 0x7f58a015b7b0 2024-09-17T03:23:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:37.836+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f58a006b540 con 0x7f5880073440 2024-09-17T03:23:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f5896a00640 1 -- 172.21.15.46:0/3104129860 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58900a5930 con 0x7f5880073440 2024-09-17T03:23:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5880073440 msgr2=0x7f5880075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5880073440 0x7f5880075950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58900af000 tx=0x7f58900195b0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 msgr2=0x7f58a0158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 0x7f58a0158ec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f588c00e7a0 tx=0x7f588c00ec60 comp rx=0 tx=0).stop 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 shutdown_connections 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5880073440 0x7f5880075950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58a015ee00 0x7f58a0159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58a015b7b0 0x7f58a0158ec0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 --2- 172.21.15.46:0/3104129860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a00cf900 0x7f58a0154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 >> 172.21.15.46:0/3104129860 conn(0x7f58a00b2040 msgr2=0x7f58a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.016+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 shutdown_connections 2024-09-17T03:23:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.020+0000 7f58a5800640 1 -- 172.21.15.46:0/3104129860 wait complete. 2024-09-17T03:23:38.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:23:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 -- 172.21.15.46:0/1878937696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 msgr2=0x7f467406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 --2- 172.21.15.46:0/1878937696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f467406d200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46740ce170 tx=0x7f466001c830 comp rx=0 tx=0).stop 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 -- 172.21.15.46:0/1878937696 shutdown_connections 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 --2- 172.21.15.46:0/1878937696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f467406d200 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 --2- 172.21.15.46:0/1878937696 >> v2:172.21.15.99:3300/0 conn(0x7f467406fc90 0x7f46740700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 --2- 172.21.15.46:0/1878937696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4674159130 0x7f467415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 -- 172.21.15.46:0/1878937696 >> 172.21.15.46:0/1878937696 conn(0x7f46740b35f0 msgr2=0x7f46740b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 -- 172.21.15.46:0/1878937696 shutdown_connections 2024-09-17T03:23:38.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 -- 172.21.15.46:0/1878937696 wait complete. 2024-09-17T03:23:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 Processor -- start 2024-09-17T03:23:38.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.200+0000 7f467be00640 1 -- start start 2024-09-17T03:23:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f46741573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467406fc90 0x7f46741543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4674159130 0x7f46741548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f467415fa10 con 0x7f467406cdf0 2024-09-17T03:23:38.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f467415f890 con 0x7f467406fc90 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f467415fb90 con 0x7f4674159130 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f46741573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f46741573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51498/0 (socket says 172.21.15.46:51498) 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 -- 172.21.15.46:0/1415755980 learned_addr learned my addr 172.21.15.46:0/1415755980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4678a00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467406fc90 0x7f46741543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679e00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4674159130 0x7f46741548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 -- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4674159130 msgr2=0x7f46741548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4674159130 0x7f46741548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 -- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467406fc90 msgr2=0x7f46741543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467406fc90 0x7f46741543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 -- 172.21.15.46:0/1415755980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467415b7a0 con 0x7f467406cdf0 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4679400640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f46741573b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f466800e7c0 tx=0x7f466800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:38.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466800ccc0 con 0x7f467406cdf0 2024-09-17T03:23:38.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f466800ce80 con 0x7f467406cdf0 2024-09-17T03:23:38.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668005280 con 0x7f467406cdf0 2024-09-17T03:23:38.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46741fd210 con 0x7f467406cdf0 2024-09-17T03:23:38.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741fcf30 con 0x7f467406cdf0 2024-09-17T03:23:38.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.204+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46740700a0 con 0x7f467406cdf0 2024-09-17T03:23:38.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.208+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f4668005420 con 0x7f467406cdf0 2024-09-17T03:23:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.208+0000 7f4671600640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073470 0x7f4654075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.208+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4668095140 con 0x7f467406cdf0 2024-09-17T03:23:38.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.208+0000 7f4678a00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073470 0x7f4654075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.208+0000 7f4678a00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073470 0x7f4654075980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46600af000 tx=0x7f466001b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:38.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.212+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f467415b7a0 con 0x7f467406cdf0 2024-09-17T03:23:38.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.408+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f467406b540 con 0x7f4654073470 2024-09-17T03:23:38.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.408+0000 7f4671600640 1 -- 172.21.15.46:0/1415755980 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f46600a6540 con 0x7f4654073470 2024-09-17T03:23:38.411 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:23:38.411 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:38.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:23:38.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:38.411 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:23:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073470 msgr2=0x7f4654075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073470 0x7f4654075980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46600af000 tx=0x7f466001b070 comp rx=0 tx=0).stop 2024-09-17T03:23:38.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 msgr2=0x7f46741573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f46741573b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f466800e7c0 tx=0x7f466800ec80 comp rx=0 tx=0).stop 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 shutdown_connections 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073470 0x7f4654075980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4674159130 0x7f46741548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467406fc90 0x7f46741543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 --2- 172.21.15.46:0/1415755980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467406cdf0 0x7f46741573b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 >> 172.21.15.46:0/1415755980 conn(0x7f46740b35f0 msgr2=0x7f46740b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 shutdown_connections 2024-09-17T03:23:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.412+0000 7f467be00640 1 -- 172.21.15.46:0/1415755980 wait complete. 2024-09-17T03:23:38.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 -- 172.21.15.46:0/1855170374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415e8c0 msgr2=0x7ff20415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 --2- 172.21.15.46:0/1855170374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415e8c0 0x7ff20415ecd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff20415cd60 tx=0x7ff1f401c8c0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 -- 172.21.15.46:0/1855170374 shutdown_connections 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 --2- 172.21.15.46:0/1855170374 >> v2:172.21.15.99:3300/0 conn(0x7ff20415bdf0 0x7ff204165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 --2- 172.21.15.46:0/1855170374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415e8c0 0x7ff20415ecd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 -- 172.21.15.46:0/1855170374 >> 172.21.15.46:0/1855170374 conn(0x7ff2040b2040 msgr2=0x7ff2040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 -- 172.21.15.46:0/1855170374 shutdown_connections 2024-09-17T03:23:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 -- 172.21.15.46:0/1855170374 wait complete. 2024-09-17T03:23:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 Processor -- start 2024-09-17T03:23:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.596+0000 7ff209600640 1 -- start start 2024-09-17T03:23:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff20415b2d0 0x7ff2041fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 0x7ff20413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff20415e8c0 0x7ff2041f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff204168c00 con 0x7ff20415bdf0 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff204168a80 con 0x7ff20415e8c0 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff204168d80 con 0x7ff20415b2d0 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 0x7ff20413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff202a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff20415b2d0 0x7ff2041fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 0x7ff20413d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51526/0 (socket says 172.21.15.46:51526) 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 -- 172.21.15.46:0/3568558727 learned_addr learned my addr 172.21.15.46:0/3568558727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff203400640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff20415e8c0 0x7ff2041f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 -- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff20415b2d0 msgr2=0x7ff2041fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff20415b2d0 0x7ff2041fbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 -- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff20415e8c0 msgr2=0x7ff2041f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff20415e8c0 0x7ff2041f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 -- 172.21.15.46:0/3568558727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2041fc4f0 con 0x7ff20415bdf0 2024-09-17T03:23:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1fbe00640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 0x7ff20413d660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00c3f0 tx=0x7ff1ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec017070 con 0x7ff20415bdf0 2024-09-17T03:23:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1ec0053c0 con 0x7ff20415bdf0 2024-09-17T03:23:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec003860 con 0x7ff20415bdf0 2024-09-17T03:23:38.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2041fc7d0 con 0x7ff20415bdf0 2024-09-17T03:23:38.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2040cf010 con 0x7ff20415bdf0 2024-09-17T03:23:38.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.600+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2040cd0c0 con 0x7ff20415bdf0 2024-09-17T03:23:38.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.604+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7ff1ec01a070 con 0x7ff20415bdf0 2024-09-17T03:23:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.604+0000 7ff1faa00640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1e0073440 0x7ff1e0075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.604+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff1ec0938c0 con 0x7ff20415bdf0 2024-09-17T03:23:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.604+0000 7ff202a00640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1e0073440 0x7ff1e0075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.604+0000 7ff202a00640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1e0073440 0x7ff1e0075950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40af000 tx=0x7ff1f40a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.608+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff1ec05df60 con 0x7ff20415bdf0 2024-09-17T03:23:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.808+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff20406b540 con 0x7ff1e0073440 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff1faa00640 1 -- 172.21.15.46:0/3568558727 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7ff1f40b3040 con 0x7ff1e0073440 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1clone1" 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:38.813 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:23:38.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1e0073440 msgr2=0x7ff1e0075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1e0073440 0x7ff1e0075950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40af000 tx=0x7ff1f40a7040 comp rx=0 tx=0).stop 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 msgr2=0x7ff20413d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 0x7ff20413d660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00c3f0 tx=0x7ff1ec00c8b0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 shutdown_connections 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1e0073440 0x7ff1e0075950 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff20415e8c0 0x7ff2041f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff20415bdf0 0x7ff20413d660 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 --2- 172.21.15.46:0/3568558727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff20415b2d0 0x7ff2041fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 >> 172.21.15.46:0/3568558727 conn(0x7ff2040b2040 msgr2=0x7ff20406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 shutdown_connections 2024-09-17T03:23:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.812+0000 7ff209600640 1 -- 172.21.15.46:0/3568558727 wait complete. 2024-09-17T03:23:38.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 -- 172.21.15.46:0/721246313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d806cdf0 msgr2=0x7f93d806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 --2- 172.21.15.46:0/721246313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d806cdf0 0x7f93d806d200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f93d80b6bf0 tx=0x7f93c001c830 comp rx=0 tx=0).stop 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 -- 172.21.15.46:0/721246313 shutdown_connections 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 --2- 172.21.15.46:0/721246313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d806cdf0 0x7f93d806d200 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 --2- 172.21.15.46:0/721246313 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d806fc90 0x7f93d80700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 --2- 172.21.15.46:0/721246313 >> v2:172.21.15.99:3300/0 conn(0x7f93d8159130 0x7f93d815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 -- 172.21.15.46:0/721246313 >> 172.21.15.46:0/721246313 conn(0x7f93d80b35f0 msgr2=0x7f93d80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 -- 172.21.15.46:0/721246313 shutdown_connections 2024-09-17T03:23:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:38.996+0000 7f93dec00640 1 -- 172.21.15.46:0/721246313 wait complete. 2024-09-17T03:23:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 Processor -- start 2024-09-17T03:23:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 -- start start 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d806cdf0 0x7f93d815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d806fc90 0x7f93d8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 0x7f93d8155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93d815fa10 con 0x7f93d8159130 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93d815f890 con 0x7f93d806fc90 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f93d815fb90 con 0x7f93d806cdf0 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 0x7f93d8155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 0x7f93d8155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51544/0 (socket says 172.21.15.46:51544) 2024-09-17T03:23:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 -- 172.21.15.46:0/2588005859 learned_addr learned my addr 172.21.15.46:0/2588005859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93cf400640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d806fc90 0x7f93d8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93d7e00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d806cdf0 0x7f93d815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 -- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d806cdf0 msgr2=0x7f93d815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d806cdf0 0x7f93d815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 -- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d806fc90 msgr2=0x7f93d8154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d806fc90 0x7f93d8154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 -- 172.21.15.46:0/2588005859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d815b490 con 0x7f93d8159130 2024-09-17T03:23:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dcc00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 0x7f93d8155450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009fd0 tx=0x7f93d000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d000cdc0 con 0x7f93d8159130 2024-09-17T03:23:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93d0004b30 con 0x7f93d8159130 2024-09-17T03:23:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0005020 con 0x7f93d8159130 2024-09-17T03:23:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d8164e40 con 0x7f93d8159130 2024-09-17T03:23:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.000+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d81fc840 con 0x7f93d8159130 2024-09-17T03:23:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.004+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d80700a0 con 0x7f93d8159130 2024-09-17T03:23:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.004+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f93d00051c0 con 0x7f93d8159130 2024-09-17T03:23:39.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.004+0000 7f93d4c00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b4073470 0x7f93b4075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:39.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.004+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f93d0095240 con 0x7f93d8159130 2024-09-17T03:23:39.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.004+0000 7f93d7e00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b4073470 0x7f93b4075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:39.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.008+0000 7f93d7e00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b4073470 0x7f93b4075980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009ac0 tx=0x7f93c80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.008+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93d005f810 con 0x7f93d8159130 2024-09-17T03:23:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:39.208+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f93d81679b0 con 0x7f93b4073470 2024-09-17T03:23:41.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.688+0000 7f93d4c00640 1 -- 172.21.15.46:0/2588005859 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f93d81679b0 con 0x7f93b4073470 2024-09-17T03:23:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b4073470 msgr2=0x7f93b4075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b4073470 0x7f93b4075980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009ac0 tx=0x7f93c80091e0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 msgr2=0x7f93d8155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 0x7f93d8155450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009fd0 tx=0x7f93d000ef10 comp rx=0 tx=0).stop 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 shutdown_connections 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b4073470 0x7f93b4075980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d8159130 0x7f93d8155450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d806fc90 0x7f93d8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 --2- 172.21.15.46:0/2588005859 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d806cdf0 0x7f93d815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 >> 172.21.15.46:0/2588005859 conn(0x7f93d80b35f0 msgr2=0x7f93d80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 shutdown_connections 2024-09-17T03:23:41.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.692+0000 7f93dec00640 1 -- 172.21.15.46:0/2588005859 wait complete. 2024-09-17T03:23:41.697 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.696+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.697 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.696+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.697 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.696+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.698 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.696+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.698 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.696+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.716+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.716+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.716+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.716+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:23:41.716+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:23:41.723 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:23:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:23:41.786 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:23:41.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:23:41.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d7d200640 1 -- 172.21.15.46:0/2741563795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 msgr2=0x7f8d5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2741563795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d5c004810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d78071fd0 tx=0x7f8d6001d1e0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d76800640 1 -- 172.21.15.46:0/2741563795 reap_dead start 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d7d200640 1 -- 172.21.15.46:0/2741563795 shutdown_connections 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d7d200640 1 -- 172.21.15.46:0/2741563795 >> 172.21.15.46:0/2741563795 conn(0x7f8d780b2040 msgr2=0x7f8d780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d7d200640 1 -- 172.21.15.46:0/2741563795 shutdown_connections 2024-09-17T03:23:41.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.956+0000 7f8d7d200640 1 -- 172.21.15.46:0/2741563795 wait complete. 2024-09-17T03:23:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 Processor -- start 2024-09-17T03:23:41.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 -- start start 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d781f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d781f6550 0x7f8d781f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d781f6960 0x7f8d78140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78167b70 con 0x7f8d5c0041f0 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781679f0 con 0x7f8d781f6550 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d7d200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8d78167cf0 con 0x7f8d781f6960 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d781f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d781f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51596/0 (socket says 172.21.15.46:51596) 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 -- 172.21.15.46:0/2947200242 learned_addr learned my addr 172.21.15.46:0/2947200242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:41.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d76800640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d781f6960 0x7f8d78140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d6fe00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d781f6550 0x7f8d781f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 -- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d781f6960 msgr2=0x7f8d78140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d781f6960 0x7f8d78140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 -- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d781f6550 msgr2=0x7f8d781f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d781f6550 0x7f8d781f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d75e00640 1 -- 172.21.15.46:0/2947200242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7815c270 con 0x7f8d5c0041f0 2024-09-17T03:23:41.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.960+0000 7f8d6fe00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d781f6550 0x7f8d781f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:41.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d75e00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d781f7940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d68001870 tx=0x7f8d680039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:41.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68015070 con 0x7f8d5c0041f0 2024-09-17T03:23:41.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d68011860 con 0x7f8d5c0041f0 2024-09-17T03:23:41.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d78141010 con 0x7f8d5c0041f0 2024-09-17T03:23:41.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68011cb0 con 0x7f8d5c0041f0 2024-09-17T03:23:41.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781f7140 con 0x7f8d5c0041f0 2024-09-17T03:23:41.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d78074e30 con 0x7f8d5c0041f0 2024-09-17T03:23:41.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f8d68018070 con 0x7f8d5c0041f0 2024-09-17T03:23:41.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.964+0000 7f8d6e000640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d54073470 0x7f8d54075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:41.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.968+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8d680952b0 con 0x7f8d5c0041f0 2024-09-17T03:23:41.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.968+0000 7f8d6fe00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d54073470 0x7f8d54075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:41.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.968+0000 7f8d6fe00640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d54073470 0x7f8d54075980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d600ad000 tx=0x7f8d60019f80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:41.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:41.972+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d6805f890 con 0x7f8d5c0041f0 2024-09-17T03:23:42.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f8d78152170 con 0x7f8d5c0041f0 2024-09-17T03:23:42.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d6e000640 1 -- 172.21.15.46:0/2947200242 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v33)=0 v33) ==== 131+0+0 (secure 0 0 0) 0x7f8d680642c0 con 0x7f8d5c0041f0 2024-09-17T03:23:42.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d54073470 msgr2=0x7f8d54075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d54073470 0x7f8d54075980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d600ad000 tx=0x7f8d60019f80 comp rx=0 tx=0).stop 2024-09-17T03:23:42.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 msgr2=0x7f8d781f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d781f7940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d68001870 tx=0x7f8d680039d0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 shutdown_connections 2024-09-17T03:23:42.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d54073470 0x7f8d54075980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d781f6960 0x7f8d78140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d781f6550 0x7f8d781f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 --2- 172.21.15.46:0/2947200242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d5c0041f0 0x7f8d781f7940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 >> 172.21.15.46:0/2947200242 conn(0x7f8d780b2040 msgr2=0x7f8d780b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:42.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 shutdown_connections 2024-09-17T03:23:42.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.148+0000 7f8d7d200640 1 -- 172.21.15.46:0/2947200242 wait complete. 2024-09-17T03:23:42.171 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:23:42.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:23:42.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:23:42.357 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:23:42.357 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:23:42.357 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:42.357 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:42.366 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:42.366 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:42.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:23:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 -- 172.21.15.46:0/1617299941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32841527b0 msgr2=0x7f328415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 --2- 172.21.15.46:0/1617299941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32841527b0 0x7f328415bb60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3284074b20 tx=0x7f327001c820 comp rx=0 tx=0).stop 2024-09-17T03:23:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 -- 172.21.15.46:0/1617299941 shutdown_connections 2024-09-17T03:23:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 --2- 172.21.15.46:0/1617299941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32841527b0 0x7f328415bb60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 -- 172.21.15.46:0/1617299941 >> 172.21.15.46:0/1617299941 conn(0x7f32840b2040 msgr2=0x7f32840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 -- 172.21.15.46:0/1617299941 shutdown_connections 2024-09-17T03:23:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.616+0000 7f328b400640 1 -- 172.21.15.46:0/1617299941 wait complete. 2024-09-17T03:23:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 Processor -- start 2024-09-17T03:23:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 -- start start 2024-09-17T03:23:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32840b4de0 0x7f32841444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 0x7f32841fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32841527b0 0x7f3284158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3284169b30 con 0x7f32840b5c70 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32841699b0 con 0x7f32841527b0 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f328b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3284169cb0 con 0x7f32840b4de0 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 0x7f32841fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 0x7f32841fa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49860/0 (socket says 172.21.15.46:49860) 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 -- 172.21.15.46:0/3064112822 learned_addr learned my addr 172.21.15.46:0/3064112822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3289400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32841527b0 0x7f3284158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3288a00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32840b4de0 0x7f32841444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 -- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32840b4de0 msgr2=0x7f32841444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32840b4de0 0x7f32841444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 -- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32841527b0 msgr2=0x7f3284158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32841527b0 0x7f3284158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 -- 172.21.15.46:0/3064112822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32840cf8b0 con 0x7f32840b5c70 2024-09-17T03:23:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.620+0000 7f3283e00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 0x7f32841fa8f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3270019430 tx=0x7f32700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32700bd070 con 0x7f32840b5c70 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3270003600 con 0x7f32840b5c70 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32700b2040 con 0x7f32840b5c70 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32841523b0 con 0x7f32840b5c70 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32840cd690 con 0x7f32840b5c70 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32840cd120 con 0x7f32840b5c70 2024-09-17T03:23:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f3270003a30 con 0x7f32840b5c70 2024-09-17T03:23:42.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3280c00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3264073470 0x7f3264075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:42.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f327012e7f0 con 0x7f32840b5c70 2024-09-17T03:23:42.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.624+0000 7f3288a00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3264073470 0x7f3264075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.628+0000 7f3288a00640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3264073470 0x7f3264075980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3278000f70 tx=0x7f3278009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.628+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32700164d0 con 0x7f32840b5c70 2024-09-17T03:23:42.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.816+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f328415bb60 con 0x7f32840b5c70 2024-09-17T03:23:42.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f3280c00640 1 -- 172.21.15.46:0/3064112822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f32700f9390 con 0x7f32840b5c70 2024-09-17T03:23:42.821 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3264073470 msgr2=0x7f3264075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3264073470 0x7f3264075980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3278000f70 tx=0x7f3278009040 comp rx=0 tx=0).stop 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 msgr2=0x7f32841fa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 0x7f32841fa8f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3270019430 tx=0x7f32700040c0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 shutdown_connections 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3264073470 0x7f3264075980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32841527b0 0x7f3284158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32840b5c70 0x7f32841fa8f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 --2- 172.21.15.46:0/3064112822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32840b4de0 0x7f32841444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 >> 172.21.15.46:0/3064112822 conn(0x7f32840b2040 msgr2=0x7f3284154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 shutdown_connections 2024-09-17T03:23:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:42.820+0000 7f328b400640 1 -- 172.21.15.46:0/3064112822 wait complete. 2024-09-17T03:23:42.843 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:23:42.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:42.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:23:44.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:23:44.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:23:44.646 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:23:44.660 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:23:44.661 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:23:44.661 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:44.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:23:44.675 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:23:44.675 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:44.675 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:44.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:23:44.717 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:23:44.717 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:23:44.717 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:44.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:23:44.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:23:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 -- 172.21.15.46:0/18116945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cd2e0 msgr2=0x7ff4c4070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 --2- 172.21.15.46:0/18116945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cd2e0 0x7ff4c4070e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009840 tx=0x7ff4b401c740 comp rx=0 tx=0).stop 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 -- 172.21.15.46:0/18116945 shutdown_connections 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 --2- 172.21.15.46:0/18116945 >> v2:172.21.15.99:3300/0 conn(0x7ff4c40cf1b0 0x7ff4c40cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 --2- 172.21.15.46:0/18116945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cd2e0 0x7ff4c4070e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 -- 172.21.15.46:0/18116945 >> 172.21.15.46:0/18116945 conn(0x7ff4c40b2c40 msgr2=0x7ff4c40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 -- 172.21.15.46:0/18116945 shutdown_connections 2024-09-17T03:23:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.984+0000 7ff4c9200640 1 -- 172.21.15.46:0/18116945 wait complete. 2024-09-17T03:23:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 Processor -- start 2024-09-17T03:23:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 -- start start 2024-09-17T03:23:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 0x7ff4c413edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cf1b0 0x7ff4c41f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c41602a0 0x7ff4c40b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4169e70 con 0x7ff4c41602a0 2024-09-17T03:23:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4169cf0 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.988+0000 7ff4c9200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c4169ff0 con 0x7ff4c40cf1b0 2024-09-17T03:23:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 0x7ff4c413edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c41602a0 0x7ff4c40b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 0x7ff4c413edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:57744/0 (socket says 172.21.15.46:57744) 2024-09-17T03:23:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4b9400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cf1b0 0x7ff4c41f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 -- 172.21.15.46:0/3591588541 learned_addr learned my addr 172.21.15.46:0/3591588541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 -- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cf1b0 msgr2=0x7ff4c41f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cf1b0 0x7ff4c41f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 -- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c41602a0 msgr2=0x7ff4c40b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c41602a0 0x7ff4c40b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 -- 172.21.15.46:0/3591588541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c413f300 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c2800640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c41602a0 0x7ff4c40b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4b9400640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cf1b0 0x7ff4c41f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c1e00640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 0x7ff4c413edc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009790 tx=0x7ff4b4005310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40ac070 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b40046d0 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c41fcc60 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004b00 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41fd220 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c40cf5c0 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.992+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7ff4b40af070 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.996+0000 7ff4baa00640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff4a0073560 0x7ff4a0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.996+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff4b412dfa0 con 0x7ff4c40cd2e0 2024-09-17T03:23:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.996+0000 7ff4b9400640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff4a0073560 0x7ff4a0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:44.996+0000 7ff4b9400640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff4a0073560 0x7ff4a0075a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac001870 tx=0x7ff4ac0079d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.000+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff4b40bf070 con 0x7ff4c40cd2e0 2024-09-17T03:23:45.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.212+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4c40b2540 con 0x7ff4c40cd2e0 2024-09-17T03:23:45.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.212+0000 7ff4baa00640 1 -- 172.21.15.46:0/3591588541 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5236 (secure 0 0 0) 0x7ff4b4016330 con 0x7ff4c40cd2e0 2024-09-17T03:23:45.215 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:45.215 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"btime":"2024-09-17T03:22:01:614270+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:52.949184+0000","modified":"2024-09-17T03:21:55.955106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244821":{"gid":244821,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3646914597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3646914597},{"type":"v1","addr":"172.21.15.99:6835","nonce":3646914597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244821,"qdb_cluster":[244821]},"id":2}]} 2024-09-17T03:23:45.215 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-09-17T03:23:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff4a0073560 msgr2=0x7ff4a0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff4a0073560 0x7ff4a0075a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac001870 tx=0x7ff4ac0079d0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 msgr2=0x7ff4c413edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 0x7ff4c413edc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009790 tx=0x7ff4b4005310 comp rx=0 tx=0).stop 2024-09-17T03:23:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 shutdown_connections 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff4a0073560 0x7ff4a0075a70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4c41602a0 0x7ff4c40b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4c40cf1b0 0x7ff4c41f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 --2- 172.21.15.46:0/3591588541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c40cd2e0 0x7ff4c413edc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 >> 172.21.15.46:0/3591588541 conn(0x7ff4c40b2c40 msgr2=0x7ff4c40b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 shutdown_connections 2024-09-17T03:23:45.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.216+0000 7ff4c9200640 1 -- 172.21.15.46:0/3591588541 wait complete. 2024-09-17T03:23:45.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:23:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 -- 172.21.15.46:0/468836956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7900b54f0 msgr2=0x7fd79015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 --2- 172.21.15.46:0/468836956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7900b54f0 0x7fd79015bb90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd790074b20 tx=0x7fd78801cb70 comp rx=0 tx=0).stop 2024-09-17T03:23:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 -- 172.21.15.46:0/468836956 shutdown_connections 2024-09-17T03:23:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 --2- 172.21.15.46:0/468836956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7900b54f0 0x7fd79015bb90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 -- 172.21.15.46:0/468836956 >> 172.21.15.46:0/468836956 conn(0x7fd7900b2040 msgr2=0x7fd7900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 -- 172.21.15.46:0/468836956 shutdown_connections 2024-09-17T03:23:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.392+0000 7fd798000640 1 -- 172.21.15.46:0/468836956 wait complete. 2024-09-17T03:23:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 Processor -- start 2024-09-17T03:23:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 -- start start 2024-09-17T03:23:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7900b5c70 0x7fd7900ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 0x7fd7900b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7900b1060 0x7fd7900cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd790169b90 con 0x7fd7901527b0 2024-09-17T03:23:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd790169a10 con 0x7fd7900b5c70 2024-09-17T03:23:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd798000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd790169d10 con 0x7fd7900b1060 2024-09-17T03:23:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd794c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 0x7fd7900b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd794c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 0x7fd7900b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49878/0 (socket says 172.21.15.46:49878) 2024-09-17T03:23:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd794c00640 1 -- 172.21.15.46:0/2817677864 learned_addr learned my addr 172.21.15.46:0/2817677864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd795600640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7900b5c70 0x7fd7900ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.396+0000 7fd796000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7900b1060 0x7fd7900cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd794c00640 1 -- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7900b1060 msgr2=0x7fd7900cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd794c00640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7900b1060 0x7fd7900cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd794c00640 1 -- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7900b5c70 msgr2=0x7fd7900ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd794c00640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7900b5c70 0x7fd7900ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd794c00640 1 -- 172.21.15.46:0/2817677864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900af1d0 con 0x7fd7901527b0 2024-09-17T03:23:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd796000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7900b1060 0x7fd7900cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd794c00640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 0x7fd7900b0b20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00c3c0 tx=0x7fd77c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c017070 con 0x7fd7901527b0 2024-09-17T03:23:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd77c0053c0 con 0x7fd7901527b0 2024-09-17T03:23:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7900aa600 con 0x7fd7901527b0 2024-09-17T03:23:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c003860 con 0x7fd7901527b0 2024-09-17T03:23:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7900aabc0 con 0x7fd7901527b0 2024-09-17T03:23:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79015c120 con 0x7fd7901527b0 2024-09-17T03:23:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.400+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd77c01a070 con 0x7fd7901527b0 2024-09-17T03:23:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.404+0000 7fd78d600640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd770073590 0x7fd770075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.404+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd77c093cb0 con 0x7fd7901527b0 2024-09-17T03:23:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.404+0000 7fd795600640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd770073590 0x7fd770075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.404+0000 7fd795600640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd770073590 0x7fd770075aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7840046c0 tx=0x7fd784007ea0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.404+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd77c021080 con 0x7fd7901527b0 2024-09-17T03:23:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.616+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd79015be40 con 0x7fd7901527b0 2024-09-17T03:23:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd78d600640 1 -- 172.21.15.46:0/2817677864 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5236 (secure 0 0 0) 0x7fd77c05e1a0 con 0x7fd7901527b0 2024-09-17T03:23:45.621 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:45.621 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"btime":"2024-09-17T03:22:01:614270+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:52.949184+0000","modified":"2024-09-17T03:21:55.955106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244821":{"gid":244821,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3646914597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3646914597},{"type":"v1","addr":"172.21.15.99:6835","nonce":3646914597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244821,"qdb_cluster":[244821]},"id":2}]} 2024-09-17T03:23:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd770073590 msgr2=0x7fd770075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd770073590 0x7fd770075aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7840046c0 tx=0x7fd784007ea0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 msgr2=0x7fd7900b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 0x7fd7900b0b20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00c3c0 tx=0x7fd77c00c880 comp rx=0 tx=0).stop 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 shutdown_connections 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd770073590 0x7fd770075aa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7900b1060 0x7fd7900cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7901527b0 0x7fd7900b0b20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 --2- 172.21.15.46:0/2817677864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7900b5c70 0x7fd7900ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 >> 172.21.15.46:0/2817677864 conn(0x7fd7900b2040 msgr2=0x7fd79015b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 shutdown_connections 2024-09-17T03:23:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.620+0000 7fd798000640 1 -- 172.21.15.46:0/2817677864 wait complete. 2024-09-17T03:23:45.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:23:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 -- 172.21.15.46:0/2052568622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c0749a0 msgr2=0x7f2c5c074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 --2- 172.21.15.46:0/2052568622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c0749a0 0x7f2c5c074db0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0726f0 tx=0x7f2c4c01c8c0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 -- 172.21.15.46:0/2052568622 shutdown_connections 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 --2- 172.21.15.46:0/2052568622 >> v2:172.21.15.99:3300/0 conn(0x7f2c5c0cefa0 0x7f2c5c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 --2- 172.21.15.46:0/2052568622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c0749a0 0x7f2c5c074db0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 -- 172.21.15.46:0/2052568622 >> 172.21.15.46:0/2052568622 conn(0x7f2c5c0b2040 msgr2=0x7f2c5c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 -- 172.21.15.46:0/2052568622 shutdown_connections 2024-09-17T03:23:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.800+0000 7f2c61c00640 1 -- 172.21.15.46:0/2052568622 wait complete. 2024-09-17T03:23:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 Processor -- start 2024-09-17T03:23:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 -- start start 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5c0749a0 0x7f2c5c13c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2c5c0cefa0 0x7f2c5c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 0x7f2c5c12d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c16a060 con 0x7f2c5c160330 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c169ee0 con 0x7f2c5c0749a0 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5c16a1e0 con 0x7f2c5c0cefa0 2024-09-17T03:23:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 0x7f2c5c12d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5c0749a0 0x7f2c5c13c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c53e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2c5c0cefa0 0x7f2c5c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 0x7f2c5c12d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49886/0 (socket says 172.21.15.46:49886) 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 -- 172.21.15.46:0/1378828010 learned_addr learned my addr 172.21.15.46:0/1378828010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 -- 172.21.15.46:0/1378828010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2c5c0cefa0 msgr2=0x7f2c5c1432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2c5c0cefa0 0x7f2c5c1432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 -- 172.21.15.46:0/1378828010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5c0749a0 msgr2=0x7f2c5c13c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5c0749a0 0x7f2c5c13c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 -- 172.21.15.46:0/1378828010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c13cfa0 con 0x7f2c5c160330 2024-09-17T03:23:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c5b400640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 0x7f2c5c12d890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c730 tx=0x7f2c5400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54003b80 con 0x7f2c5c160330 2024-09-17T03:23:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c54003d40 con 0x7f2c5c160330 2024-09-17T03:23:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c540052d0 con 0x7f2c5c160330 2024-09-17T03:23:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c13d280 con 0x7f2c5c160330 2024-09-17T03:23:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.804+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c13d840 con 0x7f2c5c160330 2024-09-17T03:23:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.808+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5c074db0 con 0x7f2c5c160330 2024-09-17T03:23:45.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.808+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2c54005680 con 0x7f2c5c160330 2024-09-17T03:23:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.808+0000 7f2c52a00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2c38073590 0x7f2c38075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.808+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2c54094660 con 0x7f2c5c160330 2024-09-17T03:23:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.808+0000 7f2c5aa00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2c38073590 0x7f2c38075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.808+0000 7f2c5aa00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2c38073590 0x7f2c38075aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0af000 tx=0x7f2c4c01b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:45.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.812+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2c5405eb30 con 0x7f2c5c160330 2024-09-17T03:23:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.988+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c5c13ca20 con 0x7f2c5c160330 2024-09-17T03:23:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.988+0000 7f2c52a00640 1 -- 172.21.15.46:0/1378828010 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16693 (secure 0 0 0) 0x7f2c54063560 con 0x7f2c5c160330 2024-09-17T03:23:45.992 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:45.992 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:54.529415+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:51.251673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:51.902649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:23:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2c38073590 msgr2=0x7f2c38075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2c38073590 0x7f2c38075aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0af000 tx=0x7f2c4c01b100 comp rx=0 tx=0).stop 2024-09-17T03:23:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 msgr2=0x7f2c5c12d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 0x7f2c5c12d890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c730 tx=0x7f2c5400cbf0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 shutdown_connections 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2c38073590 0x7f2c38075aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c5c160330 0x7f2c5c12d890 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2c5c0cefa0 0x7f2c5c1432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 --2- 172.21.15.46:0/1378828010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c5c0749a0 0x7f2c5c13c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 >> 172.21.15.46:0/1378828010 conn(0x7f2c5c0b2040 msgr2=0x7f2c5c15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 shutdown_connections 2024-09-17T03:23:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:45.992+0000 7f2c61c00640 1 -- 172.21.15.46:0/1378828010 wait complete. 2024-09-17T03:23:46.015 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:23:46.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:23:46.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 -- 172.21.15.46:0/32188370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58069500 msgr2=0x7fdd580cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/32188370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58069500 0x7fdd580cfd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009900 tx=0x7fdd4401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 -- 172.21.15.46:0/32188370 shutdown_connections 2024-09-17T03:23:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/32188370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd58069500 0x7fdd580cfd50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 -- 172.21.15.46:0/32188370 >> 172.21.15.46:0/32188370 conn(0x7fdd580b35f0 msgr2=0x7fdd580b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 -- 172.21.15.46:0/32188370 shutdown_connections 2024-09-17T03:23:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 -- 172.21.15.46:0/32188370 wait complete. 2024-09-17T03:23:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 Processor -- start 2024-09-17T03:23:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.172+0000 7fdd5fa00640 1 -- start start 2024-09-17T03:23:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd580d0290 0x7fdd581546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd58160c20 0x7fdd5806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 0x7fdd5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5816b380 con 0x7fdd5814c210 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5816b200 con 0x7fdd580d0290 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5816b500 con 0x7fdd58160c20 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 0x7fdd5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 0x7fdd5806d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49898/0 (socket says 172.21.15.46:49898) 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 -- 172.21.15.46:0/2375779274 learned_addr learned my addr 172.21.15.46:0/2375779274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5d000640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd580d0290 0x7fdd581546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd57e00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd58160c20 0x7fdd5806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 -- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd58160c20 msgr2=0x7fdd5806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd58160c20 0x7fdd5806cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 -- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd580d0290 msgr2=0x7fdd581546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd580d0290 0x7fdd581546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 -- 172.21.15.46:0/2375779274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd580b49c0 con 0x7fdd5814c210 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd57e00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd58160c20 0x7fdd5806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5d000640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd580d0290 0x7fdd581546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5da00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 0x7fdd5806d330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c730 tx=0x7fdd5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003b80 con 0x7fdd5814c210 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd50003d40 con 0x7fdd5814c210 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd500052d0 con 0x7fdd5814c210 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd58156df0 con 0x7fdd5814c210 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd580b64e0 con 0x7fdd5814c210 2024-09-17T03:23:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.176+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd580cfd50 con 0x7fdd5814c210 2024-09-17T03:23:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.180+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fdd50005680 con 0x7fdd5814c210 2024-09-17T03:23:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.180+0000 7fdd54c00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd38073560 0x7fdd38075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.180+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdd50094880 con 0x7fdd5814c210 2024-09-17T03:23:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.180+0000 7fdd5d000640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd38073560 0x7fdd38075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.180+0000 7fdd5d000640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd38073560 0x7fdd38075a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c007fd0 tx=0x7fdd4c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.184+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdd5005ebf0 con 0x7fdd5814c210 2024-09-17T03:23:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.392+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd580b1070 con 0x7fdd5814c210 2024-09-17T03:23:46.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.392+0000 7fdd54c00640 1 -- 172.21.15.46:0/2375779274 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5236 (secure 0 0 0) 0x7fdd50063620 con 0x7fdd5814c210 2024-09-17T03:23:46.395 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:46.396 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"btime":"2024-09-17T03:22:01:614270+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:52.949184+0000","modified":"2024-09-17T03:21:55.955106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244821":{"gid":244821,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3646914597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3646914597},{"type":"v1","addr":"172.21.15.99:6835","nonce":3646914597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244821,"qdb_cluster":[244821]},"id":2}]} 2024-09-17T03:23:46.396 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-09-17T03:23:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd38073560 msgr2=0x7fdd38075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd38073560 0x7fdd38075a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c007fd0 tx=0x7fdd4c009040 comp rx=0 tx=0).stop 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 msgr2=0x7fdd5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 0x7fdd5806d330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c730 tx=0x7fdd5000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 shutdown_connections 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd38073560 0x7fdd38075a70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd5814c210 0x7fdd5806d330 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd58160c20 0x7fdd5806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 --2- 172.21.15.46:0/2375779274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd580d0290 0x7fdd581546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 >> 172.21.15.46:0/2375779274 conn(0x7fdd580b35f0 msgr2=0x7fdd580d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 shutdown_connections 2024-09-17T03:23:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.396+0000 7fdd5fa00640 1 -- 172.21.15.46:0/2375779274 wait complete. 2024-09-17T03:23:46.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 -- 172.21.15.46:0/1331578399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac157030 msgr2=0x7f62ac0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 --2- 172.21.15.46:0/1331578399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac157030 0x7f62ac0b5a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f629c009840 tx=0x7f629c01c790 comp rx=0 tx=0).stop 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 -- 172.21.15.46:0/1331578399 shutdown_connections 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 --2- 172.21.15.46:0/1331578399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac157030 0x7f62ac0b5a80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 -- 172.21.15.46:0/1331578399 >> 172.21.15.46:0/1331578399 conn(0x7f62ac0708c0 msgr2=0x7f62ac070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 -- 172.21.15.46:0/1331578399 shutdown_connections 2024-09-17T03:23:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.576+0000 7f62b1400640 1 -- 172.21.15.46:0/1331578399 wait complete. 2024-09-17T03:23:46.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 Processor -- start 2024-09-17T03:23:46.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 -- start start 2024-09-17T03:23:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62ac157030 0x7f62ac13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 0x7f62ac13bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62ac1354d0 0x7f62ac1358e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac16b850 con 0x7f62ac160d50 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac16b6d0 con 0x7f62ac157030 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62b1400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac16b9d0 con 0x7f62ac1354d0 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 0x7f62ac13bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 0x7f62ac13bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49922/0 (socket says 172.21.15.46:49922) 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62a3e00640 1 -- 172.21.15.46:0/2013320788 learned_addr learned my addr 172.21.15.46:0/2013320788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62aa800640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62ac157030 0x7f62ac13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.580+0000 7f62ab200640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62ac1354d0 0x7f62ac1358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a3e00640 1 -- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62ac1354d0 msgr2=0x7f62ac1358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a3e00640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62ac1354d0 0x7f62ac1358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a3e00640 1 -- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62ac157030 msgr2=0x7f62ac13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a3e00640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62ac157030 0x7f62ac13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a3e00640 1 -- 172.21.15.46:0/2013320788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac0d2040 con 0x7f62ac160d50 2024-09-17T03:23:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a3e00640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 0x7f62ac13bca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f629400c3c0 tx=0x7f629400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294017070 con 0x7f62ac160d50 2024-09-17T03:23:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6294003600 con 0x7f62ac160d50 2024-09-17T03:23:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac0b4270 con 0x7f62ac160d50 2024-09-17T03:23:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294003a50 con 0x7f62ac160d50 2024-09-17T03:23:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac0b39d0 con 0x7f62ac160d50 2024-09-17T03:23:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac0d1660 con 0x7f62ac160d50 2024-09-17T03:23:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.584+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f629401a070 con 0x7f62ac160d50 2024-09-17T03:23:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.588+0000 7f62a2a00640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6288073590 0x7f6288075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.588+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6294093a90 con 0x7f62ac160d50 2024-09-17T03:23:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.588+0000 7f62aa800640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6288073590 0x7f6288075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.588+0000 7f62aa800640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6288073590 0x7f6288075aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f629c0af000 tx=0x7f629c0b2040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.592+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f629405df00 con 0x7f62ac160d50 2024-09-17T03:23:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.804+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62ac0d1380 con 0x7f62ac160d50 2024-09-17T03:23:46.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.804+0000 7f62a2a00640 1 -- 172.21.15.46:0/2013320788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5236 (secure 0 0 0) 0x7f6294062930 con 0x7f62ac160d50 2024-09-17T03:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:46.806 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"btime":"2024-09-17T03:22:01:614270+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:21:52.949184+0000","modified":"2024-09-17T03:21:55.955106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":244821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_244821":{"gid":244821,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3646914597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3646914597},{"type":"v1","addr":"172.21.15.99:6835","nonce":3646914597}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":244821,"qdb_cluster":[244821]},"id":2}]} 2024-09-17T03:23:46.807 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-09-17T03:23:46.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.804+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6288073590 msgr2=0x7f6288075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.804+0000 7f62b1400640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6288073590 0x7f6288075aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f629c0af000 tx=0x7f629c0b2040 comp rx=0 tx=0).stop 2024-09-17T03:23:46.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.804+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 msgr2=0x7f62ac13bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.804+0000 7f62b1400640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 0x7f62ac13bca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f629400c3c0 tx=0x7f629400c880 comp rx=0 tx=0).stop 2024-09-17T03:23:46.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 shutdown_connections 2024-09-17T03:23:46.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6288073590 0x7f6288075aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62ac1354d0 0x7f62ac1358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ac160d50 0x7f62ac13bca0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 --2- 172.21.15.46:0/2013320788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62ac157030 0x7f62ac13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 >> 172.21.15.46:0/2013320788 conn(0x7f62ac0708c0 msgr2=0x7f62ac14f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:46.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 shutdown_connections 2024-09-17T03:23:46.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.808+0000 7f62b1400640 1 -- 172.21.15.46:0/2013320788 wait complete. 2024-09-17T03:23:46.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:23:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 -- 172.21.15.46:0/902132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 msgr2=0x7fb9c015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/902132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 0x7fb9c015bb90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009840 tx=0x7fb9b001c6b0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 -- 172.21.15.46:0/902132705 shutdown_connections 2024-09-17T03:23:46.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/902132705 >> v2:172.21.15.99:3300/0 conn(0x7fb9c006cdf0 0x7fb9c006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/902132705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 0x7fb9c015bb90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 -- 172.21.15.46:0/902132705 >> 172.21.15.46:0/902132705 conn(0x7fb9c00b35f0 msgr2=0x7fb9c00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:46.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 -- 172.21.15.46:0/902132705 shutdown_connections 2024-09-17T03:23:46.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 -- 172.21.15.46:0/902132705 wait complete. 2024-09-17T03:23:46.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 Processor -- start 2024-09-17T03:23:46.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.988+0000 7fb9c6e00640 1 -- start start 2024-09-17T03:23:46.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c006cdf0 0x7fb9c01f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 0x7fb9c014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 0x7fb9c00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c0168e70 con 0x7fb9c006fc90 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c0168cf0 con 0x7fb9c006cdf0 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c0168ff0 con 0x7fb9c0159130 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 0x7fb9c014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c006cdf0 0x7fb9c01f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 0x7fb9c014cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49930/0 (socket says 172.21.15.46:49930) 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c006cdf0 0x7fb9c01f77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:57810/0 (socket says 172.21.15.46:57810) 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 -- 172.21.15.46:0/1006303423 learned_addr learned my addr 172.21.15.46:0/1006303423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c4e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 0x7fb9c00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 -- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 msgr2=0x7fb9c00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 0x7fb9c00b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 -- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c006cdf0 msgr2=0x7fb9c01f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c006cdf0 0x7fb9c01f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 -- 172.21.15.46:0/1006303423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c0150e90 con 0x7fb9c006fc90 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9b7400640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 0x7fb9c014cd40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800e9f0 tx=0x7fb9a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800ccc0 con 0x7fb9c006fc90 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9a8004020 con 0x7fb9c006fc90 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c01508d0 con 0x7fb9c006fc90 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800ce60 con 0x7fb9c006fc90 2024-09-17T03:23:46.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c0154720 con 0x7fb9c006fc90 2024-09-17T03:23:46.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.992+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c006d200 con 0x7fb9c006fc90 2024-09-17T03:23:46.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.996+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fb9a8003600 con 0x7fb9c006fc90 2024-09-17T03:23:46.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.996+0000 7fb9bcc00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb99c073560 0x7fb99c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:46.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.996+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb9a8095120 con 0x7fb9c006fc90 2024-09-17T03:23:46.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.996+0000 7fb9bfe00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb99c073560 0x7fb99c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:46.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:46.996+0000 7fb9bfe00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb99c073560 0x7fb99c075a70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00ae000 tx=0x7fb9b001aef0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:47.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.000+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb9a805f4d0 con 0x7fb9c006fc90 2024-09-17T03:23:47.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.176+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9c01576f0 con 0x7fb9c006fc90 2024-09-17T03:23:47.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.176+0000 7fb9bcc00640 1 -- 172.21.15.46:0/1006303423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16693 (secure 0 0 0) 0x7fb9a8063f00 con 0x7fb9c006fc90 2024-09-17T03:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:47.179 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:21:54.529415+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:21:51.251673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:21:51.902649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:23:47.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb99c073560 msgr2=0x7fb99c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb99c073560 0x7fb99c075a70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00ae000 tx=0x7fb9b001aef0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 msgr2=0x7fb9c014cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 0x7fb9c014cd40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800e9f0 tx=0x7fb9a800eeb0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 shutdown_connections 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb99c073560 0x7fb99c075a70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9c0159130 0x7fb9c00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9c006fc90 0x7fb9c014cd40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 --2- 172.21.15.46:0/1006303423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c006cdf0 0x7fb9c01f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 >> 172.21.15.46:0/1006303423 conn(0x7fb9c00b35f0 msgr2=0x7fb9c0076030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 shutdown_connections 2024-09-17T03:23:47.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.180+0000 7fb9c6e00640 1 -- 172.21.15.46:0/1006303423 wait complete. 2024-09-17T03:23:47.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:23:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 -- 172.21.15.46:0/11392974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 msgr2=0x7f8980074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 --2- 172.21.15.46:0/11392974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 0x7f8980074410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f896c009840 tx=0x7f896c01c740 comp rx=0 tx=0).stop 2024-09-17T03:23:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 -- 172.21.15.46:0/11392974 shutdown_connections 2024-09-17T03:23:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 --2- 172.21.15.46:0/11392974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8980074950 0x7f898015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 --2- 172.21.15.46:0/11392974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 0x7f8980074410 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 -- 172.21.15.46:0/11392974 >> 172.21.15.46:0/11392974 conn(0x7f89800708c0 msgr2=0x7f8980070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:47.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 -- 172.21.15.46:0/11392974 shutdown_connections 2024-09-17T03:23:47.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 -- 172.21.15.46:0/11392974 wait complete. 2024-09-17T03:23:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 Processor -- start 2024-09-17T03:23:47.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.364+0000 7f8987e00640 1 -- start start 2024-09-17T03:23:47.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8980074950 0x7f89800b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 0x7f8980154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 0x7f898006df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f898016a310 con 0x7f8980157030 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f898016a190 con 0x7f89800b5a80 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f898016a490 con 0x7f8980074950 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 0x7f898006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8984a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 0x7f8980154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 0x7f898006df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49944/0 (socket says 172.21.15.46:49944) 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 -- 172.21.15.46:0/550609888 learned_addr learned my addr 172.21.15.46:0/550609888 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985400640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8980074950 0x7f89800b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 -- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8980074950 msgr2=0x7f89800b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8980074950 0x7f89800b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 -- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 msgr2=0x7f8980154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 0x7f8980154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 -- 172.21.15.46:0/550609888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898013f810 con 0x7f8980157030 2024-09-17T03:23:47.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8985e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 0x7f898006df70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f897800eca0 tx=0x7f897800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978003b80 con 0x7f8980157030 2024-09-17T03:23:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8978003d20 con 0x7f8980157030 2024-09-17T03:23:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978004280 con 0x7f8980157030 2024-09-17T03:23:47.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898013faf0 con 0x7f8980157030 2024-09-17T03:23:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8980141650 con 0x7f8980157030 2024-09-17T03:23:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.368+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89800b4e50 con 0x7f8980157030 2024-09-17T03:23:47.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.372+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8978004420 con 0x7f8980157030 2024-09-17T03:23:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.372+0000 7f897d600640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8960073560 0x7f8960075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.372+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8978095550 con 0x7f8980157030 2024-09-17T03:23:47.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.372+0000 7f8985400640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8960073560 0x7f8960075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.372+0000 7f8985400640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8960073560 0x7f8960075a70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8974007990 tx=0x7f89740012e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.376+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f897805f950 con 0x7f8980157030 2024-09-17T03:23:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.588+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f898015cf90 con 0x7f8980157030 2024-09-17T03:23:47.592 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:23:47.592+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:23:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.592+0000 7f897d600640 1 -- 172.21.15.46:0/550609888 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f8978064380 con 0x7f8980157030 2024-09-17T03:23:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:23:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8960073560 msgr2=0x7f8960075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8960073560 0x7f8960075a70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8974007990 tx=0x7f89740012e0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 msgr2=0x7f898006df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 0x7f898006df70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f897800eca0 tx=0x7f897800c6a0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 shutdown_connections 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8960073560 0x7f8960075a70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8980157030 0x7f898006df70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89800b5a80 0x7f8980154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 --2- 172.21.15.46:0/550609888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8980074950 0x7f89800b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 >> 172.21.15.46:0/550609888 conn(0x7f89800708c0 msgr2=0x7f898015c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 shutdown_connections 2024-09-17T03:23:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.600+0000 7f8987e00640 1 -- 172.21.15.46:0/550609888 wait complete. 2024-09-17T03:23:47.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:23:47.731 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:23:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 -- 172.21.15.46:0/106225473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc1527b0 msgr2=0x7fb9fc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 --2- 172.21.15.46:0/106225473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc15bb60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009900 tx=0x7fb9e801c8b0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 -- 172.21.15.46:0/106225473 shutdown_connections 2024-09-17T03:23:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 --2- 172.21.15.46:0/106225473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9fc0b4de0 0x7fb9fc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 --2- 172.21.15.46:0/106225473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc15bb60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 -- 172.21.15.46:0/106225473 >> 172.21.15.46:0/106225473 conn(0x7fb9fc0b2040 msgr2=0x7fb9fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 -- 172.21.15.46:0/106225473 shutdown_connections 2024-09-17T03:23:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.780+0000 7fba04600640 1 -- 172.21.15.46:0/106225473 wait complete. 2024-09-17T03:23:47.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 Processor -- start 2024-09-17T03:23:47.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 -- start start 2024-09-17T03:23:47.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9fc0b4de0 0x7fb9fc1fca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 0x7fb9fc0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc168a00 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc168880 con 0x7fb9fc1527b0 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba04600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb9fc168b80 con 0x7fb9fc0b4de0 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 0x7fb9fc0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba02600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 0x7fb9fc0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49958/0 (socket says 172.21.15.46:49958) 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 -- 172.21.15.46:0/2567169781 learned_addr learned my addr 172.21.15.46:0/2567169781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01c00640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9fc0b4de0 0x7fb9fc1fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 -- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9fc0b4de0 msgr2=0x7fb9fc1fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9fc0b4de0 0x7fb9fc1fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 -- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9fc1527b0 msgr2=0x7fb9fc1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 -- 172.21.15.46:0/2567169781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc0ccec0 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba02600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.784+0000 7fba01200640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 0x7fb9fc0cba50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9e801a4e0 tx=0x7fb9e8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80b2040 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e8005320 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc0cd1a0 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80046d0 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc0cd760 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9fc0b51f0 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fb9e8004870 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fb9f9600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9dc073560 0x7fb9dc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:47.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7fb9e812e870 con 0x7fb9fc0b5c70 2024-09-17T03:23:47.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.788+0000 7fba01c00640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9dc073560 0x7fb9dc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:47.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.792+0000 7fba01c00640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9dc073560 0x7fb9dc075a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0007fd0 tx=0x7fb9f0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:47.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:47.792+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb9e80bc060 con 0x7fb9fc0b5c70 2024-09-17T03:23:48.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.012+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb9fc0ccc50 con 0x7fb9fc0b5c70 2024-09-17T03:23:48.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fb9f9600640 1 -- 172.21.15.46:0/2567169781 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fb9e80164d0 con 0x7fb9fc0b5c70 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9dc073560 msgr2=0x7fb9dc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9dc073560 0x7fb9dc075a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0007fd0 tx=0x7fb9f0009040 comp rx=0 tx=0).stop 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 msgr2=0x7fb9fc0cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 0x7fb9fc0cba50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9e801a4e0 tx=0x7fb9e8005760 comp rx=0 tx=0).stop 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 shutdown_connections 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9dc073560 0x7fb9dc075a70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9fc1527b0 0x7fb9fc1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9fc0b5c70 0x7fb9fc0cba50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 --2- 172.21.15.46:0/2567169781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9fc0b4de0 0x7fb9fc1fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 >> 172.21.15.46:0/2567169781 conn(0x7fb9fc0b2040 msgr2=0x7fb9fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:48.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 shutdown_connections 2024-09-17T03:23:48.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.596+0000 7fba04600640 1 -- 172.21.15.46:0/2567169781 wait complete. 2024-09-17T03:23:48.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 -- 172.21.15.46:0/4235321562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 msgr2=0x7fbd780b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/4235321562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd780b51f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009900 tx=0x7fbd7001c820 comp rx=0 tx=0).stop 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 -- 172.21.15.46:0/4235321562 shutdown_connections 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/4235321562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd780b51f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/4235321562 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 0x7fbd7815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 -- 172.21.15.46:0/4235321562 >> 172.21.15.46:0/4235321562 conn(0x7fbd780b2040 msgr2=0x7fbd780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 -- 172.21.15.46:0/4235321562 shutdown_connections 2024-09-17T03:23:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 -- 172.21.15.46:0/4235321562 wait complete. 2024-09-17T03:23:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.784+0000 7fbd7cc00640 1 Processor -- start 2024-09-17T03:23:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- start start 2024-09-17T03:23:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd78158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 0x7fbd780cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd781527b0 0x7fbd780ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78168ab0 con 0x7fbd780b4de0 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78168930 con 0x7fbd781527b0 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbd78168c30 con 0x7fbd780b5c70 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd78158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd78158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49990/0 (socket says 172.21.15.46:49990) 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 -- 172.21.15.46:0/1319967117 learned_addr learned my addr 172.21.15.46:0/1319967117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd76800640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd781527b0 0x7fbd780ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd6d400640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 0x7fbd780cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 -- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 msgr2=0x7fbd780cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 0x7fbd780cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 -- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd781527b0 msgr2=0x7fbd780ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd781527b0 0x7fbd780ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 -- 172.21.15.46:0/1319967117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd780cf350 con 0x7fbd780b4de0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd6d400640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 0x7fbd780cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd75e00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd78158c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800ea20 tx=0x7fbd6800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800cde0 con 0x7fbd780b4de0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd68004c20 con 0x7fbd780b4de0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68005020 con 0x7fbd780b4de0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd780cf630 con 0x7fbd780b4de0 2024-09-17T03:23:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd78156870 con 0x7fbd780b4de0 2024-09-17T03:23:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.788+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd780b51f0 con 0x7fbd780b4de0 2024-09-17T03:23:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.792+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fbd680051c0 con 0x7fbd780b4de0 2024-09-17T03:23:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.792+0000 7fbd6ea00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd54073590 0x7fbd54075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.792+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7fbd68095360 con 0x7fbd780b4de0 2024-09-17T03:23:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.792+0000 7fbd6d400640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd54073590 0x7fbd54075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.792+0000 7fbd6d400640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd54073590 0x7fbd54075aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd7814c730 tx=0x7fbd60009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:48.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.796+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd68016030 con 0x7fbd780b4de0 2024-09-17T03:23:48.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:48.972+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbd780b4aa0 con 0x7fbd780b4de0 2024-09-17T03:23:49.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.596+0000 7fbd6ea00640 1 -- 172.21.15.46:0/1319967117 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7fbd6805f670 con 0x7fbd780b4de0 2024-09-17T03:23:49.598 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd54073590 msgr2=0x7fbd54075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd54073590 0x7fbd54075aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd7814c730 tx=0x7fbd60009040 comp rx=0 tx=0).stop 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 msgr2=0x7fbd78158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd78158c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800ea20 tx=0x7fbd6800eee0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 shutdown_connections 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd54073590 0x7fbd54075aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd781527b0 0x7fbd780ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd780b5c70 0x7fbd780cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 --2- 172.21.15.46:0/1319967117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd780b4de0 0x7fbd78158c00 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800ea20 tx=0x7fbd6800eee0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 >> 172.21.15.46:0/1319967117 conn(0x7fbd780b2040 msgr2=0x7fbd7815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 shutdown_connections 2024-09-17T03:23:49.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.600+0000 7fbd7cc00640 1 -- 172.21.15.46:0/1319967117 wait complete. 2024-09-17T03:23:49.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:23:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 -- 172.21.15.46:0/4205777348 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 msgr2=0x7f5cd00cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 --2- 172.21.15.46:0/4205777348 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 0x7f5cd00cf3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009840 tx=0x7f5cb801c820 comp rx=0 tx=0).stop 2024-09-17T03:23:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 -- 172.21.15.46:0/4205777348 shutdown_connections 2024-09-17T03:23:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 --2- 172.21.15.46:0/4205777348 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 0x7f5cd00cf3b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 --2- 172.21.15.46:0/4205777348 >> v2:172.21.15.99:3300/0 conn(0x7f5cd00749a0 0x7f5cd0074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 -- 172.21.15.46:0/4205777348 >> 172.21.15.46:0/4205777348 conn(0x7f5cd00b2040 msgr2=0x7f5cd00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 -- 172.21.15.46:0/4205777348 shutdown_connections 2024-09-17T03:23:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.800+0000 7f5cd6800640 1 -- 172.21.15.46:0/4205777348 wait complete. 2024-09-17T03:23:49.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 Processor -- start 2024-09-17T03:23:49.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 -- start start 2024-09-17T03:23:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 0x7f5cd013d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 0x7f5cd0156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd0160330 0x7f5cd00cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd016a060 con 0x7f5cd00749a0 2024-09-17T03:23:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0169ee0 con 0x7f5cd0160330 2024-09-17T03:23:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cd6800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd016a1e0 con 0x7f5cd00cefa0 2024-09-17T03:23:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 0x7f5cd013d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 0x7f5cd013d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50002/0 (socket says 172.21.15.46:50002) 2024-09-17T03:23:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 -- 172.21.15.46:0/1775700821 learned_addr learned my addr 172.21.15.46:0/1775700821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5cc6a00640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 0x7f5cd0156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccfe00640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd0160330 0x7f5cd00cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 -- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 msgr2=0x7f5cd0156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 0x7f5cd0156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 -- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd0160330 msgr2=0x7f5cd00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd0160330 0x7f5cd00cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 -- 172.21.15.46:0/1775700821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd0156f50 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.804+0000 7f5ccf400640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 0x7f5cd013d820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e970 tx=0x7f5cc000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000cd80 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cc0004020 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0004470 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd00b1700 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd01590e0 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd0074db0 con 0x7f5cd00749a0 2024-09-17T03:23:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5cc0003600 con 0x7f5cd00749a0 2024-09-17T03:23:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc7e00640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cac073560 0x7f5cac075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7f5cc00950d0 con 0x7f5cd00749a0 2024-09-17T03:23:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.808+0000 7f5cc6a00640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cac073560 0x7f5cac075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.812+0000 7f5cc6a00640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cac073560 0x7f5cac075a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80af000 tx=0x7f5cb801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.812+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5cc005f5a0 con 0x7f5cd00749a0 2024-09-17T03:23:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:49.988+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5cd00b4130 con 0x7f5cd00749a0 2024-09-17T03:23:50.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.596+0000 7f5cc7e00640 1 -- 172.21.15.46:0/1775700821 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7f5cc0063fd0 con 0x7f5cd00749a0 2024-09-17T03:23:50.600 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:23:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.600+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cac073560 msgr2=0x7f5cac075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.600+0000 7f5cd6800640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cac073560 0x7f5cac075a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80af000 tx=0x7f5cb801b060 comp rx=0 tx=0).stop 2024-09-17T03:23:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.600+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 msgr2=0x7f5cd013d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.600+0000 7f5cd6800640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 0x7f5cd013d820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000e970 tx=0x7f5cc000ee30 comp rx=0 tx=0).stop 2024-09-17T03:23:50.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 shutdown_connections 2024-09-17T03:23:50.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5cac073560 0x7f5cac075a70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd0160330 0x7f5cd00cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5cd00cefa0 0x7f5cd0156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 --2- 172.21.15.46:0/1775700821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd00749a0 0x7f5cd013d820 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 >> 172.21.15.46:0/1775700821 conn(0x7f5cd00b2040 msgr2=0x7f5cd00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:50.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 shutdown_connections 2024-09-17T03:23:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.604+0000 7f5cd6800640 1 -- 172.21.15.46:0/1775700821 wait complete. 2024-09-17T03:23:50.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:23:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 -- 172.21.15.46:0/3217517672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd800052a80 msgr2=0x7fd8000df680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 --2- 172.21.15.46:0/3217517672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd800052a80 0x7fd8000df680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd80802ebe0 tx=0x7fd8080992b0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 -- 172.21.15.46:0/3217517672 shutdown_connections 2024-09-17T03:23:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 --2- 172.21.15.46:0/3217517672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd800052a80 0x7fd8000df680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 --2- 172.21.15.46:0/3217517672 >> v2:172.21.15.46:3300/0 conn(0x7fd80001cfe0 0x7fd800052540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 -- 172.21.15.46:0/3217517672 >> 172.21.15.46:0/3217517672 conn(0x7fd800054970 msgr2=0x7fd800054d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 -- 172.21.15.46:0/3217517672 shutdown_connections 2024-09-17T03:23:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 -- 172.21.15.46:0/3217517672 wait complete. 2024-09-17T03:23:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.944+0000 7fd80e800640 1 Processor -- start 2024-09-17T03:23:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 -- start start 2024-09-17T03:23:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 0x7fd8000db230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd80001cfe0 0x7fd8000db770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd800052a80 0x7fd800185da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:50.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8000ecc90 con 0x7fd80001cfe0 2024-09-17T03:23:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8000ecb10 con 0x7fd800052a80 2024-09-17T03:23:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd8000ece10 con 0x7fd80000b9d0 2024-09-17T03:23:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80d200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 0x7fd8000db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80d200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 0x7fd8000db230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:52634/0 (socket says 172.21.15.46:52634) 2024-09-17T03:23:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.948+0000 7fd80d200640 1 -- 172.21.15.46:0/3620960884 learned_addr learned my addr 172.21.15.46:0/3620960884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80d200640 1 -- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd800052a80 msgr2=0x7fd800185da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80d200640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd800052a80 0x7fd800185da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80d200640 1 -- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd80001cfe0 msgr2=0x7fd8000db770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80d200640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd80001cfe0 0x7fd8000db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80d200640 1 -- 172.21.15.46:0/3620960884 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80004e3e0 con 0x7fd80000b9d0 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80d200640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 0x7fd8000db230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c3c0 tx=0x7fd7fc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc017070 con 0x7fd80000b9d0 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7fc0053c0 con 0x7fd80000b9d0 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc003860 con 0x7fd80000b9d0 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80004e6c0 con 0x7fd80000b9d0 2024-09-17T03:23:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8001850b0 con 0x7fd80000b9d0 2024-09-17T03:23:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.952+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd800053570 con 0x7fd80000b9d0 2024-09-17T03:23:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.956+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd7fc01a070 con 0x7fd80000b9d0 2024-09-17T03:23:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.956+0000 7fd7fb400640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7e8073560 0x7fd7e8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.956+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fd7fc0937c0 con 0x7fd80000b9d0 2024-09-17T03:23:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.956+0000 7fd806a00640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7e8073560 0x7fd7e8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.960+0000 7fd806a00640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7e8073560 0x7fd7e8075a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40058d0 tx=0x7fd7f4005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:50.960+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd800052e90 con 0x7fd80000b9d0 2024-09-17T03:23:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.168+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fd800183970 con 0x7fd80000b9d0 2024-09-17T03:23:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd7fb400640 1 -- 172.21.15.46:0/3620960884 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v34) ==== 71+0+0 (secure 0 0 0) 0x7fd7fc05e6a0 con 0x7fd80000b9d0 2024-09-17T03:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7e8073560 msgr2=0x7fd7e8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7e8073560 0x7fd7e8075a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40058d0 tx=0x7fd7f4005860 comp rx=0 tx=0).stop 2024-09-17T03:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 msgr2=0x7fd8000db230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 0x7fd8000db230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c3c0 tx=0x7fd7fc00c880 comp rx=0 tx=0).stop 2024-09-17T03:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 shutdown_connections 2024-09-17T03:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7e8073560 0x7fd7e8075a70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd800052a80 0x7fd800185da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd80001cfe0 0x7fd8000db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 --2- 172.21.15.46:0/3620960884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd80000b9d0 0x7fd8000db230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 >> 172.21.15.46:0/3620960884 conn(0x7fd800054970 msgr2=0x7fd8000dddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 shutdown_connections 2024-09-17T03:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.176+0000 7fd80e800640 1 -- 172.21.15.46:0/3620960884 wait complete. 2024-09-17T03:23:51.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 -- 172.21.15.46:0/1390255787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 msgr2=0x7f242c0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 --2- 172.21.15.46:0/1390255787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c0cf0e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f242c15cc60 tx=0x7f241c01c8c0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 -- 172.21.15.46:0/1390255787 shutdown_connections 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 --2- 172.21.15.46:0/1390255787 >> v2:172.21.15.99:3300/0 conn(0x7f242c15e190 0x7f242c15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 --2- 172.21.15.46:0/1390255787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c0cf0e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 -- 172.21.15.46:0/1390255787 >> 172.21.15.46:0/1390255787 conn(0x7f242c0b2040 msgr2=0x7f242c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 -- 172.21.15.46:0/1390255787 shutdown_connections 2024-09-17T03:23:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 -- 172.21.15.46:0/1390255787 wait complete. 2024-09-17T03:23:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 Processor -- start 2024-09-17T03:23:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.424+0000 7f2432400640 1 -- start start 2024-09-17T03:23:51.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f242c0cf620 0x7f242c13d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:51.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f242c15e190 0x7f242c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:51.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:51.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f242c169170 con 0x7f242c15eeb0 2024-09-17T03:23:51.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f242c168ff0 con 0x7f242c0cf620 2024-09-17T03:23:51.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f242c1692f0 con 0x7f242c15e190 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50056/0 (socket says 172.21.15.46:50056) 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 -- 172.21.15.46:0/2591799097 learned_addr learned my addr 172.21.15.46:0/2591799097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242b400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f242c0cf620 0x7f242c13d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2423e00640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f242c15e190 0x7f242c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 -- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f242c15e190 msgr2=0x7f242c154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f242c15e190 0x7f242c154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 -- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f242c0cf620 msgr2=0x7f242c13d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f242c0cf620 0x7f242c13d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 -- 172.21.15.46:0/2591799097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c1f9150 con 0x7f242c15eeb0 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2423e00640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f242c15e190 0x7f242c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f242be00640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c154960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f242400e9a0 tx=0x7f242400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:51.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424004d30 con 0x7f242c15eeb0 2024-09-17T03:23:51.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f242400cb20 con 0x7f242c15eeb0 2024-09-17T03:23:51.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c1f9430 con 0x7f242c15eeb0 2024-09-17T03:23:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424005020 con 0x7f242c15eeb0 2024-09-17T03:23:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c13dba0 con 0x7f242c15eeb0 2024-09-17T03:23:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.428+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c1f9d20 con 0x7f242c15eeb0 2024-09-17T03:23:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.432+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f24240051c0 con 0x7f242c15eeb0 2024-09-17T03:23:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.432+0000 7f2423400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2408073590 0x7f2408075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.432+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f2424094ea0 con 0x7f242c15eeb0 2024-09-17T03:23:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.432+0000 7f242b400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2408073590 0x7f2408075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.432+0000 7f242b400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2408073590 0x7f2408075aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f241c0af000 tx=0x7f241c01b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.436+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2424016030 con 0x7f242c15eeb0 2024-09-17T03:23:51.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:51.676+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f242c0cf0e0 con 0x7f242c15eeb0 2024-09-17T03:23:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.600+0000 7f2423400640 1 -- 172.21.15.46:0/2591799097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v146) ==== 184+0+0 (secure 0 0 0) 0x7f242405f550 con 0x7f242c15eeb0 2024-09-17T03:23:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2408073590 msgr2=0x7f2408075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2408073590 0x7f2408075aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f241c0af000 tx=0x7f241c01b100 comp rx=0 tx=0).stop 2024-09-17T03:23:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 msgr2=0x7f242c154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c154960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f242400e9a0 tx=0x7f242400ee60 comp rx=0 tx=0).stop 2024-09-17T03:23:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 shutdown_connections 2024-09-17T03:23:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2408073590 0x7f2408075aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f242c15eeb0 0x7f242c154960 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f242c15e190 0x7f242c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 --2- 172.21.15.46:0/2591799097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f242c0cf620 0x7f242c13d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 >> 172.21.15.46:0/2591799097 conn(0x7f242c0b2040 msgr2=0x7f242c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 shutdown_connections 2024-09-17T03:23:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.604+0000 7f2432400640 1 -- 172.21.15.46:0/2591799097 wait complete. 2024-09-17T03:23:52.629 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-17T03:23:52.629 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-17T03:23:52.630 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:23:52.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 -- 172.21.15.46:0/3665994937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 msgr2=0x7fa76c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 --2- 172.21.15.46:0/3665994937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c15bb60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa76c074b20 tx=0x7fa75401c820 comp rx=0 tx=0).stop 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 -- 172.21.15.46:0/3665994937 shutdown_connections 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 --2- 172.21.15.46:0/3665994937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c15bb60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 -- 172.21.15.46:0/3665994937 >> 172.21.15.46:0/3665994937 conn(0x7fa76c0b2040 msgr2=0x7fa76c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 -- 172.21.15.46:0/3665994937 shutdown_connections 2024-09-17T03:23:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 -- 172.21.15.46:0/3665994937 wait complete. 2024-09-17T03:23:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.788+0000 7fa771200640 1 Processor -- start 2024-09-17T03:23:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 -- start start 2024-09-17T03:23:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa76c0b4de0 0x7fa76c13fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa76c0b5c70 0x7fa76c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c169bd0 con 0x7fa76c1527b0 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c169a50 con 0x7fa76c0b5c70 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c169d50 con 0x7fa76c0b4de0 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c0cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43944/0 (socket says 172.21.15.46:43944) 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 -- 172.21.15.46:0/532593732 learned_addr learned my addr 172.21.15.46:0/532593732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa769e00640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa76c0b4de0 0x7fa76c13fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa763e00640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa76c0b5c70 0x7fa76c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 -- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa76c0b4de0 msgr2=0x7fa76c13fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa76c0b4de0 0x7fa76c13fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 -- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa76c0b5c70 msgr2=0x7fa76c0cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa76c0b5c70 0x7fa76c0cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 -- 172.21.15.46:0/532593732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c0cf7d0 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa76a800640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c0cdb80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c6b0 tx=0x7fa76400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764003b80 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa764003d20 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764005330 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c0cfab0 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.792+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c0cce40 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.796+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c15bb60 con 0x7fa76c1527b0 2024-09-17T03:23:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.796+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fa7640054d0 con 0x7fa76c1527b0 2024-09-17T03:23:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.796+0000 7fa762000640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa748073560 0x7fa748075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.796+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fa764094360 con 0x7fa76c1527b0 2024-09-17T03:23:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.796+0000 7fa769e00640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa748073560 0x7fa748075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.796+0000 7fa769e00640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa748073560 0x7fa748075a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa75c007fd0 tx=0x7fa75c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.800+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa76405e9d0 con 0x7fa76c1527b0 2024-09-17T03:23:52.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.972+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa76c151d20 con 0x7fa76c1527b0 2024-09-17T03:23:52.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa762000640 1 -- 172.21.15.46:0/532593732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+12996 (secure 0 0 0) 0x7fa764063400 con 0x7fa76c1527b0 2024-09-17T03:23:52.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:52.977 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:23:50.600058+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6835/3646914597":"2024-09-18T03:23:47.594317+0000","172.21.15.99:6833/3646914597":"2024-09-18T03:23:47.594317+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa748073560 msgr2=0x7fa748075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa748073560 0x7fa748075a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa75c007fd0 tx=0x7fa75c009040 comp rx=0 tx=0).stop 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 msgr2=0x7fa76c0cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c0cdb80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c6b0 tx=0x7fa76400cb70 comp rx=0 tx=0).stop 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 shutdown_connections 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa748073560 0x7fa748075a70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa76c1527b0 0x7fa76c0cdb80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa76c0b5c70 0x7fa76c0cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 --2- 172.21.15.46:0/532593732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa76c0b4de0 0x7fa76c13fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 >> 172.21.15.46:0/532593732 conn(0x7fa76c0b2040 msgr2=0x7fa76c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 shutdown_connections 2024-09-17T03:23:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:52.976+0000 7fa771200640 1 -- 172.21.15.46:0/532593732 wait complete. 2024-09-17T03:23:53.000 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:23:53.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 -- 172.21.15.46:0/3162624021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 msgr2=0x7f30d806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 --2- 172.21.15.46:0/3162624021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 0x7f30d806e520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009840 tx=0x7f30c801c820 comp rx=0 tx=0).stop 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 -- 172.21.15.46:0/3162624021 shutdown_connections 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 --2- 172.21.15.46:0/3162624021 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d815f180 0x7f30d815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 --2- 172.21.15.46:0/3162624021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 0x7f30d806e520 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 -- 172.21.15.46:0/3162624021 >> 172.21.15.46:0/3162624021 conn(0x7f30d80708c0 msgr2=0x7f30d8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 -- 172.21.15.46:0/3162624021 shutdown_connections 2024-09-17T03:23:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.160+0000 7f30dd800640 1 -- 172.21.15.46:0/3162624021 wait complete. 2024-09-17T03:23:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 Processor -- start 2024-09-17T03:23:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 -- start start 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d80d29c0 0x7f30d8145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 0x7f30d81579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 0x7f30d8157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30d816c4a0 con 0x7f30d815f180 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30d816c320 con 0x7f30d8155ad0 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f30d816c620 con 0x7f30d80d29c0 2024-09-17T03:23:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 0x7f30d8157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 0x7f30d8157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43960/0 (socket says 172.21.15.46:43960) 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 -- 172.21.15.46:0/2796437455 learned_addr learned my addr 172.21.15.46:0/2796437455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d6a00640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d80d29c0 0x7f30d8145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d6000640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 0x7f30d81579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 -- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d80d29c0 msgr2=0x7f30d8145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d80d29c0 0x7f30d8145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 -- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 msgr2=0x7f30d81579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 0x7f30d81579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 -- 172.21.15.46:0/2796437455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d8145d40 con 0x7f30d815f180 2024-09-17T03:23:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30d7400640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 0x7f30d8157ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30d000ea80 tx=0x7f30d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000cea0 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30d0004b30 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0005020 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d81fc500 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.164+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81fcac0 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.168+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d815f590 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.168+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f30d00051c0 con 0x7f30d815f180 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.168+0000 7f30cea00640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b8073560 0x7f30b8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.168+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f30d0095130 con 0x7f30d815f180 2024-09-17T03:23:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.168+0000 7f30d6a00640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b8073560 0x7f30b8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.172+0000 7f30d6a00640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b8073560 0x7f30b8075a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30c80af000 tx=0x7f30c801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.172+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30d005f830 con 0x7f30d815f180 2024-09-17T03:23:53.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.348+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30d80d2f70 con 0x7f30d815f180 2024-09-17T03:23:53.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.348+0000 7f30cea00640 1 -- 172.21.15.46:0/2796437455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f30d0064260 con 0x7f30d815f180 2024-09-17T03:23:53.351 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:23:53.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b8073560 msgr2=0x7f30b8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b8073560 0x7f30b8075a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f30c80af000 tx=0x7f30c801b060 comp rx=0 tx=0).stop 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 msgr2=0x7f30d8157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 0x7f30d8157ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30d000ea80 tx=0x7f30d000ef40 comp rx=0 tx=0).stop 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 shutdown_connections 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f30b8073560 0x7f30b8075a70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d815f180 0x7f30d8157ee0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30d8155ad0 0x7f30d81579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 --2- 172.21.15.46:0/2796437455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30d80d29c0 0x7f30d8145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 >> 172.21.15.46:0/2796437455 conn(0x7f30d80708c0 msgr2=0x7f30d814bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 shutdown_connections 2024-09-17T03:23:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.352+0000 7f30dd800640 1 -- 172.21.15.46:0/2796437455 wait complete. 2024-09-17T03:23:53.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-17T03:23:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 -- 172.21.15.46:0/4142227099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fd0005260 msgr2=0x7f5fd000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/4142227099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fd0005260 0x7f5fd000b9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009b30 tx=0x7f5fe401d270 comp rx=0 tx=0).stop 2024-09-17T03:23:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 -- 172.21.15.46:0/4142227099 shutdown_connections 2024-09-17T03:23:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/4142227099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fd0005260 0x7f5fd000b9d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 -- 172.21.15.46:0/4142227099 >> 172.21.15.46:0/4142227099 conn(0x7f5fec0b2040 msgr2=0x7f5fec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 -- 172.21.15.46:0/4142227099 shutdown_connections 2024-09-17T03:23:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.532+0000 7f5ff1a00640 1 -- 172.21.15.46:0/4142227099 wait complete. 2024-09-17T03:23:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 Processor -- start 2024-09-17T03:23:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 -- start start 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fd0005260 0x7f5fec1f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5fec13f080 0x7f5fec13e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 0x7f5fec1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec075140 con 0x7f5fec13d3b0 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec074fc0 con 0x7f5fd0005260 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5ff1a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec0752c0 con 0x7f5fec13f080 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 0x7f5fec1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 0x7f5fec1452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43982/0 (socket says 172.21.15.46:43982) 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 -- 172.21.15.46:0/1503346403 learned_addr learned my addr 172.21.15.46:0/1503346403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5fe2000640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5fec13f080 0x7f5fec13e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feaa00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fd0005260 0x7f5fec1f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 -- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5fec13f080 msgr2=0x7f5fec13e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5fec13f080 0x7f5fec13e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 -- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fd0005260 msgr2=0x7f5fec1f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fd0005260 0x7f5fec1f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 -- 172.21.15.46:0/1503346403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec0ccf30 con 0x7f5fec13d3b0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5feb400640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 0x7f5fec1452b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5fe401db10 tx=0x7f5fe40acac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40bd070 con 0x7f5fec13d3b0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fe4005030 con 0x7f5fec13d3b0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.536+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4005440 con 0x7f5fec13d3b0 2024-09-17T03:23:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec140050 con 0x7f5fec13d3b0 2024-09-17T03:23:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec13fa90 con 0x7f5fec13d3b0 2024-09-17T03:23:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec13f520 con 0x7f5fec13d3b0 2024-09-17T03:23:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5fe40c0070 con 0x7f5fec13d3b0 2024-09-17T03:23:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5fe3400640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5fc8073560 0x7f5fc8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f5fe412f550 con 0x7f5fec13d3b0 2024-09-17T03:23:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.540+0000 7f5feaa00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5fc8073560 0x7f5fc8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.544+0000 7f5feaa00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5fc8073560 0x7f5fc8075a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc001ac0 tx=0x7f5fdc006b30 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:53.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.544+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5fe4016f80 con 0x7f5fec13d3b0 2024-09-17T03:23:53.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:53.788+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f5fec0cf630 con 0x7f5fec13d3b0 2024-09-17T03:23:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.608+0000 7f5fe3400640 1 -- 172.21.15.46:0/1503346403 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v148) ==== 169+0+0 (secure 0 0 0) 0x7f5fe40055e0 con 0x7f5fec13d3b0 2024-09-17T03:23:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5fc8073560 msgr2=0x7f5fc8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5fc8073560 0x7f5fc8075a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc001ac0 tx=0x7f5fdc006b30 comp rx=0 tx=0).stop 2024-09-17T03:23:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 msgr2=0x7f5fec1452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 0x7f5fec1452b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5fe401db10 tx=0x7f5fe40acac0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 shutdown_connections 2024-09-17T03:23:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5fc8073560 0x7f5fc8075a70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec13d3b0 0x7f5fec1452b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5fec13f080 0x7f5fec13e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 --2- 172.21.15.46:0/1503346403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fd0005260 0x7f5fec1f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 >> 172.21.15.46:0/1503346403 conn(0x7f5fec0b2040 msgr2=0x7f5fec0b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 shutdown_connections 2024-09-17T03:23:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.612+0000 7f5ff1a00640 1 -- 172.21.15.46:0/1503346403 wait complete. 2024-09-17T03:23:54.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:23:54.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.800+0000 7fa162200640 1 -- 172.21.15.46:0/634007841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c06f7c0 msgr2=0x7fa15c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.800+0000 7fa162200640 1 --2- 172.21.15.46:0/634007841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c06f7c0 0x7fa15c0ccf60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0098f0 tx=0x7fa14c01c8b0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 -- 172.21.15.46:0/634007841 shutdown_connections 2024-09-17T03:23:54.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 --2- 172.21.15.46:0/634007841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c06f7c0 0x7fa15c0ccf60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 -- 172.21.15.46:0/634007841 >> 172.21.15.46:0/634007841 conn(0x7fa15c06d1f0 msgr2=0x7fa15c06d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:54.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 -- 172.21.15.46:0/634007841 shutdown_connections 2024-09-17T03:23:54.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 -- 172.21.15.46:0/634007841 wait complete. 2024-09-17T03:23:54.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 Processor -- start 2024-09-17T03:23:54.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 -- start start 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa15c06f7c0 0x7fa15c1fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.804+0000 7fa162200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 0x7fa15c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa15c15f170 0x7fa15c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c168da0 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c168c20 con 0x7fa15c15f170 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c168f20 con 0x7fa15c06f7c0 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 0x7fa15c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 0x7fa15c0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44008/0 (socket says 172.21.15.46:44008) 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 -- 172.21.15.46:0/2614297368 learned_addr learned my addr 172.21.15.46:0/2614297368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:54.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa160e00640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa15c06f7c0 0x7fa15c1fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa161800640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa15c15f170 0x7fa15c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 -- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa15c06f7c0 msgr2=0x7fa15c1fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa15c06f7c0 0x7fa15c1fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 -- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa15c15f170 msgr2=0x7fa15c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa15c15f170 0x7fa15c0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 -- 172.21.15.46:0/2614297368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1fc460 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa153e00640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 0x7fa15c0b43f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa14400c3f0 tx=0x7fa14400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144017070 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1440053c0 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144003860 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c15acb0 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c0b3910 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c15aa20 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.808+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fa14401a070 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.812+0000 7fa159600640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa138073590 0x7fa138075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:54.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.812+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fa1440937e0 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.812+0000 7fa160e00640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa138073590 0x7fa138075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:54.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.812+0000 7fa160e00640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa138073590 0x7fa138075aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0af000 tx=0x7fa14c01b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.816+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa14405e020 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa15c0ccf60 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa159600640 1 -- 172.21.15.46:0/2614297368 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) ==== 59+0+2294 (secure 0 0 0) 0x7fa144062a50 con 0x7fa15c0cd4a0 2024-09-17T03:23:54.993 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:23:54.994 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:23:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:23:54.996 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:23:54.996 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:23:54.996 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:23:54.996 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa138073590 msgr2=0x7fa138075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa138073590 0x7fa138075aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0af000 tx=0x7fa14c01b0f0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 msgr2=0x7fa15c0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 0x7fa15c0b43f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa14400c3f0 tx=0x7fa14400c8b0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 shutdown_connections 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa138073590 0x7fa138075aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa15c15f170 0x7fa15c0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15c0cd4a0 0x7fa15c0b43f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 --2- 172.21.15.46:0/2614297368 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa15c06f7c0 0x7fa15c1fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 >> 172.21.15.46:0/2614297368 conn(0x7fa15c06d1f0 msgr2=0x7fa15c0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 shutdown_connections 2024-09-17T03:23:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:54.992+0000 7fa162200640 1 -- 172.21.15.46:0/2614297368 wait complete. 2024-09-17T03:23:55.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:23:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 -- 172.21.15.46:0/2763992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b5c70 msgr2=0x7fc38c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 --2- 172.21.15.46:0/2763992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b5c70 0x7fc38c15bb60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0cca90 tx=0x7fc37c01c800 comp rx=0 tx=0).stop 2024-09-17T03:23:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 -- 172.21.15.46:0/2763992376 shutdown_connections 2024-09-17T03:23:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 --2- 172.21.15.46:0/2763992376 >> v2:172.21.15.99:3300/0 conn(0x7fc38c0b4de0 0x7fc38c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 --2- 172.21.15.46:0/2763992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b5c70 0x7fc38c15bb60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 -- 172.21.15.46:0/2763992376 >> 172.21.15.46:0/2763992376 conn(0x7fc38c0b2040 msgr2=0x7fc38c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 -- 172.21.15.46:0/2763992376 shutdown_connections 2024-09-17T03:23:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.176+0000 7fc393200640 1 -- 172.21.15.46:0/2763992376 wait complete. 2024-09-17T03:23:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 Processor -- start 2024-09-17T03:23:55.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 -- start start 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 0x7fc38c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c0b5c70 0x7fc38c150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc38c1527b0 0x7fc38c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c168970 con 0x7fc38c0b4de0 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c1687f0 con 0x7fc38c0b5c70 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc38c168af0 con 0x7fc38c1527b0 2024-09-17T03:23:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 0x7fc38c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 0x7fc38c0ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44038/0 (socket says 172.21.15.46:44038) 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 -- 172.21.15.46:0/2280939711 learned_addr learned my addr 172.21.15.46:0/2280939711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc383e00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c0b5c70 0x7fc38c150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc391200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc38c1527b0 0x7fc38c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 -- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc38c1527b0 msgr2=0x7fc38c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc38c1527b0 0x7fc38c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 -- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c0b5c70 msgr2=0x7fc38c150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c0b5c70 0x7fc38c150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 -- 172.21.15.46:0/2280939711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c0ae8b0 con 0x7fc38c0b4de0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc38be00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 0x7fc38c0ade30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0b0040 tx=0x7fc37c0a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0bb070 con 0x7fc38c0b4de0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc37c0044b0 con 0x7fc38c0b4de0 2024-09-17T03:23:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c0aeb90 con 0x7fc38c0b4de0 2024-09-17T03:23:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0b2040 con 0x7fc38c0b4de0 2024-09-17T03:23:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.180+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c0ab5c0 con 0x7fc38c0b4de0 2024-09-17T03:23:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.184+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c0b51f0 con 0x7fc38c0b4de0 2024-09-17T03:23:55.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.184+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fc37c005290 con 0x7fc38c0b4de0 2024-09-17T03:23:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.184+0000 7fc388c00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc368073560 0x7fc368075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.184+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fc37c12df40 con 0x7fc38c0b4de0 2024-09-17T03:23:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.184+0000 7fc383e00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc368073560 0x7fc368075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.184+0000 7fc383e00640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc368073560 0x7fc368075a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc38c158f70 tx=0x7fc374009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:55.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.188+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc37c0164b0 con 0x7fc38c0b4de0 2024-09-17T03:23:55.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.364+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc38c0ae370 con 0x7fc38c0b4de0 2024-09-17T03:23:55.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.368+0000 7fc388c00640 1 -- 172.21.15.46:0/2280939711 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fc37c0f8c40 con 0x7fc38c0b4de0 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc368073560 msgr2=0x7fc368075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc368073560 0x7fc368075a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc38c158f70 tx=0x7fc374009040 comp rx=0 tx=0).stop 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 msgr2=0x7fc38c0ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 0x7fc38c0ade30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0b0040 tx=0x7fc37c0a5820 comp rx=0 tx=0).stop 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 shutdown_connections 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc368073560 0x7fc368075a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc38c1527b0 0x7fc38c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c0b5c70 0x7fc38c150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 --2- 172.21.15.46:0/2280939711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc38c0b4de0 0x7fc38c0ade30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 >> 172.21.15.46:0/2280939711 conn(0x7fc38c0b2040 msgr2=0x7fc38c159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 shutdown_connections 2024-09-17T03:23:55.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.372+0000 7fc393200640 1 -- 172.21.15.46:0/2280939711 wait complete. 2024-09-17T03:23:55.394 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:23:55.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:23:55.466 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:23:55.581 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:23:55.581 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:55.581 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:55.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:23:55.590 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:23:55.590 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:55.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:23:55.600 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:23:55.600 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:55.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:23:55.640 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:23:55.640 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:23:55.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:23:55.654 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:23:55.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 -- 172.21.15.46:0/1046533388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00749a0 msgr2=0x7f2ce0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 --2- 172.21.15.46:0/1046533388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00749a0 0x7f2ce0074db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009900 tx=0x7f2cd001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 -- 172.21.15.46:0/1046533388 shutdown_connections 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 --2- 172.21.15.46:0/1046533388 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ce00b3340 0x7f2ce00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 --2- 172.21.15.46:0/1046533388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00749a0 0x7f2ce0074db0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 -- 172.21.15.46:0/1046533388 >> 172.21.15.46:0/1046533388 conn(0x7f2ce00b2040 msgr2=0x7f2ce00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:55.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 -- 172.21.15.46:0/1046533388 shutdown_connections 2024-09-17T03:23:55.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.840+0000 7f2ce5200640 1 -- 172.21.15.46:0/1046533388 wait complete. 2024-09-17T03:23:55.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 Processor -- start 2024-09-17T03:23:55.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 -- start start 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ce00749a0 0x7f2ce00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 0x7f2ce013cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ce00cefa0 0x7f2ce00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0169e60 con 0x7f2ce00b3340 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0169ce0 con 0x7f2ce00749a0 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce0169fe0 con 0x7f2ce00cefa0 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 0x7f2ce013cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 0x7f2ce013cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44044/0 (socket says 172.21.15.46:44044) 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 -- 172.21.15.46:0/58308095 learned_addr learned my addr 172.21.15.46:0/58308095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdde00640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ce00749a0 0x7f2ce00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cde800640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ce00cefa0 0x7f2ce00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 -- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ce00cefa0 msgr2=0x7f2ce00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ce00cefa0 0x7f2ce00ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 -- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ce00749a0 msgr2=0x7f2ce00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ce00749a0 0x7f2ce00cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 -- 172.21.15.46:0/58308095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce00ce010 con 0x7f2ce00b3340 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cdd400640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 0x7f2ce013cdc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b3e0 tx=0x7f2ccc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc015020 con 0x7f2ce00b3340 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ccc011920 con 0x7f2ce00b3340 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc011d90 con 0x7f2ce00b3340 2024-09-17T03:23:55.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce00ce2a0 con 0x7f2ce00b3340 2024-09-17T03:23:55.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.844+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce00ceb00 con 0x7f2ce00b3340 2024-09-17T03:23:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.848+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce0074db0 con 0x7f2ce00b3340 2024-09-17T03:23:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.848+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2ccc018020 con 0x7f2ce00b3340 2024-09-17T03:23:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.848+0000 7f2cd6000640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2cc0073560 0x7f2cc0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.848+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f2ccc094920 con 0x7f2ce00b3340 2024-09-17T03:23:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.848+0000 7f2cdde00640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2cc0073560 0x7f2cc0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:55.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.852+0000 7f2cdde00640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2cc0073560 0x7f2cc0075a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00af000 tx=0x7f2cd001b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:55.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:55.852+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ccc01f020 con 0x7f2ce00b3340 2024-09-17T03:23:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ce015a6e0 con 0x7f2ce00b3340 2024-09-17T03:23:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2cd6000640 1 -- 172.21.15.46:0/58308095 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3916 (secure 0 0 0) 0x7f2ce00ce2a0 con 0x7f2ce00b3340 2024-09-17T03:23:56.070 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:56.070 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"btime":"2024-09-17T03:23:48:596519+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2855420485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2855420485},{"type":"v1","addr":"172.21.15.46:6835","nonce":2855420485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":244771,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3524788228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3524788228},{"type":"v1","addr":"172.21.15.46:6837","nonce":3524788228}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/567948786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":567948786},{"type":"v1","addr":"172.21.15.99:6834","nonce":567948786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":244880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3236614680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3236614680},{"type":"v1","addr":"172.21.15.99:6835","nonce":3236614680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-17T03:23:56.070 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 20 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2cc0073560 msgr2=0x7f2cc0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2cc0073560 0x7f2cc0075a70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00af000 tx=0x7f2cd001b120 comp rx=0 tx=0).stop 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 msgr2=0x7f2ce013cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 0x7f2ce013cdc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b3e0 tx=0x7f2ccc00b8a0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 shutdown_connections 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2cc0073560 0x7f2cc0075a70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ce00cefa0 0x7f2ce00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce00b3340 0x7f2ce013cdc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 --2- 172.21.15.46:0/58308095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ce00749a0 0x7f2ce00cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 >> 172.21.15.46:0/58308095 conn(0x7f2ce00b2040 msgr2=0x7f2ce0160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:56.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 shutdown_connections 2024-09-17T03:23:56.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.068+0000 7f2ce5200640 1 -- 172.21.15.46:0/58308095 wait complete. 2024-09-17T03:23:56.093 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:23:56.093 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:23:56.093 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:23:56.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:23:56.094 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:23:56.094 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:23:56.094 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:23:56.094 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:23:56.094 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:23:56.094 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:23:56.094 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:23:56.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:23:56.095 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:23:56.095 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:23:56.095 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:23:56.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:23:56.107 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:23:56.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:23:56.109 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:23:56.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:23:56.111 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:23:56.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:23:56.118 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:23:56.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:23:56.154 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:23:56.154 DEBUG:teuthology.parallel:result is None 2024-09-17T03:23:56.156 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:23:56.156 DEBUG:teuthology.parallel:result is None 2024-09-17T03:23:56.164 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:23:56.164 DEBUG:teuthology.parallel:result is None 2024-09-17T03:23:56.166 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:23:56.166 DEBUG:teuthology.parallel:result is None 2024-09-17T03:23:56.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:23:56.255 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:23:56.256 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:23:56.269 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:23:56.269 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 -- 172.21.15.46:0/1268121932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b4de0 msgr2=0x7fd6240b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 --2- 172.21.15.46:0/1268121932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b4de0 0x7fd6240b51f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009bc0 tx=0x7fd61c01cae0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 -- 172.21.15.46:0/1268121932 shutdown_connections 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 --2- 172.21.15.46:0/1268121932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b4de0 0x7fd6240b51f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 --2- 172.21.15.46:0/1268121932 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6240b5c70 0x7fd62415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 -- 172.21.15.46:0/1268121932 >> 172.21.15.46:0/1268121932 conn(0x7fd6240b2040 msgr2=0x7fd6240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 -- 172.21.15.46:0/1268121932 shutdown_connections 2024-09-17T03:23:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 -- 172.21.15.46:0/1268121932 wait complete. 2024-09-17T03:23:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 Processor -- start 2024-09-17T03:23:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.332+0000 7fd62b200640 1 -- start start 2024-09-17T03:23:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6240b4de0 0x7fd62413f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 0x7fd62413fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6241527b0 0x7fd6240ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd624168a00 con 0x7fd6240b5c70 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd624168880 con 0x7fd6240b4de0 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd624168b80 con 0x7fd6241527b0 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 0x7fd62413fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 0x7fd62413fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44090/0 (socket says 172.21.15.46:44090) 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 -- 172.21.15.46:0/1502786452 learned_addr learned my addr 172.21.15.46:0/1502786452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd623e00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6240b4de0 0x7fd62413f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd629200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6241527b0 0x7fd6240ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 -- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6241527b0 msgr2=0x7fd6240ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6241527b0 0x7fd6240ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 -- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6240b4de0 msgr2=0x7fd62413f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6240b4de0 0x7fd62413f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 -- 172.21.15.46:0/1502786452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624151080 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd61be00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 0x7fd62413fc10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c370 tx=0x7fd60c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017070 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd60c003600 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c003ab0 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd624150da0 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6240ccd40 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6240b51f0 con 0x7fd6240b5c70 2024-09-17T03:23:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.336+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd60c01a070 con 0x7fd6240b5c70 2024-09-17T03:23:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.340+0000 7fd620c00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd600073560 0x7fd600075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.340+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fd60c093800 con 0x7fd6240b5c70 2024-09-17T03:23:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.340+0000 7fd623e00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd600073560 0x7fd600075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.340+0000 7fd623e00640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd600073560 0x7fd600075a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd614004630 tx=0x7fd61400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:56.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.344+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6240b5730 con 0x7fd6240b5c70 2024-09-17T03:23:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.528+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd624159160 con 0x7fd6240b5c70 2024-09-17T03:23:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.628+0000 7fd620c00640 1 -- 172.21.15.46:0/1502786452 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) ==== 97+0+0 (secure 0 0 0) 0x7fd60c05df70 con 0x7fd6240b5c70 2024-09-17T03:23:56.629 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:23:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd600073560 msgr2=0x7fd600075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd600073560 0x7fd600075a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd614004630 tx=0x7fd61400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 msgr2=0x7fd62413fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 0x7fd62413fc10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c370 tx=0x7fd60c00c830 comp rx=0 tx=0).stop 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 shutdown_connections 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd600073560 0x7fd600075a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6241527b0 0x7fd6240ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240b5c70 0x7fd62413fc10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 --2- 172.21.15.46:0/1502786452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6240b4de0 0x7fd62413f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 >> 172.21.15.46:0/1502786452 conn(0x7fd6240b2040 msgr2=0x7fd6241583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 shutdown_connections 2024-09-17T03:23:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.632+0000 7fd62b200640 1 -- 172.21.15.46:0/1502786452 wait complete. 2024-09-17T03:23:56.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:23:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 -- 172.21.15.46:0/1608185705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0cefa0 msgr2=0x7fdf9c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 --2- 172.21.15.46:0/1608185705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cf3b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009900 tx=0x7fdf8801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 -- 172.21.15.46:0/1608185705 shutdown_connections 2024-09-17T03:23:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 --2- 172.21.15.46:0/1608185705 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0b3340 0x7fdf9c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 --2- 172.21.15.46:0/1608185705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cf3b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 -- 172.21.15.46:0/1608185705 >> 172.21.15.46:0/1608185705 conn(0x7fdf9c0b2040 msgr2=0x7fdf9c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 -- 172.21.15.46:0/1608185705 shutdown_connections 2024-09-17T03:23:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.812+0000 7fdfa3800640 1 -- 172.21.15.46:0/1608185705 wait complete. 2024-09-17T03:23:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 Processor -- start 2024-09-17T03:23:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 -- start start 2024-09-17T03:23:56.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 0x7fdf9c140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0b3340 0x7fdf9c13c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c169e10 con 0x7fdf9c0749a0 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c169c90 con 0x7fdf9c0b3340 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa3800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c169f90 con 0x7fdf9c0cefa0 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 0x7fdf9c140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 0x7fdf9c140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44104/0 (socket says 172.21.15.46:44104) 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 -- 172.21.15.46:0/2366501884 learned_addr learned my addr 172.21.15.46:0/2366501884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdf9be00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0b3340 0x7fdf9c13c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa1800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 -- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0cefa0 msgr2=0x7fdf9c0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 -- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0b3340 msgr2=0x7fdf9c13c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0b3340 0x7fdf9c13c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 -- 172.21.15.46:0/2366501884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1405a0 con 0x7fdf9c0749a0 2024-09-17T03:23:56.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa1800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:56.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.816+0000 7fdfa0e00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 0x7fdf9c140ec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000c3c0 tx=0x7fdf9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:56.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90017070 con 0x7fdf9c0749a0 2024-09-17T03:23:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf900053c0 con 0x7fdf9c0749a0 2024-09-17T03:23:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c0b0160 con 0x7fdf9c0749a0 2024-09-17T03:23:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90003860 con 0x7fdf9c0749a0 2024-09-17T03:23:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c0b0720 con 0x7fdf9c0749a0 2024-09-17T03:23:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c143310 con 0x7fdf9c0749a0 2024-09-17T03:23:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.820+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fdf9001a070 con 0x7fdf9c0749a0 2024-09-17T03:23:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.824+0000 7fdf98c00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf7c073560 0x7fdf7c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.824+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7fdf90093860 con 0x7fdf9c0749a0 2024-09-17T03:23:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.824+0000 7fdf9be00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf7c073560 0x7fdf7c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.824+0000 7fdf9be00640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf7c073560 0x7fdf7c075a70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf880af000 tx=0x7fdf8801b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:56.828+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf9005e020 con 0x7fdf9c0749a0 2024-09-17T03:23:57.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdf9c0b3750 con 0x7fdf9c0749a0 2024-09-17T03:23:57.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdf98c00640 1 -- 172.21.15.46:0/2366501884 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7fdf90062a50 con 0x7fdf9c0749a0 2024-09-17T03:23:57.061 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:57.061 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:23:57.061 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:23:57.061 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.061 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:23:57.064 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.066 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.068 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.069 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:23:57.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.072 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:23:57.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:23:57.074 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:23:57.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf7c073560 msgr2=0x7fdf7c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf7c073560 0x7fdf7c075a70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf880af000 tx=0x7fdf8801b120 comp rx=0 tx=0).stop 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 msgr2=0x7fdf9c140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 0x7fdf9c140ec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000c3c0 tx=0x7fdf9000c880 comp rx=0 tx=0).stop 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 shutdown_connections 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf7c073560 0x7fdf7c075a70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf9c0cefa0 0x7fdf9c0cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0b3340 0x7fdf9c13c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 --2- 172.21.15.46:0/2366501884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf9c0749a0 0x7fdf9c140ec0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 >> 172.21.15.46:0/2366501884 conn(0x7fdf9c0b2040 msgr2=0x7fdf9c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 shutdown_connections 2024-09-17T03:23:57.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.060+0000 7fdfa3800640 1 -- 172.21.15.46:0/2366501884 wait complete. 2024-09-17T03:23:57.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:23:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 -- 172.21.15.46:0/3510158481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b5c70 msgr2=0x7f39e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 --2- 172.21.15.46:0/3510158481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b5c70 0x7f39e015bb60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39e0070710 tx=0x7f39d001c800 comp rx=0 tx=0).stop 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 -- 172.21.15.46:0/3510158481 shutdown_connections 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 --2- 172.21.15.46:0/3510158481 >> v2:172.21.15.99:3300/0 conn(0x7f39e01527b0 0x7f39e0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 --2- 172.21.15.46:0/3510158481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b5c70 0x7f39e015bb60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 -- 172.21.15.46:0/3510158481 >> 172.21.15.46:0/3510158481 conn(0x7f39e00b2040 msgr2=0x7f39e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 -- 172.21.15.46:0/3510158481 shutdown_connections 2024-09-17T03:23:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.244+0000 7f39e5600640 1 -- 172.21.15.46:0/3510158481 wait complete. 2024-09-17T03:23:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 Processor -- start 2024-09-17T03:23:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 -- start start 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 0x7f39e01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39e00b5c70 0x7f39e014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e01527b0 0x7f39e00cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0168b20 con 0x7f39e00b4de0 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01689a0 con 0x7f39e01527b0 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f39e0168ca0 con 0x7f39e00b5c70 2024-09-17T03:23:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 0x7f39e01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 0x7f39e01564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44126/0 (socket says 172.21.15.46:44126) 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 -- 172.21.15.46:0/1412085577 learned_addr learned my addr 172.21.15.46:0/1412085577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39d7e00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39e00b5c70 0x7f39e014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39df400640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e01527b0 0x7f39e00cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 -- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39e00b5c70 msgr2=0x7f39e014ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39e00b5c70 0x7f39e014ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 -- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e01527b0 msgr2=0x7f39e00cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e01527b0 0x7f39e00cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 -- 172.21.15.46:0/1412085577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e00ccd60 con 0x7f39e00b4de0 2024-09-17T03:23:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39d7e00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39e00b5c70 0x7f39e014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:23:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39dea00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 0x7f39e01564a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39d0019410 tx=0x7f39d00a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00bc070 con 0x7f39e00b4de0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39d00044b0 con 0x7f39e00b4de0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00b2040 con 0x7f39e00b4de0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e00cd040 con 0x7f39e00b4de0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.248+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e00cd600 con 0x7f39e00b4de0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.252+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e015bb60 con 0x7f39e00b4de0 2024-09-17T03:23:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.252+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f39d0005290 con 0x7f39e00b4de0 2024-09-17T03:23:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.252+0000 7f39d6a00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39bc073560 0x7f39bc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.252+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f39d012df10 con 0x7f39e00b4de0 2024-09-17T03:23:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.252+0000 7f39d7e00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39bc073560 0x7f39bc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.256+0000 7f39d7e00640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39bc073560 0x7f39bc075a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005fd0 tx=0x7f39c8005f20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.256+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39e0075580 con 0x7f39e00b4de0 2024-09-17T03:23:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.432+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f39e014f240 con 0x7f39e00b4de0 2024-09-17T03:23:57.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.432+0000 7f39d6a00640 1 -- 172.21.15.46:0/1412085577 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7f39d00164b0 con 0x7f39e00b4de0 2024-09-17T03:23:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:23:57.436 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:23:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:23:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:23:57.436 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:23:57.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:23:57.437 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39bc073560 msgr2=0x7f39bc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39bc073560 0x7f39bc075a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005fd0 tx=0x7f39c8005f20 comp rx=0 tx=0).stop 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 msgr2=0x7f39e01564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 0x7f39e01564a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39d0019410 tx=0x7f39d00a5840 comp rx=0 tx=0).stop 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 shutdown_connections 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39bc073560 0x7f39bc075a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e01527b0 0x7f39e00cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39e00b5c70 0x7f39e014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 --2- 172.21.15.46:0/1412085577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39e00b4de0 0x7f39e01564a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 >> 172.21.15.46:0/1412085577 conn(0x7f39e00b2040 msgr2=0x7f39e01509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 shutdown_connections 2024-09-17T03:23:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.436+0000 7f39e5600640 1 -- 172.21.15.46:0/1412085577 wait complete. 2024-09-17T03:23:57.459 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:23:57.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8be6000640 1 -- 172.21.15.46:0/624796245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 msgr2=0x7f8bc400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8be6000640 1 --2- 172.21.15.46:0/624796245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8bc400b9d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009ee0 tx=0x7f8bd801d610 comp rx=0 tx=0).stop 2024-09-17T03:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8bdfe00640 1 -- 172.21.15.46:0/624796245 reap_dead start 2024-09-17T03:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8be6000640 1 -- 172.21.15.46:0/624796245 shutdown_connections 2024-09-17T03:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8be6000640 1 -- 172.21.15.46:0/624796245 >> 172.21.15.46:0/624796245 conn(0x7f8be00708c0 msgr2=0x7f8be0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8be6000640 1 -- 172.21.15.46:0/624796245 shutdown_connections 2024-09-17T03:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.620+0000 7f8be6000640 1 -- 172.21.15.46:0/624796245 wait complete. 2024-09-17T03:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 Processor -- start 2024-09-17T03:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 -- start start 2024-09-17T03:23:57.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8be01f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8be01f52f0 0x7f8be01467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8be00b4250 0x7f8be00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8be015cf90 con 0x7f8bc40050d0 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0074d30 con 0x7f8be00b4250 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8be6000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8be015d110 con 0x7f8be01f52f0 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8be01f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8be01f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44148/0 (socket says 172.21.15.46:44148) 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8bdf400640 1 -- 172.21.15.46:0/2644508665 learned_addr learned my addr 172.21.15.46:0/2644508665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8bdfe00640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8be00b4250 0x7f8be00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.624+0000 7f8bd7e00640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8be01f52f0 0x7f8be01467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bdf400640 1 -- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8be01f52f0 msgr2=0x7f8be01467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bdf400640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8be01f52f0 0x7f8be01467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bdf400640 1 -- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8be00b4250 msgr2=0x7f8be00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bdf400640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8be00b4250 0x7f8be00b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bdf400640 1 -- 172.21.15.46:0/2644508665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be00b4ba0 con 0x7f8bc40050d0 2024-09-17T03:23:57.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bdf400640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8be01f93e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006660 tx=0x7f8bd00017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:57.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011700 con 0x7f8bc40050d0 2024-09-17T03:23:57.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bd0016020 con 0x7f8bc40050d0 2024-09-17T03:23:57.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01f5ae0 con 0x7f8bc40050d0 2024-09-17T03:23:57.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011b40 con 0x7f8bc40050d0 2024-09-17T03:23:57.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.628+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01f60a0 con 0x7f8bc40050d0 2024-09-17T03:23:57.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.632+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be00b6110 con 0x7f8bc40050d0 2024-09-17T03:23:57.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.632+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8bd0011ce0 con 0x7f8bc40050d0 2024-09-17T03:23:57.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.632+0000 7f8bd7400640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8bbc073560 0x7f8bbc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:57.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.632+0000 7f8bd7e00640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8bbc073560 0x7f8bbc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:57.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.632+0000 7f8bd7e00640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8bbc073560 0x7f8bbc075a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80018b0 tx=0x7f8bc8001120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:57.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.636+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f8bd0066910 con 0x7f8bc40050d0 2024-09-17T03:23:57.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.640+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8bd0020020 con 0x7f8bc40050d0 2024-09-17T03:23:57.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:57.816+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8be00d2260 con 0x7f8bc40050d0 2024-09-17T03:23:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.644+0000 7f8bd7400640 1 -- 172.21.15.46:0/2644508665 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7f8be00b4ba0 con 0x7f8bc40050d0 2024-09-17T03:23:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:23:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8bbc073560 msgr2=0x7f8bbc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8bbc073560 0x7f8bbc075a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80018b0 tx=0x7f8bc8001120 comp rx=0 tx=0).stop 2024-09-17T03:23:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 msgr2=0x7f8be01f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8be01f93e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006660 tx=0x7f8bd00017e0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 shutdown_connections 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8bbc073560 0x7f8bbc075a70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8be00b4250 0x7f8be00b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8be01f52f0 0x7f8be01467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 --2- 172.21.15.46:0/2644508665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bc40050d0 0x7f8be01f93e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 >> 172.21.15.46:0/2644508665 conn(0x7f8be00708c0 msgr2=0x7f8be00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 shutdown_connections 2024-09-17T03:23:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.648+0000 7f8be6000640 1 -- 172.21.15.46:0/2644508665 wait complete. 2024-09-17T03:23:58.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:23:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 -- 172.21.15.46:0/1462175465 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 msgr2=0x7fb1a0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 --2- 172.21.15.46:0/1462175465 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a0074db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb190009870 tx=0x7fb19001c780 comp rx=0 tx=0).stop 2024-09-17T03:23:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 -- 172.21.15.46:0/1462175465 shutdown_connections 2024-09-17T03:23:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 --2- 172.21.15.46:0/1462175465 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a0074db0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 -- 172.21.15.46:0/1462175465 >> 172.21.15.46:0/1462175465 conn(0x7fb1a00b2040 msgr2=0x7fb1a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 -- 172.21.15.46:0/1462175465 shutdown_connections 2024-09-17T03:23:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 -- 172.21.15.46:0/1462175465 wait complete. 2024-09-17T03:23:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.848+0000 7fb1a6800640 1 Processor -- start 2024-09-17T03:23:58.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 -- start start 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a013d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a00b3340 0x7fb1a01577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 0x7fb1a0145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a016b340 con 0x7fb1a00cefa0 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a016b1c0 con 0x7fb1a00b3340 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a016b4c0 con 0x7fb1a00749a0 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 0x7fb1a0145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a013d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a013d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:34970/0 (socket says 172.21.15.46:34970) 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 -- 172.21.15.46:0/3872698142 learned_addr learned my addr 172.21.15.46:0/3872698142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 0x7fb1a0145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44162/0 (socket says 172.21.15.46:44162) 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb196a00640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a00b3340 0x7fb1a01577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 -- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a00b3340 msgr2=0x7fb1a01577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a00b3340 0x7fb1a01577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 -- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 msgr2=0x7fb1a0145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 0x7fb1a0145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 -- 172.21.15.46:0/3872698142 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a013b9e0 con 0x7fb1a00749a0 2024-09-17T03:23:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19fe00640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 0x7fb1a0145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:23:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb19f400640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a013d2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb19001ae30 tx=0x7fb190005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900a7a90 con 0x7fb1a00749a0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1900bd070 con 0x7fb1a00749a0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190003600 con 0x7fb1a00749a0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a0144fc0 con 0x7fb1a00749a0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.852+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a013d810 con 0x7fb1a00749a0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.856+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a0074db0 con 0x7fb1a00749a0 2024-09-17T03:23:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.856+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fb1900a7070 con 0x7fb1a00749a0 2024-09-17T03:23:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.856+0000 7fb197e00640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb17c073560 0x7fb17c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.856+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7fb19012f9d0 con 0x7fb1a00749a0 2024-09-17T03:23:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.856+0000 7fb196a00640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb17c073560 0x7fb17c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.856+0000 7fb196a00640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb17c073560 0x7fb17c075a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1a01fa0e0 tx=0x7fb188009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:58.860+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb190016370 con 0x7fb1a00749a0 2024-09-17T03:23:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.036+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb1a015fef0 con 0x7fb1a00749a0 2024-09-17T03:23:59.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:23:59.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:23:59.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.664+0000 7fb197e00640 1 -- 172.21.15.46:0/3872698142 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7fb1900fa2a0 con 0x7fb1a00749a0 2024-09-17T03:23:59.669 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:23:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb17c073560 msgr2=0x7fb17c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb17c073560 0x7fb17c075a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb1a01fa0e0 tx=0x7fb188009040 comp rx=0 tx=0).stop 2024-09-17T03:23:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 msgr2=0x7fb1a013d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a013d2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb19001ae30 tx=0x7fb190005d40 comp rx=0 tx=0).stop 2024-09-17T03:23:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 shutdown_connections 2024-09-17T03:23:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb17c073560 0x7fb17c075a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a00cefa0 0x7fb1a0145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a00b3340 0x7fb1a01577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 --2- 172.21.15.46:0/3872698142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb1a00749a0 0x7fb1a013d2d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.672+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 >> 172.21.15.46:0/3872698142 conn(0x7fb1a00b2040 msgr2=0x7fb1a015b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.676+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 shutdown_connections 2024-09-17T03:23:59.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.676+0000 7fb1a6800640 1 -- 172.21.15.46:0/3872698142 wait complete. 2024-09-17T03:23:59.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 -- 172.21.15.46:0/2635880037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 msgr2=0x7f401c0cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 --2- 172.21.15.46:0/2635880037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0cd260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f401c15e410 tx=0x7f401001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 -- 172.21.15.46:0/2635880037 shutdown_connections 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 --2- 172.21.15.46:0/2635880037 >> v2:172.21.15.99:3300/0 conn(0x7f401c0cdbb0 0x7f401c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 --2- 172.21.15.46:0/2635880037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0cd260 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 -- 172.21.15.46:0/2635880037 >> 172.21.15.46:0/2635880037 conn(0x7f401c0b2040 msgr2=0x7f401c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:23:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 -- 172.21.15.46:0/2635880037 shutdown_connections 2024-09-17T03:23:59.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.864+0000 7f4024400640 1 -- 172.21.15.46:0/2635880037 wait complete. 2024-09-17T03:23:59.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 Processor -- start 2024-09-17T03:23:59.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 -- start start 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f401c0cd7a0 0x7f401c0b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f401c0cdbb0 0x7f401c072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f401c165e70 con 0x7f401c15aa50 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f401c0b5f40 con 0x7f401c0cd7a0 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4024400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f401c165ff0 con 0x7f401c0cdbb0 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4022400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4022400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44180/0 (socket says 172.21.15.46:44180) 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.868+0000 7f4022400640 1 -- 172.21.15.46:0/2295204502 learned_addr learned my addr 172.21.15.46:0/2295204502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4021000640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f401c0cdbb0 0x7f401c072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4021a00640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f401c0cd7a0 0x7f401c0b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4022400640 1 -- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f401c0cdbb0 msgr2=0x7f401c072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4022400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f401c0cdbb0 0x7f401c072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4022400640 1 -- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f401c0cd7a0 msgr2=0x7f401c0b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4022400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f401c0cd7a0 0x7f401c0b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:23:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4022400640 1 -- 172.21.15.46:0/2295204502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c1558c0 con 0x7f401c15aa50 2024-09-17T03:23:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4022400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0b3510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f401400ea80 tx=0x7f401400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014004b30 con 0x7f401c15aa50 2024-09-17T03:23:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4014004cf0 con 0x7f401c15aa50 2024-09-17T03:23:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c073810 con 0x7f401c15aa50 2024-09-17T03:23:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014005280 con 0x7f401c15aa50 2024-09-17T03:23:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c0b0720 con 0x7f401c15aa50 2024-09-17T03:23:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c159780 con 0x7f401c15aa50 2024-09-17T03:23:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.872+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f4014005420 con 0x7f401c15aa50 2024-09-17T03:23:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.876+0000 7f4019600640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ffc073560 0x7f3ffc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:23:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.876+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7f40140958a0 con 0x7f401c15aa50 2024-09-17T03:23:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.876+0000 7f4021a00640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ffc073560 0x7f3ffc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:23:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.876+0000 7f4021a00640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ffc073560 0x7f3ffc075a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f40100af000 tx=0x7f401001b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:23:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:23:59.880+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f401405fd80 con 0x7f401c15aa50 2024-09-17T03:24:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.096+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f401c156b50 con 0x7f401c15aa50 2024-09-17T03:24:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.652+0000 7f4019600640 1 -- 172.21.15.46:0/2295204502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f40140647b0 con 0x7f401c15aa50 2024-09-17T03:24:00.655 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 6 and data pool 7 2024-09-17T03:24:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.660+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ffc073560 msgr2=0x7f3ffc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.660+0000 7f4024400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ffc073560 0x7f3ffc075a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f40100af000 tx=0x7f401001b100 comp rx=0 tx=0).stop 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.660+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 msgr2=0x7f401c0b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.660+0000 7f4024400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0b3510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f401400ea80 tx=0x7f401400ef40 comp rx=0 tx=0).stop 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 shutdown_connections 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ffc073560 0x7f3ffc075a70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f401c15aa50 0x7f401c0b3510 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f401c0cdbb0 0x7f401c072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 --2- 172.21.15.46:0/2295204502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f401c0cd7a0 0x7f401c0b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 >> 172.21.15.46:0/2295204502 conn(0x7f401c0b2040 msgr2=0x7f401c156020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 shutdown_connections 2024-09-17T03:24:00.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.664+0000 7f4024400640 1 -- 172.21.15.46:0/2295204502 wait complete. 2024-09-17T03:24:00.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:24:00.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 -- 172.21.15.46:0/2541688035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c0b54f0 msgr2=0x7f212c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 --2- 172.21.15.46:0/2541688035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c0b54f0 0x7f212c15bb60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2124009bc0 tx=0x7f212401ca00 comp rx=0 tx=0).stop 2024-09-17T03:24:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 -- 172.21.15.46:0/2541688035 shutdown_connections 2024-09-17T03:24:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 --2- 172.21.15.46:0/2541688035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c0b54f0 0x7f212c15bb60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 -- 172.21.15.46:0/2541688035 >> 172.21.15.46:0/2541688035 conn(0x7f212c0b2040 msgr2=0x7f212c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 -- 172.21.15.46:0/2541688035 shutdown_connections 2024-09-17T03:24:00.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.876+0000 7f2132c00640 1 -- 172.21.15.46:0/2541688035 wait complete. 2024-09-17T03:24:00.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 Processor -- start 2024-09-17T03:24:00.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 -- start start 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f212c0b5c70 0x7f212c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c1527b0 0x7f212c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 0x7f212c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f212c169b30 con 0x7f212c151fd0 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f212c1699b0 con 0x7f212c1527b0 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f212c169cb0 con 0x7f212c0b5c70 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 0x7f212c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:00.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 0x7f212c0c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44190/0 (socket says 172.21.15.46:44190) 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 -- 172.21.15.46:0/630882051 learned_addr learned my addr 172.21.15.46:0/630882051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2123e00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c1527b0 0x7f212c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f212be00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f212c0b5c70 0x7f212c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 -- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f212c0b5c70 msgr2=0x7f212c151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f212c0b5c70 0x7f212c151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 -- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c1527b0 msgr2=0x7f212c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c1527b0 0x7f212c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 -- 172.21.15.46:0/630882051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c151890 con 0x7f212c151fd0 2024-09-17T03:24:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2130c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 0x7f212c0c52b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2124019610 tx=0x7f2124009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21240bb070 con 0x7f212c151fd0 2024-09-17T03:24:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2124003df0 con 0x7f212c151fd0 2024-09-17T03:24:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21240b2040 con 0x7f212c151fd0 2024-09-17T03:24:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.880+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c157750 con 0x7f212c151fd0 2024-09-17T03:24:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.884+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c153a80 con 0x7f212c151fd0 2024-09-17T03:24:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.884+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c157920 con 0x7f212c151fd0 2024-09-17T03:24:00.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.884+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f21240be070 con 0x7f212c151fd0 2024-09-17T03:24:00.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.884+0000 7f2128c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2108073560 0x7f2108075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:00.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.884+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f212412e2d0 con 0x7f212c151fd0 2024-09-17T03:24:00.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.884+0000 7f212be00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2108073560 0x7f2108075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:00.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.888+0000 7f212be00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2108073560 0x7f2108075a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f211c001390 tx=0x7f211c001730 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:00.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:00.888+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21240166b0 con 0x7f212c151fd0 2024-09-17T03:24:01.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.064+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f212c1538a0 con 0x7f212c151fd0 2024-09-17T03:24:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2128c00640 1 -- 172.21.15.46:0/630882051 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16650 (secure 0 0 0) 0x7f212c1538a0 con 0x7f212c151fd0 2024-09-17T03:24:01.069 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:01.069 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:24:00.099134+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:23:57.829182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T03:23:59.049469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2108073560 msgr2=0x7f2108075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2108073560 0x7f2108075a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f211c001390 tx=0x7f211c001730 comp rx=0 tx=0).stop 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 msgr2=0x7f212c0c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 0x7f212c0c52b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2124019610 tx=0x7f2124009310 comp rx=0 tx=0).stop 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 shutdown_connections 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2108073560 0x7f2108075a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c151fd0 0x7f212c0c52b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212c1527b0 0x7f212c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 --2- 172.21.15.46:0/630882051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f212c0b5c70 0x7f212c151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 >> 172.21.15.46:0/630882051 conn(0x7f212c0b2040 msgr2=0x7f212c159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 shutdown_connections 2024-09-17T03:24:01.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.068+0000 7f2132c00640 1 -- 172.21.15.46:0/630882051 wait complete. 2024-09-17T03:24:01.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 -- 172.21.15.46:0/409244259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b4de0 msgr2=0x7f34300b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 --2- 172.21.15.46:0/409244259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b4de0 0x7f34300b51f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3428009bc0 tx=0x7f342801cae0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 -- 172.21.15.46:0/409244259 shutdown_connections 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 --2- 172.21.15.46:0/409244259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b4de0 0x7f34300b51f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 --2- 172.21.15.46:0/409244259 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34300b5c70 0x7f343015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 -- 172.21.15.46:0/409244259 >> 172.21.15.46:0/409244259 conn(0x7f34300b2040 msgr2=0x7f34300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 -- 172.21.15.46:0/409244259 shutdown_connections 2024-09-17T03:24:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.248+0000 7f3438400640 1 -- 172.21.15.46:0/409244259 wait complete. 2024-09-17T03:24:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 Processor -- start 2024-09-17T03:24:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 -- start start 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34300b4de0 0x7f34301498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 0x7f34300ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34301527b0 0x7f34301f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3430168ab0 con 0x7f34300b5c70 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3430168930 con 0x7f34300b4de0 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3430168c30 con 0x7f34301527b0 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 0x7f34300ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 0x7f34300ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44202/0 (socket says 172.21.15.46:44202) 2024-09-17T03:24:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 -- 172.21.15.46:0/1357618976 learned_addr learned my addr 172.21.15.46:0/1357618976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435a00640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34300b4de0 0x7f34301498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3436400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34301527b0 0x7f34301f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 -- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34301527b0 msgr2=0x7f34301f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34301527b0 0x7f34301f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 -- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34300b4de0 msgr2=0x7f34301498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34300b4de0 0x7f34301498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 -- 172.21.15.46:0/1357618976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3430158f90 con 0x7f34300b5c70 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3435000640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 0x7f34300ceaf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c370 tx=0x7f341c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c017070 con 0x7f34300b5c70 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f341c003600 con 0x7f34300b5c70 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.252+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3430159270 con 0x7f34300b5c70 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c003ab0 con 0x7f34300b5c70 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3430149df0 con 0x7f34300b5c70 2024-09-17T03:24:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34300b51f0 con 0x7f34300b5c70 2024-09-17T03:24:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f341c01a070 con 0x7f34300b5c70 2024-09-17T03:24:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f342d600640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3410073560 0x7f3410075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f341c093b00 con 0x7f34300b5c70 2024-09-17T03:24:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.256+0000 7f3435a00640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3410073560 0x7f3410075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.260+0000 7f3435a00640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3410073560 0x7f3410075a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3424009b40 tx=0x7f34240091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.260+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34300b5730 con 0x7f34300b5c70 2024-09-17T03:24:01.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.436+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f343015b640 con 0x7f34300b5c70 2024-09-17T03:24:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f342d600640 1 -- 172.21.15.46:0/1357618976 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16650 (secure 0 0 0) 0x7f341c093e20 con 0x7f34300b5c70 2024-09-17T03:24:01.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:01.441 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:24:00.099134+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:23:57.829182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T03:23:59.049469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:24:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3410073560 msgr2=0x7f3410075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3410073560 0x7f3410075a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3424009b40 tx=0x7f34240091e0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 msgr2=0x7f34300ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 0x7f34300ceaf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c370 tx=0x7f341c00c830 comp rx=0 tx=0).stop 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 shutdown_connections 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3410073560 0x7f3410075a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34301527b0 0x7f34301f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34300b5c70 0x7f34300ceaf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 --2- 172.21.15.46:0/1357618976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34300b4de0 0x7f34301498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 >> 172.21.15.46:0/1357618976 conn(0x7f34300b2040 msgr2=0x7f34300b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 shutdown_connections 2024-09-17T03:24:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.440+0000 7f3438400640 1 -- 172.21.15.46:0/1357618976 wait complete. 2024-09-17T03:24:01.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:24:01.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 -- 172.21.15.46:0/4143127025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 msgr2=0x7f558c1606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 --2- 172.21.15.46:0/4143127025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c1606b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5578009900 tx=0x7f557801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 -- 172.21.15.46:0/4143127025 shutdown_connections 2024-09-17T03:24:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 --2- 172.21.15.46:0/4143127025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c1606b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 -- 172.21.15.46:0/4143127025 >> 172.21.15.46:0/4143127025 conn(0x7f558c0b2c40 msgr2=0x7f558c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 -- 172.21.15.46:0/4143127025 shutdown_connections 2024-09-17T03:24:01.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.624+0000 7f5594600640 1 -- 172.21.15.46:0/4143127025 wait complete. 2024-09-17T03:24:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 Processor -- start 2024-09-17T03:24:01.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 -- start start 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f558c0cd2e0 0x7f558c13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f558c0cf1b0 0x7f558c152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c13d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f558c16ade0 con 0x7f558c1602a0 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f558c16ac60 con 0x7f558c0cd2e0 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5594600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f558c16af60 con 0x7f558c0cf1b0 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c13d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c13d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44212/0 (socket says 172.21.15.46:44212) 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 -- 172.21.15.46:0/3334635424 learned_addr learned my addr 172.21.15.46:0/3334635424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5591200640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f558c0cf1b0 0x7f558c152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5591c00640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f558c0cd2e0 0x7f558c13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 -- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f558c0cf1b0 msgr2=0x7f558c152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f558c0cf1b0 0x7f558c152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 -- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f558c0cd2e0 msgr2=0x7f558c13e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f558c0cd2e0 0x7f558c13e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5592600640 1 -- 172.21.15.46:0/3334635424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c152bb0 con 0x7f558c1602a0 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.628+0000 7f5591c00640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f558c0cd2e0 0x7f558c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:01.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5592600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c13d7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f558400c760 tx=0x7f558400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584003db0 con 0x7f558c1602a0 2024-09-17T03:24:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f558400ce10 con 0x7f558c1602a0 2024-09-17T03:24:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c13eaf0 con 0x7f558c1602a0 2024-09-17T03:24:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55840052d0 con 0x7f558c1602a0 2024-09-17T03:24:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c13cfd0 con 0x7f558c1602a0 2024-09-17T03:24:01.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c1606b0 con 0x7f558c1602a0 2024-09-17T03:24:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5584005470 con 0x7f558c1602a0 2024-09-17T03:24:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5589600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f556c073590 0x7f556c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f5584094600 con 0x7f558c1602a0 2024-09-17T03:24:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.632+0000 7f5591c00640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f556c073590 0x7f556c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.636+0000 7f5591c00640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f556c073590 0x7f556c075aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5580000f70 tx=0x7f5580009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:01.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.640+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f558c152bb0 con 0x7f558c1602a0 2024-09-17T03:24:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:01.852+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f558c06ec30 con 0x7f558c1602a0 2024-09-17T03:24:02.636 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-09-17T03:24:02.636+0000 7f24d8800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:24:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.684+0000 7f5589600640 1 -- 172.21.15.46:0/3334635424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f558c160bf0 con 0x7f558c1602a0 2024-09-17T03:24:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f556c073590 msgr2=0x7f556c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f556c073590 0x7f556c075aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5580000f70 tx=0x7f5580009040 comp rx=0 tx=0).stop 2024-09-17T03:24:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 msgr2=0x7f558c13d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c13d7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f558400c760 tx=0x7f558400cc20 comp rx=0 tx=0).stop 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 shutdown_connections 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f556c073590 0x7f556c075aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f558c1602a0 0x7f558c13d7d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f558c0cf1b0 0x7f558c152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 --2- 172.21.15.46:0/3334635424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f558c0cd2e0 0x7f558c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 >> 172.21.15.46:0/3334635424 conn(0x7f558c0b2c40 msgr2=0x7f558c070590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 shutdown_connections 2024-09-17T03:24:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.688+0000 7f5594600640 1 -- 172.21.15.46:0/3334635424 wait complete. 2024-09-17T03:24:02.715 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:24:02.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7c600640 1 -- 172.21.15.46:0/3494337978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c004810 msgr2=0x7fbf5c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7c600640 1 --2- 172.21.15.46:0/3494337978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c004810 0x7fbf5c004c20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbf74071fd0 tx=0x7fbf6001cef0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7a600640 1 -- 172.21.15.46:0/3494337978 reap_dead start 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7c600640 1 -- 172.21.15.46:0/3494337978 shutdown_connections 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7c600640 1 -- 172.21.15.46:0/3494337978 >> 172.21.15.46:0/3494337978 conn(0x7fbf740b2040 msgr2=0x7fbf740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7c600640 1 -- 172.21.15.46:0/3494337978 shutdown_connections 2024-09-17T03:24:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.880+0000 7fbf7c600640 1 -- 172.21.15.46:0/3494337978 wait complete. 2024-09-17T03:24:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 Processor -- start 2024-09-17T03:24:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 -- start start 2024-09-17T03:24:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf5c004810 0x7fbf741575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 0x7fbf7414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbf7414d030 0x7fbf741f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740b4340 con 0x7fbf7414c6e0 2024-09-17T03:24:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740b41c0 con 0x7fbf5c004810 2024-09-17T03:24:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740b44c0 con 0x7fbf7414d030 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 0x7fbf7414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 0x7fbf7414caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32772/0 (socket says 172.21.15.46:32772) 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 -- 172.21.15.46:0/1127823274 learned_addr learned my addr 172.21.15.46:0/1127823274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79c00640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf5c004810 0x7fbf741575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7a600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbf7414d030 0x7fbf741f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 -- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbf7414d030 msgr2=0x7fbf741f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbf7414d030 0x7fbf741f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 -- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf5c004810 msgr2=0x7fbf741575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf5c004810 0x7fbf741575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 -- 172.21.15.46:0/1127823274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7414f870 con 0x7fbf7414c6e0 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf7a600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbf7414d030 0x7fbf741f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf79200640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 0x7fbf7414caf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c10 tx=0x7fbf60001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.884+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60006090 con 0x7fbf7414c6e0 2024-09-17T03:24:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf60006320 con 0x7fbf7414c6e0 2024-09-17T03:24:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf74158860 con 0x7fbf7414c6e0 2024-09-17T03:24:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003600 con 0x7fbf7414c6e0 2024-09-17T03:24:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf74158de0 con 0x7fbf7414c6e0 2024-09-17T03:24:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf740b4a50 con 0x7fbf7414c6e0 2024-09-17T03:24:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fbf600037a0 con 0x7fbf7414c6e0 2024-09-17T03:24:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf71600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbf54073560 0x7fbf54075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fbf6012e7a0 con 0x7fbf7414c6e0 2024-09-17T03:24:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.888+0000 7fbf79c00640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbf54073560 0x7fbf54075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.892+0000 7fbf79c00640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbf54073560 0x7fbf54075a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009b40 tx=0x7fbf680091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:02.892+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf74158860 con 0x7fbf7414c6e0 2024-09-17T03:24:03.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.108+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbf7415a660 con 0x7fbf7414c6e0 2024-09-17T03:24:03.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf71600640 1 -- 172.21.15.46:0/1127823274 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fbf740b5730 con 0x7fbf7414c6e0 2024-09-17T03:24:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbf54073560 msgr2=0x7fbf54075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbf54073560 0x7fbf54075a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009b40 tx=0x7fbf680091e0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 msgr2=0x7fbf7414caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:03.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 0x7fbf7414caf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c10 tx=0x7fbf60001260 comp rx=0 tx=0).stop 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 shutdown_connections 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbf54073560 0x7fbf54075a70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbf7414d030 0x7fbf741f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf7414c6e0 0x7fbf7414caf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 --2- 172.21.15.46:0/1127823274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf5c004810 0x7fbf741575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 >> 172.21.15.46:0/1127823274 conn(0x7fbf740b2040 msgr2=0x7fbf7415f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 shutdown_connections 2024-09-17T03:24:03.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.696+0000 7fbf7c600640 1 -- 172.21.15.46:0/1127823274 wait complete. 2024-09-17T03:24:03.724 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:24:03.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:24:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58c200640 1 -- 172.21.15.46:0/3136520884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 msgr2=0x7fa56c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58c200640 1 --2- 172.21.15.46:0/3136520884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa56c004c20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa5840ce3e0 tx=0x7fa57001cb60 comp rx=0 tx=0).stop 2024-09-17T03:24:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58a200640 1 -- 172.21.15.46:0/3136520884 reap_dead start 2024-09-17T03:24:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58c200640 1 -- 172.21.15.46:0/3136520884 shutdown_connections 2024-09-17T03:24:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58c200640 1 -- 172.21.15.46:0/3136520884 >> 172.21.15.46:0/3136520884 conn(0x7fa5840b35f0 msgr2=0x7fa5840b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58c200640 1 -- 172.21.15.46:0/3136520884 shutdown_connections 2024-09-17T03:24:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.888+0000 7fa58c200640 1 -- 172.21.15.46:0/3136520884 wait complete. 2024-09-17T03:24:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 Processor -- start 2024-09-17T03:24:03.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 -- start start 2024-09-17T03:24:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa58415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840b1390 0x7fa5841597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5840b17a0 0x7fa5841614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa584167bb0 con 0x7fa56c004810 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa584167a30 con 0x7fa5840b1390 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa584167d30 con 0x7fa5840b17a0 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa58415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa58415dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32786/0 (socket says 172.21.15.46:32786) 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 -- 172.21.15.46:0/1272681512 learned_addr learned my addr 172.21.15.46:0/1272681512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa588e00640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840b1390 0x7fa5841597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58a200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5840b17a0 0x7fa5841614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 -- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5840b17a0 msgr2=0x7fa5841614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5840b17a0 0x7fa5841614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 -- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840b1390 msgr2=0x7fa5841597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840b1390 0x7fa5841597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 -- 172.21.15.46:0/1272681512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840b1f70 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa589800640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa58415dbb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa578006600 tx=0x7fa5780039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578015070 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa578011920 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578011d70 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5840b2250 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.892+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58415d3b0 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.896+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5840700a0 con 0x7fa56c004810 2024-09-17T03:24:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.896+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fa578018070 con 0x7fa56c004810 2024-09-17T03:24:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.896+0000 7fa581600640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa564073590 0x7fa564075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.896+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fa578095690 con 0x7fa56c004810 2024-09-17T03:24:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.896+0000 7fa588e00640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa564073590 0x7fa564075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.896+0000 7fa588e00640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa564073590 0x7fa564075aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa5700b0000 tx=0x7fa5700a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:03.900+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa57805fa00 con 0x7fa56c004810 2024-09-17T03:24:04.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.108+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa58406de30 con 0x7fa56c004810 2024-09-17T03:24:04.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.108+0000 7fa581600640 1 -- 172.21.15.46:0/1272681512 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7fa578064430 con 0x7fa56c004810 2024-09-17T03:24:04.111 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:04.111 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"btime":"2024-09-17T03:24:03:689855+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:24:00.099078+0000","modified":"2024-09-17T03:24:03.111473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245313":{"gid":245313,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4290314221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4290314221},{"type":"v1","addr":"172.21.15.46:6836","nonce":4290314221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245313,"qdb_cluster":[245313]},"id":3}]} 2024-09-17T03:24:04.112 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa564073590 msgr2=0x7fa564075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa564073590 0x7fa564075aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa5700b0000 tx=0x7fa5700a7040 comp rx=0 tx=0).stop 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 msgr2=0x7fa58415dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa58415dbb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa578006600 tx=0x7fa5780039d0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 shutdown_connections 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa564073590 0x7fa564075aa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5840b17a0 0x7fa5841614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840b1390 0x7fa5841597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 --2- 172.21.15.46:0/1272681512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c004810 0x7fa58415dbb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 >> 172.21.15.46:0/1272681512 conn(0x7fa5840b35f0 msgr2=0x7fa58415bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:04.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 shutdown_connections 2024-09-17T03:24:04.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.112+0000 7fa58c200640 1 -- 172.21.15.46:0/1272681512 wait complete. 2024-09-17T03:24:04.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:24:04.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 -- 172.21.15.46:0/1981052279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33001527b0 msgr2=0x7f330015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 --2- 172.21.15.46:0/1981052279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33001527b0 0x7f330015bb60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3300074b20 tx=0x7f32e801c820 comp rx=0 tx=0).stop 2024-09-17T03:24:04.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 -- 172.21.15.46:0/1981052279 shutdown_connections 2024-09-17T03:24:04.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 --2- 172.21.15.46:0/1981052279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33001527b0 0x7f330015bb60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 -- 172.21.15.46:0/1981052279 >> 172.21.15.46:0/1981052279 conn(0x7f33000b2040 msgr2=0x7f33000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:04.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 -- 172.21.15.46:0/1981052279 shutdown_connections 2024-09-17T03:24:04.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 -- 172.21.15.46:0/1981052279 wait complete. 2024-09-17T03:24:04.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 Processor -- start 2024-09-17T03:24:04.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.292+0000 7f3304c00640 1 -- start start 2024-09-17T03:24:04.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 0x7f3300151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000b5c70 0x7f330014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33001527b0 0x7f330014e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3300169b60 con 0x7f33000b4de0 2024-09-17T03:24:04.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33001699e0 con 0x7f33001527b0 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3300169ce0 con 0x7f33000b5c70 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 0x7f3300151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 0x7f3300151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32808/0 (socket says 172.21.15.46:32808) 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 -- 172.21.15.46:0/483012088 learned_addr learned my addr 172.21.15.46:0/483012088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32f7e00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000b5c70 0x7f330014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fe800640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33001527b0 0x7f330014e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 -- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000b5c70 msgr2=0x7f330014e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000b5c70 0x7f330014e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 -- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33001527b0 msgr2=0x7f330014e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33001527b0 0x7f330014e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 -- 172.21.15.46:0/483012088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33001f9b50 con 0x7f33000b4de0 2024-09-17T03:24:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32f7e00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000b5c70 0x7f330014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32fde00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 0x7f3300151800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c420 tx=0x7f32f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0017070 con 0x7f33000b4de0 2024-09-17T03:24:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32f00053c0 con 0x7f33000b4de0 2024-09-17T03:24:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0003860 con 0x7f33000b4de0 2024-09-17T03:24:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33001f9e30 con 0x7f33000b4de0 2024-09-17T03:24:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33001fa3f0 con 0x7f33000b4de0 2024-09-17T03:24:04.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.296+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330015bb60 con 0x7f33000b4de0 2024-09-17T03:24:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.300+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f32f001a070 con 0x7f33000b4de0 2024-09-17T03:24:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.300+0000 7f32f6000640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32dc073590 0x7f32dc075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.300+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f32f0093f70 con 0x7f33000b4de0 2024-09-17T03:24:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.300+0000 7f32f7e00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32dc073590 0x7f32dc075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.300+0000 7f32f7e00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32dc073590 0x7f32dc075aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f32e80af000 tx=0x7f32e801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.304+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32f0021020 con 0x7f33000b4de0 2024-09-17T03:24:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.476+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3300155cc0 con 0x7f33000b4de0 2024-09-17T03:24:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f32f6000640 1 -- 172.21.15.46:0/483012088 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16637 (secure 0 0 0) 0x7f32f005eb70 con 0x7f33000b4de0 2024-09-17T03:24:04.481 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:04.481 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:24:01.671822+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:23:57.829182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T03:23:59.049469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:24:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32dc073590 msgr2=0x7f32dc075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32dc073590 0x7f32dc075aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f32e80af000 tx=0x7f32e801b060 comp rx=0 tx=0).stop 2024-09-17T03:24:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 msgr2=0x7f3300151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 0x7f3300151800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c420 tx=0x7f32f000c8e0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 shutdown_connections 2024-09-17T03:24:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32dc073590 0x7f32dc075aa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33001527b0 0x7f330014e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000b5c70 0x7f330014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 --2- 172.21.15.46:0/483012088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33000b4de0 0x7f3300151800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 >> 172.21.15.46:0/483012088 conn(0x7f33000b2040 msgr2=0x7f33000cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 shutdown_connections 2024-09-17T03:24:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.480+0000 7f3304c00640 1 -- 172.21.15.46:0/483012088 wait complete. 2024-09-17T03:24:04.504 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:24:04.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:24:04.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 -- 172.21.15.46:0/441075183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200cd2e0 msgr2=0x7f8120070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 --2- 172.21.15.46:0/441075183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200cd2e0 0x7f8120070e80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f81201464a0 tx=0x7f811401c8c0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 -- 172.21.15.46:0/441075183 shutdown_connections 2024-09-17T03:24:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 --2- 172.21.15.46:0/441075183 >> v2:172.21.15.99:3300/0 conn(0x7f81201602a0 0x7f81201606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 --2- 172.21.15.46:0/441075183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81200cd2e0 0x7f8120070e80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 -- 172.21.15.46:0/441075183 >> 172.21.15.46:0/441075183 conn(0x7f81200b2c40 msgr2=0x7f81200b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 -- 172.21.15.46:0/441075183 shutdown_connections 2024-09-17T03:24:04.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 -- 172.21.15.46:0/441075183 wait complete. 2024-09-17T03:24:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 Processor -- start 2024-09-17T03:24:04.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.664+0000 7f8128800640 1 -- start start 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81200cd2e0 0x7f81201fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 0x7f81201fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81201602a0 0x7f8120158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f812016a2b0 con 0x7f812015cd20 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f812016a130 con 0x7f81200cd2e0 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f812016a430 con 0x7f81201602a0 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 0x7f81201fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 0x7f81201fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32820/0 (socket says 172.21.15.46:32820) 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 -- 172.21.15.46:0/444303686 learned_addr learned my addr 172.21.15.46:0/444303686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125e00640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81200cd2e0 0x7f81201fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8126800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81201602a0 0x7f8120158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 -- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81201602a0 msgr2=0x7f8120158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81201602a0 0x7f8120158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 -- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81200cd2e0 msgr2=0x7f81201fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81200cd2e0 0x7f81201fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 -- 172.21.15.46:0/444303686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8120157490 con 0x7f812015cd20 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8126800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81201602a0 0x7f8120158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8125400640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 0x7f81201fcc40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f811000d750 tx=0x7f811000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110017070 con 0x7f812015cd20 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8110009d20 con 0x7f812015cd20 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110005280 con 0x7f812015cd20 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8120156700 con 0x7f812015cd20 2024-09-17T03:24:04.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812015aa50 con 0x7f812015cd20 2024-09-17T03:24:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81201530d0 con 0x7f812015cd20 2024-09-17T03:24:04.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.668+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8110005420 con 0x7f812015cd20 2024-09-17T03:24:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.672+0000 7f811a000640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8104073560 0x7f8104075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.672+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8110094a20 con 0x7f812015cd20 2024-09-17T03:24:04.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.672+0000 7f8125e00640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8104073560 0x7f8104075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:04.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.672+0000 7f8125e00640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8104073560 0x7f8104075a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f81140af000 tx=0x7f81140a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:04.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.676+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f811000de60 con 0x7f812015cd20 2024-09-17T03:24:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.852+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f812006b540 con 0x7f8104073560 2024-09-17T03:24:04.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.856+0000 7f811a000640 1 -- 172.21.15.46:0/444303686 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f81140b3040 con 0x7f8104073560 2024-09-17T03:24:04.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:24:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8104073560 msgr2=0x7f8104075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8104073560 0x7f8104075a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f81140af000 tx=0x7f81140a7040 comp rx=0 tx=0).stop 2024-09-17T03:24:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 msgr2=0x7f81201fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 0x7f81201fcc40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f811000d750 tx=0x7f811000dc10 comp rx=0 tx=0).stop 2024-09-17T03:24:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 shutdown_connections 2024-09-17T03:24:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8104073560 0x7f8104075a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81201602a0 0x7f8120158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f812015cd20 0x7f81201fcc40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 --2- 172.21.15.46:0/444303686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81200cd2e0 0x7f81201fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 >> 172.21.15.46:0/444303686 conn(0x7f81200b2c40 msgr2=0x7f8120153440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 shutdown_connections 2024-09-17T03:24:04.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:04.860+0000 7f8128800640 1 -- 172.21.15.46:0/444303686 wait complete. 2024-09-17T03:24:04.882 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-17T03:24:03.248974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279468,"kb_used_data":4968,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754695252,"statfs":{"total":773094113280,"available":772807938048,"internally_reserved":0,"allocated":5087232,"data_stored":2710034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.600875"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.686134+0000","last_change":"2024-09-17T03:24:00.679712+0000","last_active":"2024-09-17T03:24:00.686134+0000","last_peered":"2024-09-17T03:24:00.686134+0000","last_clean":"2024-09-17T03:24:00.686134+0000","last_became_active":"2024-09-17T03:24:00.679384+0000","last_became_peered":"2024-09-17T03:24:00.679384+0000","last_unstale":"2024-09-17T03:24:00.686134+0000","last_undegraded":"2024-09-17T03:24:00.686134+0000","last_fullsized":"2024-09-17T03:24:00.686134+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:09.852780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.715551+0000","last_change":"2024-09-17T03:24:00.711936+0000","last_active":"2024-09-17T03:24:00.715551+0000","last_peered":"2024-09-17T03:24:00.715551+0000","last_clean":"2024-09-17T03:24:00.715551+0000","last_became_active":"2024-09-17T03:24:00.711688+0000","last_became_peered":"2024-09-17T03:24:00.711688+0000","last_unstale":"2024-09-17T03:24:00.715551+0000","last_undegraded":"2024-09-17T03:24:00.715551+0000","last_fullsized":"2024-09-17T03:24:00.715551+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:32.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682528+0000","last_change":"2024-09-17T03:24:00.695337+0000","last_active":"2024-09-17T03:24:01.682528+0000","last_peered":"2024-09-17T03:24:01.682528+0000","last_clean":"2024-09-17T03:24:01.682528+0000","last_became_active":"2024-09-17T03:24:00.694995+0000","last_became_peered":"2024-09-17T03:24:00.694995+0000","last_unstale":"2024-09-17T03:24:01.682528+0000","last_undegraded":"2024-09-17T03:24:01.682528+0000","last_fullsized":"2024-09-17T03:24:01.682528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.187680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.707651+0000","last_change":"2024-09-17T03:24:00.703706+0000","last_active":"2024-09-17T03:24:00.707651+0000","last_peered":"2024-09-17T03:24:00.707651+0000","last_clean":"2024-09-17T03:24:00.707651+0000","last_became_active":"2024-09-17T03:24:00.701364+0000","last_became_peered":"2024-09-17T03:24:00.701364+0000","last_unstale":"2024-09-17T03:24:00.707651+0000","last_undegraded":"2024-09-17T03:24:00.707651+0000","last_fullsized":"2024-09-17T03:24:00.707651+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:40.120192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.712745+0000","last_change":"2024-09-17T03:24:00.708883+0000","last_active":"2024-09-17T03:24:00.712745+0000","last_peered":"2024-09-17T03:24:00.712745+0000","last_clean":"2024-09-17T03:24:00.712745+0000","last_became_active":"2024-09-17T03:24:00.708572+0000","last_became_peered":"2024-09-17T03:24:00.708572+0000","last_unstale":"2024-09-17T03:24:00.712745+0000","last_undegraded":"2024-09-17T03:24:00.712745+0000","last_fullsized":"2024-09-17T03:24:00.712745+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:26.128412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678990+0000","last_change":"2024-09-17T03:24:00.674630+0000","last_active":"2024-09-17T03:24:01.678990+0000","last_peered":"2024-09-17T03:24:01.678990+0000","last_clean":"2024-09-17T03:24:01.678990+0000","last_became_active":"2024-09-17T03:24:00.673905+0000","last_became_peered":"2024-09-17T03:24:00.673905+0000","last_unstale":"2024-09-17T03:24:01.678990+0000","last_undegraded":"2024-09-17T03:24:01.678990+0000","last_fullsized":"2024-09-17T03:24:01.678990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:31.006160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681723+0000","last_change":"2024-09-17T03:24:00.675168+0000","last_active":"2024-09-17T03:24:00.681723+0000","last_peered":"2024-09-17T03:24:00.681723+0000","last_clean":"2024-09-17T03:24:00.681723+0000","last_became_active":"2024-09-17T03:24:00.674820+0000","last_became_peered":"2024-09-17T03:24:00.674820+0000","last_unstale":"2024-09-17T03:24:00.681723+0000","last_undegraded":"2024-09-17T03:24:00.681723+0000","last_fullsized":"2024-09-17T03:24:00.681723+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:11.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.715720+0000","last_change":"2024-09-17T03:24:00.714581+0000","last_active":"2024-09-17T03:24:00.715720+0000","last_peered":"2024-09-17T03:24:00.715720+0000","last_clean":"2024-09-17T03:24:00.715720+0000","last_became_active":"2024-09-17T03:24:00.712060+0000","last_became_peered":"2024-09-17T03:24:00.712060+0000","last_unstale":"2024-09-17T03:24:00.715720+0000","last_undegraded":"2024-09-17T03:24:00.715720+0000","last_fullsized":"2024-09-17T03:24:00.715720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.566668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.711156+0000","last_change":"2024-09-17T03:24:00.708244+0000","last_active":"2024-09-17T03:24:00.711156+0000","last_peered":"2024-09-17T03:24:00.711156+0000","last_clean":"2024-09-17T03:24:00.711156+0000","last_became_active":"2024-09-17T03:24:00.707863+0000","last_became_peered":"2024-09-17T03:24:00.707863+0000","last_unstale":"2024-09-17T03:24:00.711156+0000","last_undegraded":"2024-09-17T03:24:00.711156+0000","last_fullsized":"2024-09-17T03:24:00.711156+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:58.331752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.713554+0000","last_change":"2024-09-17T03:24:00.710911+0000","last_active":"2024-09-17T03:24:00.713554+0000","last_peered":"2024-09-17T03:24:00.713554+0000","last_clean":"2024-09-17T03:24:00.713554+0000","last_became_active":"2024-09-17T03:24:00.710612+0000","last_became_peered":"2024-09-17T03:24:00.710612+0000","last_unstale":"2024-09-17T03:24:00.713554+0000","last_undegraded":"2024-09-17T03:24:00.713554+0000","last_fullsized":"2024-09-17T03:24:00.713554+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:04.935975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.711142+0000","last_change":"2024-09-17T03:24:00.708153+0000","last_active":"2024-09-17T03:24:00.711142+0000","last_peered":"2024-09-17T03:24:00.711142+0000","last_clean":"2024-09-17T03:24:00.711142+0000","last_became_active":"2024-09-17T03:24:00.707818+0000","last_became_peered":"2024-09-17T03:24:00.707818+0000","last_unstale":"2024-09-17T03:24:00.711142+0000","last_undegraded":"2024-09-17T03:24:00.711142+0000","last_fullsized":"2024-09-17T03:24:00.711142+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:24.740141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703380+0000","last_change":"2024-09-17T03:24:00.700049+0000","last_active":"2024-09-17T03:24:00.703380+0000","last_peered":"2024-09-17T03:24:00.703380+0000","last_clean":"2024-09-17T03:24:00.703380+0000","last_became_active":"2024-09-17T03:24:00.699761+0000","last_became_peered":"2024-09-17T03:24:00.699761+0000","last_unstale":"2024-09-17T03:24:00.703380+0000","last_undegraded":"2024-09-17T03:24:00.703380+0000","last_fullsized":"2024-09-17T03:24:00.703380+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:39.865635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675751+0000","last_change":"2024-09-17T03:24:00.696346+0000","last_active":"2024-09-17T03:24:01.675751+0000","last_peered":"2024-09-17T03:24:01.675751+0000","last_clean":"2024-09-17T03:24:01.675751+0000","last_became_active":"2024-09-17T03:24:00.696043+0000","last_became_peered":"2024-09-17T03:24:00.696043+0000","last_unstale":"2024-09-17T03:24:01.675751+0000","last_undegraded":"2024-09-17T03:24:01.675751+0000","last_fullsized":"2024-09-17T03:24:01.675751+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:37.766883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703364+0000","last_change":"2024-09-17T03:24:00.698152+0000","last_active":"2024-09-17T03:24:00.703364+0000","last_peered":"2024-09-17T03:24:00.703364+0000","last_clean":"2024-09-17T03:24:00.703364+0000","last_became_active":"2024-09-17T03:24:00.694472+0000","last_became_peered":"2024-09-17T03:24:00.694472+0000","last_unstale":"2024-09-17T03:24:00.703364+0000","last_undegraded":"2024-09-17T03:24:00.703364+0000","last_fullsized":"2024-09-17T03:24:00.703364+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:00.826322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678808+0000","last_change":"2024-09-17T03:24:00.676915+0000","last_active":"2024-09-17T03:24:01.678808+0000","last_peered":"2024-09-17T03:24:01.678808+0000","last_clean":"2024-09-17T03:24:01.678808+0000","last_became_active":"2024-09-17T03:24:00.676270+0000","last_became_peered":"2024-09-17T03:24:00.676270+0000","last_unstale":"2024-09-17T03:24:01.678808+0000","last_undegraded":"2024-09-17T03:24:01.678808+0000","last_fullsized":"2024-09-17T03:24:01.678808+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:40.255358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.722604+0000","last_change":"2024-09-17T03:24:00.718486+0000","last_active":"2024-09-17T03:24:00.722604+0000","last_peered":"2024-09-17T03:24:00.722604+0000","last_clean":"2024-09-17T03:24:00.722604+0000","last_became_active":"2024-09-17T03:24:00.718080+0000","last_became_peered":"2024-09-17T03:24:00.718080+0000","last_unstale":"2024-09-17T03:24:00.722604+0000","last_undegraded":"2024-09-17T03:24:00.722604+0000","last_fullsized":"2024-09-17T03:24:00.722604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:44.277028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682771+0000","last_change":"2024-09-17T03:24:00.677675+0000","last_active":"2024-09-17T03:24:01.682771+0000","last_peered":"2024-09-17T03:24:01.682771+0000","last_clean":"2024-09-17T03:24:01.682771+0000","last_became_active":"2024-09-17T03:24:00.677384+0000","last_became_peered":"2024-09-17T03:24:00.677384+0000","last_unstale":"2024-09-17T03:24:01.682771+0000","last_undegraded":"2024-09-17T03:24:01.682771+0000","last_fullsized":"2024-09-17T03:24:01.682771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:41.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682586+0000","last_change":"2024-09-17T03:24:00.705343+0000","last_active":"2024-09-17T03:24:01.682586+0000","last_peered":"2024-09-17T03:24:01.682586+0000","last_clean":"2024-09-17T03:24:01.682586+0000","last_became_active":"2024-09-17T03:24:00.705008+0000","last_became_peered":"2024-09-17T03:24:00.705008+0000","last_unstale":"2024-09-17T03:24:01.682586+0000","last_undegraded":"2024-09-17T03:24:01.682586+0000","last_fullsized":"2024-09-17T03:24:01.682586+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:27.373160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.710264+0000","last_change":"2024-09-17T03:24:00.705686+0000","last_active":"2024-09-17T03:24:00.710264+0000","last_peered":"2024-09-17T03:24:00.710264+0000","last_clean":"2024-09-17T03:24:00.710264+0000","last_became_active":"2024-09-17T03:24:00.704768+0000","last_became_peered":"2024-09-17T03:24:00.704768+0000","last_unstale":"2024-09-17T03:24:00.710264+0000","last_undegraded":"2024-09-17T03:24:00.710264+0000","last_fullsized":"2024-09-17T03:24:00.710264+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:00.659956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676111+0000","last_change":"2024-09-17T03:24:00.706410+0000","last_active":"2024-09-17T03:24:01.676111+0000","last_peered":"2024-09-17T03:24:01.676111+0000","last_clean":"2024-09-17T03:24:01.676111+0000","last_became_active":"2024-09-17T03:24:00.705984+0000","last_became_peered":"2024-09-17T03:24:00.705984+0000","last_unstale":"2024-09-17T03:24:01.676111+0000","last_undegraded":"2024-09-17T03:24:01.676111+0000","last_fullsized":"2024-09-17T03:24:01.676111+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:07.078551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678908+0000","last_change":"2024-09-17T03:24:00.676215+0000","last_active":"2024-09-17T03:24:01.678908+0000","last_peered":"2024-09-17T03:24:01.678908+0000","last_clean":"2024-09-17T03:24:01.678908+0000","last_became_active":"2024-09-17T03:24:00.675886+0000","last_became_peered":"2024-09-17T03:24:00.675886+0000","last_unstale":"2024-09-17T03:24:01.678908+0000","last_undegraded":"2024-09-17T03:24:01.678908+0000","last_fullsized":"2024-09-17T03:24:01.678908+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:19.556922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703396+0000","last_change":"2024-09-17T03:24:00.698862+0000","last_active":"2024-09-17T03:24:00.703396+0000","last_peered":"2024-09-17T03:24:00.703396+0000","last_clean":"2024-09-17T03:24:00.703396+0000","last_became_active":"2024-09-17T03:24:00.694829+0000","last_became_peered":"2024-09-17T03:24:00.694829+0000","last_unstale":"2024-09-17T03:24:00.703396+0000","last_undegraded":"2024-09-17T03:24:00.703396+0000","last_fullsized":"2024-09-17T03:24:00.703396+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:32.893125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689609+0000","last_change":"2024-09-17T03:24:00.685397+0000","last_active":"2024-09-17T03:24:00.689609+0000","last_peered":"2024-09-17T03:24:00.689609+0000","last_clean":"2024-09-17T03:24:00.689609+0000","last_became_active":"2024-09-17T03:24:00.685063+0000","last_became_peered":"2024-09-17T03:24:00.685063+0000","last_unstale":"2024-09-17T03:24:00.689609+0000","last_undegraded":"2024-09-17T03:24:00.689609+0000","last_fullsized":"2024-09-17T03:24:00.689609+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:07.078551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689258+0000","last_change":"2024-09-17T03:24:00.684388+0000","last_active":"2024-09-17T03:24:00.689258+0000","last_peered":"2024-09-17T03:24:00.689258+0000","last_clean":"2024-09-17T03:24:00.689258+0000","last_became_active":"2024-09-17T03:24:00.684093+0000","last_became_peered":"2024-09-17T03:24:00.684093+0000","last_unstale":"2024-09-17T03:24:00.689258+0000","last_undegraded":"2024-09-17T03:24:00.689258+0000","last_fullsized":"2024-09-17T03:24:00.689258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:37.766883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694437+0000","last_change":"2024-09-17T03:24:00.686880+0000","last_active":"2024-09-17T03:24:00.694437+0000","last_peered":"2024-09-17T03:24:00.694437+0000","last_clean":"2024-09-17T03:24:00.694437+0000","last_became_active":"2024-09-17T03:24:00.686601+0000","last_became_peered":"2024-09-17T03:24:00.686601+0000","last_unstale":"2024-09-17T03:24:00.694437+0000","last_undegraded":"2024-09-17T03:24:00.694437+0000","last_fullsized":"2024-09-17T03:24:00.694437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:23.386007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676016+0000","last_change":"2024-09-17T03:24:00.688599+0000","last_active":"2024-09-17T03:24:01.676016+0000","last_peered":"2024-09-17T03:24:01.676016+0000","last_clean":"2024-09-17T03:24:01.676016+0000","last_became_active":"2024-09-17T03:24:00.688229+0000","last_became_peered":"2024-09-17T03:24:00.688229+0000","last_unstale":"2024-09-17T03:24:01.676016+0000","last_undegraded":"2024-09-17T03:24:01.676016+0000","last_fullsized":"2024-09-17T03:24:01.676016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:06.920757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682684+0000","last_change":"2024-09-17T03:24:00.679002+0000","last_active":"2024-09-17T03:24:01.682684+0000","last_peered":"2024-09-17T03:24:01.682684+0000","last_clean":"2024-09-17T03:24:01.682684+0000","last_became_active":"2024-09-17T03:24:00.677828+0000","last_became_peered":"2024-09-17T03:24:00.677828+0000","last_unstale":"2024-09-17T03:24:01.682684+0000","last_undegraded":"2024-09-17T03:24:01.682684+0000","last_fullsized":"2024-09-17T03:24:01.682684+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:35.766678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703406+0000","last_change":"2024-09-17T03:24:00.700037+0000","last_active":"2024-09-17T03:24:00.703406+0000","last_peered":"2024-09-17T03:24:00.703406+0000","last_clean":"2024-09-17T03:24:00.703406+0000","last_became_active":"2024-09-17T03:24:00.699470+0000","last_became_peered":"2024-09-17T03:24:00.699470+0000","last_unstale":"2024-09-17T03:24:00.703406+0000","last_undegraded":"2024-09-17T03:24:00.703406+0000","last_fullsized":"2024-09-17T03:24:00.703406+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:30.514015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689389+0000","last_change":"2024-09-17T03:24:00.685325+0000","last_active":"2024-09-17T03:24:00.689389+0000","last_peered":"2024-09-17T03:24:00.689389+0000","last_clean":"2024-09-17T03:24:00.689389+0000","last_became_active":"2024-09-17T03:24:00.685036+0000","last_became_peered":"2024-09-17T03:24:00.685036+0000","last_unstale":"2024-09-17T03:24:00.689389+0000","last_undegraded":"2024-09-17T03:24:00.689389+0000","last_fullsized":"2024-09-17T03:24:00.689389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:06.920757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682379+0000","last_change":"2024-09-17T03:24:00.683130+0000","last_active":"2024-09-17T03:24:01.682379+0000","last_peered":"2024-09-17T03:24:01.682379+0000","last_clean":"2024-09-17T03:24:01.682379+0000","last_became_active":"2024-09-17T03:24:00.682800+0000","last_became_peered":"2024-09-17T03:24:00.682800+0000","last_unstale":"2024-09-17T03:24:01.682379+0000","last_undegraded":"2024-09-17T03:24:01.682379+0000","last_fullsized":"2024-09-17T03:24:01.682379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:09.498797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678877+0000","last_change":"2024-09-17T03:24:00.671210+0000","last_active":"2024-09-17T03:24:01.678877+0000","last_peered":"2024-09-17T03:24:01.678877+0000","last_clean":"2024-09-17T03:24:01.678877+0000","last_became_active":"2024-09-17T03:24:00.670854+0000","last_became_peered":"2024-09-17T03:24:00.670854+0000","last_unstale":"2024-09-17T03:24:01.678877+0000","last_undegraded":"2024-09-17T03:24:01.678877+0000","last_fullsized":"2024-09-17T03:24:01.678877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675557+0000","last_change":"2024-09-17T03:24:00.683911+0000","last_active":"2024-09-17T03:24:01.675557+0000","last_peered":"2024-09-17T03:24:01.675557+0000","last_clean":"2024-09-17T03:24:01.675557+0000","last_became_active":"2024-09-17T03:24:00.683579+0000","last_became_peered":"2024-09-17T03:24:00.683579+0000","last_unstale":"2024-09-17T03:24:01.675557+0000","last_undegraded":"2024-09-17T03:24:01.675557+0000","last_fullsized":"2024-09-17T03:24:01.675557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:18.956672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680290+0000","last_change":"2024-09-17T03:24:00.674960+0000","last_active":"2024-09-17T03:24:00.680290+0000","last_peered":"2024-09-17T03:24:00.680290+0000","last_clean":"2024-09-17T03:24:00.680290+0000","last_became_active":"2024-09-17T03:24:00.674637+0000","last_became_peered":"2024-09-17T03:24:00.674637+0000","last_unstale":"2024-09-17T03:24:00.680290+0000","last_undegraded":"2024-09-17T03:24:00.680290+0000","last_fullsized":"2024-09-17T03:24:00.680290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:19.199176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679409+0000","last_change":"2024-09-17T03:23:59.683374+0000","last_active":"2024-09-17T03:24:01.679409+0000","last_peered":"2024-09-17T03:24:01.679409+0000","last_clean":"2024-09-17T03:24:01.679409+0000","last_became_active":"2024-09-17T03:23:59.682927+0000","last_became_peered":"2024-09-17T03:23:59.682927+0000","last_unstale":"2024-09-17T03:24:01.679409+0000","last_undegraded":"2024-09-17T03:24:01.679409+0000","last_fullsized":"2024-09-17T03:24:01.679409+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:10.720193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681126+0000","last_change":"2024-09-17T03:24:00.679188+0000","last_active":"2024-09-17T03:24:00.681126+0000","last_peered":"2024-09-17T03:24:00.681126+0000","last_clean":"2024-09-17T03:24:00.681126+0000","last_became_active":"2024-09-17T03:24:00.678835+0000","last_became_peered":"2024-09-17T03:24:00.678835+0000","last_unstale":"2024-09-17T03:24:00.681126+0000","last_undegraded":"2024-09-17T03:24:00.681126+0000","last_fullsized":"2024-09-17T03:24:00.681126+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:18.956672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682960+0000","last_change":"2024-09-17T03:23:59.686402+0000","last_active":"2024-09-17T03:24:01.682960+0000","last_peered":"2024-09-17T03:24:01.682960+0000","last_clean":"2024-09-17T03:24:01.682960+0000","last_became_active":"2024-09-17T03:23:59.686069+0000","last_became_peered":"2024-09-17T03:23:59.686069+0000","last_unstale":"2024-09-17T03:24:01.682960+0000","last_undegraded":"2024-09-17T03:24:01.682960+0000","last_fullsized":"2024-09-17T03:24:01.682960+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:49.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680715+0000","last_change":"2024-09-17T03:24:00.677750+0000","last_active":"2024-09-17T03:24:00.680715+0000","last_peered":"2024-09-17T03:24:00.680715+0000","last_clean":"2024-09-17T03:24:00.680715+0000","last_became_active":"2024-09-17T03:24:00.677432+0000","last_became_peered":"2024-09-17T03:24:00.677432+0000","last_unstale":"2024-09-17T03:24:00.680715+0000","last_undegraded":"2024-09-17T03:24:00.680715+0000","last_fullsized":"2024-09-17T03:24:00.680715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:08.338281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.687369+0000","last_change":"2024-09-17T03:23:59.674742+0000","last_active":"2024-09-17T03:24:00.687369+0000","last_peered":"2024-09-17T03:24:00.687369+0000","last_clean":"2024-09-17T03:24:00.687369+0000","last_became_active":"2024-09-17T03:23:59.674135+0000","last_became_peered":"2024-09-17T03:23:59.674135+0000","last_unstale":"2024-09-17T03:24:00.687369+0000","last_undegraded":"2024-09-17T03:24:00.687369+0000","last_fullsized":"2024-09-17T03:24:00.687369+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:41.361148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681542+0000","last_change":"2024-09-17T03:24:00.678804+0000","last_active":"2024-09-17T03:24:00.681542+0000","last_peered":"2024-09-17T03:24:00.681542+0000","last_clean":"2024-09-17T03:24:00.681542+0000","last_became_active":"2024-09-17T03:24:00.678369+0000","last_became_peered":"2024-09-17T03:24:00.678369+0000","last_unstale":"2024-09-17T03:24:00.681542+0000","last_undegraded":"2024-09-17T03:24:00.681542+0000","last_fullsized":"2024-09-17T03:24:00.681542+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:09.232872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.695811+0000","last_change":"2024-09-17T03:23:59.684650+0000","last_active":"2024-09-17T03:24:00.695811+0000","last_peered":"2024-09-17T03:24:00.695811+0000","last_clean":"2024-09-17T03:24:00.695811+0000","last_became_active":"2024-09-17T03:23:59.684235+0000","last_became_peered":"2024-09-17T03:23:59.684235+0000","last_unstale":"2024-09-17T03:24:00.695811+0000","last_undegraded":"2024-09-17T03:24:00.695811+0000","last_fullsized":"2024-09-17T03:24:00.695811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:48.454565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682397+0000","last_change":"2024-09-17T03:24:00.680552+0000","last_active":"2024-09-17T03:24:01.682397+0000","last_peered":"2024-09-17T03:24:01.682397+0000","last_clean":"2024-09-17T03:24:01.682397+0000","last_became_active":"2024-09-17T03:24:00.680263+0000","last_became_peered":"2024-09-17T03:24:00.680263+0000","last_unstale":"2024-09-17T03:24:01.682397+0000","last_undegraded":"2024-09-17T03:24:01.682397+0000","last_fullsized":"2024-09-17T03:24:01.682397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.566668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679371+0000","last_change":"2024-09-17T03:23:59.676497+0000","last_active":"2024-09-17T03:24:01.679371+0000","last_peered":"2024-09-17T03:24:01.679371+0000","last_clean":"2024-09-17T03:24:01.679371+0000","last_became_active":"2024-09-17T03:23:59.676157+0000","last_became_peered":"2024-09-17T03:23:59.676157+0000","last_unstale":"2024-09-17T03:24:01.679371+0000","last_undegraded":"2024-09-17T03:24:01.679371+0000","last_fullsized":"2024-09-17T03:24:01.679371+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:08.849517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.714267+0000","last_change":"2024-09-17T03:24:00.710747+0000","last_active":"2024-09-17T03:24:00.714267+0000","last_peered":"2024-09-17T03:24:00.714267+0000","last_clean":"2024-09-17T03:24:00.714267+0000","last_became_active":"2024-09-17T03:24:00.708450+0000","last_became_peered":"2024-09-17T03:24:00.708450+0000","last_unstale":"2024-09-17T03:24:00.714267+0000","last_undegraded":"2024-09-17T03:24:00.714267+0000","last_fullsized":"2024-09-17T03:24:00.714267+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:26:13.822513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682524+0000","last_change":"2024-09-17T03:23:59.675958+0000","last_active":"2024-09-17T03:24:01.682524+0000","last_peered":"2024-09-17T03:24:01.682524+0000","last_clean":"2024-09-17T03:24:01.682524+0000","last_became_active":"2024-09-17T03:23:59.675261+0000","last_became_peered":"2024-09-17T03:23:59.675261+0000","last_unstale":"2024-09-17T03:24:01.682524+0000","last_undegraded":"2024-09-17T03:24:01.682524+0000","last_fullsized":"2024-09-17T03:24:01.682524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:11.374505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.714843+0000","last_change":"2024-09-17T03:24:00.712231+0000","last_active":"2024-09-17T03:24:00.714843+0000","last_peered":"2024-09-17T03:24:00.714843+0000","last_clean":"2024-09-17T03:24:00.714843+0000","last_became_active":"2024-09-17T03:24:00.711807+0000","last_became_peered":"2024-09-17T03:24:00.711807+0000","last_unstale":"2024-09-17T03:24:00.714843+0000","last_undegraded":"2024-09-17T03:24:00.714843+0000","last_fullsized":"2024-09-17T03:24:00.714843+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:09.498797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.670083+0000","last_change":"2024-09-17T03:23:59.673810+0000","last_active":"2024-09-17T03:24:00.670083+0000","last_peered":"2024-09-17T03:24:00.670083+0000","last_clean":"2024-09-17T03:24:00.670083+0000","last_became_active":"2024-09-17T03:23:59.673100+0000","last_became_peered":"2024-09-17T03:23:59.673100+0000","last_unstale":"2024-09-17T03:24:00.670083+0000","last_undegraded":"2024-09-17T03:24:00.670083+0000","last_fullsized":"2024-09-17T03:24:00.670083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:03.157480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694479+0000","last_change":"2024-09-17T03:24:00.687279+0000","last_active":"2024-09-17T03:24:00.694479+0000","last_peered":"2024-09-17T03:24:00.694479+0000","last_clean":"2024-09-17T03:24:00.694479+0000","last_became_active":"2024-09-17T03:24:00.685315+0000","last_became_peered":"2024-09-17T03:24:00.685315+0000","last_unstale":"2024-09-17T03:24:00.694479+0000","last_undegraded":"2024-09-17T03:24:00.694479+0000","last_fullsized":"2024-09-17T03:24:00.694479+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:37:43.575452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.700035+0000","last_change":"2024-09-17T03:23:59.687636+0000","last_active":"2024-09-17T03:24:00.700035+0000","last_peered":"2024-09-17T03:24:00.700035+0000","last_clean":"2024-09-17T03:24:00.700035+0000","last_became_active":"2024-09-17T03:23:59.687322+0000","last_became_peered":"2024-09-17T03:23:59.687322+0000","last_unstale":"2024-09-17T03:24:00.700035+0000","last_undegraded":"2024-09-17T03:24:00.700035+0000","last_fullsized":"2024-09-17T03:24:00.700035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:21.035374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682826+0000","last_change":"2024-09-17T03:24:00.705466+0000","last_active":"2024-09-17T03:24:01.682826+0000","last_peered":"2024-09-17T03:24:01.682826+0000","last_clean":"2024-09-17T03:24:01.682826+0000","last_became_active":"2024-09-17T03:24:00.705180+0000","last_became_peered":"2024-09-17T03:24:00.705180+0000","last_unstale":"2024-09-17T03:24:01.682826+0000","last_undegraded":"2024-09-17T03:24:01.682826+0000","last_fullsized":"2024-09-17T03:24:01.682826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:40.123607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682970+0000","last_change":"2024-09-17T03:23:59.673722+0000","last_active":"2024-09-17T03:24:01.682970+0000","last_peered":"2024-09-17T03:24:01.682970+0000","last_clean":"2024-09-17T03:24:01.682970+0000","last_became_active":"2024-09-17T03:23:59.673369+0000","last_became_peered":"2024-09-17T03:23:59.673369+0000","last_unstale":"2024-09-17T03:24:01.682970+0000","last_undegraded":"2024-09-17T03:24:01.682970+0000","last_fullsized":"2024-09-17T03:24:01.682970+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:08.453921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680520+0000","last_change":"2024-09-17T03:24:00.675419+0000","last_active":"2024-09-17T03:24:00.680520+0000","last_peered":"2024-09-17T03:24:00.680520+0000","last_clean":"2024-09-17T03:24:00.680520+0000","last_became_active":"2024-09-17T03:24:00.675106+0000","last_became_peered":"2024-09-17T03:24:00.675106+0000","last_unstale":"2024-09-17T03:24:00.680520+0000","last_undegraded":"2024-09-17T03:24:00.680520+0000","last_fullsized":"2024-09-17T03:24:00.680520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:20.595904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.707144+0000","last_change":"2024-09-17T03:23:59.686562+0000","last_active":"2024-09-17T03:24:00.707144+0000","last_peered":"2024-09-17T03:24:00.707144+0000","last_clean":"2024-09-17T03:24:00.707144+0000","last_became_active":"2024-09-17T03:23:59.686246+0000","last_became_peered":"2024-09-17T03:23:59.686246+0000","last_unstale":"2024-09-17T03:24:00.707144+0000","last_undegraded":"2024-09-17T03:24:00.707144+0000","last_fullsized":"2024-09-17T03:24:00.707144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:01.226260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679258+0000","last_change":"2024-09-17T03:24:00.696303+0000","last_active":"2024-09-17T03:24:01.679258+0000","last_peered":"2024-09-17T03:24:01.679258+0000","last_clean":"2024-09-17T03:24:01.679258+0000","last_became_active":"2024-09-17T03:24:00.695937+0000","last_became_peered":"2024-09-17T03:24:00.695937+0000","last_unstale":"2024-09-17T03:24:01.679258+0000","last_undegraded":"2024-09-17T03:24:01.679258+0000","last_fullsized":"2024-09-17T03:24:01.679258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:51.673689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.696367+0000","last_change":"2024-09-17T03:23:59.688206+0000","last_active":"2024-09-17T03:24:00.696367+0000","last_peered":"2024-09-17T03:24:00.696367+0000","last_clean":"2024-09-17T03:24:00.696367+0000","last_became_active":"2024-09-17T03:23:59.687858+0000","last_became_peered":"2024-09-17T03:23:59.687858+0000","last_unstale":"2024-09-17T03:24:00.696367+0000","last_undegraded":"2024-09-17T03:24:00.696367+0000","last_fullsized":"2024-09-17T03:24:00.696367+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:50.097008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680478+0000","last_change":"2024-09-17T03:24:00.674939+0000","last_active":"2024-09-17T03:24:00.680478+0000","last_peered":"2024-09-17T03:24:00.680478+0000","last_clean":"2024-09-17T03:24:00.680478+0000","last_became_active":"2024-09-17T03:24:00.673963+0000","last_became_peered":"2024-09-17T03:24:00.673963+0000","last_unstale":"2024-09-17T03:24:00.680478+0000","last_undegraded":"2024-09-17T03:24:00.680478+0000","last_fullsized":"2024-09-17T03:24:00.680478+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:01.987364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676156+0000","last_change":"2024-09-17T03:23:59.676386+0000","last_active":"2024-09-17T03:24:01.676156+0000","last_peered":"2024-09-17T03:24:01.676156+0000","last_clean":"2024-09-17T03:24:01.676156+0000","last_became_active":"2024-09-17T03:23:59.675884+0000","last_became_peered":"2024-09-17T03:23:59.675884+0000","last_unstale":"2024-09-17T03:24:01.676156+0000","last_undegraded":"2024-09-17T03:24:01.676156+0000","last_fullsized":"2024-09-17T03:24:01.676156+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:27.081555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.712641+0000","last_change":"2024-09-17T03:24:00.708903+0000","last_active":"2024-09-17T03:24:00.712641+0000","last_peered":"2024-09-17T03:24:00.712641+0000","last_clean":"2024-09-17T03:24:00.712641+0000","last_became_active":"2024-09-17T03:24:00.708564+0000","last_became_peered":"2024-09-17T03:24:00.708564+0000","last_unstale":"2024-09-17T03:24:00.712641+0000","last_undegraded":"2024-09-17T03:24:00.712641+0000","last_fullsized":"2024-09-17T03:24:00.712641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:09.337210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679495+0000","last_change":"2024-09-17T03:23:59.686453+0000","last_active":"2024-09-17T03:24:01.679495+0000","last_peered":"2024-09-17T03:24:01.679495+0000","last_clean":"2024-09-17T03:24:01.679495+0000","last_became_active":"2024-09-17T03:23:59.685855+0000","last_became_peered":"2024-09-17T03:23:59.685855+0000","last_unstale":"2024-09-17T03:24:01.679495+0000","last_undegraded":"2024-09-17T03:24:01.679495+0000","last_fullsized":"2024-09-17T03:24:01.679495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:22.785717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.714297+0000","last_change":"2024-09-17T03:24:00.711203+0000","last_active":"2024-09-17T03:24:00.714297+0000","last_peered":"2024-09-17T03:24:00.714297+0000","last_clean":"2024-09-17T03:24:00.714297+0000","last_became_active":"2024-09-17T03:24:00.707999+0000","last_became_peered":"2024-09-17T03:24:00.707999+0000","last_unstale":"2024-09-17T03:24:00.714297+0000","last_undegraded":"2024-09-17T03:24:00.714297+0000","last_fullsized":"2024-09-17T03:24:00.714297+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:48.279400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.664677+0000","last_change":"2024-09-17T03:23:59.681355+0000","last_active":"2024-09-17T03:24:00.664677+0000","last_peered":"2024-09-17T03:24:00.664677+0000","last_clean":"2024-09-17T03:24:00.664677+0000","last_became_active":"2024-09-17T03:23:59.681014+0000","last_became_peered":"2024-09-17T03:23:59.681014+0000","last_unstale":"2024-09-17T03:24:00.664677+0000","last_undegraded":"2024-09-17T03:24:00.664677+0000","last_fullsized":"2024-09-17T03:24:00.664677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:41.740765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678814+0000","last_change":"2024-09-17T03:24:00.686271+0000","last_active":"2024-09-17T03:24:01.678814+0000","last_peered":"2024-09-17T03:24:01.678814+0000","last_clean":"2024-09-17T03:24:01.678814+0000","last_became_active":"2024-09-17T03:24:00.685903+0000","last_became_peered":"2024-09-17T03:24:00.685903+0000","last_unstale":"2024-09-17T03:24:01.678814+0000","last_undegraded":"2024-09-17T03:24:01.678814+0000","last_fullsized":"2024-09-17T03:24:01.678814+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:08.432503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678906+0000","last_change":"2024-09-17T03:23:59.686981+0000","last_active":"2024-09-17T03:24:01.678906+0000","last_peered":"2024-09-17T03:24:01.678906+0000","last_clean":"2024-09-17T03:24:01.678906+0000","last_became_active":"2024-09-17T03:23:59.686678+0000","last_became_peered":"2024-09-17T03:23:59.686678+0000","last_unstale":"2024-09-17T03:24:01.678906+0000","last_undegraded":"2024-09-17T03:24:01.678906+0000","last_fullsized":"2024-09-17T03:24:01.678906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.956844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.692081+0000","last_change":"2024-09-17T03:24:00.681726+0000","last_active":"2024-09-17T03:24:00.692081+0000","last_peered":"2024-09-17T03:24:00.692081+0000","last_clean":"2024-09-17T03:24:00.692081+0000","last_became_active":"2024-09-17T03:24:00.681446+0000","last_became_peered":"2024-09-17T03:24:00.681446+0000","last_unstale":"2024-09-17T03:24:00.692081+0000","last_undegraded":"2024-09-17T03:24:00.692081+0000","last_fullsized":"2024-09-17T03:24:00.692081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:23.788980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.718055+0000","last_change":"2024-09-17T03:23:59.687662+0000","last_active":"2024-09-17T03:24:00.718055+0000","last_peered":"2024-09-17T03:24:00.718055+0000","last_clean":"2024-09-17T03:24:00.718055+0000","last_became_active":"2024-09-17T03:23:59.687309+0000","last_became_peered":"2024-09-17T03:23:59.687309+0000","last_unstale":"2024-09-17T03:24:00.718055+0000","last_undegraded":"2024-09-17T03:24:00.718055+0000","last_fullsized":"2024-09-17T03:24:00.718055+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:47.557243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676116+0000","last_change":"2024-09-17T03:24:00.688548+0000","last_active":"2024-09-17T03:24:01.676116+0000","last_peered":"2024-09-17T03:24:01.676116+0000","last_clean":"2024-09-17T03:24:01.676116+0000","last_became_active":"2024-09-17T03:24:00.688213+0000","last_became_peered":"2024-09-17T03:24:00.688213+0000","last_unstale":"2024-09-17T03:24:01.676116+0000","last_undegraded":"2024-09-17T03:24:01.676116+0000","last_fullsized":"2024-09-17T03:24:01.676116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:22.324581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675981+0000","last_change":"2024-09-17T03:23:59.676504+0000","last_active":"2024-09-17T03:24:01.675981+0000","last_peered":"2024-09-17T03:24:01.675981+0000","last_clean":"2024-09-17T03:24:01.675981+0000","last_became_active":"2024-09-17T03:23:59.675819+0000","last_became_peered":"2024-09-17T03:23:59.675819+0000","last_unstale":"2024-09-17T03:24:01.675981+0000","last_undegraded":"2024-09-17T03:24:01.675981+0000","last_fullsized":"2024-09-17T03:24:01.675981+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:03.658474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.711867+0000","last_change":"2024-09-17T03:24:00.710362+0000","last_active":"2024-09-17T03:24:00.711867+0000","last_peered":"2024-09-17T03:24:00.711867+0000","last_clean":"2024-09-17T03:24:00.711867+0000","last_became_active":"2024-09-17T03:24:00.710091+0000","last_became_peered":"2024-09-17T03:24:00.710091+0000","last_unstale":"2024-09-17T03:24:00.711867+0000","last_undegraded":"2024-09-17T03:24:00.711867+0000","last_fullsized":"2024-09-17T03:24:00.711867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:17.523408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.683081+0000","last_change":"2024-09-17T03:23:59.668703+0000","last_active":"2024-09-17T03:24:01.683081+0000","last_peered":"2024-09-17T03:24:01.683081+0000","last_clean":"2024-09-17T03:24:01.683081+0000","last_became_active":"2024-09-17T03:23:59.668365+0000","last_became_peered":"2024-09-17T03:23:59.668365+0000","last_unstale":"2024-09-17T03:24:01.683081+0000","last_undegraded":"2024-09-17T03:24:01.683081+0000","last_fullsized":"2024-09-17T03:24:01.683081+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:19.808133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689193+0000","last_change":"2024-09-17T03:24:00.681885+0000","last_active":"2024-09-17T03:24:00.689193+0000","last_peered":"2024-09-17T03:24:00.689193+0000","last_clean":"2024-09-17T03:24:00.689193+0000","last_became_active":"2024-09-17T03:24:00.681578+0000","last_became_peered":"2024-09-17T03:24:00.681578+0000","last_unstale":"2024-09-17T03:24:00.689193+0000","last_undegraded":"2024-09-17T03:24:00.689193+0000","last_fullsized":"2024-09-17T03:24:00.689193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:56.945734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694350+0000","last_change":"2024-09-17T03:23:59.679798+0000","last_active":"2024-09-17T03:24:00.694350+0000","last_peered":"2024-09-17T03:24:00.694350+0000","last_clean":"2024-09-17T03:24:00.694350+0000","last_became_active":"2024-09-17T03:23:59.679466+0000","last_became_peered":"2024-09-17T03:23:59.679466+0000","last_unstale":"2024-09-17T03:24:00.694350+0000","last_undegraded":"2024-09-17T03:24:00.694350+0000","last_fullsized":"2024-09-17T03:24:00.694350+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:17.297194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694572+0000","last_change":"2024-09-17T03:24:00.687851+0000","last_active":"2024-09-17T03:24:00.694572+0000","last_peered":"2024-09-17T03:24:00.694572+0000","last_clean":"2024-09-17T03:24:00.694572+0000","last_became_active":"2024-09-17T03:24:00.687400+0000","last_became_peered":"2024-09-17T03:24:00.687400+0000","last_unstale":"2024-09-17T03:24:00.694572+0000","last_undegraded":"2024-09-17T03:24:00.694572+0000","last_fullsized":"2024-09-17T03:24:00.694572+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:32.389400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679380+0000","last_change":"2024-09-17T03:23:59.679174+0000","last_active":"2024-09-17T03:24:01.679380+0000","last_peered":"2024-09-17T03:24:01.679380+0000","last_clean":"2024-09-17T03:24:01.679380+0000","last_became_active":"2024-09-17T03:23:59.678916+0000","last_became_peered":"2024-09-17T03:23:59.678916+0000","last_unstale":"2024-09-17T03:24:01.679380+0000","last_undegraded":"2024-09-17T03:24:01.679380+0000","last_fullsized":"2024-09-17T03:24:01.679380+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:19.592642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679087+0000","last_change":"2024-09-17T03:24:00.696312+0000","last_active":"2024-09-17T03:24:01.679087+0000","last_peered":"2024-09-17T03:24:01.679087+0000","last_clean":"2024-09-17T03:24:01.679087+0000","last_became_active":"2024-09-17T03:24:00.695942+0000","last_became_peered":"2024-09-17T03:24:00.695942+0000","last_unstale":"2024-09-17T03:24:01.679087+0000","last_undegraded":"2024-09-17T03:24:01.679087+0000","last_fullsized":"2024-09-17T03:24:01.679087+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:18.733200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679611+0000","last_change":"2024-09-17T03:23:59.675028+0000","last_active":"2024-09-17T03:24:01.679611+0000","last_peered":"2024-09-17T03:24:01.679611+0000","last_clean":"2024-09-17T03:24:01.679611+0000","last_became_active":"2024-09-17T03:23:59.674728+0000","last_became_peered":"2024-09-17T03:23:59.674728+0000","last_unstale":"2024-09-17T03:24:01.679611+0000","last_undegraded":"2024-09-17T03:24:01.679611+0000","last_fullsized":"2024-09-17T03:24:01.679611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:09.687494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681638+0000","last_change":"2024-09-17T03:24:00.676089+0000","last_active":"2024-09-17T03:24:00.681638+0000","last_peered":"2024-09-17T03:24:00.681638+0000","last_clean":"2024-09-17T03:24:00.681638+0000","last_became_active":"2024-09-17T03:24:00.675610+0000","last_became_peered":"2024-09-17T03:24:00.675610+0000","last_unstale":"2024-09-17T03:24:00.681638+0000","last_undegraded":"2024-09-17T03:24:00.681638+0000","last_fullsized":"2024-09-17T03:24:00.681638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.379254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.669156+0000","last_change":"2024-09-17T03:23:59.682548+0000","last_active":"2024-09-17T03:24:00.669156+0000","last_peered":"2024-09-17T03:24:00.669156+0000","last_clean":"2024-09-17T03:24:00.669156+0000","last_became_active":"2024-09-17T03:23:59.682201+0000","last_became_peered":"2024-09-17T03:23:59.682201+0000","last_unstale":"2024-09-17T03:24:00.669156+0000","last_undegraded":"2024-09-17T03:24:00.669156+0000","last_fullsized":"2024-09-17T03:24:00.669156+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:29.085472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675591+0000","last_change":"2024-09-17T03:24:00.687458+0000","last_active":"2024-09-17T03:24:01.675591+0000","last_peered":"2024-09-17T03:24:01.675591+0000","last_clean":"2024-09-17T03:24:01.675591+0000","last_became_active":"2024-09-17T03:24:00.687164+0000","last_became_peered":"2024-09-17T03:24:00.687164+0000","last_unstale":"2024-09-17T03:24:01.675591+0000","last_undegraded":"2024-09-17T03:24:01.675591+0000","last_fullsized":"2024-09-17T03:24:01.675591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:56.945734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.693276+0000","last_change":"2024-09-17T03:23:59.685224+0000","last_active":"2024-09-17T03:24:00.693276+0000","last_peered":"2024-09-17T03:24:00.693276+0000","last_clean":"2024-09-17T03:24:00.693276+0000","last_became_active":"2024-09-17T03:23:59.681143+0000","last_became_peered":"2024-09-17T03:23:59.681143+0000","last_unstale":"2024-09-17T03:24:00.693276+0000","last_undegraded":"2024-09-17T03:24:00.693276+0000","last_fullsized":"2024-09-17T03:24:00.693276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:47.586031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.699373+0000","last_change":"2024-09-17T03:24:00.692868+0000","last_active":"2024-09-17T03:24:00.699373+0000","last_peered":"2024-09-17T03:24:00.699373+0000","last_clean":"2024-09-17T03:24:00.699373+0000","last_became_active":"2024-09-17T03:24:00.692545+0000","last_became_peered":"2024-09-17T03:24:00.692545+0000","last_unstale":"2024-09-17T03:24:00.699373+0000","last_undegraded":"2024-09-17T03:24:00.699373+0000","last_fullsized":"2024-09-17T03:24:00.699373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:54.312118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.664621+0000","last_change":"2024-09-17T03:23:59.682305+0000","last_active":"2024-09-17T03:24:00.664621+0000","last_peered":"2024-09-17T03:24:00.664621+0000","last_clean":"2024-09-17T03:24:00.664621+0000","last_became_active":"2024-09-17T03:23:59.681437+0000","last_became_peered":"2024-09-17T03:23:59.681437+0000","last_unstale":"2024-09-17T03:24:00.664621+0000","last_undegraded":"2024-09-17T03:24:00.664621+0000","last_fullsized":"2024-09-17T03:24:00.664621+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:52.874111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.721592+0000","last_change":"2024-09-17T03:24:00.718196+0000","last_active":"2024-09-17T03:24:00.721592+0000","last_peered":"2024-09-17T03:24:00.721592+0000","last_clean":"2024-09-17T03:24:00.721592+0000","last_became_active":"2024-09-17T03:24:00.717873+0000","last_became_peered":"2024-09-17T03:24:00.717873+0000","last_unstale":"2024-09-17T03:24:00.721592+0000","last_undegraded":"2024-09-17T03:24:00.721592+0000","last_fullsized":"2024-09-17T03:24:00.721592+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:12.377768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679092+0000","last_change":"2024-09-17T03:23:59.672475+0000","last_active":"2024-09-17T03:24:01.679092+0000","last_peered":"2024-09-17T03:24:01.679092+0000","last_clean":"2024-09-17T03:24:01.679092+0000","last_became_active":"2024-09-17T03:23:59.669909+0000","last_became_peered":"2024-09-17T03:23:59.669909+0000","last_unstale":"2024-09-17T03:24:01.679092+0000","last_undegraded":"2024-09-17T03:24:01.679092+0000","last_fullsized":"2024-09-17T03:24:01.679092+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:00.984101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.707613+0000","last_change":"2024-09-17T03:24:00.703373+0000","last_active":"2024-09-17T03:24:00.707613+0000","last_peered":"2024-09-17T03:24:00.707613+0000","last_clean":"2024-09-17T03:24:00.707613+0000","last_became_active":"2024-09-17T03:24:00.700966+0000","last_became_peered":"2024-09-17T03:24:00.700966+0000","last_unstale":"2024-09-17T03:24:00.707613+0000","last_undegraded":"2024-09-17T03:24:00.707613+0000","last_fullsized":"2024-09-17T03:24:00.707613+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:25.868561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.665347+0000","last_change":"2024-09-17T03:23:59.688084+0000","last_active":"2024-09-17T03:24:00.665347+0000","last_peered":"2024-09-17T03:24:00.665347+0000","last_clean":"2024-09-17T03:24:00.665347+0000","last_became_active":"2024-09-17T03:23:59.687731+0000","last_became_peered":"2024-09-17T03:23:59.687731+0000","last_unstale":"2024-09-17T03:24:00.665347+0000","last_undegraded":"2024-09-17T03:24:00.665347+0000","last_fullsized":"2024-09-17T03:24:00.665347+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.956844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.700356+0000","last_change":"2024-09-17T03:24:00.696252+0000","last_active":"2024-09-17T03:24:00.700356+0000","last_peered":"2024-09-17T03:24:00.700356+0000","last_clean":"2024-09-17T03:24:00.700356+0000","last_became_active":"2024-09-17T03:24:00.695908+0000","last_became_peered":"2024-09-17T03:24:00.695908+0000","last_unstale":"2024-09-17T03:24:00.700356+0000","last_undegraded":"2024-09-17T03:24:00.700356+0000","last_fullsized":"2024-09-17T03:24:00.700356+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:10.690757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.662037+0000","last_change":"2024-09-17T03:23:59.670815+0000","last_active":"2024-09-17T03:24:00.662037+0000","last_peered":"2024-09-17T03:24:00.662037+0000","last_clean":"2024-09-17T03:24:00.662037+0000","last_became_active":"2024-09-17T03:23:59.670399+0000","last_became_peered":"2024-09-17T03:23:59.670399+0000","last_unstale":"2024-09-17T03:24:00.662037+0000","last_undegraded":"2024-09-17T03:24:00.662037+0000","last_fullsized":"2024-09-17T03:24:00.662037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:35.133861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679254+0000","last_change":"2024-09-17T03:24:00.671158+0000","last_active":"2024-09-17T03:24:01.679254+0000","last_peered":"2024-09-17T03:24:01.679254+0000","last_clean":"2024-09-17T03:24:01.679254+0000","last_became_active":"2024-09-17T03:24:00.670802+0000","last_became_peered":"2024-09-17T03:24:00.670802+0000","last_unstale":"2024-09-17T03:24:01.679254+0000","last_undegraded":"2024-09-17T03:24:01.679254+0000","last_fullsized":"2024-09-17T03:24:01.679254+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:14.452407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.665157+0000","last_change":"2024-09-17T03:23:59.689197+0000","last_active":"2024-09-17T03:24:00.665157+0000","last_peered":"2024-09-17T03:24:00.665157+0000","last_clean":"2024-09-17T03:24:00.665157+0000","last_became_active":"2024-09-17T03:23:59.688837+0000","last_became_peered":"2024-09-17T03:23:59.688837+0000","last_unstale":"2024-09-17T03:24:00.665157+0000","last_undegraded":"2024-09-17T03:24:00.665157+0000","last_fullsized":"2024-09-17T03:24:00.665157+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:24.954127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679167+0000","last_change":"2024-09-17T03:24:00.688830+0000","last_active":"2024-09-17T03:24:01.679167+0000","last_peered":"2024-09-17T03:24:01.679167+0000","last_clean":"2024-09-17T03:24:01.679167+0000","last_became_active":"2024-09-17T03:24:00.686003+0000","last_became_peered":"2024-09-17T03:24:00.686003+0000","last_unstale":"2024-09-17T03:24:01.679167+0000","last_undegraded":"2024-09-17T03:24:01.679167+0000","last_fullsized":"2024-09-17T03:24:01.679167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:58.275617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682528+0000","last_change":"2024-09-17T03:23:59.679373+0000","last_active":"2024-09-17T03:24:01.682528+0000","last_peered":"2024-09-17T03:24:01.682528+0000","last_clean":"2024-09-17T03:24:01.682528+0000","last_became_active":"2024-09-17T03:23:59.679013+0000","last_became_peered":"2024-09-17T03:23:59.679013+0000","last_unstale":"2024-09-17T03:24:01.682528+0000","last_undegraded":"2024-09-17T03:24:01.682528+0000","last_fullsized":"2024-09-17T03:24:01.682528+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:43.273765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.691697+0000","last_change":"2024-09-17T03:24:00.688162+0000","last_active":"2024-09-17T03:24:00.691697+0000","last_peered":"2024-09-17T03:24:00.691697+0000","last_clean":"2024-09-17T03:24:00.691697+0000","last_became_active":"2024-09-17T03:24:00.686096+0000","last_became_peered":"2024-09-17T03:24:00.686096+0000","last_unstale":"2024-09-17T03:24:00.691697+0000","last_undegraded":"2024-09-17T03:24:00.691697+0000","last_fullsized":"2024-09-17T03:24:00.691697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:50.278679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.661803+0000","last_change":"2024-09-17T03:23:59.682062+0000","last_active":"2024-09-17T03:24:00.661803+0000","last_peered":"2024-09-17T03:24:00.661803+0000","last_clean":"2024-09-17T03:24:00.661803+0000","last_became_active":"2024-09-17T03:23:59.681682+0000","last_became_peered":"2024-09-17T03:23:59.681682+0000","last_unstale":"2024-09-17T03:24:00.661803+0000","last_undegraded":"2024-09-17T03:24:00.661803+0000","last_fullsized":"2024-09-17T03:24:00.661803+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:15.268791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.683034+0000","last_change":"2024-09-17T03:24:00.677168+0000","last_active":"2024-09-17T03:24:01.683034+0000","last_peered":"2024-09-17T03:24:01.683034+0000","last_clean":"2024-09-17T03:24:01.683034+0000","last_became_active":"2024-09-17T03:24:00.676863+0000","last_became_peered":"2024-09-17T03:24:00.676863+0000","last_unstale":"2024-09-17T03:24:01.683034+0000","last_undegraded":"2024-09-17T03:24:01.683034+0000","last_fullsized":"2024-09-17T03:24:01.683034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:07.271152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.700578+0000","last_change":"2024-09-17T03:23:59.686849+0000","last_active":"2024-09-17T03:24:00.700578+0000","last_peered":"2024-09-17T03:24:00.700578+0000","last_clean":"2024-09-17T03:24:00.700578+0000","last_became_active":"2024-09-17T03:23:59.684414+0000","last_became_peered":"2024-09-17T03:23:59.684414+0000","last_unstale":"2024-09-17T03:24:00.700578+0000","last_undegraded":"2024-09-17T03:24:00.700578+0000","last_fullsized":"2024-09-17T03:24:00.700578+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:50.231599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.709959+0000","last_change":"2024-09-17T03:24:00.703424+0000","last_active":"2024-09-17T03:24:00.709959+0000","last_peered":"2024-09-17T03:24:00.709959+0000","last_clean":"2024-09-17T03:24:00.709959+0000","last_became_active":"2024-09-17T03:24:00.701027+0000","last_became_peered":"2024-09-17T03:24:00.701027+0000","last_unstale":"2024-09-17T03:24:00.709959+0000","last_undegraded":"2024-09-17T03:24:00.709959+0000","last_fullsized":"2024-09-17T03:24:00.709959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:29.787699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.691827+0000","last_change":"2024-09-17T03:23:59.678761+0000","last_active":"2024-09-17T03:24:00.691827+0000","last_peered":"2024-09-17T03:24:00.691827+0000","last_clean":"2024-09-17T03:24:00.691827+0000","last_became_active":"2024-09-17T03:23:59.678428+0000","last_became_peered":"2024-09-17T03:23:59.678428+0000","last_unstale":"2024-09-17T03:24:00.691827+0000","last_undegraded":"2024-09-17T03:24:00.691827+0000","last_fullsized":"2024-09-17T03:24:00.691827+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:02.207045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":150,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.664380+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:24:00.664380+0000","last_peered":"2024-09-17T03:24:00.664380+0000","last_clean":"2024-09-17T03:24:00.664380+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:24:00.664380+0000","last_undegraded":"2024-09-17T03:24:00.664380+0000","last_fullsized":"2024-09-17T03:24:00.664380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1081344,"data_stored":782729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":491520,"data_stored":192943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":495616,"data_stored":193883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35828,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336012,"statfs":{"total":96636764160,"available":96600076288,"internally_reserved":0,"allocated":495616,"data_stored":192909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":499712,"data_stored":193905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1081344,"data_stored":782673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":185496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39972,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331868,"statfs":{"total":96636764160,"available":96595832832,"internally_reserved":0,"allocated":471040,"data_stored":185496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:24:04.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:24:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 -- 172.21.15.46:0/3692936246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 msgr2=0x7fc42c0b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 --2- 172.21.15.46:0/3692936246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c0b3710 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc42c071800 tx=0x7fc41c01c800 comp rx=0 tx=0).stop 2024-09-17T03:24:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 -- 172.21.15.46:0/3692936246 shutdown_connections 2024-09-17T03:24:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 --2- 172.21.15.46:0/3692936246 >> v2:172.21.15.99:3300/0 conn(0x7fc42c073f90 0x7fc42c0743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 --2- 172.21.15.46:0/3692936246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c0b3710 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 -- 172.21.15.46:0/3692936246 >> 172.21.15.46:0/3692936246 conn(0x7fc42c0d06e0 msgr2=0x7fc42c0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 -- 172.21.15.46:0/3692936246 shutdown_connections 2024-09-17T03:24:05.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.080+0000 7fc433200640 1 -- 172.21.15.46:0/3692936246 wait complete. 2024-09-17T03:24:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 Processor -- start 2024-09-17T03:24:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 -- start start 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc42c073f90 0x7fc42c1fc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc42c0af520 0x7fc42c0aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c06f6e0 con 0x7fc42c15c730 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c06f560 con 0x7fc42c0af520 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c06f860 con 0x7fc42c073f90 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c1f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32842/0 (socket says 172.21.15.46:32842) 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 -- 172.21.15.46:0/1439816188 learned_addr learned my addr 172.21.15.46:0/1439816188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc423400640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc42c0af520 0x7fc42c0aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc42be00640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc42c073f90 0x7fc42c1fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 -- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc42c073f90 msgr2=0x7fc42c1fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc42c073f90 0x7fc42c1fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 -- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc42c0af520 msgr2=0x7fc42c0aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc42c0af520 0x7fc42c0aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 -- 172.21.15.46:0/1439816188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c0cf4e0 con 0x7fc42c15c730 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc42be00640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc42c073f90 0x7fc42c1fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc431200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c1f89e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc42400cae0 tx=0x7fc42400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424016020 con 0x7fc42c15c730 2024-09-17T03:24:05.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc42400ab40 con 0x7fc42c15c730 2024-09-17T03:24:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42400ce20 con 0x7fc42c15c730 2024-09-17T03:24:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1fc190 con 0x7fc42c15c730 2024-09-17T03:24:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.084+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c0aeff0 con 0x7fc42c15c730 2024-09-17T03:24:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.088+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c0b3710 con 0x7fc42c15c730 2024-09-17T03:24:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.088+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fc424019020 con 0x7fc42c15c730 2024-09-17T03:24:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.088+0000 7fc428c00640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc408073560 0x7fc408075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.088+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fc424094980 con 0x7fc42c15c730 2024-09-17T03:24:05.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.088+0000 7fc42be00640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc408073560 0x7fc408075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.088+0000 7fc42be00640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc408073560 0x7fc408075a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0af000 tx=0x7fc41c01b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:05.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.092+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc424020020 con 0x7fc42c15c730 2024-09-17T03:24:05.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.264+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc42c151ad0 con 0x7fc408073560 2024-09-17T03:24:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.272+0000 7fc428c00640 1 -- 172.21.15.46:0/1439816188 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7fc42c151ad0 con 0x7fc408073560 2024-09-17T03:24:05.276 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:05.277 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:24:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc408073560 msgr2=0x7fc408075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc408073560 0x7fc408075a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0af000 tx=0x7fc41c01b040 comp rx=0 tx=0).stop 2024-09-17T03:24:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 msgr2=0x7fc42c1f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c1f89e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc42400cae0 tx=0x7fc42400cbd0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 shutdown_connections 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc408073560 0x7fc408075a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc42c15c730 0x7fc42c1f89e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc42c0af520 0x7fc42c0aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 --2- 172.21.15.46:0/1439816188 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc42c073f90 0x7fc42c1fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 >> 172.21.15.46:0/1439816188 conn(0x7fc42c0d06e0 msgr2=0x7fc42c1552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 shutdown_connections 2024-09-17T03:24:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.276+0000 7fc433200640 1 -- 172.21.15.46:0/1439816188 wait complete. 2024-09-17T03:24:05.297 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-17T03:24:05.249503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279468,"kb_used_data":4968,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754695252,"statfs":{"total":773094113280,"available":772807938048,"internally_reserved":0,"allocated":5087232,"data_stored":2710034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001349"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.686134+0000","last_change":"2024-09-17T03:24:00.679712+0000","last_active":"2024-09-17T03:24:00.686134+0000","last_peered":"2024-09-17T03:24:00.686134+0000","last_clean":"2024-09-17T03:24:00.686134+0000","last_became_active":"2024-09-17T03:24:00.679384+0000","last_became_peered":"2024-09-17T03:24:00.679384+0000","last_unstale":"2024-09-17T03:24:00.686134+0000","last_undegraded":"2024-09-17T03:24:00.686134+0000","last_fullsized":"2024-09-17T03:24:00.686134+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:09.852780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.715551+0000","last_change":"2024-09-17T03:24:00.711936+0000","last_active":"2024-09-17T03:24:00.715551+0000","last_peered":"2024-09-17T03:24:00.715551+0000","last_clean":"2024-09-17T03:24:00.715551+0000","last_became_active":"2024-09-17T03:24:00.711688+0000","last_became_peered":"2024-09-17T03:24:00.711688+0000","last_unstale":"2024-09-17T03:24:00.715551+0000","last_undegraded":"2024-09-17T03:24:00.715551+0000","last_fullsized":"2024-09-17T03:24:00.715551+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:32.852422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682528+0000","last_change":"2024-09-17T03:24:00.695337+0000","last_active":"2024-09-17T03:24:01.682528+0000","last_peered":"2024-09-17T03:24:01.682528+0000","last_clean":"2024-09-17T03:24:01.682528+0000","last_became_active":"2024-09-17T03:24:00.694995+0000","last_became_peered":"2024-09-17T03:24:00.694995+0000","last_unstale":"2024-09-17T03:24:01.682528+0000","last_undegraded":"2024-09-17T03:24:01.682528+0000","last_fullsized":"2024-09-17T03:24:01.682528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:02.187680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.707651+0000","last_change":"2024-09-17T03:24:00.703706+0000","last_active":"2024-09-17T03:24:00.707651+0000","last_peered":"2024-09-17T03:24:00.707651+0000","last_clean":"2024-09-17T03:24:00.707651+0000","last_became_active":"2024-09-17T03:24:00.701364+0000","last_became_peered":"2024-09-17T03:24:00.701364+0000","last_unstale":"2024-09-17T03:24:00.707651+0000","last_undegraded":"2024-09-17T03:24:00.707651+0000","last_fullsized":"2024-09-17T03:24:00.707651+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:40.120192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.712745+0000","last_change":"2024-09-17T03:24:00.708883+0000","last_active":"2024-09-17T03:24:00.712745+0000","last_peered":"2024-09-17T03:24:00.712745+0000","last_clean":"2024-09-17T03:24:00.712745+0000","last_became_active":"2024-09-17T03:24:00.708572+0000","last_became_peered":"2024-09-17T03:24:00.708572+0000","last_unstale":"2024-09-17T03:24:00.712745+0000","last_undegraded":"2024-09-17T03:24:00.712745+0000","last_fullsized":"2024-09-17T03:24:00.712745+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:26.128412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678990+0000","last_change":"2024-09-17T03:24:00.674630+0000","last_active":"2024-09-17T03:24:01.678990+0000","last_peered":"2024-09-17T03:24:01.678990+0000","last_clean":"2024-09-17T03:24:01.678990+0000","last_became_active":"2024-09-17T03:24:00.673905+0000","last_became_peered":"2024-09-17T03:24:00.673905+0000","last_unstale":"2024-09-17T03:24:01.678990+0000","last_undegraded":"2024-09-17T03:24:01.678990+0000","last_fullsized":"2024-09-17T03:24:01.678990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:31.006160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681723+0000","last_change":"2024-09-17T03:24:00.675168+0000","last_active":"2024-09-17T03:24:00.681723+0000","last_peered":"2024-09-17T03:24:00.681723+0000","last_clean":"2024-09-17T03:24:00.681723+0000","last_became_active":"2024-09-17T03:24:00.674820+0000","last_became_peered":"2024-09-17T03:24:00.674820+0000","last_unstale":"2024-09-17T03:24:00.681723+0000","last_undegraded":"2024-09-17T03:24:00.681723+0000","last_fullsized":"2024-09-17T03:24:00.681723+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:11.723455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.715720+0000","last_change":"2024-09-17T03:24:00.714581+0000","last_active":"2024-09-17T03:24:00.715720+0000","last_peered":"2024-09-17T03:24:00.715720+0000","last_clean":"2024-09-17T03:24:00.715720+0000","last_became_active":"2024-09-17T03:24:00.712060+0000","last_became_peered":"2024-09-17T03:24:00.712060+0000","last_unstale":"2024-09-17T03:24:00.715720+0000","last_undegraded":"2024-09-17T03:24:00.715720+0000","last_fullsized":"2024-09-17T03:24:00.715720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.566668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.711156+0000","last_change":"2024-09-17T03:24:00.708244+0000","last_active":"2024-09-17T03:24:00.711156+0000","last_peered":"2024-09-17T03:24:00.711156+0000","last_clean":"2024-09-17T03:24:00.711156+0000","last_became_active":"2024-09-17T03:24:00.707863+0000","last_became_peered":"2024-09-17T03:24:00.707863+0000","last_unstale":"2024-09-17T03:24:00.711156+0000","last_undegraded":"2024-09-17T03:24:00.711156+0000","last_fullsized":"2024-09-17T03:24:00.711156+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:58.331752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.713554+0000","last_change":"2024-09-17T03:24:00.710911+0000","last_active":"2024-09-17T03:24:00.713554+0000","last_peered":"2024-09-17T03:24:00.713554+0000","last_clean":"2024-09-17T03:24:00.713554+0000","last_became_active":"2024-09-17T03:24:00.710612+0000","last_became_peered":"2024-09-17T03:24:00.710612+0000","last_unstale":"2024-09-17T03:24:00.713554+0000","last_undegraded":"2024-09-17T03:24:00.713554+0000","last_fullsized":"2024-09-17T03:24:00.713554+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:04.935975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.711142+0000","last_change":"2024-09-17T03:24:00.708153+0000","last_active":"2024-09-17T03:24:00.711142+0000","last_peered":"2024-09-17T03:24:00.711142+0000","last_clean":"2024-09-17T03:24:00.711142+0000","last_became_active":"2024-09-17T03:24:00.707818+0000","last_became_peered":"2024-09-17T03:24:00.707818+0000","last_unstale":"2024-09-17T03:24:00.711142+0000","last_undegraded":"2024-09-17T03:24:00.711142+0000","last_fullsized":"2024-09-17T03:24:00.711142+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:24.740141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703380+0000","last_change":"2024-09-17T03:24:00.700049+0000","last_active":"2024-09-17T03:24:00.703380+0000","last_peered":"2024-09-17T03:24:00.703380+0000","last_clean":"2024-09-17T03:24:00.703380+0000","last_became_active":"2024-09-17T03:24:00.699761+0000","last_became_peered":"2024-09-17T03:24:00.699761+0000","last_unstale":"2024-09-17T03:24:00.703380+0000","last_undegraded":"2024-09-17T03:24:00.703380+0000","last_fullsized":"2024-09-17T03:24:00.703380+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:39.865635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675751+0000","last_change":"2024-09-17T03:24:00.696346+0000","last_active":"2024-09-17T03:24:01.675751+0000","last_peered":"2024-09-17T03:24:01.675751+0000","last_clean":"2024-09-17T03:24:01.675751+0000","last_became_active":"2024-09-17T03:24:00.696043+0000","last_became_peered":"2024-09-17T03:24:00.696043+0000","last_unstale":"2024-09-17T03:24:01.675751+0000","last_undegraded":"2024-09-17T03:24:01.675751+0000","last_fullsized":"2024-09-17T03:24:01.675751+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:37.766883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703364+0000","last_change":"2024-09-17T03:24:00.698152+0000","last_active":"2024-09-17T03:24:00.703364+0000","last_peered":"2024-09-17T03:24:00.703364+0000","last_clean":"2024-09-17T03:24:00.703364+0000","last_became_active":"2024-09-17T03:24:00.694472+0000","last_became_peered":"2024-09-17T03:24:00.694472+0000","last_unstale":"2024-09-17T03:24:00.703364+0000","last_undegraded":"2024-09-17T03:24:00.703364+0000","last_fullsized":"2024-09-17T03:24:00.703364+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:00.826322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678808+0000","last_change":"2024-09-17T03:24:00.676915+0000","last_active":"2024-09-17T03:24:01.678808+0000","last_peered":"2024-09-17T03:24:01.678808+0000","last_clean":"2024-09-17T03:24:01.678808+0000","last_became_active":"2024-09-17T03:24:00.676270+0000","last_became_peered":"2024-09-17T03:24:00.676270+0000","last_unstale":"2024-09-17T03:24:01.678808+0000","last_undegraded":"2024-09-17T03:24:01.678808+0000","last_fullsized":"2024-09-17T03:24:01.678808+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:40.255358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.722604+0000","last_change":"2024-09-17T03:24:00.718486+0000","last_active":"2024-09-17T03:24:00.722604+0000","last_peered":"2024-09-17T03:24:00.722604+0000","last_clean":"2024-09-17T03:24:00.722604+0000","last_became_active":"2024-09-17T03:24:00.718080+0000","last_became_peered":"2024-09-17T03:24:00.718080+0000","last_unstale":"2024-09-17T03:24:00.722604+0000","last_undegraded":"2024-09-17T03:24:00.722604+0000","last_fullsized":"2024-09-17T03:24:00.722604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:44.277028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682771+0000","last_change":"2024-09-17T03:24:00.677675+0000","last_active":"2024-09-17T03:24:01.682771+0000","last_peered":"2024-09-17T03:24:01.682771+0000","last_clean":"2024-09-17T03:24:01.682771+0000","last_became_active":"2024-09-17T03:24:00.677384+0000","last_became_peered":"2024-09-17T03:24:00.677384+0000","last_unstale":"2024-09-17T03:24:01.682771+0000","last_undegraded":"2024-09-17T03:24:01.682771+0000","last_fullsized":"2024-09-17T03:24:01.682771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:41.583671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682586+0000","last_change":"2024-09-17T03:24:00.705343+0000","last_active":"2024-09-17T03:24:01.682586+0000","last_peered":"2024-09-17T03:24:01.682586+0000","last_clean":"2024-09-17T03:24:01.682586+0000","last_became_active":"2024-09-17T03:24:00.705008+0000","last_became_peered":"2024-09-17T03:24:00.705008+0000","last_unstale":"2024-09-17T03:24:01.682586+0000","last_undegraded":"2024-09-17T03:24:01.682586+0000","last_fullsized":"2024-09-17T03:24:01.682586+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:27.373160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.710264+0000","last_change":"2024-09-17T03:24:00.705686+0000","last_active":"2024-09-17T03:24:00.710264+0000","last_peered":"2024-09-17T03:24:00.710264+0000","last_clean":"2024-09-17T03:24:00.710264+0000","last_became_active":"2024-09-17T03:24:00.704768+0000","last_became_peered":"2024-09-17T03:24:00.704768+0000","last_unstale":"2024-09-17T03:24:00.710264+0000","last_undegraded":"2024-09-17T03:24:00.710264+0000","last_fullsized":"2024-09-17T03:24:00.710264+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:00.659956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676111+0000","last_change":"2024-09-17T03:24:00.706410+0000","last_active":"2024-09-17T03:24:01.676111+0000","last_peered":"2024-09-17T03:24:01.676111+0000","last_clean":"2024-09-17T03:24:01.676111+0000","last_became_active":"2024-09-17T03:24:00.705984+0000","last_became_peered":"2024-09-17T03:24:00.705984+0000","last_unstale":"2024-09-17T03:24:01.676111+0000","last_undegraded":"2024-09-17T03:24:01.676111+0000","last_fullsized":"2024-09-17T03:24:01.676111+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:07.078551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678908+0000","last_change":"2024-09-17T03:24:00.676215+0000","last_active":"2024-09-17T03:24:01.678908+0000","last_peered":"2024-09-17T03:24:01.678908+0000","last_clean":"2024-09-17T03:24:01.678908+0000","last_became_active":"2024-09-17T03:24:00.675886+0000","last_became_peered":"2024-09-17T03:24:00.675886+0000","last_unstale":"2024-09-17T03:24:01.678908+0000","last_undegraded":"2024-09-17T03:24:01.678908+0000","last_fullsized":"2024-09-17T03:24:01.678908+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:19.556922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703396+0000","last_change":"2024-09-17T03:24:00.698862+0000","last_active":"2024-09-17T03:24:00.703396+0000","last_peered":"2024-09-17T03:24:00.703396+0000","last_clean":"2024-09-17T03:24:00.703396+0000","last_became_active":"2024-09-17T03:24:00.694829+0000","last_became_peered":"2024-09-17T03:24:00.694829+0000","last_unstale":"2024-09-17T03:24:00.703396+0000","last_undegraded":"2024-09-17T03:24:00.703396+0000","last_fullsized":"2024-09-17T03:24:00.703396+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:32.893125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689609+0000","last_change":"2024-09-17T03:24:00.685397+0000","last_active":"2024-09-17T03:24:00.689609+0000","last_peered":"2024-09-17T03:24:00.689609+0000","last_clean":"2024-09-17T03:24:00.689609+0000","last_became_active":"2024-09-17T03:24:00.685063+0000","last_became_peered":"2024-09-17T03:24:00.685063+0000","last_unstale":"2024-09-17T03:24:00.689609+0000","last_undegraded":"2024-09-17T03:24:00.689609+0000","last_fullsized":"2024-09-17T03:24:00.689609+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:07.078551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689258+0000","last_change":"2024-09-17T03:24:00.684388+0000","last_active":"2024-09-17T03:24:00.689258+0000","last_peered":"2024-09-17T03:24:00.689258+0000","last_clean":"2024-09-17T03:24:00.689258+0000","last_became_active":"2024-09-17T03:24:00.684093+0000","last_became_peered":"2024-09-17T03:24:00.684093+0000","last_unstale":"2024-09-17T03:24:00.689258+0000","last_undegraded":"2024-09-17T03:24:00.689258+0000","last_fullsized":"2024-09-17T03:24:00.689258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:37.766883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694437+0000","last_change":"2024-09-17T03:24:00.686880+0000","last_active":"2024-09-17T03:24:00.694437+0000","last_peered":"2024-09-17T03:24:00.694437+0000","last_clean":"2024-09-17T03:24:00.694437+0000","last_became_active":"2024-09-17T03:24:00.686601+0000","last_became_peered":"2024-09-17T03:24:00.686601+0000","last_unstale":"2024-09-17T03:24:00.694437+0000","last_undegraded":"2024-09-17T03:24:00.694437+0000","last_fullsized":"2024-09-17T03:24:00.694437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:23.386007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676016+0000","last_change":"2024-09-17T03:24:00.688599+0000","last_active":"2024-09-17T03:24:01.676016+0000","last_peered":"2024-09-17T03:24:01.676016+0000","last_clean":"2024-09-17T03:24:01.676016+0000","last_became_active":"2024-09-17T03:24:00.688229+0000","last_became_peered":"2024-09-17T03:24:00.688229+0000","last_unstale":"2024-09-17T03:24:01.676016+0000","last_undegraded":"2024-09-17T03:24:01.676016+0000","last_fullsized":"2024-09-17T03:24:01.676016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:06.920757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682684+0000","last_change":"2024-09-17T03:24:00.679002+0000","last_active":"2024-09-17T03:24:01.682684+0000","last_peered":"2024-09-17T03:24:01.682684+0000","last_clean":"2024-09-17T03:24:01.682684+0000","last_became_active":"2024-09-17T03:24:00.677828+0000","last_became_peered":"2024-09-17T03:24:00.677828+0000","last_unstale":"2024-09-17T03:24:01.682684+0000","last_undegraded":"2024-09-17T03:24:01.682684+0000","last_fullsized":"2024-09-17T03:24:01.682684+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:35.766678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.703406+0000","last_change":"2024-09-17T03:24:00.700037+0000","last_active":"2024-09-17T03:24:00.703406+0000","last_peered":"2024-09-17T03:24:00.703406+0000","last_clean":"2024-09-17T03:24:00.703406+0000","last_became_active":"2024-09-17T03:24:00.699470+0000","last_became_peered":"2024-09-17T03:24:00.699470+0000","last_unstale":"2024-09-17T03:24:00.703406+0000","last_undegraded":"2024-09-17T03:24:00.703406+0000","last_fullsized":"2024-09-17T03:24:00.703406+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:30.514015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689389+0000","last_change":"2024-09-17T03:24:00.685325+0000","last_active":"2024-09-17T03:24:00.689389+0000","last_peered":"2024-09-17T03:24:00.689389+0000","last_clean":"2024-09-17T03:24:00.689389+0000","last_became_active":"2024-09-17T03:24:00.685036+0000","last_became_peered":"2024-09-17T03:24:00.685036+0000","last_unstale":"2024-09-17T03:24:00.689389+0000","last_undegraded":"2024-09-17T03:24:00.689389+0000","last_fullsized":"2024-09-17T03:24:00.689389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:06.920757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682379+0000","last_change":"2024-09-17T03:24:00.683130+0000","last_active":"2024-09-17T03:24:01.682379+0000","last_peered":"2024-09-17T03:24:01.682379+0000","last_clean":"2024-09-17T03:24:01.682379+0000","last_became_active":"2024-09-17T03:24:00.682800+0000","last_became_peered":"2024-09-17T03:24:00.682800+0000","last_unstale":"2024-09-17T03:24:01.682379+0000","last_undegraded":"2024-09-17T03:24:01.682379+0000","last_fullsized":"2024-09-17T03:24:01.682379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:09.498797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678877+0000","last_change":"2024-09-17T03:24:00.671210+0000","last_active":"2024-09-17T03:24:01.678877+0000","last_peered":"2024-09-17T03:24:01.678877+0000","last_clean":"2024-09-17T03:24:01.678877+0000","last_became_active":"2024-09-17T03:24:00.670854+0000","last_became_peered":"2024-09-17T03:24:00.670854+0000","last_unstale":"2024-09-17T03:24:01.678877+0000","last_undegraded":"2024-09-17T03:24:01.678877+0000","last_fullsized":"2024-09-17T03:24:01.678877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:47.878595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675557+0000","last_change":"2024-09-17T03:24:00.683911+0000","last_active":"2024-09-17T03:24:01.675557+0000","last_peered":"2024-09-17T03:24:01.675557+0000","last_clean":"2024-09-17T03:24:01.675557+0000","last_became_active":"2024-09-17T03:24:00.683579+0000","last_became_peered":"2024-09-17T03:24:00.683579+0000","last_unstale":"2024-09-17T03:24:01.675557+0000","last_undegraded":"2024-09-17T03:24:01.675557+0000","last_fullsized":"2024-09-17T03:24:01.675557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:18.956672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680290+0000","last_change":"2024-09-17T03:24:00.674960+0000","last_active":"2024-09-17T03:24:00.680290+0000","last_peered":"2024-09-17T03:24:00.680290+0000","last_clean":"2024-09-17T03:24:00.680290+0000","last_became_active":"2024-09-17T03:24:00.674637+0000","last_became_peered":"2024-09-17T03:24:00.674637+0000","last_unstale":"2024-09-17T03:24:00.680290+0000","last_undegraded":"2024-09-17T03:24:00.680290+0000","last_fullsized":"2024-09-17T03:24:00.680290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:19.199176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679409+0000","last_change":"2024-09-17T03:23:59.683374+0000","last_active":"2024-09-17T03:24:01.679409+0000","last_peered":"2024-09-17T03:24:01.679409+0000","last_clean":"2024-09-17T03:24:01.679409+0000","last_became_active":"2024-09-17T03:23:59.682927+0000","last_became_peered":"2024-09-17T03:23:59.682927+0000","last_unstale":"2024-09-17T03:24:01.679409+0000","last_undegraded":"2024-09-17T03:24:01.679409+0000","last_fullsized":"2024-09-17T03:24:01.679409+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:10.720193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681126+0000","last_change":"2024-09-17T03:24:00.679188+0000","last_active":"2024-09-17T03:24:00.681126+0000","last_peered":"2024-09-17T03:24:00.681126+0000","last_clean":"2024-09-17T03:24:00.681126+0000","last_became_active":"2024-09-17T03:24:00.678835+0000","last_became_peered":"2024-09-17T03:24:00.678835+0000","last_unstale":"2024-09-17T03:24:00.681126+0000","last_undegraded":"2024-09-17T03:24:00.681126+0000","last_fullsized":"2024-09-17T03:24:00.681126+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:18.956672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682960+0000","last_change":"2024-09-17T03:23:59.686402+0000","last_active":"2024-09-17T03:24:01.682960+0000","last_peered":"2024-09-17T03:24:01.682960+0000","last_clean":"2024-09-17T03:24:01.682960+0000","last_became_active":"2024-09-17T03:23:59.686069+0000","last_became_peered":"2024-09-17T03:23:59.686069+0000","last_unstale":"2024-09-17T03:24:01.682960+0000","last_undegraded":"2024-09-17T03:24:01.682960+0000","last_fullsized":"2024-09-17T03:24:01.682960+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:49.128013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680715+0000","last_change":"2024-09-17T03:24:00.677750+0000","last_active":"2024-09-17T03:24:00.680715+0000","last_peered":"2024-09-17T03:24:00.680715+0000","last_clean":"2024-09-17T03:24:00.680715+0000","last_became_active":"2024-09-17T03:24:00.677432+0000","last_became_peered":"2024-09-17T03:24:00.677432+0000","last_unstale":"2024-09-17T03:24:00.680715+0000","last_undegraded":"2024-09-17T03:24:00.680715+0000","last_fullsized":"2024-09-17T03:24:00.680715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:08.338281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.687369+0000","last_change":"2024-09-17T03:23:59.674742+0000","last_active":"2024-09-17T03:24:00.687369+0000","last_peered":"2024-09-17T03:24:00.687369+0000","last_clean":"2024-09-17T03:24:00.687369+0000","last_became_active":"2024-09-17T03:23:59.674135+0000","last_became_peered":"2024-09-17T03:23:59.674135+0000","last_unstale":"2024-09-17T03:24:00.687369+0000","last_undegraded":"2024-09-17T03:24:00.687369+0000","last_fullsized":"2024-09-17T03:24:00.687369+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:41.361148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681542+0000","last_change":"2024-09-17T03:24:00.678804+0000","last_active":"2024-09-17T03:24:00.681542+0000","last_peered":"2024-09-17T03:24:00.681542+0000","last_clean":"2024-09-17T03:24:00.681542+0000","last_became_active":"2024-09-17T03:24:00.678369+0000","last_became_peered":"2024-09-17T03:24:00.678369+0000","last_unstale":"2024-09-17T03:24:00.681542+0000","last_undegraded":"2024-09-17T03:24:00.681542+0000","last_fullsized":"2024-09-17T03:24:00.681542+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:09.232872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.695811+0000","last_change":"2024-09-17T03:23:59.684650+0000","last_active":"2024-09-17T03:24:00.695811+0000","last_peered":"2024-09-17T03:24:00.695811+0000","last_clean":"2024-09-17T03:24:00.695811+0000","last_became_active":"2024-09-17T03:23:59.684235+0000","last_became_peered":"2024-09-17T03:23:59.684235+0000","last_unstale":"2024-09-17T03:24:00.695811+0000","last_undegraded":"2024-09-17T03:24:00.695811+0000","last_fullsized":"2024-09-17T03:24:00.695811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:48.454565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682397+0000","last_change":"2024-09-17T03:24:00.680552+0000","last_active":"2024-09-17T03:24:01.682397+0000","last_peered":"2024-09-17T03:24:01.682397+0000","last_clean":"2024-09-17T03:24:01.682397+0000","last_became_active":"2024-09-17T03:24:00.680263+0000","last_became_peered":"2024-09-17T03:24:00.680263+0000","last_unstale":"2024-09-17T03:24:01.682397+0000","last_undegraded":"2024-09-17T03:24:01.682397+0000","last_fullsized":"2024-09-17T03:24:01.682397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.566668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679371+0000","last_change":"2024-09-17T03:23:59.676497+0000","last_active":"2024-09-17T03:24:01.679371+0000","last_peered":"2024-09-17T03:24:01.679371+0000","last_clean":"2024-09-17T03:24:01.679371+0000","last_became_active":"2024-09-17T03:23:59.676157+0000","last_became_peered":"2024-09-17T03:23:59.676157+0000","last_unstale":"2024-09-17T03:24:01.679371+0000","last_undegraded":"2024-09-17T03:24:01.679371+0000","last_fullsized":"2024-09-17T03:24:01.679371+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:08.849517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.714267+0000","last_change":"2024-09-17T03:24:00.710747+0000","last_active":"2024-09-17T03:24:00.714267+0000","last_peered":"2024-09-17T03:24:00.714267+0000","last_clean":"2024-09-17T03:24:00.714267+0000","last_became_active":"2024-09-17T03:24:00.708450+0000","last_became_peered":"2024-09-17T03:24:00.708450+0000","last_unstale":"2024-09-17T03:24:00.714267+0000","last_undegraded":"2024-09-17T03:24:00.714267+0000","last_fullsized":"2024-09-17T03:24:00.714267+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:26:13.822513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682524+0000","last_change":"2024-09-17T03:23:59.675958+0000","last_active":"2024-09-17T03:24:01.682524+0000","last_peered":"2024-09-17T03:24:01.682524+0000","last_clean":"2024-09-17T03:24:01.682524+0000","last_became_active":"2024-09-17T03:23:59.675261+0000","last_became_peered":"2024-09-17T03:23:59.675261+0000","last_unstale":"2024-09-17T03:24:01.682524+0000","last_undegraded":"2024-09-17T03:24:01.682524+0000","last_fullsized":"2024-09-17T03:24:01.682524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:11.374505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.714843+0000","last_change":"2024-09-17T03:24:00.712231+0000","last_active":"2024-09-17T03:24:00.714843+0000","last_peered":"2024-09-17T03:24:00.714843+0000","last_clean":"2024-09-17T03:24:00.714843+0000","last_became_active":"2024-09-17T03:24:00.711807+0000","last_became_peered":"2024-09-17T03:24:00.711807+0000","last_unstale":"2024-09-17T03:24:00.714843+0000","last_undegraded":"2024-09-17T03:24:00.714843+0000","last_fullsized":"2024-09-17T03:24:00.714843+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:09.498797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.670083+0000","last_change":"2024-09-17T03:23:59.673810+0000","last_active":"2024-09-17T03:24:00.670083+0000","last_peered":"2024-09-17T03:24:00.670083+0000","last_clean":"2024-09-17T03:24:00.670083+0000","last_became_active":"2024-09-17T03:23:59.673100+0000","last_became_peered":"2024-09-17T03:23:59.673100+0000","last_unstale":"2024-09-17T03:24:00.670083+0000","last_undegraded":"2024-09-17T03:24:00.670083+0000","last_fullsized":"2024-09-17T03:24:00.670083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:03.157480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694479+0000","last_change":"2024-09-17T03:24:00.687279+0000","last_active":"2024-09-17T03:24:00.694479+0000","last_peered":"2024-09-17T03:24:00.694479+0000","last_clean":"2024-09-17T03:24:00.694479+0000","last_became_active":"2024-09-17T03:24:00.685315+0000","last_became_peered":"2024-09-17T03:24:00.685315+0000","last_unstale":"2024-09-17T03:24:00.694479+0000","last_undegraded":"2024-09-17T03:24:00.694479+0000","last_fullsized":"2024-09-17T03:24:00.694479+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:37:43.575452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.700035+0000","last_change":"2024-09-17T03:23:59.687636+0000","last_active":"2024-09-17T03:24:00.700035+0000","last_peered":"2024-09-17T03:24:00.700035+0000","last_clean":"2024-09-17T03:24:00.700035+0000","last_became_active":"2024-09-17T03:23:59.687322+0000","last_became_peered":"2024-09-17T03:23:59.687322+0000","last_unstale":"2024-09-17T03:24:00.700035+0000","last_undegraded":"2024-09-17T03:24:00.700035+0000","last_fullsized":"2024-09-17T03:24:00.700035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:21.035374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682826+0000","last_change":"2024-09-17T03:24:00.705466+0000","last_active":"2024-09-17T03:24:01.682826+0000","last_peered":"2024-09-17T03:24:01.682826+0000","last_clean":"2024-09-17T03:24:01.682826+0000","last_became_active":"2024-09-17T03:24:00.705180+0000","last_became_peered":"2024-09-17T03:24:00.705180+0000","last_unstale":"2024-09-17T03:24:01.682826+0000","last_undegraded":"2024-09-17T03:24:01.682826+0000","last_fullsized":"2024-09-17T03:24:01.682826+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:40.123607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682970+0000","last_change":"2024-09-17T03:23:59.673722+0000","last_active":"2024-09-17T03:24:01.682970+0000","last_peered":"2024-09-17T03:24:01.682970+0000","last_clean":"2024-09-17T03:24:01.682970+0000","last_became_active":"2024-09-17T03:23:59.673369+0000","last_became_peered":"2024-09-17T03:23:59.673369+0000","last_unstale":"2024-09-17T03:24:01.682970+0000","last_undegraded":"2024-09-17T03:24:01.682970+0000","last_fullsized":"2024-09-17T03:24:01.682970+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:08.453921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680520+0000","last_change":"2024-09-17T03:24:00.675419+0000","last_active":"2024-09-17T03:24:00.680520+0000","last_peered":"2024-09-17T03:24:00.680520+0000","last_clean":"2024-09-17T03:24:00.680520+0000","last_became_active":"2024-09-17T03:24:00.675106+0000","last_became_peered":"2024-09-17T03:24:00.675106+0000","last_unstale":"2024-09-17T03:24:00.680520+0000","last_undegraded":"2024-09-17T03:24:00.680520+0000","last_fullsized":"2024-09-17T03:24:00.680520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:20.595904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.707144+0000","last_change":"2024-09-17T03:23:59.686562+0000","last_active":"2024-09-17T03:24:00.707144+0000","last_peered":"2024-09-17T03:24:00.707144+0000","last_clean":"2024-09-17T03:24:00.707144+0000","last_became_active":"2024-09-17T03:23:59.686246+0000","last_became_peered":"2024-09-17T03:23:59.686246+0000","last_unstale":"2024-09-17T03:24:00.707144+0000","last_undegraded":"2024-09-17T03:24:00.707144+0000","last_fullsized":"2024-09-17T03:24:00.707144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:01.226260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679258+0000","last_change":"2024-09-17T03:24:00.696303+0000","last_active":"2024-09-17T03:24:01.679258+0000","last_peered":"2024-09-17T03:24:01.679258+0000","last_clean":"2024-09-17T03:24:01.679258+0000","last_became_active":"2024-09-17T03:24:00.695937+0000","last_became_peered":"2024-09-17T03:24:00.695937+0000","last_unstale":"2024-09-17T03:24:01.679258+0000","last_undegraded":"2024-09-17T03:24:01.679258+0000","last_fullsized":"2024-09-17T03:24:01.679258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:51.673689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.696367+0000","last_change":"2024-09-17T03:23:59.688206+0000","last_active":"2024-09-17T03:24:00.696367+0000","last_peered":"2024-09-17T03:24:00.696367+0000","last_clean":"2024-09-17T03:24:00.696367+0000","last_became_active":"2024-09-17T03:23:59.687858+0000","last_became_peered":"2024-09-17T03:23:59.687858+0000","last_unstale":"2024-09-17T03:24:00.696367+0000","last_undegraded":"2024-09-17T03:24:00.696367+0000","last_fullsized":"2024-09-17T03:24:00.696367+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:50.097008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.680478+0000","last_change":"2024-09-17T03:24:00.674939+0000","last_active":"2024-09-17T03:24:00.680478+0000","last_peered":"2024-09-17T03:24:00.680478+0000","last_clean":"2024-09-17T03:24:00.680478+0000","last_became_active":"2024-09-17T03:24:00.673963+0000","last_became_peered":"2024-09-17T03:24:00.673963+0000","last_unstale":"2024-09-17T03:24:00.680478+0000","last_undegraded":"2024-09-17T03:24:00.680478+0000","last_fullsized":"2024-09-17T03:24:00.680478+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:01.987364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676156+0000","last_change":"2024-09-17T03:23:59.676386+0000","last_active":"2024-09-17T03:24:01.676156+0000","last_peered":"2024-09-17T03:24:01.676156+0000","last_clean":"2024-09-17T03:24:01.676156+0000","last_became_active":"2024-09-17T03:23:59.675884+0000","last_became_peered":"2024-09-17T03:23:59.675884+0000","last_unstale":"2024-09-17T03:24:01.676156+0000","last_undegraded":"2024-09-17T03:24:01.676156+0000","last_fullsized":"2024-09-17T03:24:01.676156+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:27.081555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.712641+0000","last_change":"2024-09-17T03:24:00.708903+0000","last_active":"2024-09-17T03:24:00.712641+0000","last_peered":"2024-09-17T03:24:00.712641+0000","last_clean":"2024-09-17T03:24:00.712641+0000","last_became_active":"2024-09-17T03:24:00.708564+0000","last_became_peered":"2024-09-17T03:24:00.708564+0000","last_unstale":"2024-09-17T03:24:00.712641+0000","last_undegraded":"2024-09-17T03:24:00.712641+0000","last_fullsized":"2024-09-17T03:24:00.712641+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:09.337210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679495+0000","last_change":"2024-09-17T03:23:59.686453+0000","last_active":"2024-09-17T03:24:01.679495+0000","last_peered":"2024-09-17T03:24:01.679495+0000","last_clean":"2024-09-17T03:24:01.679495+0000","last_became_active":"2024-09-17T03:23:59.685855+0000","last_became_peered":"2024-09-17T03:23:59.685855+0000","last_unstale":"2024-09-17T03:24:01.679495+0000","last_undegraded":"2024-09-17T03:24:01.679495+0000","last_fullsized":"2024-09-17T03:24:01.679495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:22.785717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.714297+0000","last_change":"2024-09-17T03:24:00.711203+0000","last_active":"2024-09-17T03:24:00.714297+0000","last_peered":"2024-09-17T03:24:00.714297+0000","last_clean":"2024-09-17T03:24:00.714297+0000","last_became_active":"2024-09-17T03:24:00.707999+0000","last_became_peered":"2024-09-17T03:24:00.707999+0000","last_unstale":"2024-09-17T03:24:00.714297+0000","last_undegraded":"2024-09-17T03:24:00.714297+0000","last_fullsized":"2024-09-17T03:24:00.714297+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:48.279400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.664677+0000","last_change":"2024-09-17T03:23:59.681355+0000","last_active":"2024-09-17T03:24:00.664677+0000","last_peered":"2024-09-17T03:24:00.664677+0000","last_clean":"2024-09-17T03:24:00.664677+0000","last_became_active":"2024-09-17T03:23:59.681014+0000","last_became_peered":"2024-09-17T03:23:59.681014+0000","last_unstale":"2024-09-17T03:24:00.664677+0000","last_undegraded":"2024-09-17T03:24:00.664677+0000","last_fullsized":"2024-09-17T03:24:00.664677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:41.740765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678814+0000","last_change":"2024-09-17T03:24:00.686271+0000","last_active":"2024-09-17T03:24:01.678814+0000","last_peered":"2024-09-17T03:24:01.678814+0000","last_clean":"2024-09-17T03:24:01.678814+0000","last_became_active":"2024-09-17T03:24:00.685903+0000","last_became_peered":"2024-09-17T03:24:00.685903+0000","last_unstale":"2024-09-17T03:24:01.678814+0000","last_undegraded":"2024-09-17T03:24:01.678814+0000","last_fullsized":"2024-09-17T03:24:01.678814+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:08.432503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.678906+0000","last_change":"2024-09-17T03:23:59.686981+0000","last_active":"2024-09-17T03:24:01.678906+0000","last_peered":"2024-09-17T03:24:01.678906+0000","last_clean":"2024-09-17T03:24:01.678906+0000","last_became_active":"2024-09-17T03:23:59.686678+0000","last_became_peered":"2024-09-17T03:23:59.686678+0000","last_unstale":"2024-09-17T03:24:01.678906+0000","last_undegraded":"2024-09-17T03:24:01.678906+0000","last_fullsized":"2024-09-17T03:24:01.678906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.956844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.692081+0000","last_change":"2024-09-17T03:24:00.681726+0000","last_active":"2024-09-17T03:24:00.692081+0000","last_peered":"2024-09-17T03:24:00.692081+0000","last_clean":"2024-09-17T03:24:00.692081+0000","last_became_active":"2024-09-17T03:24:00.681446+0000","last_became_peered":"2024-09-17T03:24:00.681446+0000","last_unstale":"2024-09-17T03:24:00.692081+0000","last_undegraded":"2024-09-17T03:24:00.692081+0000","last_fullsized":"2024-09-17T03:24:00.692081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:23.788980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.718055+0000","last_change":"2024-09-17T03:23:59.687662+0000","last_active":"2024-09-17T03:24:00.718055+0000","last_peered":"2024-09-17T03:24:00.718055+0000","last_clean":"2024-09-17T03:24:00.718055+0000","last_became_active":"2024-09-17T03:23:59.687309+0000","last_became_peered":"2024-09-17T03:23:59.687309+0000","last_unstale":"2024-09-17T03:24:00.718055+0000","last_undegraded":"2024-09-17T03:24:00.718055+0000","last_fullsized":"2024-09-17T03:24:00.718055+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:47.557243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.676116+0000","last_change":"2024-09-17T03:24:00.688548+0000","last_active":"2024-09-17T03:24:01.676116+0000","last_peered":"2024-09-17T03:24:01.676116+0000","last_clean":"2024-09-17T03:24:01.676116+0000","last_became_active":"2024-09-17T03:24:00.688213+0000","last_became_peered":"2024-09-17T03:24:00.688213+0000","last_unstale":"2024-09-17T03:24:01.676116+0000","last_undegraded":"2024-09-17T03:24:01.676116+0000","last_fullsized":"2024-09-17T03:24:01.676116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:22.324581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675981+0000","last_change":"2024-09-17T03:23:59.676504+0000","last_active":"2024-09-17T03:24:01.675981+0000","last_peered":"2024-09-17T03:24:01.675981+0000","last_clean":"2024-09-17T03:24:01.675981+0000","last_became_active":"2024-09-17T03:23:59.675819+0000","last_became_peered":"2024-09-17T03:23:59.675819+0000","last_unstale":"2024-09-17T03:24:01.675981+0000","last_undegraded":"2024-09-17T03:24:01.675981+0000","last_fullsized":"2024-09-17T03:24:01.675981+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:03.658474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.711867+0000","last_change":"2024-09-17T03:24:00.710362+0000","last_active":"2024-09-17T03:24:00.711867+0000","last_peered":"2024-09-17T03:24:00.711867+0000","last_clean":"2024-09-17T03:24:00.711867+0000","last_became_active":"2024-09-17T03:24:00.710091+0000","last_became_peered":"2024-09-17T03:24:00.710091+0000","last_unstale":"2024-09-17T03:24:00.711867+0000","last_undegraded":"2024-09-17T03:24:00.711867+0000","last_fullsized":"2024-09-17T03:24:00.711867+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:17.523408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.683081+0000","last_change":"2024-09-17T03:23:59.668703+0000","last_active":"2024-09-17T03:24:01.683081+0000","last_peered":"2024-09-17T03:24:01.683081+0000","last_clean":"2024-09-17T03:24:01.683081+0000","last_became_active":"2024-09-17T03:23:59.668365+0000","last_became_peered":"2024-09-17T03:23:59.668365+0000","last_unstale":"2024-09-17T03:24:01.683081+0000","last_undegraded":"2024-09-17T03:24:01.683081+0000","last_fullsized":"2024-09-17T03:24:01.683081+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:19.808133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.689193+0000","last_change":"2024-09-17T03:24:00.681885+0000","last_active":"2024-09-17T03:24:00.689193+0000","last_peered":"2024-09-17T03:24:00.689193+0000","last_clean":"2024-09-17T03:24:00.689193+0000","last_became_active":"2024-09-17T03:24:00.681578+0000","last_became_peered":"2024-09-17T03:24:00.681578+0000","last_unstale":"2024-09-17T03:24:00.689193+0000","last_undegraded":"2024-09-17T03:24:00.689193+0000","last_fullsized":"2024-09-17T03:24:00.689193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:56.945734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694350+0000","last_change":"2024-09-17T03:23:59.679798+0000","last_active":"2024-09-17T03:24:00.694350+0000","last_peered":"2024-09-17T03:24:00.694350+0000","last_clean":"2024-09-17T03:24:00.694350+0000","last_became_active":"2024-09-17T03:23:59.679466+0000","last_became_peered":"2024-09-17T03:23:59.679466+0000","last_unstale":"2024-09-17T03:24:00.694350+0000","last_undegraded":"2024-09-17T03:24:00.694350+0000","last_fullsized":"2024-09-17T03:24:00.694350+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:17.297194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.694572+0000","last_change":"2024-09-17T03:24:00.687851+0000","last_active":"2024-09-17T03:24:00.694572+0000","last_peered":"2024-09-17T03:24:00.694572+0000","last_clean":"2024-09-17T03:24:00.694572+0000","last_became_active":"2024-09-17T03:24:00.687400+0000","last_became_peered":"2024-09-17T03:24:00.687400+0000","last_unstale":"2024-09-17T03:24:00.694572+0000","last_undegraded":"2024-09-17T03:24:00.694572+0000","last_fullsized":"2024-09-17T03:24:00.694572+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:32.389400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679380+0000","last_change":"2024-09-17T03:23:59.679174+0000","last_active":"2024-09-17T03:24:01.679380+0000","last_peered":"2024-09-17T03:24:01.679380+0000","last_clean":"2024-09-17T03:24:01.679380+0000","last_became_active":"2024-09-17T03:23:59.678916+0000","last_became_peered":"2024-09-17T03:23:59.678916+0000","last_unstale":"2024-09-17T03:24:01.679380+0000","last_undegraded":"2024-09-17T03:24:01.679380+0000","last_fullsized":"2024-09-17T03:24:01.679380+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:19.592642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679087+0000","last_change":"2024-09-17T03:24:00.696312+0000","last_active":"2024-09-17T03:24:01.679087+0000","last_peered":"2024-09-17T03:24:01.679087+0000","last_clean":"2024-09-17T03:24:01.679087+0000","last_became_active":"2024-09-17T03:24:00.695942+0000","last_became_peered":"2024-09-17T03:24:00.695942+0000","last_unstale":"2024-09-17T03:24:01.679087+0000","last_undegraded":"2024-09-17T03:24:01.679087+0000","last_fullsized":"2024-09-17T03:24:01.679087+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:18.733200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679611+0000","last_change":"2024-09-17T03:23:59.675028+0000","last_active":"2024-09-17T03:24:01.679611+0000","last_peered":"2024-09-17T03:24:01.679611+0000","last_clean":"2024-09-17T03:24:01.679611+0000","last_became_active":"2024-09-17T03:23:59.674728+0000","last_became_peered":"2024-09-17T03:23:59.674728+0000","last_unstale":"2024-09-17T03:24:01.679611+0000","last_undegraded":"2024-09-17T03:24:01.679611+0000","last_fullsized":"2024-09-17T03:24:01.679611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:09.687494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.681638+0000","last_change":"2024-09-17T03:24:00.676089+0000","last_active":"2024-09-17T03:24:00.681638+0000","last_peered":"2024-09-17T03:24:00.681638+0000","last_clean":"2024-09-17T03:24:00.681638+0000","last_became_active":"2024-09-17T03:24:00.675610+0000","last_became_peered":"2024-09-17T03:24:00.675610+0000","last_unstale":"2024-09-17T03:24:00.681638+0000","last_undegraded":"2024-09-17T03:24:00.681638+0000","last_fullsized":"2024-09-17T03:24:00.681638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.379254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.669156+0000","last_change":"2024-09-17T03:23:59.682548+0000","last_active":"2024-09-17T03:24:00.669156+0000","last_peered":"2024-09-17T03:24:00.669156+0000","last_clean":"2024-09-17T03:24:00.669156+0000","last_became_active":"2024-09-17T03:23:59.682201+0000","last_became_peered":"2024-09-17T03:23:59.682201+0000","last_unstale":"2024-09-17T03:24:00.669156+0000","last_undegraded":"2024-09-17T03:24:00.669156+0000","last_fullsized":"2024-09-17T03:24:00.669156+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:29.085472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.675591+0000","last_change":"2024-09-17T03:24:00.687458+0000","last_active":"2024-09-17T03:24:01.675591+0000","last_peered":"2024-09-17T03:24:01.675591+0000","last_clean":"2024-09-17T03:24:01.675591+0000","last_became_active":"2024-09-17T03:24:00.687164+0000","last_became_peered":"2024-09-17T03:24:00.687164+0000","last_unstale":"2024-09-17T03:24:01.675591+0000","last_undegraded":"2024-09-17T03:24:01.675591+0000","last_fullsized":"2024-09-17T03:24:01.675591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:56.945734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.693276+0000","last_change":"2024-09-17T03:23:59.685224+0000","last_active":"2024-09-17T03:24:00.693276+0000","last_peered":"2024-09-17T03:24:00.693276+0000","last_clean":"2024-09-17T03:24:00.693276+0000","last_became_active":"2024-09-17T03:23:59.681143+0000","last_became_peered":"2024-09-17T03:23:59.681143+0000","last_unstale":"2024-09-17T03:24:00.693276+0000","last_undegraded":"2024-09-17T03:24:00.693276+0000","last_fullsized":"2024-09-17T03:24:00.693276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:47.586031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.699373+0000","last_change":"2024-09-17T03:24:00.692868+0000","last_active":"2024-09-17T03:24:00.699373+0000","last_peered":"2024-09-17T03:24:00.699373+0000","last_clean":"2024-09-17T03:24:00.699373+0000","last_became_active":"2024-09-17T03:24:00.692545+0000","last_became_peered":"2024-09-17T03:24:00.692545+0000","last_unstale":"2024-09-17T03:24:00.699373+0000","last_undegraded":"2024-09-17T03:24:00.699373+0000","last_fullsized":"2024-09-17T03:24:00.699373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:54.312118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.664621+0000","last_change":"2024-09-17T03:23:59.682305+0000","last_active":"2024-09-17T03:24:00.664621+0000","last_peered":"2024-09-17T03:24:00.664621+0000","last_clean":"2024-09-17T03:24:00.664621+0000","last_became_active":"2024-09-17T03:23:59.681437+0000","last_became_peered":"2024-09-17T03:23:59.681437+0000","last_unstale":"2024-09-17T03:24:00.664621+0000","last_undegraded":"2024-09-17T03:24:00.664621+0000","last_fullsized":"2024-09-17T03:24:00.664621+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:52.874111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.721592+0000","last_change":"2024-09-17T03:24:00.718196+0000","last_active":"2024-09-17T03:24:00.721592+0000","last_peered":"2024-09-17T03:24:00.721592+0000","last_clean":"2024-09-17T03:24:00.721592+0000","last_became_active":"2024-09-17T03:24:00.717873+0000","last_became_peered":"2024-09-17T03:24:00.717873+0000","last_unstale":"2024-09-17T03:24:00.721592+0000","last_undegraded":"2024-09-17T03:24:00.721592+0000","last_fullsized":"2024-09-17T03:24:00.721592+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:12.377768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679092+0000","last_change":"2024-09-17T03:23:59.672475+0000","last_active":"2024-09-17T03:24:01.679092+0000","last_peered":"2024-09-17T03:24:01.679092+0000","last_clean":"2024-09-17T03:24:01.679092+0000","last_became_active":"2024-09-17T03:23:59.669909+0000","last_became_peered":"2024-09-17T03:23:59.669909+0000","last_unstale":"2024-09-17T03:24:01.679092+0000","last_undegraded":"2024-09-17T03:24:01.679092+0000","last_fullsized":"2024-09-17T03:24:01.679092+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:00.984101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.707613+0000","last_change":"2024-09-17T03:24:00.703373+0000","last_active":"2024-09-17T03:24:00.707613+0000","last_peered":"2024-09-17T03:24:00.707613+0000","last_clean":"2024-09-17T03:24:00.707613+0000","last_became_active":"2024-09-17T03:24:00.700966+0000","last_became_peered":"2024-09-17T03:24:00.700966+0000","last_unstale":"2024-09-17T03:24:00.707613+0000","last_undegraded":"2024-09-17T03:24:00.707613+0000","last_fullsized":"2024-09-17T03:24:00.707613+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:25.868561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.665347+0000","last_change":"2024-09-17T03:23:59.688084+0000","last_active":"2024-09-17T03:24:00.665347+0000","last_peered":"2024-09-17T03:24:00.665347+0000","last_clean":"2024-09-17T03:24:00.665347+0000","last_became_active":"2024-09-17T03:23:59.687731+0000","last_became_peered":"2024-09-17T03:23:59.687731+0000","last_unstale":"2024-09-17T03:24:00.665347+0000","last_undegraded":"2024-09-17T03:24:00.665347+0000","last_fullsized":"2024-09-17T03:24:00.665347+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.956844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.700356+0000","last_change":"2024-09-17T03:24:00.696252+0000","last_active":"2024-09-17T03:24:00.700356+0000","last_peered":"2024-09-17T03:24:00.700356+0000","last_clean":"2024-09-17T03:24:00.700356+0000","last_became_active":"2024-09-17T03:24:00.695908+0000","last_became_peered":"2024-09-17T03:24:00.695908+0000","last_unstale":"2024-09-17T03:24:00.700356+0000","last_undegraded":"2024-09-17T03:24:00.700356+0000","last_fullsized":"2024-09-17T03:24:00.700356+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:10.690757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.662037+0000","last_change":"2024-09-17T03:23:59.670815+0000","last_active":"2024-09-17T03:24:00.662037+0000","last_peered":"2024-09-17T03:24:00.662037+0000","last_clean":"2024-09-17T03:24:00.662037+0000","last_became_active":"2024-09-17T03:23:59.670399+0000","last_became_peered":"2024-09-17T03:23:59.670399+0000","last_unstale":"2024-09-17T03:24:00.662037+0000","last_undegraded":"2024-09-17T03:24:00.662037+0000","last_fullsized":"2024-09-17T03:24:00.662037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:35.133861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679254+0000","last_change":"2024-09-17T03:24:00.671158+0000","last_active":"2024-09-17T03:24:01.679254+0000","last_peered":"2024-09-17T03:24:01.679254+0000","last_clean":"2024-09-17T03:24:01.679254+0000","last_became_active":"2024-09-17T03:24:00.670802+0000","last_became_peered":"2024-09-17T03:24:00.670802+0000","last_unstale":"2024-09-17T03:24:01.679254+0000","last_undegraded":"2024-09-17T03:24:01.679254+0000","last_fullsized":"2024-09-17T03:24:01.679254+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:14.452407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.665157+0000","last_change":"2024-09-17T03:23:59.689197+0000","last_active":"2024-09-17T03:24:00.665157+0000","last_peered":"2024-09-17T03:24:00.665157+0000","last_clean":"2024-09-17T03:24:00.665157+0000","last_became_active":"2024-09-17T03:23:59.688837+0000","last_became_peered":"2024-09-17T03:23:59.688837+0000","last_unstale":"2024-09-17T03:24:00.665157+0000","last_undegraded":"2024-09-17T03:24:00.665157+0000","last_fullsized":"2024-09-17T03:24:00.665157+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:24.954127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.679167+0000","last_change":"2024-09-17T03:24:00.688830+0000","last_active":"2024-09-17T03:24:01.679167+0000","last_peered":"2024-09-17T03:24:01.679167+0000","last_clean":"2024-09-17T03:24:01.679167+0000","last_became_active":"2024-09-17T03:24:00.686003+0000","last_became_peered":"2024-09-17T03:24:00.686003+0000","last_unstale":"2024-09-17T03:24:01.679167+0000","last_undegraded":"2024-09-17T03:24:01.679167+0000","last_fullsized":"2024-09-17T03:24:01.679167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:58.275617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.682528+0000","last_change":"2024-09-17T03:23:59.679373+0000","last_active":"2024-09-17T03:24:01.682528+0000","last_peered":"2024-09-17T03:24:01.682528+0000","last_clean":"2024-09-17T03:24:01.682528+0000","last_became_active":"2024-09-17T03:23:59.679013+0000","last_became_peered":"2024-09-17T03:23:59.679013+0000","last_unstale":"2024-09-17T03:24:01.682528+0000","last_undegraded":"2024-09-17T03:24:01.682528+0000","last_fullsized":"2024-09-17T03:24:01.682528+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:43.273765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.691697+0000","last_change":"2024-09-17T03:24:00.688162+0000","last_active":"2024-09-17T03:24:00.691697+0000","last_peered":"2024-09-17T03:24:00.691697+0000","last_clean":"2024-09-17T03:24:00.691697+0000","last_became_active":"2024-09-17T03:24:00.686096+0000","last_became_peered":"2024-09-17T03:24:00.686096+0000","last_unstale":"2024-09-17T03:24:00.691697+0000","last_undegraded":"2024-09-17T03:24:00.691697+0000","last_fullsized":"2024-09-17T03:24:00.691697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:50.278679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.661803+0000","last_change":"2024-09-17T03:23:59.682062+0000","last_active":"2024-09-17T03:24:00.661803+0000","last_peered":"2024-09-17T03:24:00.661803+0000","last_clean":"2024-09-17T03:24:00.661803+0000","last_became_active":"2024-09-17T03:23:59.681682+0000","last_became_peered":"2024-09-17T03:23:59.681682+0000","last_unstale":"2024-09-17T03:24:00.661803+0000","last_undegraded":"2024-09-17T03:24:00.661803+0000","last_fullsized":"2024-09-17T03:24:00.661803+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:15.268791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T03:24:01.683034+0000","last_change":"2024-09-17T03:24:00.677168+0000","last_active":"2024-09-17T03:24:01.683034+0000","last_peered":"2024-09-17T03:24:01.683034+0000","last_clean":"2024-09-17T03:24:01.683034+0000","last_became_active":"2024-09-17T03:24:00.676863+0000","last_became_peered":"2024-09-17T03:24:00.676863+0000","last_unstale":"2024-09-17T03:24:01.683034+0000","last_undegraded":"2024-09-17T03:24:01.683034+0000","last_fullsized":"2024-09-17T03:24:01.683034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:07.271152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.700578+0000","last_change":"2024-09-17T03:23:59.686849+0000","last_active":"2024-09-17T03:24:00.700578+0000","last_peered":"2024-09-17T03:24:00.700578+0000","last_clean":"2024-09-17T03:24:00.700578+0000","last_became_active":"2024-09-17T03:23:59.684414+0000","last_became_peered":"2024-09-17T03:23:59.684414+0000","last_unstale":"2024-09-17T03:24:00.700578+0000","last_undegraded":"2024-09-17T03:24:00.700578+0000","last_fullsized":"2024-09-17T03:24:00.700578+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:50.231599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.709959+0000","last_change":"2024-09-17T03:24:00.703424+0000","last_active":"2024-09-17T03:24:00.709959+0000","last_peered":"2024-09-17T03:24:00.709959+0000","last_clean":"2024-09-17T03:24:00.709959+0000","last_became_active":"2024-09-17T03:24:00.701027+0000","last_became_peered":"2024-09-17T03:24:00.701027+0000","last_unstale":"2024-09-17T03:24:00.709959+0000","last_undegraded":"2024-09-17T03:24:00.709959+0000","last_fullsized":"2024-09-17T03:24:00.709959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:59.647895+0000","last_clean_scrub_stamp":"2024-09-17T03:23:59.647895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:29.787699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.691827+0000","last_change":"2024-09-17T03:23:59.678761+0000","last_active":"2024-09-17T03:24:00.691827+0000","last_peered":"2024-09-17T03:24:00.691827+0000","last_clean":"2024-09-17T03:24:00.691827+0000","last_became_active":"2024-09-17T03:23:59.678428+0000","last_became_peered":"2024-09-17T03:23:59.678428+0000","last_unstale":"2024-09-17T03:24:00.691827+0000","last_undegraded":"2024-09-17T03:24:00.691827+0000","last_fullsized":"2024-09-17T03:24:00.691827+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:23:58.644632+0000","last_clean_scrub_stamp":"2024-09-17T03:23:58.644632+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:02.207045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":150,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T03:24:00.664380+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:24:00.664380+0000","last_peered":"2024-09-17T03:24:00.664380+0000","last_clean":"2024-09-17T03:24:00.664380+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:24:00.664380+0000","last_undegraded":"2024-09-17T03:24:00.664380+0000","last_fullsized":"2024-09-17T03:24:00.664380+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1081344,"data_stored":782729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":491520,"data_stored":192943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":495616,"data_stored":193883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35828,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336012,"statfs":{"total":96636764160,"available":96600076288,"internally_reserved":0,"allocated":495616,"data_stored":192909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":499712,"data_stored":193905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1081344,"data_stored":782673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":471040,"data_stored":185496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39972,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331868,"statfs":{"total":96636764160,"available":96595832832,"internally_reserved":0,"allocated":471040,"data_stored":185496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:24:05.300 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:24:05.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 -- 172.21.15.46:0/508075458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc81527b0 msgr2=0x7f1cc815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 --2- 172.21.15.46:0/508075458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc81527b0 0x7f1cc815bb60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009870 tx=0x7f1cb001c7a0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 -- 172.21.15.46:0/508075458 shutdown_connections 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 --2- 172.21.15.46:0/508075458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc81527b0 0x7f1cc815bb60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 -- 172.21.15.46:0/508075458 >> 172.21.15.46:0/508075458 conn(0x7f1cc80b2040 msgr2=0x7f1cc80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 -- 172.21.15.46:0/508075458 shutdown_connections 2024-09-17T03:24:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 -- 172.21.15.46:0/508075458 wait complete. 2024-09-17T03:24:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.504+0000 7f1ccda00640 1 Processor -- start 2024-09-17T03:24:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- start start 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc80b4de0 0x7f1cc81f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc80b5c70 0x7f1cc813d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 0x7f1cc81fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8169c60 con 0x7f1cc81527b0 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8169ae0 con 0x7f1cc80b5c70 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc8169de0 con 0x7f1cc80b4de0 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 0x7f1cc81fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 0x7f1cc81fc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32852/0 (socket says 172.21.15.46:32852) 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 -- 172.21.15.46:0/1496455280 learned_addr learned my addr 172.21.15.46:0/1496455280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cbfe00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc80b5c70 0x7f1cc813d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc6a00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc80b4de0 0x7f1cc81f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 -- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc80b4de0 msgr2=0x7f1cc81f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc80b4de0 0x7f1cc81f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 -- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc80b5c70 msgr2=0x7f1cc813d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc80b5c70 0x7f1cc813d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 -- 172.21.15.46:0/1496455280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc81f94b0 con 0x7f1cc81527b0 2024-09-17T03:24:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cc7400640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 0x7f1cc81fc070 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c700 tx=0x7f1cc000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003b80 con 0x7f1cc81527b0 2024-09-17T03:24:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cc0003d20 con 0x7f1cc81527b0 2024-09-17T03:24:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000aab0 con 0x7f1cc81527b0 2024-09-17T03:24:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc81f9790 con 0x7f1cc81527b0 2024-09-17T03:24:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc81f8fc0 con 0x7f1cc81527b0 2024-09-17T03:24:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.508+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc815bb60 con 0x7f1cc81527b0 2024-09-17T03:24:05.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.512+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f1cc000ac50 con 0x7f1cc81527b0 2024-09-17T03:24:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.512+0000 7f1cbea00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca4073560 0x7f1ca4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.512+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f1cc0094580 con 0x7f1cc81527b0 2024-09-17T03:24:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.512+0000 7f1cc6a00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca4073560 0x7f1ca4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.512+0000 7f1cc6a00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca4073560 0x7f1ca4075a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8007f30 tx=0x7f1cb8007ec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:05.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.516+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1cc005ea10 con 0x7f1cc81527b0 2024-09-17T03:24:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.760+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1cc8159790 con 0x7f1cc81527b0 2024-09-17T03:24:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.764+0000 7f1cbea00640 1 -- 172.21.15.46:0/1496455280 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7f1cc0063440 con 0x7f1cc81527b0 2024-09-17T03:24:05.767 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:24:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:24:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:24:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:24:05.767 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:24:05.768 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca4073560 msgr2=0x7f1ca4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca4073560 0x7f1ca4075a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8007f30 tx=0x7f1cb8007ec0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 msgr2=0x7f1cc81fc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 0x7f1cc81fc070 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c700 tx=0x7f1cc000cbc0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 shutdown_connections 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca4073560 0x7f1ca4075a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc81527b0 0x7f1cc81fc070 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc80b5c70 0x7f1cc813d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 --2- 172.21.15.46:0/1496455280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc80b4de0 0x7f1cc81f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 >> 172.21.15.46:0/1496455280 conn(0x7f1cc80b2040 msgr2=0x7f1cc8154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 shutdown_connections 2024-09-17T03:24:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.768+0000 7f1ccda00640 1 -- 172.21.15.46:0/1496455280 wait complete. 2024-09-17T03:24:05.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:24:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 -- 172.21.15.46:0/1554699948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 msgr2=0x7fbadc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 --2- 172.21.15.46:0/1554699948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc15bb60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbadc070710 tx=0x7fbacc01c890 comp rx=0 tx=0).stop 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 -- 172.21.15.46:0/1554699948 shutdown_connections 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 --2- 172.21.15.46:0/1554699948 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbadc0b4de0 0x7fbadc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 --2- 172.21.15.46:0/1554699948 >> v2:172.21.15.99:3300/0 conn(0x7fbadc1527b0 0x7fbadc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 --2- 172.21.15.46:0/1554699948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc15bb60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 -- 172.21.15.46:0/1554699948 >> 172.21.15.46:0/1554699948 conn(0x7fbadc0b2040 msgr2=0x7fbadc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 -- 172.21.15.46:0/1554699948 shutdown_connections 2024-09-17T03:24:05.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.948+0000 7fbae2400640 1 -- 172.21.15.46:0/1554699948 wait complete. 2024-09-17T03:24:05.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 Processor -- start 2024-09-17T03:24:05.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 -- start start 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbadc0b4de0 0x7fbadc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbadc1527b0 0x7fbadc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc15f6b0 con 0x7fbadc0b5c70 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc15f530 con 0x7fbadc1527b0 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbadc15f830 con 0x7fbadc0b4de0 2024-09-17T03:24:05.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32858/0 (socket says 172.21.15.46:32858) 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 -- 172.21.15.46:0/1270256929 learned_addr learned my addr 172.21.15.46:0/1270256929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbadb400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbadc0b4de0 0x7fbadc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbadbe00640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbadc1527b0 0x7fbadc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 -- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbadc0b4de0 msgr2=0x7fbadc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbadc0b4de0 0x7fbadc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 -- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbadc1527b0 msgr2=0x7fbadc0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbadc1527b0 0x7fbadc0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 -- 172.21.15.46:0/1270256929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc0ce1d0 con 0x7fbadc0b5c70 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbadbe00640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbadc1527b0 0x7fbadc0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad2a00640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc144b20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbac40058d0 tx=0x7fbac400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4017070 con 0x7fbadc0b5c70 2024-09-17T03:24:05.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac4009d20 con 0x7fbadc0b5c70 2024-09-17T03:24:05.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40106c0 con 0x7fbadc0b5c70 2024-09-17T03:24:05.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc0ce4b0 con 0x7fbadc0b5c70 2024-09-17T03:24:05.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.952+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc0cea70 con 0x7fbadc0b5c70 2024-09-17T03:24:05.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.956+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbadc15fdd0 con 0x7fbadc0b5c70 2024-09-17T03:24:05.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.956+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fbac401a070 con 0x7fbadc0b5c70 2024-09-17T03:24:05.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.956+0000 7fbad3e00640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbab8073560 0x7fbab8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:05.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.956+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fbac4094a90 con 0x7fbadc0b5c70 2024-09-17T03:24:05.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.956+0000 7fbadb400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbab8073560 0x7fbab8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:05.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.960+0000 7fbadb400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbab8073560 0x7fbab8075a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0af000 tx=0x7fbacc0a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:05.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:05.960+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbac405ee00 con 0x7fbadc0b5c70 2024-09-17T03:24:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.176+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbadc0cdf60 con 0x7fbadc0b5c70 2024-09-17T03:24:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.176+0000 7fbad3e00640 1 -- 172.21.15.46:0/1270256929 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7fbac4063830 con 0x7fbadc0b5c70 2024-09-17T03:24:06.179 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:24:06.179 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"btime":"2024-09-17T03:24:03:689855+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:24:00.099078+0000","modified":"2024-09-17T03:24:03.111473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245313":{"gid":245313,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4290314221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4290314221},{"type":"v1","addr":"172.21.15.46:6836","nonce":4290314221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245313,"qdb_cluster":[245313]},"id":3}]} 2024-09-17T03:24:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbab8073560 msgr2=0x7fbab8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbab8073560 0x7fbab8075a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0af000 tx=0x7fbacc0a7040 comp rx=0 tx=0).stop 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 msgr2=0x7fbadc144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc144b20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbac40058d0 tx=0x7fbac400dde0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 shutdown_connections 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbab8073560 0x7fbab8075a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbadc1527b0 0x7fbadc0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbadc0b5c70 0x7fbadc144b20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 --2- 172.21.15.46:0/1270256929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbadc0b4de0 0x7fbadc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 >> 172.21.15.46:0/1270256929 conn(0x7fbadc0b2040 msgr2=0x7fbadc159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 shutdown_connections 2024-09-17T03:24:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.180+0000 7fbae2400640 1 -- 172.21.15.46:0/1270256929 wait complete. 2024-09-17T03:24:06.605 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:24:00.099078+0000', 'modified': '2024-09-17T03:24:03.111473+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 245313}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_245313': {'gid': 245313, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/4290314221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6835', 'nonce': 4290314221}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 4290314221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 245313, 'qdb_cluster': [245313]} 2024-09-17T03:24:06.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:24:06.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T03:24:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.768+0000 7f0db4000640 1 -- 172.21.15.46:0/2168347591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d94005310 msgr2=0x7f0d9400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.768+0000 7f0db4000640 1 --2- 172.21.15.46:0/2168347591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d94005310 0x7f0d9400b9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0dac15bbc0 tx=0x7f0da401c910 comp rx=0 tx=0).stop 2024-09-17T03:24:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- 172.21.15.46:0/2168347591 shutdown_connections 2024-09-17T03:24:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 --2- 172.21.15.46:0/2168347591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d94005310 0x7f0d9400b9d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- 172.21.15.46:0/2168347591 >> 172.21.15.46:0/2168347591 conn(0x7f0dac0b2c40 msgr2=0x7f0dac0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- 172.21.15.46:0/2168347591 shutdown_connections 2024-09-17T03:24:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- 172.21.15.46:0/2168347591 wait complete. 2024-09-17T03:24:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 Processor -- start 2024-09-17T03:24:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- start start 2024-09-17T03:24:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d94005310 0x7f0dac14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dac14f7c0 0x7f0dac1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 0x7f0dac1f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac073f40 con 0x7f0dac1f8d80 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac073dc0 con 0x7f0dac14f7c0 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.772+0000 7f0db4000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac0740c0 con 0x7f0d94005310 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 0x7f0dac1f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 0x7f0dac1f9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32874/0 (socket says 172.21.15.46:32874) 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 -- 172.21.15.46:0/3472842892 learned_addr learned my addr 172.21.15.46:0/3472842892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db1600640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d94005310 0x7f0dac14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dac14f7c0 0x7f0dac1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 -- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d94005310 msgr2=0x7f0dac14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d94005310 0x7f0dac14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 -- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dac14f7c0 msgr2=0x7f0dac1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dac14f7c0 0x7f0dac1445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 -- 172.21.15.46:0/3472842892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac1f9c20 con 0x7f0dac1f8d80 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dac14f7c0 0x7f0dac1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db2000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 0x7f0dac1f9190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0da401d1b0 tx=0x7f0da4006d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0da9600640 1 -- 172.21.15.46:0/3472842892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40a5770 con 0x7f0dac1f8d80 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0da9600640 1 -- 172.21.15.46:0/3472842892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0da40b5020 con 0x7f0dac1f8d80 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0da9600640 1 -- 172.21.15.46:0/3472842892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40060d0 con 0x7f0dac1f8d80 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac1e5f20 con 0x7f0dac1f8d80 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1e64e0 con 0x7f0dac1f8d80 2024-09-17T03:24:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.776+0000 7f0da9600640 1 -- 172.21.15.46:0/3472842892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f0da4006270 con 0x7f0dac1f8d80 2024-09-17T03:24:06.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0da9600640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c073590 0x7f0d8c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0da9600640 1 -- 172.21.15.46:0/3472842892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f0da412e960 con 0x7f0dac1f8d80 2024-09-17T03:24:06.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db1600640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c073590 0x7f0d8c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 Processor -- start 2024-09-17T03:24:06.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 -- start start 2024-09-17T03:24:06.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db1600640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c073590 0x7f0d8c075aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0da0005fd0 tx=0x7f0da000b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565612ef460 0x5565618306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561830be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616c8ca0 0x556561831120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556561705410 con 0x5565616df950 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556561642b40 con 0x5565612ef460 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x556561631f70 con 0x5565616c8ca0 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561830be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561830be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32886/0 (socket says 172.21.15.46:32886) 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 -- 172.21.15.46:0/467871829 learned_addr learned my addr 172.21.15.46:0/467871829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db1600640 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565612ef460 0x5565618306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db2000640 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616c8ca0 0x556561831120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 -- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616c8ca0 msgr2=0x556561831120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616c8ca0 0x556561831120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 -- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565612ef460 msgr2=0x5565618306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565612ef460 0x5565618306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 -- 172.21.15.46:0/467871829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565615f95b0 con 0x5565616df950 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0db0c00640 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561830be0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0d98006c90 tx=0x7f0d98006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff9037ceac4be0d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.780+0000 7f0d9dc00640 1 -- 172.21.15.46:0/467871829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98012440 con 0x5565616df950 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0d9dc00640 1 -- 172.21.15.46:0/467871829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d98016070 con 0x5565616df950 2024-09-17T03:24:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0d9dc00640 1 -- 172.21.15.46:0/467871829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98012820 con 0x5565616df950 2024-09-17T03:24:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/467871829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 msgr2=0x556561830be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561830be0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0d98006c90 tx=0x7f0d98006430 comp rx=0 tx=0).stop 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/467871829 shutdown_connections 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616c8ca0 0x556561831120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561830be0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/467871829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565612ef460 0x5565618306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/467871829 >> 172.21.15.46:0/467871829 conn(0x5565617b5d90 msgr2=0x5565617e5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/467871829 shutdown_connections 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/467871829 wait complete. 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- start start 2024-09-17T03:24:06.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 Processor -- start 2024-09-17T03:24:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 0x556561837920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x556561837e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616df950 0x5565618383a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5565615cc5f0 con 0x5565612ef460 2024-09-17T03:24:06.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556561635d80 con 0x5565616c8ca0 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x5565615d8520 con 0x5565616df950 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0dbb6c1000 10 client.0 _ll_register_callbacks cb 0x5565615dc830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0db1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 0x556561837920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0db1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 0x556561837920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32898/0 (socket says 172.21.15.46:32898) 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.784+0000 7f0db1600640 1 -- 172.21.15.46:0/3310284349 learned_addr learned my addr 172.21.15.46:0/3310284349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x556561837e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db2000640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616df950 0x5565618383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db1600640 1 -- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616df950 msgr2=0x5565618383a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db1600640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616df950 0x5565618383a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db1600640 1 -- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 msgr2=0x556561837e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db1600640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x556561837e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db1600640 1 -- 172.21.15.46:0/3310284349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565615d0190 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x556561837e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db1600640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 0x556561837920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0da000bb70 tx=0x7f0da0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81ce76402c5d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 1 -- 172.21.15.46:0/3310284349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da002d060 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565615ca110 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 1 -- 172.21.15.46:0/3310284349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f0da0017030 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 1 -- 172.21.15.46:0/3310284349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0da0033050 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 10 client.245448 fetch_fsmap learned FSMap version 26 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5565616b6f10 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 1 -- 172.21.15.46:0/3310284349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5565615ca110 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 1 -- 172.21.15.46:0/3310284349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f0da0036020 con 0x5565612ef460 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 10 client.245448 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 10 client.245448 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:24:06.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 20 client.245448 populate_metadata read hostname 'smithi046' 2024-09-17T03:24:06.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612d47d0 0x556561841d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 4 client.245448 mds_command: new command op to 245313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:24:06.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 --> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556561644a90 con 0x5565612d47d0 2024-09-17T03:24:06.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612d47d0 0x556561841d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612d47d0 0x556561841d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325fae1585beceb server_cookie=7b549877707affcb in_seq=0 out_seq=0 2024-09-17T03:24:06.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 10 client.245448 ms_handle_connect on v2:172.21.15.46:6835/4290314221 2024-09-17T03:24:06.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 1 -- 172.21.15.46:0/3310284349 <== mds.0 v2:172.21.15.46:6835/4290314221 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556561644a90 con 0x5565612d47d0 2024-09-17T03:24:06.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 10 client.245448 handle_command_reply: tid=0 2024-09-17T03:24:06.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 client.245448 shutdown 2024-09-17T03:24:06.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 msgr2=0x556561837920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 0x556561837920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0da000bb70 tx=0x7f0da0004740 comp rx=0 tx=0).stop 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 shutdown_connections 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612d47d0 0x556561841d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565616df950 0x5565618383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x556561837e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3310284349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565612ef460 0x556561837920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0d92000640 0 client.245448 ms_handle_reset on v2:172.21.15.46:6835/4290314221 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 >> 172.21.15.46:0/3310284349 conn(0x5565617b5d90 msgr2=0x5565617ec440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.788+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 shutdown_connections 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.792+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3310284349 wait complete. 2024-09-17T03:24:06.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.792+0000 7f0dbb6c1000 20 client.245448 trim_cache size 0 max 16384 2024-09-17T03:24:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.808+0000 7f0dbb6c1000 1 Processor -- start 2024-09-17T03:24:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.808+0000 7f0dbb6c1000 1 -- start start 2024-09-17T03:24:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x5565618a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x5565618a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac2028c0 con 0x5565616df950 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac201900 con 0x5565616c8ca0 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac151890 con 0x5565612ef460 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x5565618a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x5565618a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32902/0 (socket says 172.21.15.46:32902) 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 -- 172.21.15.46:0/3852765118 learned_addr learned my addr 172.21.15.46:0/3852765118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db1600640 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x5565618a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 -- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 msgr2=0x5565618a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x5565618a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 -- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 msgr2=0x5565618a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 -- 172.21.15.46:0/3852765118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656168ddb0 con 0x5565616df950 2024-09-17T03:24:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db0c00640 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0db2000640 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x5565618a30b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0da401b010 tx=0x7f0da40f8c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2da85096578ece server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0d9d200640 1 -- 172.21.15.46:0/3852765118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40fd040 con 0x5565616df950 2024-09-17T03:24:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0d9d200640 1 -- 172.21.15.46:0/3852765118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0da4006850 con 0x5565616df950 2024-09-17T03:24:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0d9d200640 1 -- 172.21.15.46:0/3852765118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40f8260 con 0x5565616df950 2024-09-17T03:24:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3852765118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 msgr2=0x5565618a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x5565618a30b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0da401b010 tx=0x7f0da40f8c50 comp rx=0 tx=0).stop 2024-09-17T03:24:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3852765118 shutdown_connections 2024-09-17T03:24:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x5565618a30b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/3852765118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x5565618a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3852765118 >> 172.21.15.46:0/3852765118 conn(0x5565617b5d90 msgr2=0x5565617e4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3852765118 shutdown_connections 2024-09-17T03:24:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/3852765118 wait complete. 2024-09-17T03:24:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 -- start start 2024-09-17T03:24:06.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.812+0000 7f0dbb6c1000 1 Processor -- start 2024-09-17T03:24:06.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x556561861260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561861ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac2012b0 con 0x5565616df950 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565615fdee0 con 0x5565616c8ca0 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55656163a3b0 con 0x5565612ef460 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 10 client.0 _ll_register_callbacks cb 0x7f0da000b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561861ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561861ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32906/0 (socket says 172.21.15.46:32906) 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 -- 172.21.15.46:0/83618628 learned_addr learned my addr 172.21.15.46:0/83618628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 -- 172.21.15.46:0/83618628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 msgr2=0x556561861260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x556561861260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 -- 172.21.15.46:0/83618628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 msgr2=0x5565618617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 -- 172.21.15.46:0/83618628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565615f8ab0 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db2000640 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561861ce0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0da401cdf0 tx=0x7f0da40f8630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ac3c4e5ea5fe62 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0d92000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0d92000640 1 -- 172.21.15.46:0/83618628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40f8220 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0d92000640 1 -- 172.21.15.46:0/83618628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f0da40f4710 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556561640540 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0d92000640 1 -- 172.21.15.46:0/83618628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0da40f94c0 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 10 client.245454 fetch_fsmap learned FSMap version 26 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5565615d7070 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0d92000640 1 -- 172.21.15.46:0/83618628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556561640540 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0d92000640 1 -- 172.21.15.46:0/83618628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f0da40f7a90 con 0x5565616df950 2024-09-17T03:24:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 10 client.245454 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 10 client.245454 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 20 client.245454 populate_metadata read hostname 'smithi046' 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612c2830 0x556561862640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 4 client.245454 mds_command: new command op to 245313 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 --> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] -- command(tid 0: {"prefix": "status"}) -- 0x55656163d970 con 0x5565612c2830 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.816+0000 7f0db1600640 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612c2830 0x556561862640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db1600640 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612c2830 0x556561862640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd51ebcae57423 server_cookie=fd8da33b457c77eb in_seq=0 out_seq=0 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0d92000640 10 client.245454 ms_handle_connect on v2:172.21.15.46:6835/4290314221 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0d92000640 1 -- 172.21.15.46:0/83618628 <== mds.0 v2:172.21.15.46:6835/4290314221 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55656163d970 con 0x5565612c2830 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0d92000640 10 client.245454 handle_command_reply: tid=0 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 client.245454 shutdown 2024-09-17T03:24:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 msgr2=0x556561861ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561861ce0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0da401cdf0 tx=0x7f0da40f8630 comp rx=0 tx=0).stop 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 shutdown_connections 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:6835/4290314221,v1:172.21.15.46:6836/4290314221] conn(0x5565612c2830 0x556561862640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565616df950 0x556561861ce0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565616c8ca0 0x5565618617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0d92000640 0 client.245454 ms_handle_reset on v2:172.21.15.46:6835/4290314221 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 --2- 172.21.15.46:0/83618628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5565612ef460 0x556561861260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 >> 172.21.15.46:0/83618628 conn(0x5565617b5d90 msgr2=0x5565618416a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 shutdown_connections 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 1 -- 172.21.15.46:0/83618628 wait complete. 2024-09-17T03:24:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0dbb6c1000 20 client.245454 trim_cache size 0 max 16384 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 245313, 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.1638404089999996, 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 43, 2024-09-17T03:24:06.828 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 43, 2024-09-17T03:24:06.829 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.561065642999999 2024-09-17T03:24:06.829 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:24:06.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c073590 msgr2=0x7f0d8c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c073590 0x7f0d8c075aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0da0005fd0 tx=0x7f0da000b260 comp rx=0 tx=0).stop 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 msgr2=0x7f0dac1f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 0x7f0dac1f9190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0da401d1b0 tx=0x7f0da4006d30 comp rx=0 tx=0).stop 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 shutdown_connections 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c073590 0x7f0d8c075aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac1f8d80 0x7f0dac1f9190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dac14f7c0 0x7f0dac1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 --2- 172.21.15.46:0/3472842892 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d94005310 0x7f0dac14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 >> 172.21.15.46:0/3472842892 conn(0x7f0dac0b2c40 msgr2=0x7f0dac15a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 shutdown_connections 2024-09-17T03:24:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:06.820+0000 7f0db4000640 1 -- 172.21.15.46:0/3472842892 wait complete. 2024-09-17T03:24:06.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 -- 172.21.15.46:0/3568384235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 msgr2=0x7f6d8815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 --2- 172.21.15.46:0/3568384235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d8815bb60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d88070710 tx=0x7f6d7801c800 comp rx=0 tx=0).stop 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 -- 172.21.15.46:0/3568384235 shutdown_connections 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 --2- 172.21.15.46:0/3568384235 >> v2:172.21.15.99:3300/0 conn(0x7f6d881527b0 0x7f6d88074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 --2- 172.21.15.46:0/3568384235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d8815bb60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 -- 172.21.15.46:0/3568384235 >> 172.21.15.46:0/3568384235 conn(0x7f6d880b2040 msgr2=0x7f6d880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 -- 172.21.15.46:0/3568384235 shutdown_connections 2024-09-17T03:24:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 -- 172.21.15.46:0/3568384235 wait complete. 2024-09-17T03:24:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 Processor -- start 2024-09-17T03:24:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.000+0000 7f6d8f400640 1 -- start start 2024-09-17T03:24:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d880b4de0 0x7f6d88141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d881fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d881527b0 0x7f6d881fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88168b20 con 0x7f6d880b5c70 2024-09-17T03:24:07.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d881689a0 con 0x7f6d880b4de0 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6d88168ca0 con 0x7f6d881527b0 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d881fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d881fc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32926/0 (socket says 172.21.15.46:32926) 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 -- 172.21.15.46:0/625333537 learned_addr learned my addr 172.21.15.46:0/625333537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8ca00640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d880b4de0 0x7f6d88141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8d400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d881527b0 0x7f6d881fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 -- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d881527b0 msgr2=0x7f6d881fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d881527b0 0x7f6d881fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 -- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d880b4de0 msgr2=0x7f6d88141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d880b4de0 0x7f6d88141ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 -- 172.21.15.46:0/625333537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d881423e0 con 0x7f6d880b5c70 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8d400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d881527b0 0x7f6d881fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d7fe00640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d881fc3d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c9f0 tx=0x7f6d7000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70007bb0 con 0x7f6d880b5c70 2024-09-17T03:24:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d70007e40 con 0x7f6d880b5c70 2024-09-17T03:24:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000f690 con 0x7f6d880b5c70 2024-09-17T03:24:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d88153c30 con 0x7f6d880b5c70 2024-09-17T03:24:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d88153150 con 0x7f6d880b5c70 2024-09-17T03:24:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.004+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d881fcec0 con 0x7f6d880b5c70 2024-09-17T03:24:07.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.008+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6d7000f830 con 0x7f6d880b5c70 2024-09-17T03:24:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.008+0000 7f6d85600640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d64073560 0x7f6d64075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.008+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f6d70095030 con 0x7f6d880b5c70 2024-09-17T03:24:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.008+0000 7f6d8ca00640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d64073560 0x7f6d64075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.008+0000 7f6d8ca00640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d64073560 0x7f6d64075a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d780af000 tx=0x7f6d7801b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.012+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d70014030 con 0x7f6d880b5c70 2024-09-17T03:24:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.220+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d8815bb60 con 0x7f6d880b5c70 2024-09-17T03:24:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d85600640 1 -- 172.21.15.46:0/625333537 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f6d7005fb80 con 0x7f6d880b5c70 2024-09-17T03:24:07.224 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:24:07.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d64073560 msgr2=0x7f6d64075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:07.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d64073560 0x7f6d64075a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d780af000 tx=0x7f6d7801b040 comp rx=0 tx=0).stop 2024-09-17T03:24:07.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 msgr2=0x7f6d881fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d881fc3d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c9f0 tx=0x7f6d7000ceb0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 shutdown_connections 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d64073560 0x7f6d64075a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d881527b0 0x7f6d881fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880b5c70 0x7f6d881fc3d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 --2- 172.21.15.46:0/625333537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d880b4de0 0x7f6d88141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 >> 172.21.15.46:0/625333537 conn(0x7f6d880b2040 msgr2=0x7f6d880cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 shutdown_connections 2024-09-17T03:24:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07.224+0000 7f6d8f400640 1 -- 172.21.15.46:0/625333537 wait complete. 2024-09-17T03:24:07.246 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:24:07.247 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:24:07.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.247 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:24:07.257 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:24:07.257 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:24:07.257 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:24:07.257 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.257 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42241sec preferred_lft 42241sec 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.258 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.259 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:24:07.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.261 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:24:07.318 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:24:07.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.320 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:24:07.372 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:24:07.373 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:24:07.373 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:24:07.373 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:24:07.373 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:24:07.373 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:24:07.373 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:24:07.374 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:24:07.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:24:07.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:24:07.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:24:07.374 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:24:07.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:24:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:24:07.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.512 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:24:07.521 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:24:07.521 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:24:07.522 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:24:07.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:24:07.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:24:07.584 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:24:07.636 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:24:07.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:24:07.636 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:24:07.691 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:24:07.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:24:07.793 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:24:07.793 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:24:07.809 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:24:07.809 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:24:07.809 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:24:07.809 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:24:07.809 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:24:07.813 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:24:07.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:07.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:24:07.834 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:24:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:24:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:24:07.940 INFO:teuthology.orchestra.run.smithi046.stdout:client245036 2024-09-17T03:24:07.941 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:07.941 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:07.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:24:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 -- 172.21.15.46:0/4015984042 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c0b5c70 msgr2=0x7f3d4c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 --2- 172.21.15.46:0/4015984042 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c0b5c70 0x7f3d4c15bb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009870 tx=0x7f3d3c01c6c0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 -- 172.21.15.46:0/4015984042 shutdown_connections 2024-09-17T03:24:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 --2- 172.21.15.46:0/4015984042 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c0b5c70 0x7f3d4c15bb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 -- 172.21.15.46:0/4015984042 >> 172.21.15.46:0/4015984042 conn(0x7f3d4c0b2040 msgr2=0x7f3d4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:08.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 -- 172.21.15.46:0/4015984042 shutdown_connections 2024-09-17T03:24:08.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.152+0000 7f3d51a00640 1 -- 172.21.15.46:0/4015984042 wait complete. 2024-09-17T03:24:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 Processor -- start 2024-09-17T03:24:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 -- start start 2024-09-17T03:24:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d4c0b4de0 0x7f3d4c1fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 0x7f3d4c0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c1527b0 0x7f3d4c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c169cb0 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c169b30 con 0x7f3d4c0b4de0 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4c169e30 con 0x7f3d4c1527b0 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 0x7f3d4c0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 0x7f3d4c0aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32938/0 (socket says 172.21.15.46:32938) 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 -- 172.21.15.46:0/1447110437 learned_addr learned my addr 172.21.15.46:0/1447110437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d4b400640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c1527b0 0x7f3d4c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d4aa00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d4c0b4de0 0x7f3d4c1fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 -- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c1527b0 msgr2=0x7f3d4c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c1527b0 0x7f3d4c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 -- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d4c0b4de0 msgr2=0x7f3d4c1fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d4c0b4de0 0x7f3d4c1fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 -- 172.21.15.46:0/1447110437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c1f88c0 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d43e00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 0x7f3d4c0aec40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c4b0 tx=0x7f3d3400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34016020 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d34012920 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4c1fcc90 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34012d50 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.156+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4c1f8e60 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.160+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4c1fca00 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.160+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f3d34019020 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.160+0000 7f3d42a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d28073590 0x7f3d28075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.160+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f3d340944b0 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.160+0000 7f3d4aa00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d28073590 0x7f3d28075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.160+0000 7f3d4aa00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d28073590 0x7f3d28075aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0af000 tx=0x7f3d3c01af00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.164+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3d34020030 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.376+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3d4c15bb60 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.720+0000 7f3d42a00640 1 -- 172.21.15.46:0/1447110437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f3d3405e8a0 con 0x7f3d4c0b5c70 2024-09-17T03:24:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d28073590 msgr2=0x7f3d28075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d28073590 0x7f3d28075aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0af000 tx=0x7f3d3c01af00 comp rx=0 tx=0).stop 2024-09-17T03:24:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 msgr2=0x7f3d4c0aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 0x7f3d4c0aec40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c4b0 tx=0x7f3d3400c970 comp rx=0 tx=0).stop 2024-09-17T03:24:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 shutdown_connections 2024-09-17T03:24:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3d28073590 0x7f3d28075aa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3d4c1527b0 0x7f3d4c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d4c0b5c70 0x7f3d4c0aec40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 --2- 172.21.15.46:0/1447110437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d4c0b4de0 0x7f3d4c1fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 >> 172.21.15.46:0/1447110437 conn(0x7f3d4c0b2040 msgr2=0x7f3d4c151840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 shutdown_connections 2024-09-17T03:24:08.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.724+0000 7f3d51a00640 1 -- 172.21.15.46:0/1447110437 wait complete. 2024-09-17T03:24:08.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 -- 172.21.15.46:0/558421712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00749a0 msgr2=0x7f41f0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 --2- 172.21.15.46:0/558421712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00749a0 0x7f41f0074db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009900 tx=0x7f41e001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 -- 172.21.15.46:0/558421712 shutdown_connections 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 --2- 172.21.15.46:0/558421712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00749a0 0x7f41f0074db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 -- 172.21.15.46:0/558421712 >> 172.21.15.46:0/558421712 conn(0x7f41f00b2040 msgr2=0x7f41f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 -- 172.21.15.46:0/558421712 shutdown_connections 2024-09-17T03:24:08.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 -- 172.21.15.46:0/558421712 wait complete. 2024-09-17T03:24:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 Processor -- start 2024-09-17T03:24:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.904+0000 7f41f6000640 1 -- start start 2024-09-17T03:24:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f00749a0 0x7f41f013e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 0x7f41f0144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41f00cefa0 0x7f41f01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41f016b350 con 0x7f41f00b3340 2024-09-17T03:24:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41f016b1d0 con 0x7f41f00749a0 2024-09-17T03:24:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f41f016b4d0 con 0x7f41f00cefa0 2024-09-17T03:24:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 0x7f41f0144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 0x7f41f0144070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32958/0 (socket says 172.21.15.46:32958) 2024-09-17T03:24:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 -- 172.21.15.46:0/1704227700 learned_addr learned my addr 172.21.15.46:0/1704227700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41efe00640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41f00cefa0 0x7f41f01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41ef400640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f00749a0 0x7f41f013e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 -- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41f00cefa0 msgr2=0x7f41f01445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41f00cefa0 0x7f41f01445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 -- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f00749a0 msgr2=0x7f41f013e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f00749a0 0x7f41f013e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 -- 172.21.15.46:0/1704227700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f01fd020 con 0x7f41f00b3340 2024-09-17T03:24:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41efe00640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41f00cefa0 0x7f41f01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:24:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7e00640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 0x7f41f0144070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c3a0 tx=0x7f41d800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8017070 con 0x7f41f00b3340 2024-09-17T03:24:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d8003600 con 0x7f41f00b3340 2024-09-17T03:24:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8003a50 con 0x7f41f00b3340 2024-09-17T03:24:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f01fc410 con 0x7f41f00b3340 2024-09-17T03:24:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f01fc9d0 con 0x7f41f00b3340 2024-09-17T03:24:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.908+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f0074db0 con 0x7f41f00b3340 2024-09-17T03:24:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.912+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f41d801a070 con 0x7f41f00b3340 2024-09-17T03:24:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.912+0000 7f41e7400640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41cc073590 0x7f41cc075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.912+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f41d8093b40 con 0x7f41f00b3340 2024-09-17T03:24:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.912+0000 7f41ef400640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41cc073590 0x7f41cc075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.912+0000 7f41ef400640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41cc073590 0x7f41cc075aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41e00af000 tx=0x7f41e001b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:08.916+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41d805e690 con 0x7f41f00b3340 2024-09-17T03:24:09.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.112+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f41f006b540 con 0x7f41cc073590 2024-09-17T03:24:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41e7400640 1 -- 172.21.15.46:0/1704227700 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f41e0004040 con 0x7f41cc073590 2024-09-17T03:24:09.117 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:24:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:24:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:24:09.117 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:24:09.117 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41cc073590 msgr2=0x7f41cc075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41cc073590 0x7f41cc075aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41e00af000 tx=0x7f41e001b120 comp rx=0 tx=0).stop 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 msgr2=0x7f41f0144070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 0x7f41f0144070 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c3a0 tx=0x7f41d800c860 comp rx=0 tx=0).stop 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 shutdown_connections 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41cc073590 0x7f41cc075aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41f00cefa0 0x7f41f01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41f00b3340 0x7f41f0144070 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 --2- 172.21.15.46:0/1704227700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41f00749a0 0x7f41f013e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 >> 172.21.15.46:0/1704227700 conn(0x7f41f00b2040 msgr2=0x7f41f015bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 shutdown_connections 2024-09-17T03:24:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.116+0000 7f41f6000640 1 -- 172.21.15.46:0/1704227700 wait complete. 2024-09-17T03:24:09.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:24:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 -- 172.21.15.46:0/4232827185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 msgr2=0x7f7d3015b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 --2- 172.21.15.46:0/4232827185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d3015b980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009c80 tx=0x7f7d2801cd10 comp rx=0 tx=0).stop 2024-09-17T03:24:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 -- 172.21.15.46:0/4232827185 shutdown_connections 2024-09-17T03:24:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 --2- 172.21.15.46:0/4232827185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d3015b980 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 --2- 172.21.15.46:0/4232827185 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7d300cf900 0x7f7d3015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 -- 172.21.15.46:0/4232827185 >> 172.21.15.46:0/4232827185 conn(0x7f7d300b2040 msgr2=0x7f7d300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 -- 172.21.15.46:0/4232827185 shutdown_connections 2024-09-17T03:24:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.300+0000 7f7d37000640 1 -- 172.21.15.46:0/4232827185 wait complete. 2024-09-17T03:24:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 Processor -- start 2024-09-17T03:24:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 -- start start 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d300cf900 0x7f7d3013ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d30074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7d3015ee00 0x7f7d30075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300b5020 con 0x7f7d3015e190 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300b4ea0 con 0x7f7d300cf900 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d37000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7d300b51a0 con 0x7f7d3015ee00 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d30074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d30074ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32976/0 (socket says 172.21.15.46:32976) 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d27e00640 1 -- 172.21.15.46:0/3925890803 learned_addr learned my addr 172.21.15.46:0/3925890803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:09.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d35000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7d3015ee00 0x7f7d30075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.304+0000 7f7d2fe00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d300cf900 0x7f7d3013ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d27e00640 1 -- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7d3015ee00 msgr2=0x7f7d30075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d27e00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7d3015ee00 0x7f7d30075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d27e00640 1 -- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d300cf900 msgr2=0x7f7d3013ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d27e00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d300cf900 0x7f7d3013ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d27e00640 1 -- 172.21.15.46:0/3925890803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d300cdfb0 con 0x7f7d3015e190 2024-09-17T03:24:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d27e00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d30074ae0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c430 tx=0x7f7d1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18017070 con 0x7f7d3015e190 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d180053c0 con 0x7f7d3015e190 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d300ce3b0 con 0x7f7d3015e190 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18003860 con 0x7f7d3015e190 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3013f4f0 con 0x7f7d3015e190 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d300cec20 con 0x7f7d3015e190 2024-09-17T03:24:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.308+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f7d1801a070 con 0x7f7d3015e190 2024-09-17T03:24:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.312+0000 7f7d2cc00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d0c073560 0x7f7d0c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.312+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7d18093fc0 con 0x7f7d3015e190 2024-09-17T03:24:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.312+0000 7f7d2fe00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d0c073560 0x7f7d0c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.312+0000 7f7d2fe00640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d0c073560 0x7f7d0c075a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000bb50 tx=0x7f7d2000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:09.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.312+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d300cec20 con 0x7f7d3015e190 2024-09-17T03:24:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7d300cf040 con 0x7f7d3015e190 2024-09-17T03:24:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d2cc00640 1 -- 172.21.15.46:0/3925890803 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f7d300cf040 con 0x7f7d3015e190 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d0c073560 msgr2=0x7f7d0c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d0c073560 0x7f7d0c075a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000bb50 tx=0x7f7d2000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 msgr2=0x7f7d30074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d30074ae0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c430 tx=0x7f7d1800c8f0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 shutdown_connections 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d0c073560 0x7f7d0c075a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7d3015ee00 0x7f7d30075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d3015e190 0x7f7d30074ae0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 --2- 172.21.15.46:0/3925890803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d300cf900 0x7f7d3013ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 >> 172.21.15.46:0/3925890803 conn(0x7f7d300b2040 msgr2=0x7f7d30159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 shutdown_connections 2024-09-17T03:24:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.488+0000 7f7d37000640 1 -- 172.21.15.46:0/3925890803 wait complete. 2024-09-17T03:24:09.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:24:09.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 -- 172.21.15.46:0/951277713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b4de0 msgr2=0x7f5b080b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 --2- 172.21.15.46:0/951277713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b4de0 0x7f5b080b51f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009c80 tx=0x7f5b0001cce0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 -- 172.21.15.46:0/951277713 shutdown_connections 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 --2- 172.21.15.46:0/951277713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b4de0 0x7f5b080b51f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 --2- 172.21.15.46:0/951277713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b080b5c70 0x7f5b0815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 -- 172.21.15.46:0/951277713 >> 172.21.15.46:0/951277713 conn(0x7f5b080b2040 msgr2=0x7f5b080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 -- 172.21.15.46:0/951277713 shutdown_connections 2024-09-17T03:24:09.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 -- 172.21.15.46:0/951277713 wait complete. 2024-09-17T03:24:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 Processor -- start 2024-09-17T03:24:09.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.672+0000 7f5b0e600640 1 -- start start 2024-09-17T03:24:09.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b080b4de0 0x7f5b0814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 0x7f5b080cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b081527b0 0x7f5b080cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08168a00 con 0x7f5b080b5c70 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08168880 con 0x7f5b080b4de0 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5b08168b80 con 0x7f5b081527b0 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 0x7f5b080cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 0x7f5b080cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32996/0 (socket says 172.21.15.46:32996) 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 -- 172.21.15.46:0/1157669993 learned_addr learned my addr 172.21.15.46:0/1157669993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b07e00640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b081527b0 0x7f5b080cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b07400640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b080b4de0 0x7f5b0814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 -- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b081527b0 msgr2=0x7f5b080cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b081527b0 0x7f5b080cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 -- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b080b4de0 msgr2=0x7f5b0814ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b080b4de0 0x7f5b0814ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 -- 172.21.15.46:0/1157669993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b080ad0c0 con 0x7f5b080b5c70 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5affe00640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 0x7f5b080cc950 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c3c0 tx=0x7f5af000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0017070 con 0x7f5b080b5c70 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5af00053c0 con 0x7f5b080b5c70 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b080cd660 con 0x7f5b080b5c70 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0003840 con 0x7f5b080b5c70 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b080ce8b0 con 0x7f5b080b5c70 2024-09-17T03:24:09.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.676+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b080b51f0 con 0x7f5b080b5c70 2024-09-17T03:24:09.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.680+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5af001a070 con 0x7f5b080b5c70 2024-09-17T03:24:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.680+0000 7f5aff400640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ae4073590 0x7f5ae4075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.680+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f5af0093ac0 con 0x7f5b080b5c70 2024-09-17T03:24:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.680+0000 7f5b07400640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ae4073590 0x7f5ae4075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:09.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.680+0000 7f5b07400640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ae4073590 0x7f5ae4075aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5af800bb50 tx=0x7f5af800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:09.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.684+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5b080b5730 con 0x7f5b080b5c70 2024-09-17T03:24:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f5b080cd3f0 con 0x7f5b080b5c70 2024-09-17T03:24:09.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5aff400640 1 -- 172.21.15.46:0/1157669993 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v35) ==== 116+0+2 (secure 0 0 0) 0x7f5af005df40 con 0x7f5b080b5c70 2024-09-17T03:24:09.861 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:24:09.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ae4073590 msgr2=0x7f5ae4075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ae4073590 0x7f5ae4075aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5af800bb50 tx=0x7f5af800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 msgr2=0x7f5b080cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 0x7f5b080cc950 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c3c0 tx=0x7f5af000c880 comp rx=0 tx=0).stop 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 shutdown_connections 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ae4073590 0x7f5ae4075aa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5b081527b0 0x7f5b080cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b080b5c70 0x7f5b080cc950 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 --2- 172.21.15.46:0/1157669993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b080b4de0 0x7f5b0814ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 >> 172.21.15.46:0/1157669993 conn(0x7f5b080b2040 msgr2=0x7f5b080b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 shutdown_connections 2024-09-17T03:24:09.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:09.860+0000 7f5b0e600640 1 -- 172.21.15.46:0/1157669993 wait complete. 2024-09-17T03:24:09.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:24:10.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 -- 172.21.15.46:0/3450054872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b54f0 msgr2=0x7fac7c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 --2- 172.21.15.46:0/3450054872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b54f0 0x7fac7c15bb60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c80 tx=0x7fac7401cce0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 -- 172.21.15.46:0/3450054872 shutdown_connections 2024-09-17T03:24:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 --2- 172.21.15.46:0/3450054872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b54f0 0x7fac7c15bb60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 -- 172.21.15.46:0/3450054872 >> 172.21.15.46:0/3450054872 conn(0x7fac7c0b2040 msgr2=0x7fac7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 -- 172.21.15.46:0/3450054872 shutdown_connections 2024-09-17T03:24:10.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.044+0000 7fac80c00640 1 -- 172.21.15.46:0/3450054872 wait complete. 2024-09-17T03:24:10.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 Processor -- start 2024-09-17T03:24:10.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 -- start start 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 0x7fac7c149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac7c1527b0 0x7fac7c14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c1fcb20 0x7fac7c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c169bd0 con 0x7fac7c0b5c70 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c169a50 con 0x7fac7c1fcb20 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fac7c169d50 con 0x7fac7c1527b0 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 0x7fac7c149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 0x7fac7c149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33016/0 (socket says 172.21.15.46:33016) 2024-09-17T03:24:10.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 -- 172.21.15.46:0/1892551903 learned_addr learned my addr 172.21.15.46:0/1892551903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac71400640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac7c1527b0 0x7fac7c14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac7a800640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c1fcb20 0x7fac7c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 -- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac7c1527b0 msgr2=0x7fac7c14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac7c1527b0 0x7fac7c14c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 -- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c1fcb20 msgr2=0x7fac7c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c1fcb20 0x7fac7c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 -- 172.21.15.46:0/1892551903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c06f6e0 con 0x7fac7c0b5c70 2024-09-17T03:24:10.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac79e00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 0x7fac7c149720 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b440 tx=0x7fac6c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c015020 con 0x7fac7c0b5c70 2024-09-17T03:24:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac6c011980 con 0x7fac7c0b5c70 2024-09-17T03:24:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c149c60 con 0x7fac7c0b5c70 2024-09-17T03:24:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c011dd0 con 0x7fac7c0b5c70 2024-09-17T03:24:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.048+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c14d400 con 0x7fac7c0b5c70 2024-09-17T03:24:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.052+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7c15bb60 con 0x7fac7c0b5c70 2024-09-17T03:24:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.052+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fac6c018020 con 0x7fac7c0b5c70 2024-09-17T03:24:10.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.052+0000 7fac72a00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fac580734d0 0x7fac580759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.052+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fac6c0946a0 con 0x7fac7c0b5c70 2024-09-17T03:24:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.052+0000 7fac71400640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fac580734d0 0x7fac580759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.056+0000 7fac71400640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fac580734d0 0x7fac580759e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0cf680 tx=0x7fac64009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.060+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac6c01f030 con 0x7fac7c0b5c70 2024-09-17T03:24:10.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.232+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fac7c0b25f0 con 0x7fac7c0b5c70 2024-09-17T03:24:10.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.236+0000 7fac72a00640 1 -- 172.21.15.46:0/1892551903 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v36)=0 v36) ==== 131+0+0 (secure 0 0 0) 0x7fac6c00d040 con 0x7fac7c0b5c70 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fac580734d0 msgr2=0x7fac580759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fac580734d0 0x7fac580759e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0cf680 tx=0x7fac64009040 comp rx=0 tx=0).stop 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 msgr2=0x7fac7c149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 0x7fac7c149720 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00b440 tx=0x7fac6c00b900 comp rx=0 tx=0).stop 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 shutdown_connections 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fac580734d0 0x7fac580759e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac7c1fcb20 0x7fac7c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac7c1527b0 0x7fac7c14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 --2- 172.21.15.46:0/1892551903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac7c0b5c70 0x7fac7c149720 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 >> 172.21.15.46:0/1892551903 conn(0x7fac7c0b2040 msgr2=0x7fac7c14df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 shutdown_connections 2024-09-17T03:24:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.240+0000 7fac80c00640 1 -- 172.21.15.46:0/1892551903 wait complete. 2024-09-17T03:24:10.253 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.252+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.253 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.252+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.253 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.252+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.253 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.252+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.253 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.252+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:24:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 -- 172.21.15.46:0/4172979433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2741536f0 msgr2=0x7ff27415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 --2- 172.21.15.46:0/4172979433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2741536f0 0x7ff27415d480 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff264009840 tx=0x7ff26401c620 comp rx=0 tx=0).stop 2024-09-17T03:24:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 -- 172.21.15.46:0/4172979433 shutdown_connections 2024-09-17T03:24:10.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 --2- 172.21.15.46:0/4172979433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2741536f0 0x7ff27415d480 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 -- 172.21.15.46:0/4172979433 >> 172.21.15.46:0/4172979433 conn(0x7ff2740b4560 msgr2=0x7ff2740d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 -- 172.21.15.46:0/4172979433 shutdown_connections 2024-09-17T03:24:10.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.424+0000 7ff273e00640 1 -- 172.21.15.46:0/4172979433 wait complete. 2024-09-17T03:24:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 Processor -- start 2024-09-17T03:24:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 -- start start 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2741536f0 0x7ff2741fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff274160060 0x7ff27413c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 0x7ff274150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff27416b600 con 0x7ff27414dbd0 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff27416b480 con 0x7ff274160060 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff27416b780 con 0x7ff2741536f0 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 0x7ff274150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 0x7ff274150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33040/0 (socket says 172.21.15.46:33040) 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 -- 172.21.15.46:0/4070613059 learned_addr learned my addr 172.21.15.46:0/4070613059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff272a00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2741536f0 0x7ff2741fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff26be00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff274160060 0x7ff27413c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 -- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2741536f0 msgr2=0x7ff2741fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2741536f0 0x7ff2741fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 -- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff274160060 msgr2=0x7ff27413c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff274160060 0x7ff27413c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.428+0000 7ff273400640 1 -- 172.21.15.46:0/4070613059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27414e810 con 0x7ff27414dbd0 2024-09-17T03:24:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff273400640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 0x7ff274150390 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00c730 tx=0x7ff26c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c003b80 con 0x7ff27414dbd0 2024-09-17T03:24:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff26c003d40 con 0x7ff27414dbd0 2024-09-17T03:24:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0052d0 con 0x7ff27414dbd0 2024-09-17T03:24:10.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27414eaf0 con 0x7ff27414dbd0 2024-09-17T03:24:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27413bbf0 con 0x7ff27414dbd0 2024-09-17T03:24:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27415d480 con 0x7ff27414dbd0 2024-09-17T03:24:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7ff26c005470 con 0x7ff27414dbd0 2024-09-17T03:24:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff26aa00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff250073560 0x7ff250075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.432+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7ff26c0948e0 con 0x7ff27414dbd0 2024-09-17T03:24:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.436+0000 7ff272a00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff250073560 0x7ff250075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.436+0000 7ff272a00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff250073560 0x7ff250075a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff2640ae000 tx=0x7ff26401ae60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:10.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.440+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff26c05f120 con 0x7ff27414dbd0 2024-09-17T03:24:10.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.640+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7ff27406b540 con 0x7ff250073560 2024-09-17T03:24:10.847 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.844+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.847 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.844+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.847 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.844+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.847 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.844+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.847 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:24:10.844+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:24:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.884+0000 7ff26aa00640 1 -- 172.21.15.46:0/4070613059 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2640af040 con 0x7ff250073560 2024-09-17T03:24:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff250073560 msgr2=0x7ff250075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff250073560 0x7ff250075a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff2640ae000 tx=0x7ff26401ae60 comp rx=0 tx=0).stop 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 msgr2=0x7ff274150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 0x7ff274150390 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00c730 tx=0x7ff26c00cbf0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 shutdown_connections 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff250073560 0x7ff250075a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27414dbd0 0x7ff274150390 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff274160060 0x7ff27413c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 --2- 172.21.15.46:0/4070613059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2741536f0 0x7ff2741fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 >> 172.21.15.46:0/4070613059 conn(0x7ff2740b4560 msgr2=0x7ff2740b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 shutdown_connections 2024-09-17T03:24:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:10.888+0000 7ff273e00640 1 -- 172.21.15.46:0/4070613059 wait complete. 2024-09-17T03:24:10.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:24:11.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 -- 172.21.15.46:0/2184846711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 msgr2=0x7f26b00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 --2- 172.21.15.46:0/2184846711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b00b51f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26b0071cb0 tx=0x7f26a801ccc0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 -- 172.21.15.46:0/2184846711 shutdown_connections 2024-09-17T03:24:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 --2- 172.21.15.46:0/2184846711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b00b51f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 --2- 172.21.15.46:0/2184846711 >> v2:172.21.15.99:3300/0 conn(0x7f26b00b5c70 0x7f26b015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 -- 172.21.15.46:0/2184846711 >> 172.21.15.46:0/2184846711 conn(0x7f26b00b2040 msgr2=0x7f26b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 -- 172.21.15.46:0/2184846711 shutdown_connections 2024-09-17T03:24:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.068+0000 7f26b4c00640 1 -- 172.21.15.46:0/2184846711 wait complete. 2024-09-17T03:24:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 Processor -- start 2024-09-17T03:24:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 -- start start 2024-09-17T03:24:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b01f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26b00b5c70 0x7f26b014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b01527b0 0x7f26b015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0168920 con 0x7f26b00b4de0 2024-09-17T03:24:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01687a0 con 0x7f26b01527b0 2024-09-17T03:24:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26b4c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f26b0168aa0 con 0x7f26b00b5c70 2024-09-17T03:24:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.072+0000 7f26ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b01f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b01f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33082/0 (socket says 172.21.15.46:33082) 2024-09-17T03:24:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 -- 172.21.15.46:0/3563026296 learned_addr learned my addr 172.21.15.46:0/3563026296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:24:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26a5400640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26b00b5c70 0x7f26b014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ae800640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b01527b0 0x7f26b015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 -- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26b00b5c70 msgr2=0x7f26b014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26b00b5c70 0x7f26b014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 -- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b01527b0 msgr2=0x7f26b015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b01527b0 0x7f26b015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 -- 172.21.15.46:0/3563026296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b00cdf00 con 0x7f26b00b4de0 2024-09-17T03:24:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26ade00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b01f6740 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f26a000d750 tx=0x7f26a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0017070 con 0x7f26b00b4de0 2024-09-17T03:24:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26a0009d20 con 0x7f26b00b4de0 2024-09-17T03:24:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b00ce130 con 0x7f26b00b4de0 2024-09-17T03:24:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a000de60 con 0x7f26b00b4de0 2024-09-17T03:24:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b00cd2c0 con 0x7f26b00b4de0 2024-09-17T03:24:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b00b51f0 con 0x7f26b00b4de0 2024-09-17T03:24:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.076+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f26a0004280 con 0x7f26b00b4de0 2024-09-17T03:24:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.080+0000 7f26a6a00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f268c073560 0x7f268c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:24:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.080+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f26a0094bd0 con 0x7f26b00b4de0 2024-09-17T03:24:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.080+0000 7f26a5400640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f268c073560 0x7f268c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:24:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.080+0000 7f26a5400640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f268c073560 0x7f268c075a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26b00afa20 tx=0x7f2698009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:24:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.084+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26a0020020 con 0x7f26b00b4de0 2024-09-17T03:24:11.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.284+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f26b006b540 con 0x7f268c073560 2024-09-17T03:24:11.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.288+0000 7f26a6a00640 1 -- 172.21.15.46:0/3563026296 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2698001d20 con 0x7f268c073560 2024-09-17T03:24:11.290 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:24:11.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f268c073560 msgr2=0x7f268c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:11.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f268c073560 0x7f268c075a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26b00afa20 tx=0x7f2698009040 comp rx=0 tx=0).stop 2024-09-17T03:24:11.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 msgr2=0x7f26b01f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:24:11.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b01f6740 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f26a000d750 tx=0x7f26a000dc10 comp rx=0 tx=0).stop 2024-09-17T03:24:11.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 shutdown_connections 2024-09-17T03:24:11.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f268c073560 0x7f268c075a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b01527b0 0x7f26b015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26b00b5c70 0x7f26b014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 --2- 172.21.15.46:0/3563026296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b00b4de0 0x7f26b01f6740 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:24:11.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 >> 172.21.15.46:0/3563026296 conn(0x7f26b00b2040 msgr2=0x7f26b0151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:24:11.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 shutdown_connections 2024-09-17T03:24:11.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:24:11.292+0000 7f26b4c00640 1 -- 172.21.15.46:0/3563026296 wait complete. 2024-09-17T03:24:11.314 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:24:11.314 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:11.314 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:11.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:11.324 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:24:19.658 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:24:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:24:19.659 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.2772 s, 130 MB/s 2024-09-17T03:24:19.660 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:19.660 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:19.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:19.669 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:24:28.086 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:24:28.087 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:24:28.087 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.36552 s, 128 MB/s 2024-09-17T03:24:28.088 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:28.088 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:28.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:28.098 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:24:29.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:24:29.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:24:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:24:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:24:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.48883 s, 126 MB/s 2024-09-17T03:24:36.639 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:36.639 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:36.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:36.649 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:24:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:24:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:24:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.55481 s, 126 MB/s 2024-09-17T03:24:45.257 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:45.257 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:45.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:45.267 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:24:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:24:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:24:53.762 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.43697 s, 127 MB/s 2024-09-17T03:24:53.764 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:24:53.764 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:24:53.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:24:53.773 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:24:59.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:24:59.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:25:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:25:02.577 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:25:02.578 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.75317 s, 123 MB/s 2024-09-17T03:25:02.579 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:25:02.579 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:25:02.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:02.588 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:25:10.990 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:25:10.990 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:25:10.990 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.34911 s, 129 MB/s 2024-09-17T03:25:10.992 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:25:10.992 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:25:11.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:11.001 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:25:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:25:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:25:19.670 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.61675 s, 125 MB/s 2024-09-17T03:25:19.671 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:25:19.671 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:25:19.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:19.680 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:25:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:25:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:25:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.25442 s, 130 MB/s 2024-09-17T03:25:27.992 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:25:27.992 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:25:28.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:28.001 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:25:29.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:25:29.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:25:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:25:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:25:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.68753 s, 124 MB/s 2024-09-17T03:25:36.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:25:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 -- 172.21.15.46:0/1024681472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815e290 msgr2=0x7fcdb815bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 --2- 172.21.15.46:0/1024681472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815e290 0x7fcdb815bdf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009bc0 tx=0x7fcdb001cba0 comp rx=0 tx=0).stop 2024-09-17T03:25:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 -- 172.21.15.46:0/1024681472 shutdown_connections 2024-09-17T03:25:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 --2- 172.21.15.46:0/1024681472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815e290 0x7fcdb815bdf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 --2- 172.21.15.46:0/1024681472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb815e8c0 0x7fcdb815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 -- 172.21.15.46:0/1024681472 >> 172.21.15.46:0/1024681472 conn(0x7fcdb80b2040 msgr2=0x7fcdb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 -- 172.21.15.46:0/1024681472 shutdown_connections 2024-09-17T03:25:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.908+0000 7fcdbf600640 1 -- 172.21.15.46:0/1024681472 wait complete. 2024-09-17T03:25:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 Processor -- start 2024-09-17T03:25:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 -- start start 2024-09-17T03:25:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 0x7fcdb813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb815e290 0x7fcdb8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb815e8c0 0x7fcdb8141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8169340 con 0x7fcdb815b2d0 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81691c0 con 0x7fcdb815e8c0 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbf600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb81694c0 con 0x7fcdb815e290 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 0x7fcdb813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 0x7fcdb813c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41224/0 (socket says 172.21.15.46:41224) 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbcc00640 1 -- 172.21.15.46:0/2190034559 learned_addr learned my addr 172.21.15.46:0/2190034559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdb7e00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb815e290 0x7fcdb8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.912+0000 7fcdbd600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb815e8c0 0x7fcdb8141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbcc00640 1 -- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb815e290 msgr2=0x7fcdb8140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbcc00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb815e290 0x7fcdb8140e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbcc00640 1 -- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb815e8c0 msgr2=0x7fcdb8141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbcc00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb815e8c0 0x7fcdb8141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbcc00640 1 -- 172.21.15.46:0/2190034559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb8140650 con 0x7fcdb815b2d0 2024-09-17T03:25:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbcc00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 0x7fcdb813c250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c9f0 tx=0x7fcdac00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac007bb0 con 0x7fcdb815b2d0 2024-09-17T03:25:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdac007e40 con 0x7fcdb815b2d0 2024-09-17T03:25:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00f690 con 0x7fcdb815b2d0 2024-09-17T03:25:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb80c60a0 con 0x7fcdb815b2d0 2024-09-17T03:25:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb80c6660 con 0x7fcdb815b2d0 2024-09-17T03:25:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb815ecd0 con 0x7fcdb815b2d0 2024-09-17T03:25:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.916+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 14) ==== 97540+0+0 (secure 0 0 0) 0x7fcdac00f830 con 0x7fcdb815b2d0 2024-09-17T03:25:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.920+0000 7fcdb4c00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98073560 0x7fcd98075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.920+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fcdac094f40 con 0x7fcdb815b2d0 2024-09-17T03:25:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.920+0000 7fcdb7e00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98073560 0x7fcd98075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.920+0000 7fcdb7e00640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98073560 0x7fcd98075a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80b2a70 tx=0x7fcda4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:36.924+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcdac014030 con 0x7fcdb815b2d0 2024-09-17T03:25:37.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.124+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fcdb80cfc10 con 0x7fcd98073560 2024-09-17T03:25:37.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:37.128+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:37.131 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:37.128+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:37.131 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:37.128+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:37.131 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:37.128+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:37.131 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:37.128+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:37.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.152+0000 7fcdb4c00640 1 -- 172.21.15.46:0/2190034559 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcdb80cfc10 con 0x7fcd98073560 2024-09-17T03:25:37.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98073560 msgr2=0x7fcd98075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98073560 0x7fcd98075a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80b2a70 tx=0x7fcda4009040 comp rx=0 tx=0).stop 2024-09-17T03:25:37.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 msgr2=0x7fcdb813c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 0x7fcdb813c250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00c9f0 tx=0x7fcdac00ceb0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 shutdown_connections 2024-09-17T03:25:37.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98073560 0x7fcd98075a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb815e8c0 0x7fcdb8141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb815e290 0x7fcdb8140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 --2- 172.21.15.46:0/2190034559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb815b2d0 0x7fcdb813c250 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 >> 172.21.15.46:0/2190034559 conn(0x7fcdb80b2040 msgr2=0x7fcdb80747c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:37.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 shutdown_connections 2024-09-17T03:25:37.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.156+0000 7fcdbf600640 1 -- 172.21.15.46:0/2190034559 wait complete. 2024-09-17T03:25:37.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 -- 172.21.15.46:0/2856655773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f815efc0 msgr2=0x7f43f8070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 --2- 172.21.15.46:0/2856655773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f815efc0 0x7f43f8070910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009840 tx=0x7f43ec01c820 comp rx=0 tx=0).stop 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 -- 172.21.15.46:0/2856655773 shutdown_connections 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 --2- 172.21.15.46:0/2856655773 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43f80b0810 0x7f43f8068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 --2- 172.21.15.46:0/2856655773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f815efc0 0x7f43f8070910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 -- 172.21.15.46:0/2856655773 >> 172.21.15.46:0/2856655773 conn(0x7f43f80b35f0 msgr2=0x7f43f80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 -- 172.21.15.46:0/2856655773 shutdown_connections 2024-09-17T03:25:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.344+0000 7f43ff600640 1 -- 172.21.15.46:0/2856655773 wait complete. 2024-09-17T03:25:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 Processor -- start 2024-09-17T03:25:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 -- start start 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 0x7f43f81fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43f80b0c20 0x7f43f81f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43f815efc0 0x7f43f813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43f816cae0 con 0x7f43f80b0810 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43f816c960 con 0x7f43f80b0c20 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f43f816cc60 con 0x7f43f815efc0 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 0x7f43f81fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 0x7f43f81fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41254/0 (socket says 172.21.15.46:41254) 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 -- 172.21.15.46:0/1483273051 learned_addr learned my addr 172.21.15.46:0/1483273051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43f7e00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43f80b0c20 0x7f43f81f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fd600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43f815efc0 0x7f43f813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 -- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43f815efc0 msgr2=0x7f43f813cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43f815efc0 0x7f43f813cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 -- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43f80b0c20 msgr2=0x7f43f81f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43f80b0c20 0x7f43f81f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 -- 172.21.15.46:0/1483273051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f8150cf0 con 0x7f43f80b0810 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43fcc00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 0x7f43f81fd340 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f43ec01aed0 tx=0x7f43ec0a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0bb070 con 0x7f43f80b0810 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43ec005030 con 0x7f43f80b0810 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0b2040 con 0x7f43f80b0810 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.348+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f8150fd0 con 0x7f43f80b0810 2024-09-17T03:25:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.352+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f81fce20 con 0x7f43f80b0810 2024-09-17T03:25:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.352+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f8068df0 con 0x7f43f80b0810 2024-09-17T03:25:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.352+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 97591+0+0 (secure 0 0 0) 0x7f43ec005460 con 0x7f43f80b0810 2024-09-17T03:25:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.352+0000 7f43f4c00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f43d8073620 0x7f43d8075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.352+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f43ec12e280 con 0x7f43f80b0810 2024-09-17T03:25:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.352+0000 7f43f7e00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f43d8073620 0x7f43d8075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.356+0000 7f43f7e00640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f43d8073620 0x7f43d8075b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43f80b6900 tx=0x7f43e400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.356+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43ec0f1db0 con 0x7f43f80b0810 2024-09-17T03:25:37.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.560+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f43f806b540 con 0x7f43d8073620 2024-09-17T03:25:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.564+0000 7f43f4c00640 1 -- 172.21.15.46:0/1483273051 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f43e400b780 con 0x7f43d8073620 2024-09-17T03:25:37.567 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f43d8073620 msgr2=0x7f43d8075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f43d8073620 0x7f43d8075b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f43f80b6900 tx=0x7f43e400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 msgr2=0x7f43f81fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 0x7f43f81fd340 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f43ec01aed0 tx=0x7f43ec0a5700 comp rx=0 tx=0).stop 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 shutdown_connections 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f43d8073620 0x7f43d8075b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43f815efc0 0x7f43f813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43f80b0c20 0x7f43f81f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 --2- 172.21.15.46:0/1483273051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f80b0810 0x7f43f81fd340 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 >> 172.21.15.46:0/1483273051 conn(0x7f43f80b35f0 msgr2=0x7f43f8160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 shutdown_connections 2024-09-17T03:25:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:37.568+0000 7f43ff600640 1 -- 172.21.15.46:0/1483273051 wait complete. 2024-09-17T03:25:37.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:37.590 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:37.590 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:25:37.599 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:37.599 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10488365056" 2024-09-17T03:25:37.599 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:25:37.600 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-17T03:25:39.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:39.601 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:39.601 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:25:39.612 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:39.613 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10488365056" 2024-09-17T03:25:39.613 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:25:39.614 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-17T03:25:41.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:25:41.615 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:41.615 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:25:41.627 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/c3d1f0a5-1cbf-4faa-9c52-911c609d8e7d 2024-09-17T03:25:41.627 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10737418240" 2024-09-17T03:25:41.627 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:25:41.628 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-17T03:25:41.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone33690 2024-09-17T03:25:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 -- 172.21.15.46:0/3749891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac074950 msgr2=0x7f9aac0b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/3749891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac074950 0x7f9aac0b5260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9aac073cf0 tx=0x7f9aa401ccf0 comp rx=0 tx=0).stop 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 -- 172.21.15.46:0/3749891172 shutdown_connections 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/3749891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac074950 0x7f9aac0b5260 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/3749891172 >> v2:172.21.15.99:3300/0 conn(0x7f9aac1536f0 0x7f9aac15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 -- 172.21.15.46:0/3749891172 >> 172.21.15.46:0/3749891172 conn(0x7f9aac0b4560 msgr2=0x7f9aac0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 -- 172.21.15.46:0/3749891172 shutdown_connections 2024-09-17T03:25:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 -- 172.21.15.46:0/3749891172 wait complete. 2024-09-17T03:25:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.848+0000 7f9ab2a00640 1 Processor -- start 2024-09-17T03:25:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 -- start start 2024-09-17T03:25:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac1536f0 0x7f9aac136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9aac160060 0x7f9aac1390a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 0x7f9aac0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac169e10 con 0x7f9aac149410 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac169c90 con 0x7f9aac1536f0 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac169f90 con 0x7f9aac160060 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 0x7f9aac0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 0x7f9aac0d1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41276/0 (socket says 172.21.15.46:41276) 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 -- 172.21.15.46:0/2037005919 learned_addr learned my addr 172.21.15.46:0/2037005919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab1600640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac1536f0 0x7f9aac136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab0c00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9aac160060 0x7f9aac1390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 -- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9aac160060 msgr2=0x7f9aac1390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9aac160060 0x7f9aac1390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 -- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac1536f0 msgr2=0x7f9aac136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac1536f0 0x7f9aac136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 -- 172.21.15.46:0/2037005919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac1395e0 con 0x7f9aac149410 2024-09-17T03:25:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2000640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 0x7f9aac0d1600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009bd0 tx=0x7f9aa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40bb070 con 0x7f9aac149410 2024-09-17T03:25:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aa4004a50 con 0x7f9aac149410 2024-09-17T03:25:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac0d0560 con 0x7f9aac149410 2024-09-17T03:25:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40b2040 con 0x7f9aac149410 2024-09-17T03:25:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.852+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac0d0b20 con 0x7f9aac149410 2024-09-17T03:25:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.856+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac0b5260 con 0x7f9aac149410 2024-09-17T03:25:41.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.856+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 97591+0+0 (secure 0 0 0) 0x7f9aa40be070 con 0x7f9aac149410 2024-09-17T03:25:41.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.856+0000 7f9aa9600640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9a8c073620 0x7f9a8c075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:41.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.856+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f9aa412e770 con 0x7f9aac149410 2024-09-17T03:25:41.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.856+0000 7f9ab1600640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9a8c073620 0x7f9a8c075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:41.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.856+0000 7f9ab1600640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9a8c073620 0x7f9a8c075b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009b40 tx=0x7f9aa00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:41.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:41.860+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9aac1395e0 con 0x7f9aac149410 2024-09-17T03:25:42.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.064+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone33690", "target": ["mon-mgr", ""]}) -- 0x7f9aac06b540 con 0x7f9a8c073620 2024-09-17T03:25:42.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.140+0000 7f9aa9600640 1 -- 172.21.15.46:0/2037005919 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9aa0011be0 con 0x7f9a8c073620 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9a8c073620 msgr2=0x7f9a8c075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9a8c073620 0x7f9a8c075b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009b40 tx=0x7f9aa00091e0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 msgr2=0x7f9aac0d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 0x7f9aac0d1600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009bd0 tx=0x7f9aa4004270 comp rx=0 tx=0).stop 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 shutdown_connections 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9a8c073620 0x7f9a8c075b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac149410 0x7f9aac0d1600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9aac160060 0x7f9aac1390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 --2- 172.21.15.46:0/2037005919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9aac1536f0 0x7f9aac136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 >> 172.21.15.46:0/2037005919 conn(0x7f9aac0b4560 msgr2=0x7f9aac15c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:42.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 shutdown_connections 2024-09-17T03:25:42.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.144+0000 7f9ab2a00640 1 -- 172.21.15.46:0/2037005919 wait complete. 2024-09-17T03:25:42.146 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.144+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.146 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.144+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.147 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.144+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.147 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.144+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.147 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.144+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.165 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.164+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.165 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.164+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.165 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.164+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.165 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.164+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.165 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:42.164+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:42.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80541 2024-09-17T03:25:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 -- 172.21.15.46:0/1884705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 msgr2=0x7f73980b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 --2- 172.21.15.46:0/1884705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f73980b0730 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f73980cb7e0 tx=0x7f738801caf0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 -- 172.21.15.46:0/1884705508 shutdown_connections 2024-09-17T03:25:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 --2- 172.21.15.46:0/1884705508 >> v2:172.21.15.99:3300/0 conn(0x7f739815f5c0 0x7f739815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 --2- 172.21.15.46:0/1884705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f73980b0730 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 -- 172.21.15.46:0/1884705508 >> 172.21.15.46:0/1884705508 conn(0x7f73980b41f0 msgr2=0x7f73980b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 -- 172.21.15.46:0/1884705508 shutdown_connections 2024-09-17T03:25:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 -- 172.21.15.46:0/1884705508 wait complete. 2024-09-17T03:25:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 Processor -- start 2024-09-17T03:25:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.344+0000 7f7397200640 1 -- start start 2024-09-17T03:25:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f739806dbc0 0x7f739815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f739814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f739815f5c0 0x7f73980cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7398168ff0 con 0x7f73980d1140 2024-09-17T03:25:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7398168e70 con 0x7f739806dbc0 2024-09-17T03:25:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7398169170 con 0x7f739815f5c0 2024-09-17T03:25:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f739814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7395e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f739806dbc0 0x7f739815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f739814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41332/0 (socket says 172.21.15.46:41332) 2024-09-17T03:25:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 -- 172.21.15.46:0/2226122917 learned_addr learned my addr 172.21.15.46:0/2226122917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7396800640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f739815f5c0 0x7f73980cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 -- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f739815f5c0 msgr2=0x7f73980cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f739815f5c0 0x7f73980cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 -- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f739806dbc0 msgr2=0x7f739815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f739806dbc0 0x7f739815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 -- 172.21.15.46:0/2226122917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739809c230 con 0x7f73980d1140 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7395e00640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f739806dbc0 0x7f739815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738d400640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f739814bef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f738000c400 tx=0x7f738000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380017070 con 0x7f73980d1140 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7380003600 con 0x7f73980d1140 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739809c730 con 0x7f73980d1140 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380003a50 con 0x7f73980d1140 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739809ccf0 con 0x7f73980d1140 2024-09-17T03:25:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.348+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739809c4a0 con 0x7f73980d1140 2024-09-17T03:25:42.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.352+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 97591+0+0 (secure 0 0 0) 0x7f738001a070 con 0x7f73980d1140 2024-09-17T03:25:42.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.352+0000 7f738ea00640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73740735f0 0x7f7374075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.352+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f7380093c90 con 0x7f73980d1140 2024-09-17T03:25:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.352+0000 7f7395e00640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73740735f0 0x7f7374075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.352+0000 7f7395e00640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73740735f0 0x7f7374075b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f73880af000 tx=0x7f738801b330 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.356+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f738005df80 con 0x7f73980d1140 2024-09-17T03:25:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:42.560+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80541", "target": ["mon-mgr", ""]}) -- 0x7f73980b0730 con 0x7f73740735f0 2024-09-17T03:25:43.148 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:43.148+0000 7fbe42800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:43.148 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:43.148+0000 7fbe42800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:43.148 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:43.148+0000 7fbe42800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:43.148 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:43.148+0000 7fbe42800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:43.148 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:43.148+0000 7fbe42800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.264+0000 7f738ea00640 1 -- 172.21.15.46:0/2226122917 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73980b0730 con 0x7f73740735f0 2024-09-17T03:25:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.268+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73740735f0 msgr2=0x7f7374075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.268+0000 7f7397200640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73740735f0 0x7f7374075b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f73880af000 tx=0x7f738801b330 comp rx=0 tx=0).stop 2024-09-17T03:25:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.268+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 msgr2=0x7f739814bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.268+0000 7f7397200640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f739814bef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f738000c400 tx=0x7f738000c8c0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.272+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 shutdown_connections 2024-09-17T03:25:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.272+0000 7f7397200640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73740735f0 0x7f7374075b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.272+0000 7f7397200640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f739815f5c0 0x7f73980cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.272+0000 7f7397200640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73980d1140 0x7f739814bef0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.272+0000 7f7397200640 1 --2- 172.21.15.46:0/2226122917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f739806dbc0 0x7f739815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.272+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 >> 172.21.15.46:0/2226122917 conn(0x7f73980b41f0 msgr2=0x7f739815e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:43.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.276+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 shutdown_connections 2024-09-17T03:25:43.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.280+0000 7f7397200640 1 -- 172.21.15.46:0/2226122917 wait complete. 2024-09-17T03:25:43.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96892 2024-09-17T03:25:43.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cfe00640 1 -- 172.21.15.46:0/2860298949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d00cdf30 msgr2=0x7f29b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:43.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cfe00640 1 --2- 172.21.15.46:0/2860298949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d00cdf30 0x7f29b4004810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009b60 tx=0x7f29bc01d2d0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cf400640 1 -- 172.21.15.46:0/2860298949 reap_dead start 2024-09-17T03:25:43.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cfe00640 1 -- 172.21.15.46:0/2860298949 shutdown_connections 2024-09-17T03:25:43.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cfe00640 1 -- 172.21.15.46:0/2860298949 >> 172.21.15.46:0/2860298949 conn(0x7f29d006e4a0 msgr2=0x7f29d006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:43.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cfe00640 1 -- 172.21.15.46:0/2860298949 shutdown_connections 2024-09-17T03:25:43.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.696+0000 7f29cfe00640 1 -- 172.21.15.46:0/2860298949 wait complete. 2024-09-17T03:25:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.700+0000 7f29cfe00640 1 Processor -- start 2024-09-17T03:25:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.700+0000 7f29cfe00640 1 -- start start 2024-09-17T03:25:43.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d00cdf30 0x7f29d00c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:43.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d015ed90 0x7f29d015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:43.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d0140240 0x7f29d0140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:43.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0165360 con 0x7f29d00cdf30 2024-09-17T03:25:43.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01651e0 con 0x7f29d0140240 2024-09-17T03:25:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f29d01654e0 con 0x7f29d015ed90 2024-09-17T03:25:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d0140240 0x7f29d0140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d015ed90 0x7f29d015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d0140240 0x7f29d0140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:37484/0 (socket says 172.21.15.46:37484) 2024-09-17T03:25:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cf400640 1 -- 172.21.15.46:0/1286729437 learned_addr learned my addr 172.21.15.46:0/1286729437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 -- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d0140240 msgr2=0x7f29d0140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d0140240 0x7f29d0140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 -- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d00cdf30 msgr2=0x7f29d00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d00cdf30 0x7f29d00c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 -- 172.21.15.46:0/1286729437 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d0140b90 con 0x7f29d015ed90 2024-09-17T03:25:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29ce000640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d015ed90 0x7f29d015f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29b4010160 tx=0x7f29bc005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0b1040 con 0x7f29d015ed90 2024-09-17T03:25:43.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29bc0ac8a0 con 0x7f29d015ed90 2024-09-17T03:25:43.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0accd0 con 0x7f29d015ed90 2024-09-17T03:25:43.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.704+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d0161250 con 0x7f29d015ed90 2024-09-17T03:25:43.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.708+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d0161810 con 0x7f29d015ed90 2024-09-17T03:25:43.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.708+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d0160740 con 0x7f29d015ed90 2024-09-17T03:25:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.708+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 16) ==== 97744+0+0 (secure 0 0 0) 0x7f29bc0bf060 con 0x7f29d015ed90 2024-09-17T03:25:43.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.708+0000 7f29c6a00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f29b00737d0 0x7f29b0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:43.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.712+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f29bc12efe0 con 0x7f29d015ed90 2024-09-17T03:25:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.712+0000 7f29cea00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f29b00737d0 0x7f29b0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:43.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.712+0000 7f29cea00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f29b00737d0 0x7f29b0075ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29c0002070 tx=0x7f29c0002000 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:43.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:43.720+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f29bc016fe0 con 0x7f29d015ed90 2024-09-17T03:25:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.172+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96892", "target": ["mon-mgr", ""]}) -- 0x7f29d006b540 con 0x7f29b00737d0 2024-09-17T03:25:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.756+0000 7f29c6a00640 1 -- 172.21.15.46:0/1286729437 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29c0003e10 con 0x7f29b00737d0 2024-09-17T03:25:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f29b00737d0 msgr2=0x7f29b0075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f29b00737d0 0x7f29b0075ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29c0002070 tx=0x7f29c0002000 comp rx=0 tx=0).stop 2024-09-17T03:25:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d015ed90 msgr2=0x7f29d015f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d015ed90 0x7f29d015f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29b4010160 tx=0x7f29bc005660 comp rx=0 tx=0).stop 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 shutdown_connections 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f29b00737d0 0x7f29b0075ce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d0140240 0x7f29d0140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f29d015ed90 0x7f29d015f1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 --2- 172.21.15.46:0/1286729437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29d00cdf30 0x7f29d00c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 >> 172.21.15.46:0/1286729437 conn(0x7f29d006e4a0 msgr2=0x7f29d0072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 shutdown_connections 2024-09-17T03:25:44.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:44.764+0000 7f29cfe00640 1 -- 172.21.15.46:0/1286729437 wait complete. 2024-09-17T03:25:44.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67613 2024-09-17T03:25:44.898 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:44.888+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:44.898 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:44.888+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:44.898 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:44.888+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:44.898 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:44.888+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:44.898 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:44.888+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:45.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 -- 172.21.15.46:0/2428114173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 msgr2=0x7fe8f40ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:45.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/2428114173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f40ae8e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009900 tx=0x7fe8e001c7c0 comp rx=0 tx=0).stop 2024-09-17T03:25:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 -- 172.21.15.46:0/2428114173 shutdown_connections 2024-09-17T03:25:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/2428114173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f40ae8e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/2428114173 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8f415f8f0 0x7fe8f41557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:45.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 -- 172.21.15.46:0/2428114173 >> 172.21.15.46:0/2428114173 conn(0x7fe8f406e1d0 msgr2=0x7fe8f406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:45.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 -- 172.21.15.46:0/2428114173 shutdown_connections 2024-09-17T03:25:45.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.324+0000 7fe8f8c00640 1 -- 172.21.15.46:0/2428114173 wait complete. 2024-09-17T03:25:45.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.328+0000 7fe8f8c00640 1 Processor -- start 2024-09-17T03:25:45.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.328+0000 7fe8f8c00640 1 -- start start 2024-09-17T03:25:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.332+0000 7fe8f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f4147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.336+0000 7fe8f8c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8f40cf620 0x7fe8f4148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.336+0000 7fe8f1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f4147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.336+0000 7fe8f1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f4147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47394/0 (socket says 172.21.15.46:47394) 2024-09-17T03:25:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 -- 172.21.15.46:0/3087044554 learned_addr learned my addr 172.21.15.46:0/3087044554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f415f8f0 0x7fe8f414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f416c740 con 0x7fe8f40cf210 2024-09-17T03:25:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f416c5c0 con 0x7fe8f415f8f0 2024-09-17T03:25:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f416c8c0 con 0x7fe8f40cf620 2024-09-17T03:25:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1400640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8f40cf620 0x7fe8f4148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 -- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8f40cf620 msgr2=0x7fe8f4148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8f40cf620 0x7fe8f4148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 -- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f415f8f0 msgr2=0x7fe8f414c060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:25:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f415f8f0 0x7fe8f414c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f41490b0 con 0x7fe8f40cf210 2024-09-17T03:25:45.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8f1e00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f4147bc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400e7c0 tx=0x7fe8e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.340+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400ccc0 con 0x7fe8f40cf210 2024-09-17T03:25:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.344+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8e400ce60 con 0x7fe8f40cf210 2024-09-17T03:25:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.344+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400ccc0 con 0x7fe8f40cf210 2024-09-17T03:25:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.344+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f4149390 con 0x7fe8f40cf210 2024-09-17T03:25:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.344+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f4149c60 con 0x7fe8f40cf210 2024-09-17T03:25:45.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.348+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f4152eb0 con 0x7fe8f40cf210 2024-09-17T03:25:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.360+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 97744+0+0 (secure 0 0 0) 0x7fe8e4021430 con 0x7fe8f40cf210 2024-09-17T03:25:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.360+0000 7fe8ea000640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe8d40739a0 0x7fe8d4075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.360+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fe8e40954a0 con 0x7fe8f40cf210 2024-09-17T03:25:45.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.376+0000 7fe8f1400640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe8d40739a0 0x7fe8d4075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.384+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8e405fbd0 con 0x7fe8f40cf210 2024-09-17T03:25:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.388+0000 7fe8f1400640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe8d40739a0 0x7fe8d4075eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00ae000 tx=0x7fe8e000b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:45.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.452+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 17) ==== 97795+0+0 (secure 0 0 0) 0x7fe8e405b740 con 0x7fe8f40cf210 2024-09-17T03:25:45.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:45.940+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67613", "target": ["mon-mgr", ""]}) -- 0x7fe8f406b540 con 0x7fe8d40739a0 2024-09-17T03:25:47.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.028+0000 7fe8ea000640 1 -- 172.21.15.46:0/3087044554 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8e0005d60 con 0x7fe8d40739a0 2024-09-17T03:25:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.040+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe8d40739a0 msgr2=0x7fe8d4075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.040+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe8d40739a0 0x7fe8d4075eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00ae000 tx=0x7fe8e000b040 comp rx=0 tx=0).stop 2024-09-17T03:25:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.040+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 msgr2=0x7fe8f4147bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.040+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f4147bc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400e7c0 tx=0x7fe8e400ec80 comp rx=0 tx=0).stop 2024-09-17T03:25:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.044+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 shutdown_connections 2024-09-17T03:25:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.044+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe8d40739a0 0x7fe8d4075eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.044+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8f415f8f0 0x7fe8f414c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.044+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8f40cf620 0x7fe8f4148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.044+0000 7fe8f8c00640 1 --2- 172.21.15.46:0/3087044554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f40cf210 0x7fe8f4147bc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.044+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 >> 172.21.15.46:0/3087044554 conn(0x7fe8f406e1d0 msgr2=0x7fe8f40afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.048+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 shutdown_connections 2024-09-17T03:25:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.048+0000 7fe8f8c00640 1 -- 172.21.15.46:0/3087044554 wait complete. 2024-09-17T03:25:47.107 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:47.104+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:47.107 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:47.104+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:47.107 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:47.104+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:47.107 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:47.104+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:47.107 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:47.104+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:47.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:25:47.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.668+0000 7f49ce000640 1 -- 172.21.15.46:0/4061311208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80aeff0 msgr2=0x7f49c80b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:47.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.668+0000 7f49c6000640 1 -- 172.21.15.46:0/4061311208 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00b2070 con 0x7f49c80aeff0 2024-09-17T03:25:47.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.668+0000 7f49ce000640 1 --2- 172.21.15.46:0/4061311208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80aeff0 0x7f49c80b3340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009bc0 tx=0x7f49c001ca70 comp rx=0 tx=0).stop 2024-09-17T03:25:47.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.672+0000 7f49ce000640 1 -- 172.21.15.46:0/4061311208 shutdown_connections 2024-09-17T03:25:47.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.672+0000 7f49ce000640 1 --2- 172.21.15.46:0/4061311208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80aeff0 0x7f49c80b3340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.672+0000 7f49ce000640 1 --2- 172.21.15.46:0/4061311208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c815f8f0 0x7f49c81557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.672+0000 7f49ce000640 1 -- 172.21.15.46:0/4061311208 >> 172.21.15.46:0/4061311208 conn(0x7f49c806e1d0 msgr2=0x7f49c806e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:47.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.672+0000 7f49ce000640 1 -- 172.21.15.46:0/4061311208 shutdown_connections 2024-09-17T03:25:47.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.672+0000 7f49ce000640 1 -- 172.21.15.46:0/4061311208 wait complete. 2024-09-17T03:25:47.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.680+0000 7f49ce000640 1 Processor -- start 2024-09-17T03:25:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.680+0000 7f49ce000640 1 -- start start 2024-09-17T03:25:47.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49ce000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80cf210 0x7f49c814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:47.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49ce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c815f8f0 0x7f49c80aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:47.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49ce000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 0x7f49c8073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:47.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49ce000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80cd010 con 0x7f49c815f8f0 2024-09-17T03:25:47.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49ce000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80cce90 con 0x7f49c80cf210 2024-09-17T03:25:47.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49ce000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f49c816c5c0 con 0x7f49c80aaa10 2024-09-17T03:25:47.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49c7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 0x7f49c8073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:47.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49c7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 0x7f49c8073490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41592/0 (socket says 172.21.15.46:41592) 2024-09-17T03:25:47.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49c7e00640 1 -- 172.21.15.46:0/642283128 learned_addr learned my addr 172.21.15.46:0/642283128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:47.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49c7400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80cf210 0x7f49c814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:47.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.684+0000 7f49c7e00640 1 -- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80cf210 msgr2=0x7f49c814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:47.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49c7e00640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80cf210 0x7f49c814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49c7e00640 1 -- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c815f8f0 msgr2=0x7f49c80aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:47.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49c7e00640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c815f8f0 0x7f49c80aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:47.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49c7e00640 1 -- 172.21.15.46:0/642283128 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c814f550 con 0x7f49c80aaa10 2024-09-17T03:25:47.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49c7e00640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 0x7f49c8073490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b0040 tx=0x7f49c0005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:47.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00b2040 con 0x7f49c80aaa10 2024-09-17T03:25:47.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c0004130 con 0x7f49c80aaa10 2024-09-17T03:25:47.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.688+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0004560 con 0x7f49c80aaa10 2024-09-17T03:25:47.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.692+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c80acd90 con 0x7f49c80aaa10 2024-09-17T03:25:47.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.692+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c814ec20 con 0x7f49c80aaa10 2024-09-17T03:25:47.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.692+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 17) ==== 97795+0+0 (secure 0 0 0) 0x7f49c00bd060 con 0x7f49c80aaa10 2024-09-17T03:25:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.696+0000 7f49bf400640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49a4073930 0x7f49a4075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.696+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f49c012e680 con 0x7f49c80aaa10 2024-09-17T03:25:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.696+0000 7f49c7400640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49a4073930 0x7f49a4075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.700+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c814e6d0 con 0x7f49c80aaa10 2024-09-17T03:25:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.712+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f49c00167b0 con 0x7f49c80aaa10 2024-09-17T03:25:47.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:47.740+0000 7f49c7400640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49a4073930 0x7f49a4075e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009b40 tx=0x7f49b0009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:48.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.240+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f49c8208c80 con 0x7f49c80aaa10 2024-09-17T03:25:48.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.240+0000 7f49bf400640 1 -- 172.21.15.46:0/642283128 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2185 (secure 0 0 0) 0x7f49c00f8c50 con 0x7f49c80aaa10 2024-09-17T03:25:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:25:48.245 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:25:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:25:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:25:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:25:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:25:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:25:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:25:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:25:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:25:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:25:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:25:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:25:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:25:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:25:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:25:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:25:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:25:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 306, 2024-09-17T03:25:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:25:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:25:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:25:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:25:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:25:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 45, 2024-09-17T03:25:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:25:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:25:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:25:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:25:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:25:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:25:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:25:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:25:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:25:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:25:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:25:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:25:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:25:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 3062, 2024-09-17T03:25:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 12705965050, 2024-09-17T03:25:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 24854503424, 2024-09-17T03:25:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 748239609856, 2024-09-17T03:25:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:25:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 174187958, 2024-09-17T03:25:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 253108061, 2024-09-17T03:25:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 42, 2024-09-17T03:25:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 69 2024-09-17T03:25:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:25:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 27, 2024-09-17T03:25:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:24:08:717127+0000", 2024-09-17T03:25:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3, 2024-09-17T03:25:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:25:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:25:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:25:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:25:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 3, 2024-09-17T03:25:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:25:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:25:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:25:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 245313 2024-09-17T03:25:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:25:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:25:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:25:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:25:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:25:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:25:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:25:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:25:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:25:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:25:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:25:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:25:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 8, 2024-09-17T03:25:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:25:09.264615+0000", 2024-09-17T03:25:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:25:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:25:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:25:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "2 ongoing clones - average progress is 3.711% (1s)\n [=...........................] (remaining: 45s)", 2024-09-17T03:25:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.037110000848770142, 2024-09-17T03:25:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:25:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:48.261 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:25:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49a4073930 msgr2=0x7f49a4075e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49a4073930 0x7f49a4075e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009b40 tx=0x7f49b0009260 comp rx=0 tx=0).stop 2024-09-17T03:25:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 msgr2=0x7f49c8073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 0x7f49c8073490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b0040 tx=0x7f49c0005a60 comp rx=0 tx=0).stop 2024-09-17T03:25:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 shutdown_connections 2024-09-17T03:25:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49a4073930 0x7f49a4075e40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f49c80aaa10 0x7f49c8073490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c815f8f0 0x7f49c80aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 --2- 172.21.15.46:0/642283128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c80cf210 0x7f49c814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 >> 172.21.15.46:0/642283128 conn(0x7f49c806e1d0 msgr2=0x7f49c80727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 shutdown_connections 2024-09-17T03:25:48.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.252+0000 7f49ce000640 1 -- 172.21.15.46:0/642283128 wait complete. 2024-09-17T03:25:48.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone33690 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.768+0000 7f2218a00640 1 -- 172.21.15.46:0/2919207278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 msgr2=0x7f2214155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.768+0000 7f2218a00640 1 --2- 172.21.15.46:0/2919207278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214155900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2204013f30 tx=0x7f2204024e40 comp rx=0 tx=0).stop 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.772+0000 7f2218a00640 1 -- 172.21.15.46:0/2919207278 shutdown_connections 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.772+0000 7f2218a00640 1 --2- 172.21.15.46:0/2919207278 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 0x7f221415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.772+0000 7f2218a00640 1 --2- 172.21.15.46:0/2919207278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214155900 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.772+0000 7f2218a00640 1 -- 172.21.15.46:0/2919207278 >> 172.21.15.46:0/2919207278 conn(0x7f22140b2040 msgr2=0x7f22140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.772+0000 7f2218a00640 1 -- 172.21.15.46:0/2919207278 shutdown_connections 2024-09-17T03:25:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.772+0000 7f2218a00640 1 -- 172.21.15.46:0/2919207278 wait complete. 2024-09-17T03:25:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.780+0000 7f2218a00640 1 Processor -- start 2024-09-17T03:25:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.780+0000 7f2218a00640 1 -- start start 2024-09-17T03:25:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.784+0000 7f2218a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:48.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.784+0000 7f2218a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 0x7f221414d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:48.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.784+0000 7f2218a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 0x7f2214159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.784+0000 7f2211e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.784+0000 7f2211e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:37554/0 (socket says 172.21.15.46:37554) 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.784+0000 7f2211400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 0x7f221414d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2211400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 0x7f221414d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41610/0 (socket says 172.21.15.46:41610) 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2212800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 0x7f2214159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2212800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 0x7f2214159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47418/0 (socket says 172.21.15.46:47418) 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2218a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22140743c0 con 0x7f221415faa0 2024-09-17T03:25:48.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2218a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2214074240 con 0x7f221415c5e0 2024-09-17T03:25:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2218a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2214074540 con 0x7f221415f150 2024-09-17T03:25:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2211e00640 1 -- 172.21.15.46:0/4170520009 learned_addr learned my addr 172.21.15.46:0/4170520009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2212800640 1 -- 172.21.15.46:0/4170520009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 msgr2=0x7f221414d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2212800640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 0x7f221414d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.788+0000 7f2212800640 1 -- 172.21.15.46:0/4170520009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 msgr2=0x7f2214153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.792+0000 7f2212800640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.792+0000 7f2212800640 1 -- 172.21.15.46:0/4170520009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2214153dc0 con 0x7f221415faa0 2024-09-17T03:25:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.792+0000 7f2212800640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 0x7f2214159cd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f22140b3930 tx=0x7f220c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.792+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c019070 con 0x7f221415faa0 2024-09-17T03:25:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.792+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f220c014440 con 0x7f221415faa0 2024-09-17T03:25:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.792+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c014890 con 0x7f221415faa0 2024-09-17T03:25:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.800+0000 7f2218a00640 1 -- 172.21.15.46:0/4170520009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22141f7af0 con 0x7f221415faa0 2024-09-17T03:25:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.800+0000 7f2218a00640 1 -- 172.21.15.46:0/4170520009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22141f80b0 con 0x7f221415faa0 2024-09-17T03:25:48.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.804+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2214074ac0 con 0x7f221415faa0 2024-09-17T03:25:48.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.808+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 97795+0+0 (secure 0 0 0) 0x7f220c01c070 con 0x7f221415faa0 2024-09-17T03:25:48.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.808+0000 7f220a000640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f4073830 0x7f21f4075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:48.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.812+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f220c094e40 con 0x7f221415faa0 2024-09-17T03:25:48.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.824+0000 7f2211e00640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f4073830 0x7f21f4075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:48.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.824+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f220c05ed40 con 0x7f221415faa0 2024-09-17T03:25:48.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:48.856+0000 7f2211e00640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f4073830 0x7f21f4075d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2204024c70 tx=0x7f2204023680 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:49.476+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone33690", "target": ["mon-mgr", ""]}) -- 0x7f221406b540 con 0x7f21f4073830 2024-09-17T03:25:49.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:49.568+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 18) ==== 97846+0+0 (secure 0 0 0) 0x7f220c05afb0 con 0x7f221415faa0 2024-09-17T03:25:49.613 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:49.612+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:49.614 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:49.612+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:49.614 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:49.612+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:49.614 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:49.612+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:49.614 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:49.612+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:51.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:51.604+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f220c05efa0 con 0x7f221415faa0 2024-09-17T03:25:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.424+0000 7f220a000640 1 -- 172.21.15.46:0/4170520009 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2204004140 con 0x7f21f4073830 2024-09-17T03:25:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f4073830 msgr2=0x7f21f4075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f4073830 0x7f21f4075d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2204024c70 tx=0x7f2204023680 comp rx=0 tx=0).stop 2024-09-17T03:25:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 msgr2=0x7f2214159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 0x7f2214159cd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f22140b3930 tx=0x7f220c009ef0 comp rx=0 tx=0).stop 2024-09-17T03:25:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 shutdown_connections 2024-09-17T03:25:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f4073830 0x7f21f4075d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221415faa0 0x7f2214159cd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221415f150 0x7f221414d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 --2- 172.21.15.46:0/4170520009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221415c5e0 0x7f2214153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.436+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 >> 172.21.15.46:0/4170520009 conn(0x7f22140b2040 msgr2=0x7f22140ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.440+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 shutdown_connections 2024-09-17T03:25:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:52.440+0000 7f21ff400640 1 -- 172.21.15.46:0/4170520009 wait complete. 2024-09-17T03:25:52.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80541 2024-09-17T03:25:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.016+0000 7f20f4c00640 1 -- 172.21.15.46:0/3891402860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20d4004b70 msgr2=0x7f20d4004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.016+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3891402860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20d4004b70 0x7f20d4004fe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c310 tx=0x7f20dc01f730 comp rx=0 tx=0).stop 2024-09-17T03:25:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.020+0000 7f20f4c00640 1 -- 172.21.15.46:0/3891402860 shutdown_connections 2024-09-17T03:25:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.020+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3891402860 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20d4004b70 0x7f20d4004fe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.020+0000 7f20f4c00640 1 -- 172.21.15.46:0/3891402860 >> 172.21.15.46:0/3891402860 conn(0x7f20f00b31f0 msgr2=0x7f20f006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.020+0000 7f20f4c00640 1 -- 172.21.15.46:0/3891402860 shutdown_connections 2024-09-17T03:25:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.020+0000 7f20f4c00640 1 -- 172.21.15.46:0/3891402860 wait complete. 2024-09-17T03:25:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.024+0000 7f20f4c00640 1 Processor -- start 2024-09-17T03:25:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.024+0000 7f20f4c00640 1 -- start start 2024-09-17T03:25:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.028+0000 7f20f4c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 0x7f20f01e5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.028+0000 7f20f4c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20f0150c60 0x7f20f00aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.028+0000 7f20f4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f01e3bf0 0x7f20f00abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:53.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.028+0000 7f20ede00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 0x7f20f01e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 0x7f20f01e5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:36308/0 (socket says 172.21.15.46:36308) 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ed400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20f0150c60 0x7f20f00aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ed400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20f0150c60 0x7f20f00aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:53228/0 (socket says 172.21.15.46:53228) 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20f4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f00b3ea0 con 0x7f20f01e3bf0 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20f4c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20f00b3d20 con 0x7f20d4004b70 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20f4c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f20f00b4020 con 0x7f20f0150c60 2024-09-17T03:25:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 -- 172.21.15.46:0/3263261817 learned_addr learned my addr 172.21.15.46:0/3263261817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ee800640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f01e3bf0 0x7f20f00abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 -- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20f0150c60 msgr2=0x7f20f00aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20f0150c60 0x7f20f00aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 -- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f01e3bf0 msgr2=0x7f20f00abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f01e3bf0 0x7f20f00abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 -- 172.21.15.46:0/3263261817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f00adea0 con 0x7f20d4004b70 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20ede00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 0x7f20f01e5250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0b1040 tx=0x7f20dc0061a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0b3040 con 0x7f20d4004b70 2024-09-17T03:25:53.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.032+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20dc0bd070 con 0x7f20d4004b70 2024-09-17T03:25:53.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.036+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0b9ba0 con 0x7f20d4004b70 2024-09-17T03:25:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.036+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f00ae180 con 0x7f20d4004b70 2024-09-17T03:25:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.036+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f00ae740 con 0x7f20d4004b70 2024-09-17T03:25:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.040+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f20dc0ac070 con 0x7f20d4004b70 2024-09-17T03:25:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.040+0000 7f20e6000640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f20d007c150 0x7f20d007e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.040+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f20dc1327c0 con 0x7f20d4004b70 2024-09-17T03:25:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.044+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f015e2e0 con 0x7f20d4004b70 2024-09-17T03:25:53.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.044+0000 7f20ed400640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f20d007c150 0x7f20d007e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:53.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.044+0000 7f20ed400640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f20d007c150 0x7f20d007e660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20f0151160 tx=0x7f20e0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.056+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f20dc0c7020 con 0x7f20d4004b70 2024-09-17T03:25:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:53.676+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80541", "target": ["mon-mgr", ""]}) -- 0x7f20f006b540 con 0x7f20d007c150 2024-09-17T03:25:55.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.428+0000 7f20e6000640 1 -- 172.21.15.46:0/3263261817 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20e0001c50 con 0x7f20d007c150 2024-09-17T03:25:55.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.432+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f20d007c150 msgr2=0x7f20d007e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:55.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.432+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f20d007c150 0x7f20d007e660 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20f0151160 tx=0x7f20e0009040 comp rx=0 tx=0).stop 2024-09-17T03:25:55.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.432+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 msgr2=0x7f20f01e5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:55.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.432+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 0x7f20f01e5250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0b1040 tx=0x7f20dc0061a0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 shutdown_connections 2024-09-17T03:25:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f20d007c150 0x7f20d007e660 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f01e3bf0 0x7f20f00abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f20f0150c60 0x7f20f00aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 --2- 172.21.15.46:0/3263261817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20d4004b70 0x7f20f01e5250 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 >> 172.21.15.46:0/3263261817 conn(0x7f20f00b31f0 msgr2=0x7f20f00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 shutdown_connections 2024-09-17T03:25:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.436+0000 7f20f4c00640 1 -- 172.21.15.46:0/3263261817 wait complete. 2024-09-17T03:25:55.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96892 2024-09-17T03:25:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.948+0000 7f7385400640 1 -- 172.21.15.46:0/1810612008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73800cf210 msgr2=0x7f73800ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:55.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.948+0000 7f7385400640 1 --2- 172.21.15.46:0/1810612008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73800cf210 0x7f73800ae8e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f73800cd4b0 tx=0x7f737001c7a0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.952+0000 7f7385400640 1 -- 172.21.15.46:0/1810612008 shutdown_connections 2024-09-17T03:25:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.952+0000 7f7385400640 1 --2- 172.21.15.46:0/1810612008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf620 0x7f73800aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.952+0000 7f7385400640 1 --2- 172.21.15.46:0/1810612008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73800cf210 0x7f73800ae8e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.952+0000 7f7385400640 1 --2- 172.21.15.46:0/1810612008 >> v2:172.21.15.46:3300/0 conn(0x7f738015f8f0 0x7f73801557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.952+0000 7f7385400640 1 -- 172.21.15.46:0/1810612008 >> 172.21.15.46:0/1810612008 conn(0x7f738006e1d0 msgr2=0x7f738006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.960+0000 7f7385400640 1 -- 172.21.15.46:0/1810612008 shutdown_connections 2024-09-17T03:25:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.960+0000 7f7385400640 1 -- 172.21.15.46:0/1810612008 wait complete. 2024-09-17T03:25:55.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.972+0000 7f7385400640 1 Processor -- start 2024-09-17T03:25:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.972+0000 7f7385400640 1 -- start start 2024-09-17T03:25:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f7385400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 0x7f7380145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f7385400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73800cf620 0x7f73800afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:55.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f7385400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f738015f8f0 0x7f738014b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:55.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f7385400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73800cc8d0 con 0x7f73800cf620 2024-09-17T03:25:55.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f7385400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73800cc750 con 0x7f738015f8f0 2024-09-17T03:25:55.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f7385400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f73800cca50 con 0x7f73800cf210 2024-09-17T03:25:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f737e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 0x7f7380145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f737e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 0x7f7380145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:53254/0 (socket says 172.21.15.46:53254) 2024-09-17T03:25:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f737e800640 1 -- 172.21.15.46:0/1830014468 learned_addr learned my addr 172.21.15.46:0/1830014468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.980+0000 7f737de00640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73800cf620 0x7f73800afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737f200640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f738015f8f0 0x7f738014b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737e800640 1 -- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f738015f8f0 msgr2=0x7f738014b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737e800640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f738015f8f0 0x7f738014b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737e800640 1 -- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73800cf620 msgr2=0x7f73800afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737e800640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73800cf620 0x7f73800afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737e800640 1 -- 172.21.15.46:0/1830014468 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73800b1910 con 0x7f73800cf210 2024-09-17T03:25:55.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f737e800640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 0x7f7380145aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f737800ba50 tx=0x7f737800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378019070 con 0x7f73800cf210 2024-09-17T03:25:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f737800ccf0 con 0x7f73800cf210 2024-09-17T03:25:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.984+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73780146a0 con 0x7f73800cf210 2024-09-17T03:25:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.988+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73801475b0 con 0x7f73800cf210 2024-09-17T03:25:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.988+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801493b0 con 0x7f73800cf210 2024-09-17T03:25:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.992+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7380147780 con 0x7f73800cf210 2024-09-17T03:25:56.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:55.996+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f737801c070 con 0x7f73800cf210 2024-09-17T03:25:56.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:56.000+0000 7f7376a00640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f735c0739a0 0x7f735c075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:56.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:56.000+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f73780951f0 con 0x7f73800cf210 2024-09-17T03:25:56.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:56.008+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f737805ec40 con 0x7f73800cf210 2024-09-17T03:25:56.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:56.012+0000 7f737de00640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f735c0739a0 0x7f735c075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:56.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:56.016+0000 7f737de00640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f735c0739a0 0x7f735c075eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73700af000 tx=0x7f737001afe0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:56.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:56.552+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96892", "target": ["mon-mgr", ""]}) -- 0x7f738006b540 con 0x7f735c0739a0 2024-09-17T03:25:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.280+0000 7f7376a00640 1 -- 172.21.15.46:0/1830014468 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7370004020 con 0x7f735c0739a0 2024-09-17T03:25:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f735c0739a0 msgr2=0x7f735c075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f735c0739a0 0x7f735c075eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73700af000 tx=0x7f737001afe0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 msgr2=0x7f7380145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 0x7f7380145aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f737800ba50 tx=0x7f737800bf10 comp rx=0 tx=0).stop 2024-09-17T03:25:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 shutdown_connections 2024-09-17T03:25:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f735c0739a0 0x7f735c075eb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f738015f8f0 0x7f738014b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73800cf620 0x7f73800afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 --2- 172.21.15.46:0/1830014468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73800cf210 0x7f7380145aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 >> 172.21.15.46:0/1830014468 conn(0x7f738006e1d0 msgr2=0x7f7380072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.288+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 shutdown_connections 2024-09-17T03:25:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.292+0000 7f7385400640 1 -- 172.21.15.46:0/1830014468 wait complete. 2024-09-17T03:25:57.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67613 2024-09-17T03:25:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 -- 172.21.15.46:0/3637906707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8410075340 msgr2=0x7f841015c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 --2- 172.21.15.46:0/3637906707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8410075340 0x7f841015c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009870 tx=0x7f83fc01c7d0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 -- 172.21.15.46:0/3637906707 shutdown_connections 2024-09-17T03:25:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 --2- 172.21.15.46:0/3637906707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8410075340 0x7f841015c920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 -- 172.21.15.46:0/3637906707 >> 172.21.15.46:0/3637906707 conn(0x7f84100b41f0 msgr2=0x7f84100b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 -- 172.21.15.46:0/3637906707 shutdown_connections 2024-09-17T03:25:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.648+0000 7f8416200640 1 -- 172.21.15.46:0/3637906707 wait complete. 2024-09-17T03:25:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.652+0000 7f8416200640 1 Processor -- start 2024-09-17T03:25:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.652+0000 7f8416200640 1 -- start start 2024-09-17T03:25:57.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 0x7f841009a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84100cda80 0x7f84101543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84100ca8a0 0x7f8410154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f841016b4d0 con 0x7f84100746e0 2024-09-17T03:25:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f841016b350 con 0x7f84100cda80 2024-09-17T03:25:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f841016b650 con 0x7f84100ca8a0 2024-09-17T03:25:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 0x7f841009a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 0x7f841009a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55940/0 (socket says 172.21.15.46:55940) 2024-09-17T03:25:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 -- 172.21.15.46:0/2987476435 learned_addr learned my addr 172.21.15.46:0/2987476435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f840fe00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84100cda80 0x7f84101543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 -- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84100ca8a0 msgr2=0x7f8410154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84100ca8a0 0x7f8410154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 -- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84100cda80 msgr2=0x7f84101543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84100cda80 0x7f84101543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 -- 172.21.15.46:0/2987476435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841009aed0 con 0x7f84100746e0 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8414e00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 0x7f841009a990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f840400c430 tx=0x7f840400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404017070 con 0x7f84100746e0 2024-09-17T03:25:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84040053c0 con 0x7f84100746e0 2024-09-17T03:25:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84101550c0 con 0x7f84100746e0 2024-09-17T03:25:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404003860 con 0x7f84100746e0 2024-09-17T03:25:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.656+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84100cfc40 con 0x7f84100746e0 2024-09-17T03:25:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.660+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841015c920 con 0x7f84100746e0 2024-09-17T03:25:57.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.660+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f840401a070 con 0x7f84100746e0 2024-09-17T03:25:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.664+0000 7f840cc00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f83f0073b50 0x7f83f0076060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:57.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.664+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f8404003600 con 0x7f84100746e0 2024-09-17T03:25:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.672+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f840405e010 con 0x7f84100746e0 2024-09-17T03:25:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.676+0000 7f840fe00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f83f0073b50 0x7f83f0076060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:57.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:57.680+0000 7f840fe00640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f83f0073b50 0x7f83f0076060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0af000 tx=0x7f83fc01b010 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.048+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67613", "target": ["mon-mgr", ""]}) -- 0x7f84100cfe80 con 0x7f83f0073b50 2024-09-17T03:25:58.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f840cc00640 1 -- 172.21.15.46:0/2987476435 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83fc0a5630 con 0x7f83f0073b50 2024-09-17T03:25:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f83f0073b50 msgr2=0x7f83f0076060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f83f0073b50 0x7f83f0076060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0af000 tx=0x7f83fc01b010 comp rx=0 tx=0).stop 2024-09-17T03:25:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 msgr2=0x7f841009a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 0x7f841009a990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f840400c430 tx=0x7f840400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 shutdown_connections 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f83f0073b50 0x7f83f0076060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84100ca8a0 0x7f8410154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84100cda80 0x7f84101543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 --2- 172.21.15.46:0/2987476435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84100746e0 0x7f841009a990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.184+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 >> 172.21.15.46:0/2987476435 conn(0x7f84100b41f0 msgr2=0x7f8410150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.188+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 shutdown_connections 2024-09-17T03:25:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.188+0000 7f8416200640 1 -- 172.21.15.46:0/2987476435 wait complete. 2024-09-17T03:25:58.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:25:58.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa65b400640 1 -- 172.21.15.46:0/1541908493 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540ad680 con 0x7fa65c0b54f0 2024-09-17T03:25:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 -- 172.21.15.46:0/1541908493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0b54f0 msgr2=0x7fa65c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 --2- 172.21.15.46:0/1541908493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0b54f0 0x7fa65c15bb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa654009bf0 tx=0x7fa65401cbb0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 -- 172.21.15.46:0/1541908493 shutdown_connections 2024-09-17T03:25:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 --2- 172.21.15.46:0/1541908493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0b54f0 0x7fa65c15bb60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 -- 172.21.15.46:0/1541908493 >> 172.21.15.46:0/1541908493 conn(0x7fa65c0b2040 msgr2=0x7fa65c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 -- 172.21.15.46:0/1541908493 shutdown_connections 2024-09-17T03:25:58.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.368+0000 7fa663800640 1 -- 172.21.15.46:0/1541908493 wait complete. 2024-09-17T03:25:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 Processor -- start 2024-09-17T03:25:58.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 -- start start 2024-09-17T03:25:58.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c0b5c70 0x7fa65c14cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 0x7fa65c14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0cd120 0x7fa65c1f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c169c70 con 0x7fa65c1527b0 2024-09-17T03:25:58.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c169af0 con 0x7fa65c0b5c70 2024-09-17T03:25:58.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c169df0 con 0x7fa65c0cd120 2024-09-17T03:25:58.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 0x7fa65c14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 0x7fa65c14b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55954/0 (socket says 172.21.15.46:55954) 2024-09-17T03:25:58.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 -- 172.21.15.46:0/2870894858 learned_addr learned my addr 172.21.15.46:0/2870894858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:58.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa660e00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c0b5c70 0x7fa65c14cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa661800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0cd120 0x7fa65c1f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 -- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0cd120 msgr2=0x7fa65c1f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0cd120 0x7fa65c1f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 -- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c0b5c70 msgr2=0x7fa65c14cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c0b5c70 0x7fa65c14cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 -- 172.21.15.46:0/2870894858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c14d470 con 0x7fa65c1527b0 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa65be00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 0x7fa65c14b740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c430 tx=0x7fa64800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:58.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648011790 con 0x7fa65c1527b0 2024-09-17T03:25:58.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa648017070 con 0x7fa65c1527b0 2024-09-17T03:25:58.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c14c8a0 con 0x7fa65c1527b0 2024-09-17T03:25:58.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648011ce0 con 0x7fa65c1527b0 2024-09-17T03:25:58.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.372+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c1f8690 con 0x7fa65c1527b0 2024-09-17T03:25:58.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.376+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65c15bb60 con 0x7fa65c1527b0 2024-09-17T03:25:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.376+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fa64801c460 con 0x7fa65c1527b0 2024-09-17T03:25:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.376+0000 7fa658c00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa63c073950 0x7fa63c075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.376+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa6480951a0 con 0x7fa65c1527b0 2024-09-17T03:25:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.376+0000 7fa660e00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa63c073950 0x7fa63c075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.376+0000 7fa660e00640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa63c073950 0x7fa63c075e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa650007fd0 tx=0x7fa650009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:58.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.380+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa64805ec30 con 0x7fa65c1527b0 2024-09-17T03:25:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.576+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa65c14c650 con 0x7fa63c073950 2024-09-17T03:25:58.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.580+0000 7fa658c00640 1 -- 172.21.15.46:0/2870894858 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa65c14c650 con 0x7fa63c073950 2024-09-17T03:25:58.580 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:25:58.581 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:58.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:25:58.581 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:58.581 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.580+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa63c073950 msgr2=0x7fa63c075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.580+0000 7fa663800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa63c073950 0x7fa63c075e60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa650007fd0 tx=0x7fa650009040 comp rx=0 tx=0).stop 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.580+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 msgr2=0x7fa65c14b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.580+0000 7fa663800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 0x7fa65c14b740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c430 tx=0x7fa64800c8f0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 shutdown_connections 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa63c073950 0x7fa63c075e60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa65c0cd120 0x7fa65c1f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa65c1527b0 0x7fa65c14b740 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 --2- 172.21.15.46:0/2870894858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c0b5c70 0x7fa65c14cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 >> 172.21.15.46:0/2870894858 conn(0x7fa65c0b2040 msgr2=0x7fa65c159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:58.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 shutdown_connections 2024-09-17T03:25:58.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.584+0000 7fa663800640 1 -- 172.21.15.46:0/2870894858 wait complete. 2024-09-17T03:25:58.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:25:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 -- 172.21.15.46:0/3139141431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c1527b0 msgr2=0x7f361c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 --2- 172.21.15.46:0/3139141431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c1527b0 0x7f361c074e30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f361c071cb0 tx=0x7f360401c890 comp rx=0 tx=0).stop 2024-09-17T03:25:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 -- 172.21.15.46:0/3139141431 shutdown_connections 2024-09-17T03:25:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 --2- 172.21.15.46:0/3139141431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c1527b0 0x7f361c074e30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 --2- 172.21.15.46:0/3139141431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f361c0b4de0 0x7f361c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 --2- 172.21.15.46:0/3139141431 >> v2:172.21.15.99:3300/0 conn(0x7f361c0b5c70 0x7f361c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 -- 172.21.15.46:0/3139141431 >> 172.21.15.46:0/3139141431 conn(0x7f361c0b2040 msgr2=0x7f361c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 -- 172.21.15.46:0/3139141431 shutdown_connections 2024-09-17T03:25:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.764+0000 7f3620c00640 1 -- 172.21.15.46:0/3139141431 wait complete. 2024-09-17T03:25:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 Processor -- start 2024-09-17T03:25:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 -- start start 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f361c0b4de0 0x7f361c1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 0x7f361c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f361c1527b0 0x7f361c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f361c15f5f0 con 0x7f361c0b5c70 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f361c15f470 con 0x7f361c1527b0 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f361c15f770 con 0x7f361c0b4de0 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 0x7f361c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 0x7f361c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55974/0 (socket says 172.21.15.46:55974) 2024-09-17T03:25:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 -- 172.21.15.46:0/239817929 learned_addr learned my addr 172.21.15.46:0/239817929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f361a800640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f361c1527b0 0x7f361c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3619e00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f361c0b4de0 0x7f361c1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 -- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f361c0b4de0 msgr2=0x7f361c1448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f361c0b4de0 0x7f361c1448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 -- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f361c1527b0 msgr2=0x7f361c156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f361c1527b0 0x7f361c156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 -- 172.21.15.46:0/239817929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c158ea0 con 0x7f361c0b5c70 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3613e00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 0x7f361c156910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f36040b0040 tx=0x7f36040a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040b2040 con 0x7f361c0b5c70 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36040044b0 con 0x7f361c0b5c70 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c159110 con 0x7f361c0b5c70 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604005290 con 0x7f361c0b5c70 2024-09-17T03:25:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.768+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c157990 con 0x7f361c0b5c70 2024-09-17T03:25:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.772+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c0cdcf0 con 0x7f361c0b5c70 2024-09-17T03:25:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.772+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f3604005430 con 0x7f361c0b5c70 2024-09-17T03:25:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.772+0000 7f3612000640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f8073950 0x7f35f8075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.772+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f360412ef60 con 0x7f361c0b5c70 2024-09-17T03:25:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.772+0000 7f3619e00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f8073950 0x7f35f8075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.772+0000 7f3619e00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f8073950 0x7f35f8075e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f360c00bb50 tx=0x7f360c00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.776+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36040164b0 con 0x7f361c0b5c70 2024-09-17T03:25:58.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.980+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f361c15fcb0 con 0x7f35f8073950 2024-09-17T03:25:58.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.984+0000 7f3612000640 1 -- 172.21.15.46:0/239817929 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7f361c15fcb0 con 0x7f35f8073950 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone33690" 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:58.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone80541" 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone96892" 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone67613" 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:58.987 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:25:58.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f8073950 msgr2=0x7f35f8075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f8073950 0x7f35f8075e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f360c00bb50 tx=0x7f360c00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 msgr2=0x7f361c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:58.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 0x7f361c156910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f36040b0040 tx=0x7f36040a5870 comp rx=0 tx=0).stop 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 shutdown_connections 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f8073950 0x7f35f8075e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f361c1527b0 0x7f361c156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f361c0b5c70 0x7f361c156910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 --2- 172.21.15.46:0/239817929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f361c0b4de0 0x7f361c1448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 >> 172.21.15.46:0/239817929 conn(0x7f361c0b2040 msgr2=0x7f361c0b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 shutdown_connections 2024-09-17T03:25:58.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:58.988+0000 7f3620c00640 1 -- 172.21.15.46:0/239817929 wait complete. 2024-09-17T03:25:59.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone33690 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 -- 172.21.15.46:0/3999859296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 msgr2=0x7f612c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 --2- 172.21.15.46:0/3999859296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 0x7f612c06d200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6118009840 tx=0x7f611801c790 comp rx=0 tx=0).stop 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 -- 172.21.15.46:0/3999859296 shutdown_connections 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 --2- 172.21.15.46:0/3999859296 >> v2:172.21.15.99:3300/0 conn(0x7f612c06fc90 0x7f612c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 --2- 172.21.15.46:0/3999859296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 0x7f612c06d200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 -- 172.21.15.46:0/3999859296 >> 172.21.15.46:0/3999859296 conn(0x7f612c0b35f0 msgr2=0x7f612c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 -- 172.21.15.46:0/3999859296 shutdown_connections 2024-09-17T03:25:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 -- 172.21.15.46:0/3999859296 wait complete. 2024-09-17T03:25:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 Processor -- start 2024-09-17T03:25:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.172+0000 7f6133c00640 1 -- start start 2024-09-17T03:25:59.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 0x7f612c1f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 0x7f612c155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f612c159130 0x7f612c0af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f612c168df0 con 0x7f612c06fc90 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f612c168c70 con 0x7f612c159130 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f612c168f70 con 0x7f612c06cdf0 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 0x7f612c155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 0x7f612c155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55984/0 (socket says 172.21.15.46:55984) 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 -- 172.21.15.46:0/940454784 learned_addr learned my addr 172.21.15.46:0/940454784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6131200640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 0x7f612c1f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6131c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f612c159130 0x7f612c0af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 -- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 msgr2=0x7f612c1f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 0x7f612c1f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 -- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f612c159130 msgr2=0x7f612c0af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f612c159130 0x7f612c0af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 -- 172.21.15.46:0/940454784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c0d1030 con 0x7f612c06fc90 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f612be00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 0x7f612c155b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f611801cc70 tx=0x7f61180a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180ac070 con 0x7f612c06fc90 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c0aa670 con 0x7f612c06fc90 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6118005130 con 0x7f612c06fc90 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118005540 con 0x7f612c06fc90 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c15ce60 con 0x7f612c06fc90 2024-09-17T03:25:59.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.176+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c0b21d0 con 0x7f612c06fc90 2024-09-17T03:25:59.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.180+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f61180af070 con 0x7f612c06fc90 2024-09-17T03:25:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.180+0000 7f6128c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f610c073980 0x7f610c075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.180+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f611812f000 con 0x7f612c06fc90 2024-09-17T03:25:59.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.180+0000 7f6131200640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f610c073980 0x7f610c075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.180+0000 7f6131200640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f610c073980 0x7f610c075e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6120001870 tx=0x7f6120001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:59.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.184+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f61180c1030 con 0x7f612c06fc90 2024-09-17T03:25:59.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:25:59.379 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:25:59.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.388+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone33690", "target": ["mon-mgr", ""]}) -- 0x7f612c06b540 con 0x7f610c073980 2024-09-17T03:25:59.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.408+0000 7f6128c00640 1 -- 172.21.15.46:0/940454784 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f612000f6b0 con 0x7f610c073980 2024-09-17T03:25:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f610c073980 msgr2=0x7f610c075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f610c073980 0x7f610c075e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6120001870 tx=0x7f6120001380 comp rx=0 tx=0).stop 2024-09-17T03:25:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 msgr2=0x7f612c155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 0x7f612c155b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f611801cc70 tx=0x7f61180a6f90 comp rx=0 tx=0).stop 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 shutdown_connections 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f610c073980 0x7f610c075e90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f612c159130 0x7f612c0af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f612c06fc90 0x7f612c155b20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 --2- 172.21.15.46:0/940454784 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f612c06cdf0 0x7f612c1f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 >> 172.21.15.46:0/940454784 conn(0x7f612c0b35f0 msgr2=0x7f612c157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 shutdown_connections 2024-09-17T03:25:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.412+0000 7f6133c00640 1 -- 172.21.15.46:0/940454784 wait complete. 2024-09-17T03:25:59.414 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.412+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.414 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.412+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.414 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.412+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.414 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.412+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.414 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.412+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.435 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.432+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.435 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.432+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.435 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.432+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.435 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.432+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.436 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:25:59.432+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:25:59.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:25:59.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 -- 172.21.15.46:0/1807901569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc0d1250 msgr2=0x7ff2dc068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 --2- 172.21.15.46:0/1807901569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc0d1250 0x7ff2dc068df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009900 tx=0x7ff2c401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 -- 172.21.15.46:0/1807901569 shutdown_connections 2024-09-17T03:25:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 --2- 172.21.15.46:0/1807901569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc0d1250 0x7ff2dc068df0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 -- 172.21.15.46:0/1807901569 >> 172.21.15.46:0/1807901569 conn(0x7ff2dc0b35f0 msgr2=0x7ff2dc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 -- 172.21.15.46:0/1807901569 shutdown_connections 2024-09-17T03:25:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.640+0000 7ff2e2400640 1 -- 172.21.15.46:0/1807901569 wait complete. 2024-09-17T03:25:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 Processor -- start 2024-09-17T03:25:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 -- start start 2024-09-17T03:25:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 0x7ff2dc0b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2dc0d0e40 0x7ff2dc159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc14a7b0 0x7ff2dc14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc16b480 con 0x7ff2dc069330 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc16b300 con 0x7ff2dc14a7b0 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc16b600 con 0x7ff2dc0d0e40 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 0x7ff2dc0b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 0x7ff2dc0b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56032/0 (socket says 172.21.15.46:56032) 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 -- 172.21.15.46:0/589130217 learned_addr learned my addr 172.21.15.46:0/589130217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2d3e00640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2dc0d0e40 0x7ff2dc159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2dbe00640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc14a7b0 0x7ff2dc14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 -- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2dc0d0e40 msgr2=0x7ff2dc159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2dc0d0e40 0x7ff2dc159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 -- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc14a7b0 msgr2=0x7ff2dc14abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc14a7b0 0x7ff2dc14abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 -- 172.21.15.46:0/589130217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc14b100 con 0x7ff2dc069330 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2db400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 0x7ff2dc0b6340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c3c0 tx=0x7ff2cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc017070 con 0x7ff2dc069330 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2cc003600 con 0x7ff2dc069330 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc003a50 con 0x7ff2dc069330 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.644+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1fc810 con 0x7ff2dc069330 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.648+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1fcdd0 con 0x7ff2dc069330 2024-09-17T03:25:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.648+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc0d1250 con 0x7ff2dc069330 2024-09-17T03:25:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.648+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7ff2cc01a070 con 0x7ff2dc069330 2024-09-17T03:25:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.648+0000 7ff2d3400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2b8073950 0x7ff2b8075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:25:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.648+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7ff2cc0946c0 con 0x7ff2dc069330 2024-09-17T03:25:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.648+0000 7ff2d3e00640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2b8073950 0x7ff2b8075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:25:59.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.652+0000 7ff2d3e00640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2b8073950 0x7ff2b8075e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af000 tx=0x7ff2c401b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:25:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.652+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff2dc14b100 con 0x7ff2dc069330 2024-09-17T03:25:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.860+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2dc0b6880 con 0x7ff2b8073950 2024-09-17T03:25:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.872+0000 7ff2d3400640 1 -- 172.21.15.46:0/589130217 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff2dc0b6880 con 0x7ff2b8073950 2024-09-17T03:25:59.875 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:25:59.875 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:25:59.875 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:25:59.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:25:59.875 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:25:59.875 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2b8073950 msgr2=0x7ff2b8075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2b8073950 0x7ff2b8075e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af000 tx=0x7ff2c401b120 comp rx=0 tx=0).stop 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 msgr2=0x7ff2dc0b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 0x7ff2dc0b6340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c3c0 tx=0x7ff2cc00c880 comp rx=0 tx=0).stop 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 shutdown_connections 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2b8073950 0x7ff2b8075e60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2dc14a7b0 0x7ff2dc14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2dc0d0e40 0x7ff2dc159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 --2- 172.21.15.46:0/589130217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2dc069330 0x7ff2dc0b6340 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:25:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 >> 172.21.15.46:0/589130217 conn(0x7ff2dc0b35f0 msgr2=0x7ff2dc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:25:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 shutdown_connections 2024-09-17T03:25:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:25:59.876+0000 7ff2e2400640 1 -- 172.21.15.46:0/589130217 wait complete. 2024-09-17T03:25:59.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:26:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 -- 172.21.15.46:0/3786367656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b5c70 msgr2=0x7fa02815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 --2- 172.21.15.46:0/3786367656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b5c70 0x7fa02815bb60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009870 tx=0x7fa01c01c6c0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 -- 172.21.15.46:0/3786367656 shutdown_connections 2024-09-17T03:26:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 --2- 172.21.15.46:0/3786367656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b5c70 0x7fa02815bb60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 -- 172.21.15.46:0/3786367656 >> 172.21.15.46:0/3786367656 conn(0x7fa0280b2040 msgr2=0x7fa0280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 -- 172.21.15.46:0/3786367656 shutdown_connections 2024-09-17T03:26:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.060+0000 7fa030000640 1 -- 172.21.15.46:0/3786367656 wait complete. 2024-09-17T03:26:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 Processor -- start 2024-09-17T03:26:00.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 -- start start 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 0x7fa028154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0280b5c70 0x7fa0280b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0281527b0 0x7fa0280b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa028169cb0 con 0x7fa0281527b0 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa028169b30 con 0x7fa0280b5c70 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa030000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa028169e30 con 0x7fa0280b4de0 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0281527b0 0x7fa0280b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02d600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 0x7fa028154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02d600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 0x7fa028154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:53382/0 (socket says 172.21.15.46:53382) 2024-09-17T03:26:00.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02d600640 1 -- 172.21.15.46:0/1322614045 learned_addr learned my addr 172.21.15.46:0/1322614045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02cc00640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0280b5c70 0x7fa0280b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 -- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 msgr2=0x7fa028154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 0x7fa028154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 -- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0280b5c70 msgr2=0x7fa0280b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0280b5c70 0x7fa0280b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 -- 172.21.15.46:0/1322614045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa028150240 con 0x7fa0281527b0 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02d600640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 0x7fa028154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.064+0000 7fa02e000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0281527b0 0x7fa0280b41f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa02000c700 tx=0x7fa02000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020003b80 con 0x7fa0281527b0 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0281588d0 con 0x7fa0281527b0 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa020003d20 con 0x7fa0281527b0 2024-09-17T03:26:00.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020005330 con 0x7fa0281527b0 2024-09-17T03:26:00.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0281fc270 con 0x7fa0281527b0 2024-09-17T03:26:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa028158640 con 0x7fa0281527b0 2024-09-17T03:26:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fa0200054d0 con 0x7fa0281527b0 2024-09-17T03:26:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa025600640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa008073950 0x7fa008075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa020095260 con 0x7fa0281527b0 2024-09-17T03:26:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.068+0000 7fa02d600640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa008073950 0x7fa008075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.072+0000 7fa02d600640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa008073950 0x7fa008075e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa01c01a2f0 tx=0x7fa01c01af00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.072+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa02005eb20 con 0x7fa0281527b0 2024-09-17T03:26:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.280+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa02815bb60 con 0x7fa008073950 2024-09-17T03:26:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.308+0000 7fa025600640 1 -- 172.21.15.46:0/1322614045 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa02815bb60 con 0x7fa008073950 2024-09-17T03:26:00.309 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.308+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa008073950 msgr2=0x7fa008075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.308+0000 7fa030000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa008073950 0x7fa008075e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa01c01a2f0 tx=0x7fa01c01af00 comp rx=0 tx=0).stop 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.308+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0281527b0 msgr2=0x7fa0280b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.308+0000 7fa030000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0281527b0 0x7fa0280b41f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa02000c700 tx=0x7fa02000cb00 comp rx=0 tx=0).stop 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 shutdown_connections 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa008073950 0x7fa008075e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0281527b0 0x7fa0280b41f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0280b5c70 0x7fa0280b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 --2- 172.21.15.46:0/1322614045 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0280b4de0 0x7fa028154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 >> 172.21.15.46:0/1322614045 conn(0x7fa0280b2040 msgr2=0x7fa028154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 shutdown_connections 2024-09-17T03:26:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.312+0000 7fa030000640 1 -- 172.21.15.46:0/1322614045 wait complete. 2024-09-17T03:26:00.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:26:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 -- 172.21.15.46:0/3824118991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac161160 msgr2=0x7f94ac074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 --2- 172.21.15.46:0/3824118991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac161160 0x7f94ac074950 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0713e0 tx=0x7f94a401cd80 comp rx=0 tx=0).stop 2024-09-17T03:26:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 -- 172.21.15.46:0/3824118991 shutdown_connections 2024-09-17T03:26:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 --2- 172.21.15.46:0/3824118991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac161160 0x7f94ac074950 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 --2- 172.21.15.46:0/3824118991 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac160d50 0x7f94ac074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 --2- 172.21.15.46:0/3824118991 >> v2:172.21.15.99:3300/0 conn(0x7f94ac157030 0x7f94ac0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 -- 172.21.15.46:0/3824118991 >> 172.21.15.46:0/3824118991 conn(0x7f94ac0708c0 msgr2=0x7f94ac070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 -- 172.21.15.46:0/3824118991 shutdown_connections 2024-09-17T03:26:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.492+0000 7f94b2600640 1 -- 172.21.15.46:0/3824118991 wait complete. 2024-09-17T03:26:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 Processor -- start 2024-09-17T03:26:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 -- start start 2024-09-17T03:26:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 0x7f94ac0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac160d50 0x7f94ac0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac161160 0x7f94ac14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac0722c0 con 0x7f94ac157030 2024-09-17T03:26:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac072140 con 0x7f94ac161160 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94b2600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac072440 con 0x7f94ac160d50 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 0x7f94ac0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 0x7f94ac0b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56062/0 (socket says 172.21.15.46:56062) 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 -- 172.21.15.46:0/2709818977 learned_addr learned my addr 172.21.15.46:0/2709818977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94a3e00640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac160d50 0x7f94ac0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94abe00640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac161160 0x7f94ac14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 -- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac160d50 msgr2=0x7f94ac0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac160d50 0x7f94ac0b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 -- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac161160 msgr2=0x7f94ac14cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac161160 0x7f94ac14cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.496+0000 7f94ab400640 1 -- 172.21.15.46:0/2709818977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac0b2020 con 0x7f94ac157030 2024-09-17T03:26:00.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94ab400640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 0x7f94ac0b45b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e7f0 tx=0x7f949c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00eea0 con 0x7f94ac157030 2024-09-17T03:26:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f949c00cd50 con 0x7f94ac157030 2024-09-17T03:26:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c010690 con 0x7f94ac157030 2024-09-17T03:26:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac150200 con 0x7f94ac157030 2024-09-17T03:26:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1507c0 con 0x7f94ac157030 2024-09-17T03:26:00.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac0728f0 con 0x7f94ac157030 2024-09-17T03:26:00.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f949c010830 con 0x7f94ac157030 2024-09-17T03:26:00.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94a3400640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9488073980 0x7f9488075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.500+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f949c095c20 con 0x7f94ac157030 2024-09-17T03:26:00.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.504+0000 7f94a3e00640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9488073980 0x7f9488075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.504+0000 7f94a3e00640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9488073980 0x7f9488075e90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0b5dd0 tx=0x7f94940091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:00.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.508+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f949c05f6c0 con 0x7f94ac157030 2024-09-17T03:26:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.708+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f94ac06b540 con 0x7f9488073980 2024-09-17T03:26:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94a3400640 1 -- 172.21.15.46:0/2709818977 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9494011bb0 con 0x7f9488073980 2024-09-17T03:26:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9488073980 msgr2=0x7f9488075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9488073980 0x7f9488075e90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0b5dd0 tx=0x7f94940091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 msgr2=0x7f94ac0b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 0x7f94ac0b45b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e7f0 tx=0x7f949c00ecb0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 shutdown_connections 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9488073980 0x7f9488075e90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94ac161160 0x7f94ac14cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f94ac160d50 0x7f94ac0b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 --2- 172.21.15.46:0/2709818977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94ac157030 0x7f94ac0b45b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 >> 172.21.15.46:0/2709818977 conn(0x7f94ac0708c0 msgr2=0x7f94ac0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 shutdown_connections 2024-09-17T03:26:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.728+0000 7f94b2600640 1 -- 172.21.15.46:0/2709818977 wait complete. 2024-09-17T03:26:00.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone80541 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 -- 172.21.15.46:0/2579925273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24157030 msgr2=0x7f7f240b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 --2- 172.21.15.46:0/2579925273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24157030 0x7f7f240b5a80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009840 tx=0x7f7f1801c790 comp rx=0 tx=0).stop 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 -- 172.21.15.46:0/2579925273 shutdown_connections 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 --2- 172.21.15.46:0/2579925273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24157030 0x7f7f240b5a80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 -- 172.21.15.46:0/2579925273 >> 172.21.15.46:0/2579925273 conn(0x7f7f240708c0 msgr2=0x7f7f24070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 -- 172.21.15.46:0/2579925273 shutdown_connections 2024-09-17T03:26:00.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 -- 172.21.15.46:0/2579925273 wait complete. 2024-09-17T03:26:00.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 Processor -- start 2024-09-17T03:26:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.924+0000 7f7f2c200640 1 -- start start 2024-09-17T03:26:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 0x7f7f2414b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f24160d50 0x7f7f24158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24159400 0x7f7f24151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2416b8c0 con 0x7f7f24157030 2024-09-17T03:26:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2416b740 con 0x7f7f24159400 2024-09-17T03:26:00.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2416ba40 con 0x7f7f24160d50 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 0x7f7f2414b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 0x7f7f2414b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56064/0 (socket says 172.21.15.46:56064) 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 -- 172.21.15.46:0/3983661205 learned_addr learned my addr 172.21.15.46:0/3983661205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f28e00640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f24160d50 0x7f7f24158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2a200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24159400 0x7f7f24151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 -- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f24160d50 msgr2=0x7f7f24158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f24160d50 0x7f7f24158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 -- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24159400 msgr2=0x7f7f24151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24159400 0x7f7f24151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 -- 172.21.15.46:0/3983661205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2414bd20 con 0x7f7f24157030 2024-09-17T03:26:00.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f29800640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 0x7f7f2414b7e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009790 tx=0x7f7f180a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:00.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18005240 con 0x7f7f24157030 2024-09-17T03:26:00.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f180053e0 con 0x7f7f24157030 2024-09-17T03:26:00.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241f74e0 con 0x7f7f24157030 2024-09-17T03:26:00.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180046d0 con 0x7f7f24157030 2024-09-17T03:26:00.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241f7aa0 con 0x7f7f24157030 2024-09-17T03:26:00.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.928+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f240d2260 con 0x7f7f24157030 2024-09-17T03:26:00.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.932+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f7f180b8030 con 0x7f7f24157030 2024-09-17T03:26:00.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.932+0000 7f7f21600640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f04073950 0x7f7f04075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:00.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.932+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f7f1812f610 con 0x7f7f24157030 2024-09-17T03:26:00.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.932+0000 7f7f28e00640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f04073950 0x7f7f04075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:00.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.932+0000 7f7f28e00640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f04073950 0x7f7f04075e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7f2415aa50 tx=0x7f7f10009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:00.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:00.936+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f180bf030 con 0x7f7f24157030 2024-09-17T03:26:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.136+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone80541", "target": ["mon-mgr", ""]}) -- 0x7f7f2406b540 con 0x7f7f04073950 2024-09-17T03:26:01.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.156+0000 7f7f21600640 1 -- 172.21.15.46:0/3983661205 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f1000f7d0 con 0x7f7f04073950 2024-09-17T03:26:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f04073950 msgr2=0x7f7f04075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f04073950 0x7f7f04075e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7f2415aa50 tx=0x7f7f10009040 comp rx=0 tx=0).stop 2024-09-17T03:26:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 msgr2=0x7f7f2414b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 0x7f7f2414b7e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009790 tx=0x7f7f180a65a0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 shutdown_connections 2024-09-17T03:26:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f04073950 0x7f7f04075e60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f24159400 0x7f7f24151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f24160d50 0x7f7f24158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 --2- 172.21.15.46:0/3983661205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f24157030 0x7f7f2414b7e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 >> 172.21.15.46:0/3983661205 conn(0x7f7f240708c0 msgr2=0x7f7f24075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 shutdown_connections 2024-09-17T03:26:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.160+0000 7f7f2c200640 1 -- 172.21.15.46:0/3983661205 wait complete. 2024-09-17T03:26:01.172 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:01.172+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:01.172 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:01.172+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:01.172 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:01.172+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:01.172 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:01.172+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:01.172 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:01.172+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:01.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone96892 2024-09-17T03:26:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 -- 172.21.15.46:0/3717198529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 msgr2=0x7fc5e815f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 --2- 172.21.15.46:0/3717198529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e815f070 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009900 tx=0x7fc5d801c7e0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 -- 172.21.15.46:0/3717198529 shutdown_connections 2024-09-17T03:26:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 --2- 172.21.15.46:0/3717198529 >> v2:172.21.15.99:3300/0 conn(0x7fc5e80b24d0 0x7fc5e80b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 --2- 172.21.15.46:0/3717198529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e815f070 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 -- 172.21.15.46:0/3717198529 >> 172.21.15.46:0/3717198529 conn(0x7fc5e806cdf0 msgr2=0x7fc5e806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 -- 172.21.15.46:0/3717198529 shutdown_connections 2024-09-17T03:26:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.356+0000 7fc5ed000640 1 -- 172.21.15.46:0/3717198529 wait complete. 2024-09-17T03:26:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 Processor -- start 2024-09-17T03:26:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 -- start start 2024-09-17T03:26:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e81fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5e815f5b0 0x7fc5e80d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5e80d0bd0 0x7fc5e80ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8168a60 con 0x7fc5e8155a00 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81688e0 con 0x7fc5e815f5b0 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5ed000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e8168be0 con 0x7fc5e80d0bd0 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e81fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e81fb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56102/0 (socket says 172.21.15.46:56102) 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.360+0000 7fc5e7400640 1 -- 172.21.15.46:0/312517453 learned_addr learned my addr 172.21.15.46:0/312517453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7e00640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5e80d0bd0 0x7fc5e80ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5dfe00640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5e815f5b0 0x7fc5e80d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7400640 1 -- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5e80d0bd0 msgr2=0x7fc5e80ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7400640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5e80d0bd0 0x7fc5e80ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7400640 1 -- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5e815f5b0 msgr2=0x7fc5e80d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7400640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5e815f5b0 0x7fc5e80d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7400640 1 -- 172.21.15.46:0/312517453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e813f440 con 0x7fc5e8155a00 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5dfe00640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5e815f5b0 0x7fc5e80d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5e7400640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e81fb7f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009850 tx=0x7fc5d80a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b2040 con 0x7fc5e8155a00 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5d80043b0 con 0x7fc5e8155a00 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8005290 con 0x7fc5e8155a00 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e813f720 con 0x7fc5e8155a00 2024-09-17T03:26:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e813fce0 con 0x7fc5e8155a00 2024-09-17T03:26:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e80b3130 con 0x7fc5e8155a00 2024-09-17T03:26:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.364+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fc5d80bd060 con 0x7fc5e8155a00 2024-09-17T03:26:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.368+0000 7fc5df400640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc5c4073950 0x7fc5c4075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.368+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fc5d812e4e0 con 0x7fc5e8155a00 2024-09-17T03:26:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.368+0000 7fc5dfe00640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc5c4073950 0x7fc5c4075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.368+0000 7fc5dfe00640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc5c4073950 0x7fc5c4075e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5e81542c0 tx=0x7fc5d0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.372+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5d8016490 con 0x7fc5e8155a00 2024-09-17T03:26:01.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.392+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fc5d8005430 con 0x7fc5e8155a00 2024-09-17T03:26:01.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:01.583+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone96892", "target": ["mon-mgr", ""]}) -- 0x7fc5e8154e70 con 0x7fc5c4073950 2024-09-17T03:26:02.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5df400640 1 -- 172.21.15.46:0/312517453 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5e8154e70 con 0x7fc5c4073950 2024-09-17T03:26:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc5c4073950 msgr2=0x7fc5c4075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc5c4073950 0x7fc5c4075e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc5e81542c0 tx=0x7fc5d0009040 comp rx=0 tx=0).stop 2024-09-17T03:26:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 msgr2=0x7fc5e81fb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e81fb7f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009850 tx=0x7fc5d80a56e0 comp rx=0 tx=0).stop 2024-09-17T03:26:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 shutdown_connections 2024-09-17T03:26:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc5c4073950 0x7fc5c4075e60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5e80d0bd0 0x7fc5e80ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5e815f5b0 0x7fc5e80d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 --2- 172.21.15.46:0/312517453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8155a00 0x7fc5e81fb7f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 >> 172.21.15.46:0/312517453 conn(0x7fc5e806cdf0 msgr2=0x7fc5e815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 shutdown_connections 2024-09-17T03:26:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:02.775+0000 7fc5ed000640 1 -- 172.21.15.46:0/312517453 wait complete. 2024-09-17T03:26:02.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67613 2024-09-17T03:26:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 -- 172.21.15.46:0/1536499946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec8074950 msgr2=0x7feec80b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 --2- 172.21.15.46:0/1536499946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec8074950 0x7feec80b5260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009900 tx=0x7feeb801c870 comp rx=0 tx=0).stop 2024-09-17T03:26:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 -- 172.21.15.46:0/1536499946 shutdown_connections 2024-09-17T03:26:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 --2- 172.21.15.46:0/1536499946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec8074950 0x7feec80b5260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 -- 172.21.15.46:0/1536499946 >> 172.21.15.46:0/1536499946 conn(0x7feec80b4560 msgr2=0x7feec80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 -- 172.21.15.46:0/1536499946 shutdown_connections 2024-09-17T03:26:03.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.091+0000 7feecd000640 1 -- 172.21.15.46:0/1536499946 wait complete. 2024-09-17T03:26:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.095+0000 7feecd000640 1 Processor -- start 2024-09-17T03:26:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.095+0000 7feecd000640 1 -- start start 2024-09-17T03:26:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feecd000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 0x7feec814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feecd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec8160060 0x7feec8150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feecd000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec80d1160 0x7feec80d1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feecd000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feec816aea0 con 0x7feec8160060 2024-09-17T03:26:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feecd000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feec816ad20 con 0x7feec80d1160 2024-09-17T03:26:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feecd000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feec816b020 con 0x7feec81536f0 2024-09-17T03:26:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec80d1160 0x7feec80d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 0x7feec814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 0x7feec814a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:43700/0 (socket says 172.21.15.46:43700) 2024-09-17T03:26:03.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 -- 172.21.15.46:0/2612090079 learned_addr learned my addr 172.21.15.46:0/2612090079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec6a00640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec8160060 0x7feec8150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 -- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec80d1160 msgr2=0x7feec80d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec80d1160 0x7feec80d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 -- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec8160060 msgr2=0x7feec8150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec8160060 0x7feec8150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 -- 172.21.15.46:0/2612090079 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec814ad70 con 0x7feec81536f0 2024-09-17T03:26:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.099+0000 7feec7400640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 0x7feec814a830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3c0 tx=0x7feeb400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.103+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4017070 con 0x7feec81536f0 2024-09-17T03:26:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.103+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb40053c0 con 0x7feec81536f0 2024-09-17T03:26:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.103+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4003860 con 0x7feec81536f0 2024-09-17T03:26:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.103+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec80d1ab0 con 0x7feec81536f0 2024-09-17T03:26:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.103+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feec8152820 con 0x7feec81536f0 2024-09-17T03:26:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.107+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7feeb401a070 con 0x7feec81536f0 2024-09-17T03:26:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.107+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec80b5260 con 0x7feec81536f0 2024-09-17T03:26:03.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.107+0000 7feebf400640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feea8073b00 0x7feea8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.107+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7feeb4091d90 con 0x7feec81536f0 2024-09-17T03:26:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.111+0000 7feec6a00640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feea8073b00 0x7feea8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.111+0000 7feec6a00640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feea8073b00 0x7feea8076010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feec80d3290 tx=0x7feec0011ce0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.115+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feec80d1ab0 con 0x7feec81536f0 2024-09-17T03:26:03.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.511+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67613", "target": ["mon-mgr", ""]}) -- 0x7feec814b2b0 con 0x7feea8073b00 2024-09-17T03:26:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.547+0000 7feebf400640 1 -- 172.21.15.46:0/2612090079 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feec00154b0 con 0x7feea8073b00 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feea8073b00 msgr2=0x7feea8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feea8073b00 0x7feea8076010 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feec80d3290 tx=0x7feec0011ce0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 msgr2=0x7feec814a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 0x7feec814a830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3c0 tx=0x7feeb400c880 comp rx=0 tx=0).stop 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 shutdown_connections 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feea8073b00 0x7feea8076010 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feec80d1160 0x7feec80d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec8160060 0x7feec8150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 --2- 172.21.15.46:0/2612090079 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feec81536f0 0x7feec814a830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 >> 172.21.15.46:0/2612090079 conn(0x7feec80b4560 msgr2=0x7feec815d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 shutdown_connections 2024-09-17T03:26:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.551+0000 7feecd000640 1 -- 172.21.15.46:0/2612090079 wait complete. 2024-09-17T03:26:03.597 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:26:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:26:03.698 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:26:03.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:26:03.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:26:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3849786806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d25d0 msgr2=0x7f4d980d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3849786806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d25d0 0x7f4d980d0fa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009900 tx=0x7f4d8801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3849786806 shutdown_connections 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3849786806 >> v2:172.21.15.46:3300/0 conn(0x7f4d980d29e0 0x7f4d980d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3849786806 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d25d0 0x7f4d980d0fa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3849786806 >> 172.21.15.46:0/3849786806 conn(0x7f4d98073e00 msgr2=0x7f4d9806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3849786806 shutdown_connections 2024-09-17T03:26:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.967+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3849786806 wait complete. 2024-09-17T03:26:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.971+0000 7f4d9ca00640 1 Processor -- start 2024-09-17T03:26:03.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.971+0000 7f4d9ca00640 1 -- start start 2024-09-17T03:26:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d9ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 0x7f4d9815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d9ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980d25d0 0x7f4d980717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d9ca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d29e0 0x7f4d9815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d9ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d98170860 con 0x7f4d980b60d0 2024-09-17T03:26:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d9ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d98073260 con 0x7f4d980d25d0 2024-09-17T03:26:03.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d9ca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4d981709e0 con 0x7f4d980d29e0 2024-09-17T03:26:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 0x7f4d9815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 0x7f4d9815c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46720/0 (socket says 172.21.15.46:46720) 2024-09-17T03:26:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d95e00640 1 -- 172.21.15.46:0/3571269422 learned_addr learned my addr 172.21.15.46:0/3571269422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d8d400640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980d25d0 0x7f4d980717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.975+0000 7f4d96800640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d29e0 0x7f4d9815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8d400640 1 -- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d29e0 msgr2=0x7f4d9815ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8d400640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d29e0 0x7f4d9815ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8d400640 1 -- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 msgr2=0x7f4d9815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8d400640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 0x7f4d9815c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8d400640 1 -- 172.21.15.46:0/3571269422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d98071d30 con 0x7f4d980d25d0 2024-09-17T03:26:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d95e00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 0x7f4d9815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:26:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8d400640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980d25d0 0x7f4d980717f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d980b3e90 tx=0x7f4d8000eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000cab0 con 0x7f4d980d25d0 2024-09-17T03:26:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d8000cd40 con 0x7f4d980d25d0 2024-09-17T03:26:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80010690 con 0x7f4d980d25d0 2024-09-17T03:26:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d980ca3b0 con 0x7f4d980d25d0 2024-09-17T03:26:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d980ca970 con 0x7f4d980d25d0 2024-09-17T03:26:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.979+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d980b64e0 con 0x7f4d980d25d0 2024-09-17T03:26:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.983+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f4d80010830 con 0x7f4d980d25d0 2024-09-17T03:26:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.983+0000 7f4d8ea00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d74073b00 0x7f4d74076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.983+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7f4d80093610 con 0x7f4d980d25d0 2024-09-17T03:26:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.987+0000 7f4d95e00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d74073b00 0x7f4d74076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.987+0000 7f4d95e00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d74073b00 0x7f4d74076010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d8801af90 tx=0x7f4d8801b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:03.991+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4d8005fa70 con 0x7f4d980d25d0 2024-09-17T03:26:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.299+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4d980b9d30 con 0x7f4d980d25d0 2024-09-17T03:26:04.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.303+0000 7f4d8ea00640 1 -- 172.21.15.46:0/3571269422 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v36)=0 v36) ==== 131+0+0 (secure 0 0 0) 0x7f4d800644a0 con 0x7f4d980d25d0 2024-09-17T03:26:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.303+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d74073b00 msgr2=0x7f4d74076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.303+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d74073b00 0x7f4d74076010 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d8801af90 tx=0x7f4d8801b120 comp rx=0 tx=0).stop 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.303+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980d25d0 msgr2=0x7f4d980717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.303+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980d25d0 0x7f4d980717f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d980b3e90 tx=0x7f4d8000eb50 comp rx=0 tx=0).stop 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 shutdown_connections 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d74073b00 0x7f4d74076010 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d980d29e0 0x7f4d9815ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980d25d0 0x7f4d980717f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 --2- 172.21.15.46:0/3571269422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d980b60d0 0x7f4d9815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 >> 172.21.15.46:0/3571269422 conn(0x7f4d98073e00 msgr2=0x7f4d980b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 shutdown_connections 2024-09-17T03:26:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.307+0000 7f4d9ca00640 1 -- 172.21.15.46:0/3571269422 wait complete. 2024-09-17T03:26:04.343 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:26:04.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:26:04.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:26:04.571 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:26:04.571 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:26:04.571 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:04.572 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:04.587 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:04.587 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:04.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:26:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 -- 172.21.15.46:0/3692528127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc15eea0 msgr2=0x7fbafc15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 --2- 172.21.15.46:0/3692528127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc15eea0 0x7fbafc15f2b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0b79b0 tx=0x7fbaf401ccf0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 -- 172.21.15.46:0/3692528127 shutdown_connections 2024-09-17T03:26:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 --2- 172.21.15.46:0/3692528127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc15eea0 0x7fbafc15f2b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 --2- 172.21.15.46:0/3692528127 >> v2:172.21.15.99:3300/0 conn(0x7fbafc06f710 0x7fbafc15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 -- 172.21.15.46:0/3692528127 >> 172.21.15.46:0/3692528127 conn(0x7fbafc0704e0 msgr2=0x7fbafc0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.879+0000 7fbb02800640 1 -- 172.21.15.46:0/3692528127 shutdown_connections 2024-09-17T03:26:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.883+0000 7fbb02800640 1 -- 172.21.15.46:0/3692528127 wait complete. 2024-09-17T03:26:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.883+0000 7fbb02800640 1 Processor -- start 2024-09-17T03:26:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.883+0000 7fbb02800640 1 -- start start 2024-09-17T03:26:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb02800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbafc1536d0 0x7fbafc13ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb02800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 0x7fbafc0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb02800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 0x7fbafc14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbafc1536d0 0x7fbafc13ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbafc1536d0 0x7fbafc13ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:43734/0 (socket says 172.21.15.46:43734) 2024-09-17T03:26:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb00a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 0x7fbafc0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb00a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 0x7fbafc0b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:48580/0 (socket says 172.21.15.46:48580) 2024-09-17T03:26:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 0x7fbafc14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 0x7fbafc14d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46744/0 (socket says 172.21.15.46:46744) 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb02800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc169ec0 con 0x7fbafc14cce0 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb02800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc169d40 con 0x7fbafc15eea0 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb02800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc16a040 con 0x7fbafc1536d0 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01400640 1 -- 172.21.15.46:0/3826768060 learned_addr learned my addr 172.21.15.46:0/3826768060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 -- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbafc1536d0 msgr2=0x7fbafc13ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbafc1536d0 0x7fbafc13ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 -- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 msgr2=0x7fbafc0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 0x7fbafc0b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 -- 172.21.15.46:0/3826768060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc14d630 con 0x7fbafc14cce0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.887+0000 7fbb01e00640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 0x7fbafc14d0f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40ac000 tx=0x7fbaf4005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4003b90 con 0x7fbafc14cce0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaf4003e20 con 0x7fbafc14cce0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4004cb0 con 0x7fbafc14cce0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc148fa0 con 0x7fbafc14cce0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc149560 con 0x7fbafc14cce0 2024-09-17T03:26:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbb00a00640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 0x7fbafc0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-17T03:26:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.891+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc15d6b0 con 0x7fbafc14cce0 2024-09-17T03:26:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.895+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fbaf4004030 con 0x7fbafc14cce0 2024-09-17T03:26:04.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.895+0000 7fbaf9600640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbadc073b80 0x7fbadc076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:04.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.895+0000 7fbb01400640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbadc073b80 0x7fbadc076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:04.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.895+0000 7fbb01400640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbadc073b80 0x7fbadc076090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00016d0 tx=0x7fbaf0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:04.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.899+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7fbaf40fda10 con 0x7fbafc14cce0 2024-09-17T03:26:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:04.899+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbafc14d630 con 0x7fbafc14cce0 2024-09-17T03:26:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbafc14c7c0 con 0x7fbafc14cce0 2024-09-17T03:26:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbaf9600640 1 -- 172.21.15.46:0/3826768060 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7fbaf40be070 con 0x7fbafc14cce0 2024-09-17T03:26:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbadc073b80 msgr2=0x7fbadc076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbadc073b80 0x7fbadc076090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00016d0 tx=0x7fbaf0009040 comp rx=0 tx=0).stop 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 msgr2=0x7fbafc14d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 0x7fbafc14d0f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40ac000 tx=0x7fbaf4005aa0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 shutdown_connections 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbadc073b80 0x7fbadc076090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbafc14cce0 0x7fbafc14d0f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbafc15eea0 0x7fbafc0b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 --2- 172.21.15.46:0/3826768060 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbafc1536d0 0x7fbafc13ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 >> 172.21.15.46:0/3826768060 conn(0x7fbafc0704e0 msgr2=0x7fbafc0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 shutdown_connections 2024-09-17T03:26:05.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.191+0000 7fbb02800640 1 -- 172.21.15.46:0/3826768060 wait complete. 2024-09-17T03:26:05.215 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:26:05.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:05.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:26:05.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:26:05.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:05.274 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:26:05.298 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:26:05.298 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:26:05.299 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:05.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:26:05.308 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:26:05.308 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:05.308 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:05.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:26:05.348 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:26:05.348 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:26:05.348 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:05.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:26:05.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 -- 172.21.15.46:0/1518519986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc481527b0 msgr2=0x7fbc48074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/1518519986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc481527b0 0x7fbc48074e30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbc48071cb0 tx=0x7fbc3001c890 comp rx=0 tx=0).stop 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 -- 172.21.15.46:0/1518519986 shutdown_connections 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/1518519986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc481527b0 0x7fbc48074e30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/1518519986 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc480b4de0 0x7fbc480b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/1518519986 >> v2:172.21.15.99:3300/0 conn(0x7fbc480b5c70 0x7fbc4815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 -- 172.21.15.46:0/1518519986 >> 172.21.15.46:0/1518519986 conn(0x7fbc480b2040 msgr2=0x7fbc480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 -- 172.21.15.46:0/1518519986 shutdown_connections 2024-09-17T03:26:05.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.547+0000 7fbc4ca00640 1 -- 172.21.15.46:0/1518519986 wait complete. 2024-09-17T03:26:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 Processor -- start 2024-09-17T03:26:05.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 -- start start 2024-09-17T03:26:05.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc480b4de0 0x7fbc4813a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 0x7fbc481578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc481527b0 0x7fbc4814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4815f6b0 con 0x7fbc480b5c70 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4815f530 con 0x7fbc481527b0 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc4ca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4815f830 con 0x7fbc480b4de0 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 0x7fbc481578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 0x7fbc481578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46762/0 (socket says 172.21.15.46:46762) 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 -- 172.21.15.46:0/2130067763 learned_addr learned my addr 172.21.15.46:0/2130067763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc45e00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc480b4de0 0x7fbc4813a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc46800640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc481527b0 0x7fbc4814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 -- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc480b4de0 msgr2=0x7fbc4813a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc480b4de0 0x7fbc4813a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 -- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc481527b0 msgr2=0x7fbc4814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc481527b0 0x7fbc4814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 -- 172.21.15.46:0/2130067763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4813a980 con 0x7fbc480b5c70 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.551+0000 7fbc3d400640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 0x7fbc481578d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbc300b0040 tx=0x7fbc300a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc300b2040 con 0x7fbc480b5c70 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc300043d0 con 0x7fbc480b5c70 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc48157e10 con 0x7fbc480b5c70 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30005290 con 0x7fbc480b5c70 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc48150430 con 0x7fbc480b5c70 2024-09-17T03:26:05.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc48157260 con 0x7fbc480b5c70 2024-09-17T03:26:05.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fbc300bd060 con 0x7fbc480b5c70 2024-09-17T03:26:05.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc3ea00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc24073b00 0x7fbc24076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:05.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7fbc3012bf30 con 0x7fbc480b5c70 2024-09-17T03:26:05.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.555+0000 7fbc45e00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc24073b00 0x7fbc24076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:05.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.559+0000 7fbc45e00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc24073b00 0x7fbc24076010 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009b40 tx=0x7fbc380091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:05.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.559+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc4813a980 con 0x7fbc480b5c70 2024-09-17T03:26:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc4815fd70 con 0x7fbc480b5c70 2024-09-17T03:26:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc3ea00640 1 -- 172.21.15.46:0/2130067763 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5233 (secure 0 0 0) 0x7fbc300164b0 con 0x7fbc480b5c70 2024-09-17T03:26:05.773 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:05.773 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"btime":"2024-09-17T03:24:08:717127+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:24:00.099078+0000","modified":"2024-09-17T03:24:03.111473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245313":{"gid":245313,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4290314221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4290314221},{"type":"v1","addr":"172.21.15.46:6836","nonce":4290314221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245313,"qdb_cluster":[245313]},"id":3}]} 2024-09-17T03:26:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc24073b00 msgr2=0x7fbc24076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc24073b00 0x7fbc24076010 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009b40 tx=0x7fbc380091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 msgr2=0x7fbc481578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 0x7fbc481578d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbc300b0040 tx=0x7fbc300a5790 comp rx=0 tx=0).stop 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 shutdown_connections 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc24073b00 0x7fbc24076010 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc481527b0 0x7fbc4814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc480b5c70 0x7fbc481578d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 --2- 172.21.15.46:0/2130067763 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc480b4de0 0x7fbc4813a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 >> 172.21.15.46:0/2130067763 conn(0x7fbc480b2040 msgr2=0x7fbc4815e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 shutdown_connections 2024-09-17T03:26:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:05.771+0000 7fbc4ca00640 1 -- 172.21.15.46:0/2130067763 wait complete. 2024-09-17T03:26:05.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 -- 172.21.15.46:0/2642283985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806cdf0 msgr2=0x7f309806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 --2- 172.21.15.46:0/2642283985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806cdf0 0x7f309806d200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3088009840 tx=0x7f308801c790 comp rx=0 tx=0).stop 2024-09-17T03:26:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 -- 172.21.15.46:0/2642283985 shutdown_connections 2024-09-17T03:26:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 --2- 172.21.15.46:0/2642283985 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f309806fc90 0x7f30980700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 --2- 172.21.15.46:0/2642283985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806cdf0 0x7f309806d200 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3088009840 tx=0x7f308801c790 comp rx=0 tx=0).stop 2024-09-17T03:26:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 -- 172.21.15.46:0/2642283985 >> 172.21.15.46:0/2642283985 conn(0x7f30980b35f0 msgr2=0x7f30980b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:06.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 -- 172.21.15.46:0/2642283985 shutdown_connections 2024-09-17T03:26:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.055+0000 7f309ce00640 1 -- 172.21.15.46:0/2642283985 wait complete. 2024-09-17T03:26:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.059+0000 7f309ce00640 1 Processor -- start 2024-09-17T03:26:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.059+0000 7f309ce00640 1 -- start start 2024-09-17T03:26:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 0x7f30980b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3098159130 0x7f3098150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30980b2040 0x7f3098151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3098168da0 con 0x7f309806fc90 2024-09-17T03:26:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3098168c20 con 0x7f30980b2040 2024-09-17T03:26:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3098168f20 con 0x7f3098159130 2024-09-17T03:26:06.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 0x7f30980b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 0x7f30980b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46768/0 (socket says 172.21.15.46:46768) 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 -- 172.21.15.46:0/511588235 learned_addr learned my addr 172.21.15.46:0/511588235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095400640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3098159130 0x7f3098150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3096800640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30980b2040 0x7f3098151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 -- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3098159130 msgr2=0x7f3098150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3098159130 0x7f3098150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 -- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30980b2040 msgr2=0x7f3098151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30980b2040 0x7f3098151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 -- 172.21.15.46:0/511588235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3098158830 con 0x7f309806fc90 2024-09-17T03:26:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f3095e00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 0x7f30980b1b00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f309000cb40 tx=0x7f309000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090016020 con 0x7f309806fc90 2024-09-17T03:26:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f309000ab60 con 0x7f309806fc90 2024-09-17T03:26:06.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090012420 con 0x7f309806fc90 2024-09-17T03:26:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.063+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3098164610 con 0x7f309806fc90 2024-09-17T03:26:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.067+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3098156bb0 con 0x7f309806fc90 2024-09-17T03:26:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.067+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309806d200 con 0x7f309806fc90 2024-09-17T03:26:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.067+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f3090019020 con 0x7f309806fc90 2024-09-17T03:26:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.071+0000 7f308e000640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3078073b00 0x7f3078076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.071+0000 7f3095400640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3078073b00 0x7f3078076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.071+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7f309005a5f0 con 0x7f309806fc90 2024-09-17T03:26:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.071+0000 7f3095400640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3078073b00 0x7f3078076010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f30880ae000 tx=0x7f308801afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.079+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f309005ea80 con 0x7f309806fc90 2024-09-17T03:26:06.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.307+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30980d0930 con 0x7f309806fc90 2024-09-17T03:26:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.307+0000 7f308e000640 1 -- 172.21.15.46:0/511588235 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5233 (secure 0 0 0) 0x7f309005e380 con 0x7f309806fc90 2024-09-17T03:26:06.311 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:06.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"btime":"2024-09-17T03:24:08:717127+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:24:00.099078+0000","modified":"2024-09-17T03:24:03.111473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245313":{"gid":245313,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4290314221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4290314221},{"type":"v1","addr":"172.21.15.46:6836","nonce":4290314221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245313,"qdb_cluster":[245313]},"id":3}]} 2024-09-17T03:26:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-17T03:26:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3078073b00 msgr2=0x7f3078076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3078073b00 0x7f3078076010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f30880ae000 tx=0x7f308801afd0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 msgr2=0x7f30980b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 0x7f30980b1b00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f309000cb40 tx=0x7f309000cc30 comp rx=0 tx=0).stop 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 shutdown_connections 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3078073b00 0x7f3078076010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30980b2040 0x7f3098151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3098159130 0x7f3098150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 --2- 172.21.15.46:0/511588235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f309806fc90 0x7f30980b1b00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 >> 172.21.15.46:0/511588235 conn(0x7f30980b35f0 msgr2=0x7f3098076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 shutdown_connections 2024-09-17T03:26:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.311+0000 7f309ce00640 1 -- 172.21.15.46:0/511588235 wait complete. 2024-09-17T03:26:06.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:26:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 -- 172.21.15.46:0/668487492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 msgr2=0x7f4778074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 --2- 172.21.15.46:0/668487492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f4778074e30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4778071cb0 tx=0x7f476001c890 comp rx=0 tx=0).stop 2024-09-17T03:26:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 -- 172.21.15.46:0/668487492 shutdown_connections 2024-09-17T03:26:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 --2- 172.21.15.46:0/668487492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f4778074e30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 --2- 172.21.15.46:0/668487492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47780b4de0 0x7f47780b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 --2- 172.21.15.46:0/668487492 >> v2:172.21.15.99:3300/0 conn(0x7f47780b5c70 0x7f477815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 -- 172.21.15.46:0/668487492 >> 172.21.15.46:0/668487492 conn(0x7f47780b2040 msgr2=0x7f47780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 -- 172.21.15.46:0/668487492 shutdown_connections 2024-09-17T03:26:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.495+0000 7f477e800640 1 -- 172.21.15.46:0/668487492 wait complete. 2024-09-17T03:26:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 Processor -- start 2024-09-17T03:26:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 -- start start 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47780b4de0 0x7f477814b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47780b5c70 0x7f47780ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f47780aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f477815f6b0 con 0x7f47781527b0 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f477815f530 con 0x7f47780b5c70 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f477815f830 con 0x7f47780b4de0 2024-09-17T03:26:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f47780aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f47780aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46788/0 (socket says 172.21.15.46:46788) 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 -- 172.21.15.46:0/2571687244 learned_addr learned my addr 172.21.15.46:0/2571687244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777400640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47780b4de0 0x7f477814b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f476fe00640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47780b5c70 0x7f47780ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 -- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47780b4de0 msgr2=0x7f477814b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47780b4de0 0x7f477814b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 -- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47780b5c70 msgr2=0x7f47780ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47780b5c70 0x7f47780ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 -- 172.21.15.46:0/2571687244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47780ccae0 con 0x7f47781527b0 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f4777e00640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f47780aee20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f477000e960 tx=0x7f477000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770004d30 con 0x7f47781527b0 2024-09-17T03:26:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f477000cb20 con 0x7f47781527b0 2024-09-17T03:26:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770005020 con 0x7f47781527b0 2024-09-17T03:26:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4778153e50 con 0x7f47781527b0 2024-09-17T03:26:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.499+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4778166b40 con 0x7f47781527b0 2024-09-17T03:26:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.503+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47780cccb0 con 0x7f47781527b0 2024-09-17T03:26:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.503+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f47700051c0 con 0x7f47781527b0 2024-09-17T03:26:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.503+0000 7f476f400640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4754073b30 0x7f4754076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.503+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7f4770093360 con 0x7f47781527b0 2024-09-17T03:26:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.507+0000 7f4777400640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4754073b30 0x7f4754076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.507+0000 7f4777400640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4754073b30 0x7f4754076040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476800bb50 tx=0x7f476800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.511+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f477009a070 con 0x7f47781527b0 2024-09-17T03:26:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f477815fd50 con 0x7f47781527b0 2024-09-17T03:26:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f476f400640 1 -- 172.21.15.46:0/2571687244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16841 (secure 0 0 0) 0x7f4770016030 con 0x7f47781527b0 2024-09-17T03:26:06.689 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:06.689 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:26:01.381793+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:23:57.829182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T03:23:59.049469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2d","mappings":[{"from":0,"to":5}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4754073b30 msgr2=0x7f4754076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4754073b30 0x7f4754076040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476800bb50 tx=0x7f476800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 msgr2=0x7f47780aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f47780aee20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f477000e960 tx=0x7f477000ee20 comp rx=0 tx=0).stop 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 shutdown_connections 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4754073b30 0x7f4754076040 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47781527b0 0x7f47780aee20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47780b5c70 0x7f47780ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 --2- 172.21.15.46:0/2571687244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47780b4de0 0x7f477814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 >> 172.21.15.46:0/2571687244 conn(0x7f47780b2040 msgr2=0x7f47781581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 shutdown_connections 2024-09-17T03:26:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.687+0000 7f477e800640 1 -- 172.21.15.46:0/2571687244 wait complete. 2024-09-17T03:26:06.712 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:26:06.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 -- 172.21.15.46:0/3292600461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98157030 msgr2=0x7f1d980b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 --2- 172.21.15.46:0/3292600461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98157030 0x7f1d980b5a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009840 tx=0x7f1d8801c820 comp rx=0 tx=0).stop 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 -- 172.21.15.46:0/3292600461 shutdown_connections 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 --2- 172.21.15.46:0/3292600461 >> v2:172.21.15.99:3300/0 conn(0x7f1d98074b20 0x7f1d9815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 --2- 172.21.15.46:0/3292600461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98157030 0x7f1d980b5a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 -- 172.21.15.46:0/3292600461 >> 172.21.15.46:0/3292600461 conn(0x7f1d980708c0 msgr2=0x7f1d98070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 -- 172.21.15.46:0/3292600461 shutdown_connections 2024-09-17T03:26:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.867+0000 7f1d9e600640 1 -- 172.21.15.46:0/3292600461 wait complete. 2024-09-17T03:26:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 Processor -- start 2024-09-17T03:26:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 -- start start 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 0x7f1d98139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98160d50 0x7f1d98150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815bac0 0x7f1d9815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9816a7d0 con 0x7f1d98157030 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9816a650 con 0x7f1d9815bac0 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d9e600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9816a950 con 0x7f1d98160d50 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d97400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 0x7f1d98139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d97400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 0x7f1d98139910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46792/0 (socket says 172.21.15.46:46792) 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d97400640 1 -- 172.21.15.46:0/4137350533 learned_addr learned my addr 172.21.15.46:0/4137350533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d97e00640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815bac0 0x7f1d9815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.871+0000 7f1d8fe00640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98160d50 0x7f1d98150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d97400640 1 -- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98160d50 msgr2=0x7f1d98150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d97400640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98160d50 0x7f1d98150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d97400640 1 -- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815bac0 msgr2=0x7f1d9815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d97400640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815bac0 0x7f1d9815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d97400640 1 -- 172.21.15.46:0/4137350533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9815c410 con 0x7f1d98157030 2024-09-17T03:26:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d97400640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 0x7f1d98139910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009790 tx=0x7f1d880a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88004240 con 0x7f1d98157030 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d98139e50 con 0x7f1d98157030 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d880bf070 con 0x7f1d98157030 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88003840 con 0x7f1d98157030 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d980b3a30 con 0x7f1d98157030 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9815cf90 con 0x7f1d98157030 2024-09-17T03:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.875+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f1d880039e0 con 0x7f1d98157030 2024-09-17T03:26:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.879+0000 7f1d8f400640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d74073b00 0x7f1d74076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.879+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7f1d8812dd60 con 0x7f1d98157030 2024-09-17T03:26:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.879+0000 7f1d8fe00640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d74073b00 0x7f1d74076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.879+0000 7f1d8fe00640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d74073b00 0x7f1d74076010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d980d1db0 tx=0x7f1d80001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:06.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:06.883+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1d9815c410 con 0x7f1d98157030 2024-09-17T03:26:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d980d2260 con 0x7f1d98157030 2024-09-17T03:26:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d8f400640 1 -- 172.21.15.46:0/4137350533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5233 (secure 0 0 0) 0x7f1d88016410 con 0x7f1d98157030 2024-09-17T03:26:07.096 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:07.096 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"btime":"2024-09-17T03:24:08:717127+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:24:00.099078+0000","modified":"2024-09-17T03:24:03.111473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245313":{"gid":245313,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4290314221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4290314221},{"type":"v1","addr":"172.21.15.46:6836","nonce":4290314221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245313,"qdb_cluster":[245313]},"id":3}]} 2024-09-17T03:26:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-17T03:26:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d74073b00 msgr2=0x7f1d74076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d74073b00 0x7f1d74076010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d980d1db0 tx=0x7f1d80001380 comp rx=0 tx=0).stop 2024-09-17T03:26:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 msgr2=0x7f1d98139910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 0x7f1d98139910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009790 tx=0x7f1d880a64f0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 shutdown_connections 2024-09-17T03:26:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d74073b00 0x7f1d74076010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815bac0 0x7f1d9815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d98160d50 0x7f1d98150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 --2- 172.21.15.46:0/4137350533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d98157030 0x7f1d98139910 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 >> 172.21.15.46:0/4137350533 conn(0x7f1d980708c0 msgr2=0x7f1d9815f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 shutdown_connections 2024-09-17T03:26:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.095+0000 7f1d9e600640 1 -- 172.21.15.46:0/4137350533 wait complete. 2024-09-17T03:26:07.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:07.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 -- 172.21.15.46:0/3308981976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32600cf280 msgr2=0x7f32600cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 --2- 172.21.15.46:0/3308981976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32600cf280 0x7f32600cf690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3258009cb0 tx=0x7f325801cce0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 -- 172.21.15.46:0/3308981976 shutdown_connections 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 --2- 172.21.15.46:0/3308981976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32600cf280 0x7f32600cf690 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 --2- 172.21.15.46:0/3308981976 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f326015e890 0x7f326015eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 -- 172.21.15.46:0/3308981976 >> 172.21.15.46:0/3308981976 conn(0x7f32600b2040 msgr2=0x7f32600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 -- 172.21.15.46:0/3308981976 shutdown_connections 2024-09-17T03:26:07.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.275+0000 7f3265400640 1 -- 172.21.15.46:0/3308981976 wait complete. 2024-09-17T03:26:07.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 Processor -- start 2024-09-17T03:26:07.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 -- start start 2024-09-17T03:26:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32600cf280 0x7f3260139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326013e790 0x7f32600ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 0x7f32600aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32600b5550 con 0x7f326015e890 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32600b53d0 con 0x7f326013e790 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3260140590 con 0x7f32600cf280 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 0x7f32600aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32600cf280 0x7f3260139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3257e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326013e790 0x7f32600ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 0x7f32600aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46808/0 (socket says 172.21.15.46:46808) 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 -- 172.21.15.46:0/59018676 learned_addr learned my addr 172.21.15.46:0/59018676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 -- 172.21.15.46:0/59018676 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32600cf280 msgr2=0x7f3260139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32600cf280 0x7f3260139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 -- 172.21.15.46:0/59018676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326013e790 msgr2=0x7f32600ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326013e790 0x7f32600ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 -- 172.21.15.46:0/59018676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326015c8e0 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f325f200640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 0x7f32600aee20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f325801a910 tx=0x7f32580092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580b2040 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3258004d40 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258004260 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326015cbc0 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.279+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326015d180 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.283+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32600cf690 con 0x7f326015e890 2024-09-17T03:26:07.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.283+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f32580be060 con 0x7f326015e890 2024-09-17T03:26:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.283+0000 7f3256a00640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f323c073b00 0x7f323c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.283+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7f325812c7e0 con 0x7f326015e890 2024-09-17T03:26:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.283+0000 7f325e800640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f323c073b00 0x7f323c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.283+0000 7f325e800640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f323c073b00 0x7f323c076010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3250007fd0 tx=0x7f3250009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.287+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3258016990 con 0x7f326015e890 2024-09-17T03:26:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.499+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32600cd390 con 0x7f326015e890 2024-09-17T03:26:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3256a00640 1 -- 172.21.15.46:0/59018676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5233 (secure 0 0 0) 0x7f32580f8c00 con 0x7f326015e890 2024-09-17T03:26:07.504 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:07.504 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"btime":"2024-09-17T03:24:08:717127+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:24:00.099078+0000","modified":"2024-09-17T03:24:03.111473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245313":{"gid":245313,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4290314221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4290314221},{"type":"v1","addr":"172.21.15.46:6836","nonce":4290314221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245313,"qdb_cluster":[245313]},"id":3}]} 2024-09-17T03:26:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f323c073b00 msgr2=0x7f323c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f323c073b00 0x7f323c076010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3250007fd0 tx=0x7f3250009040 comp rx=0 tx=0).stop 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 msgr2=0x7f32600aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 0x7f32600aee20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f325801a910 tx=0x7f32580092c0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 shutdown_connections 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f323c073b00 0x7f323c076010 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f326015e890 0x7f32600aee20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326013e790 0x7f32600ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 --2- 172.21.15.46:0/59018676 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f32600cf280 0x7f3260139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 >> 172.21.15.46:0/59018676 conn(0x7f32600b2040 msgr2=0x7f326013ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 shutdown_connections 2024-09-17T03:26:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.503+0000 7f3265400640 1 -- 172.21.15.46:0/59018676 wait complete. 2024-09-17T03:26:07.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:26:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 -- 172.21.15.46:0/4155293475 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 msgr2=0x7fa9a8074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4155293475 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a8074240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa994009900 tx=0x7fa99401c850 comp rx=0 tx=0).stop 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 -- 172.21.15.46:0/4155293475 shutdown_connections 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4155293475 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a8074240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4155293475 >> v2:172.21.15.99:3300/0 conn(0x7fa9a81536f0 0x7fa9a815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 -- 172.21.15.46:0/4155293475 >> 172.21.15.46:0/4155293475 conn(0x7fa9a80b4560 msgr2=0x7fa9a80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.683+0000 7fa9ae800640 1 -- 172.21.15.46:0/4155293475 shutdown_connections 2024-09-17T03:26:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 -- 172.21.15.46:0/4155293475 wait complete. 2024-09-17T03:26:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 Processor -- start 2024-09-17T03:26:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 -- start start 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 0x7fa9a8142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a80d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8160470 0x7fa9a80d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a816a490 con 0x7fa9a81536f0 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a816a310 con 0x7fa9a8160470 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.687+0000 7fa9ae800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a816a610 con 0x7fa9a8160060 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 0x7fa9a8142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 0x7fa9a8142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46814/0 (socket says 172.21.15.46:46814) 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ad400640 1 -- 172.21.15.46:0/4085821925 learned_addr learned my addr 172.21.15.46:0/4085821925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a80d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ade00640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8160470 0x7fa9a80d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 -- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8160470 msgr2=0x7fa9a80d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8160470 0x7fa9a80d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 -- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 msgr2=0x7fa9a8142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 0x7fa9a8142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 -- 172.21.15.46:0/4085821925 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a814e830 con 0x7fa9a8160060 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ad400640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 0x7fa9a8142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9aca00640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a80d17f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8160e80 tx=0x7fa994005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940b2040 con 0x7fa9a8160060 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa994004120 con 0x7fa9a8160060 2024-09-17T03:26:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994003600 con 0x7fa9a8160060 2024-09-17T03:26:07.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a80d0810 con 0x7fa9a8160060 2024-09-17T03:26:07.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a80d0dd0 con 0x7fa9a8160060 2024-09-17T03:26:07.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a80d0580 con 0x7fa9a8160060 2024-09-17T03:26:07.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.691+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa9940be060 con 0x7fa9a8160060 2024-09-17T03:26:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.695+0000 7fa9a5600640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa988073b30 0x7fa988076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.695+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7fa99412bed0 con 0x7fa9a8160060 2024-09-17T03:26:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.695+0000 7fa9ad400640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa988073b30 0x7fa988076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.695+0000 7fa9ad400640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa988073b30 0x7fa988076040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009680 tx=0x7fa99c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.699+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa9940164d0 con 0x7fa9a8160060 2024-09-17T03:26:07.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.879+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9a815d480 con 0x7fa9a8160060 2024-09-17T03:26:07.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.879+0000 7fa9a5600640 1 -- 172.21.15.46:0/4085821925 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16841 (secure 0 0 0) 0x7fa9940f82f0 con 0x7fa9a8160060 2024-09-17T03:26:07.881 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:07.881 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:26:01.381793+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:23:57.829182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T03:23:59.049469+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2d","mappings":[{"from":0,"to":5}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa988073b30 msgr2=0x7fa988076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa988073b30 0x7fa988076040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009680 tx=0x7fa99c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 msgr2=0x7fa9a80d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a80d17f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8160e80 tx=0x7fa994005760 comp rx=0 tx=0).stop 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 shutdown_connections 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa988073b30 0x7fa988076040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9a8160470 0x7fa9a80d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9a8160060 0x7fa9a80d17f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 --2- 172.21.15.46:0/4085821925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9a81536f0 0x7fa9a8142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:07.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 >> 172.21.15.46:0/4085821925 conn(0x7fa9a80b4560 msgr2=0x7fa9a80b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:07.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 shutdown_connections 2024-09-17T03:26:07.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:07.883+0000 7fa9ae800640 1 -- 172.21.15.46:0/4085821925 wait complete. 2024-09-17T03:26:07.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:26:08.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 -- 172.21.15.46:0/4043319864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c81527b0 msgr2=0x7f76c815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 --2- 172.21.15.46:0/4043319864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c81527b0 0x7f76c815bb60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009900 tx=0x7f76b801c8b0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 -- 172.21.15.46:0/4043319864 shutdown_connections 2024-09-17T03:26:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 --2- 172.21.15.46:0/4043319864 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f76c80b4de0 0x7f76c80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 --2- 172.21.15.46:0/4043319864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c81527b0 0x7f76c815bb60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 -- 172.21.15.46:0/4043319864 >> 172.21.15.46:0/4043319864 conn(0x7f76c80b2040 msgr2=0x7f76c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 -- 172.21.15.46:0/4043319864 shutdown_connections 2024-09-17T03:26:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 -- 172.21.15.46:0/4043319864 wait complete. 2024-09-17T03:26:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 Processor -- start 2024-09-17T03:26:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.063+0000 7f76d0800640 1 -- start start 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 0x7f76c8149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f76c80b5c70 0x7f76c814bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81527b0 0x7f76c8159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8168a30 con 0x7f76c80b4de0 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76c81688b0 con 0x7f76c81527b0 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f76c8168bb0 con 0x7f76c80b5c70 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 0x7f76c8149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 0x7f76c8149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46828/0 (socket says 172.21.15.46:46828) 2024-09-17T03:26:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 -- 172.21.15.46:0/2236960700 learned_addr learned my addr 172.21.15.46:0/2236960700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cd400640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f76c80b5c70 0x7f76c814bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76ce800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81527b0 0x7f76c8159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 -- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f76c80b5c70 msgr2=0x7f76c814bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f76c80b5c70 0x7f76c814bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 -- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81527b0 msgr2=0x7f76c8159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81527b0 0x7f76c8159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 -- 172.21.15.46:0/2236960700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c80cde20 con 0x7f76c80b4de0 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76ce800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81527b0 0x7f76c8159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76cde00640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 0x7f76c8149e20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00c430 tx=0x7f76bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc017070 con 0x7f76c80b4de0 2024-09-17T03:26:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76bc003600 con 0x7f76c80b4de0 2024-09-17T03:26:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc003a50 con 0x7f76c80b4de0 2024-09-17T03:26:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c81f9c20 con 0x7f76c80b4de0 2024-09-17T03:26:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81fa1e0 con 0x7f76c80b4de0 2024-09-17T03:26:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.067+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c80b51f0 con 0x7f76c80b4de0 2024-09-17T03:26:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.071+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f76bc01a070 con 0x7f76c80b4de0 2024-09-17T03:26:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.071+0000 7f76c2000640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76ac073b30 0x7f76ac076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.071+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5823+0+0 (secure 0 0 0) 0x7f76bc094700 con 0x7f76c80b4de0 2024-09-17T03:26:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.071+0000 7f76cd400640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76ac073b30 0x7f76ac076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.071+0000 7f76cd400640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76ac073b30 0x7f76ac076040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76b80af000 tx=0x7f76b801b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.075+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76c80cde20 con 0x7f76c80b4de0 2024-09-17T03:26:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.291+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f76c8150260 con 0x7f76c80b4de0 2024-09-17T03:26:08.292 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:26:08.291+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:26:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.291+0000 7f76c2000640 1 -- 172.21.15.46:0/2236960700 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f76c80b5730 con 0x7f76c80b4de0 2024-09-17T03:26:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:26:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76ac073b30 msgr2=0x7f76ac076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76ac073b30 0x7f76ac076040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76b80af000 tx=0x7f76b801b0f0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 msgr2=0x7f76c8149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 0x7f76c8149e20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00c430 tx=0x7f76bc00c8f0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 shutdown_connections 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76ac073b30 0x7f76ac076040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c81527b0 0x7f76c8159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f76c80b5c70 0x7f76c814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 --2- 172.21.15.46:0/2236960700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76c80b4de0 0x7f76c8149e20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.295+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 >> 172.21.15.46:0/2236960700 conn(0x7f76c80b2040 msgr2=0x7f76c815b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.299+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 shutdown_connections 2024-09-17T03:26:08.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.299+0000 7f76d0800640 1 -- 172.21.15.46:0/2236960700 wait complete. 2024-09-17T03:26:08.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:26:08.442 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:26:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 -- 172.21.15.46:0/545614348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718074b20 msgr2=0x7f971815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 --2- 172.21.15.46:0/545614348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718074b20 0x7f971815cf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9710009bc0 tx=0x7f971001ca30 comp rx=0 tx=0).stop 2024-09-17T03:26:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 -- 172.21.15.46:0/545614348 shutdown_connections 2024-09-17T03:26:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 --2- 172.21.15.46:0/545614348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718074b20 0x7f971815cf90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 -- 172.21.15.46:0/545614348 >> 172.21.15.46:0/545614348 conn(0x7f97180708c0 msgr2=0x7f9718070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 -- 172.21.15.46:0/545614348 shutdown_connections 2024-09-17T03:26:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.487+0000 7f971d400640 1 -- 172.21.15.46:0/545614348 wait complete. 2024-09-17T03:26:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 Processor -- start 2024-09-17T03:26:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 -- start start 2024-09-17T03:26:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 0x7f97181f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718157030 0x7f971814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f971814ba60 0x7f971815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f971816b4e0 con 0x7f97180b5a80 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f971816b360 con 0x7f9718157030 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f971d400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f971816b660 con 0x7f971814ba60 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 0x7f97181f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 0x7f97181f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46848/0 (socket says 172.21.15.46:46848) 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 -- 172.21.15.46:0/3513924215 learned_addr learned my addr 172.21.15.46:0/3513924215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9717200640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f971814ba60 0x7f971815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f970fe00640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718157030 0x7f971814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 -- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f971814ba60 msgr2=0x7f971815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f971814ba60 0x7f971815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 -- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718157030 msgr2=0x7f971814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718157030 0x7f971814b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 -- 172.21.15.46:0/3513924215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97181f9a20 con 0x7f97180b5a80 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f970fe00640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718157030 0x7f971814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f9716800640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 0x7f97181f94e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f970800c400 tx=0x7f970800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708017070 con 0x7f97180b5a80 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.491+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9708005400 con 0x7f97180b5a80 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9718155100 con 0x7f97180b5a80 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708003860 con 0x7f97180b5a80 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181f8ce0 con 0x7f97180b5a80 2024-09-17T03:26:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97180d2260 con 0x7f97180b5a80 2024-09-17T03:26:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f970801a070 con 0x7f97180b5a80 2024-09-17T03:26:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f970ea00640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96f4073b30 0x7f96f4076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5881+0+0 (secure 0 0 0) 0x7f9708091dc0 con 0x7f97180b5a80 2024-09-17T03:26:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.495+0000 7f970fe00640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96f4073b30 0x7f96f4076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.499+0000 7f970fe00640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96f4073b30 0x7f96f4076040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9700000c10 tx=0x7f9700009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.499+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f970805e270 con 0x7f97180b5a80 2024-09-17T03:26:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:08.715+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f971815cf70 con 0x7f97180b5a80 2024-09-17T03:26:09.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.423+0000 7f970ea00640 1 -- 172.21.15.46:0/3513924215 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f9708062ca0 con 0x7f97180b5a80 2024-09-17T03:26:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96f4073b30 msgr2=0x7f96f4076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96f4073b30 0x7f96f4076040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9700000c10 tx=0x7f9700009040 comp rx=0 tx=0).stop 2024-09-17T03:26:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 msgr2=0x7f97181f94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:09.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 0x7f97181f94e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f970800c400 tx=0x7f970800c8c0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 shutdown_connections 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96f4073b30 0x7f96f4076040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f971814ba60 0x7f971815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9718157030 0x7f971814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 --2- 172.21.15.46:0/3513924215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97180b5a80 0x7f97181f94e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.427+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 >> 172.21.15.46:0/3513924215 conn(0x7f97180708c0 msgr2=0x7f971815f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.431+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 shutdown_connections 2024-09-17T03:26:09.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.431+0000 7f971d400640 1 -- 172.21.15.46:0/3513924215 wait complete. 2024-09-17T03:26:09.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 -- 172.21.15.46:0/3538892997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b5c70 msgr2=0x7fd6f815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/3538892997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b5c70 0x7fd6f815bb60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80cca90 tx=0x7fd6ec01c800 comp rx=0 tx=0).stop 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 -- 172.21.15.46:0/3538892997 shutdown_connections 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/3538892997 >> v2:172.21.15.99:3300/0 conn(0x7fd6f80b4de0 0x7fd6f80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/3538892997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b5c70 0x7fd6f815bb60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 -- 172.21.15.46:0/3538892997 >> 172.21.15.46:0/3538892997 conn(0x7fd6f80b2040 msgr2=0x7fd6f80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 -- 172.21.15.46:0/3538892997 shutdown_connections 2024-09-17T03:26:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.623+0000 7fd6ffa00640 1 -- 172.21.15.46:0/3538892997 wait complete. 2024-09-17T03:26:09.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 Processor -- start 2024-09-17T03:26:09.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 -- start start 2024-09-17T03:26:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 0x7fd6f80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6f80b5c70 0x7fd6f8150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:09.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6f81527b0 0x7fd6f815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8168970 con 0x7fd6f80b4de0 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81687f0 con 0x7fd6f81527b0 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f8168af0 con 0x7fd6f80b5c70 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 0x7fd6f80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 0x7fd6f80ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46870/0 (socket says 172.21.15.46:46870) 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 -- 172.21.15.46:0/4177104352 learned_addr learned my addr 172.21.15.46:0/4177104352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6f80b5c70 0x7fd6f8150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fda00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6f81527b0 0x7fd6f815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 -- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6f80b5c70 msgr2=0x7fd6f8150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6f80b5c70 0x7fd6f8150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 -- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6f81527b0 msgr2=0x7fd6f815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6f81527b0 0x7fd6f815da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 -- 172.21.15.46:0/4177104352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f80ae8b0 con 0x7fd6f80b4de0 2024-09-17T03:26:09.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fda00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6f81527b0 0x7fd6f815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:09.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6fd000640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 0x7fd6f80ade30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0b0040 tx=0x7fd6ec0a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:09.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0bb070 con 0x7fd6f80b4de0 2024-09-17T03:26:09.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ec0044b0 con 0x7fd6f80b4de0 2024-09-17T03:26:09.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0b2040 con 0x7fd6f80b4de0 2024-09-17T03:26:09.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f80aeb90 con 0x7fd6f80b4de0 2024-09-17T03:26:09.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.627+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f80ab5c0 con 0x7fd6f80b4de0 2024-09-17T03:26:09.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.631+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f80b51f0 con 0x7fd6f80b4de0 2024-09-17T03:26:09.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.631+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd6ec005290 con 0x7fd6f80b4de0 2024-09-17T03:26:09.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.631+0000 7fd6f4c00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd6d8073b00 0x7fd6d8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:09.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.631+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5881+0+0 (secure 0 0 0) 0x7fd6ec12c340 con 0x7fd6f80b4de0 2024-09-17T03:26:09.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.631+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd6d8073b00 0x7fd6d8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.631+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd6d8073b00 0x7fd6d8076010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8158f70 tx=0x7fd6e4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:09.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.635+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6ec0164b0 con 0x7fd6f80b4de0 2024-09-17T03:26:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:09.811+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd6f80ae370 con 0x7fd6f80b4de0 2024-09-17T03:26:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.427+0000 7fd6f4c00640 1 -- 172.21.15.46:0/4177104352 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7fd6ec0f87a0 con 0x7fd6f80b4de0 2024-09-17T03:26:10.429 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd6d8073b00 msgr2=0x7fd6d8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd6d8073b00 0x7fd6d8076010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8158f70 tx=0x7fd6e4009040 comp rx=0 tx=0).stop 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 msgr2=0x7fd6f80ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 0x7fd6f80ade30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0b0040 tx=0x7fd6ec0a5820 comp rx=0 tx=0).stop 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 shutdown_connections 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd6d8073b00 0x7fd6d8076010 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6f81527b0 0x7fd6f815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd6f80b5c70 0x7fd6f8150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 --2- 172.21.15.46:0/4177104352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f80b4de0 0x7fd6f80ade30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 >> 172.21.15.46:0/4177104352 conn(0x7fd6f80b2040 msgr2=0x7fd6f8159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 shutdown_connections 2024-09-17T03:26:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.431+0000 7fd6ffa00640 1 -- 172.21.15.46:0/4177104352 wait complete. 2024-09-17T03:26:10.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:26:10.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 -- 172.21.15.46:0/1054996815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0301527b0 msgr2=0x7fc030074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 --2- 172.21.15.46:0/1054996815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0301527b0 0x7fc030074e30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01801c820 comp rx=0 tx=0).stop 2024-09-17T03:26:10.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 -- 172.21.15.46:0/1054996815 shutdown_connections 2024-09-17T03:26:10.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 --2- 172.21.15.46:0/1054996815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0301527b0 0x7fc030074e30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 --2- 172.21.15.46:0/1054996815 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc0300b5c70 0x7fc03015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 -- 172.21.15.46:0/1054996815 >> 172.21.15.46:0/1054996815 conn(0x7fc0300b2040 msgr2=0x7fc0300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 -- 172.21.15.46:0/1054996815 shutdown_connections 2024-09-17T03:26:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.631+0000 7fc035600640 1 -- 172.21.15.46:0/1054996815 wait complete. 2024-09-17T03:26:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 Processor -- start 2024-09-17T03:26:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 -- start start 2024-09-17T03:26:10.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 0x7fc0300cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc0300b5c70 0x7fc030142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301527b0 0x7fc0300aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc030168bb0 con 0x7fc0300b4de0 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc030168a30 con 0x7fc0301527b0 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc030168d30 con 0x7fc0300b5c70 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 0x7fc0300cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 0x7fc0300cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46876/0 (socket says 172.21.15.46:46876) 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 -- 172.21.15.46:0/2470298 learned_addr learned my addr 172.21.15.46:0/2470298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc026000640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc0300b5c70 0x7fc030142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02f400640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301527b0 0x7fc0300aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:10.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 -- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc0300b5c70 msgr2=0x7fc030142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc0300b5c70 0x7fc030142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 -- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301527b0 msgr2=0x7fc0300aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301527b0 0x7fc0300aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 -- 172.21.15.46:0/2470298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0300addc0 con 0x7fc0300b4de0 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02f400640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301527b0 0x7fc0300aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc02ea00640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 0x7fc0300cdb30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc02000e7f0 tx=0x7fc02000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02000ccc0 con 0x7fc0300b4de0 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc02000ce80 con 0x7fc0300b4de0 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0300ae060 con 0x7fc0300b4de0 2024-09-17T03:26:10.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020005280 con 0x7fc0300b4de0 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.635+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0300ae620 con 0x7fc0300b4de0 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.639+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03015bb60 con 0x7fc0300b4de0 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.639+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fc020005420 con 0x7fc0300b4de0 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.639+0000 7fc027400640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc00c073b30 0x7fc00c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.639+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5443+0+0 (secure 0 0 0) 0x7fc0200934c0 con 0x7fc0300b4de0 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.639+0000 7fc026000640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc00c073b30 0x7fc00c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.639+0000 7fc026000640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc00c073b30 0x7fc00c076040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0180af000 tx=0x7fc01801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:10.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.643+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc02005ff30 con 0x7fc0300b4de0 2024-09-17T03:26:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.819+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc030151550 con 0x7fc0300b4de0 2024-09-17T03:26:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.959+0000 7fc027400640 1 -- 172.21.15.46:0/2470298 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7fc02005f830 con 0x7fc0300b4de0 2024-09-17T03:26:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:26:10.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.963+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc00c073b30 msgr2=0x7fc00c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.963+0000 7fc035600640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc00c073b30 0x7fc00c076040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0180af000 tx=0x7fc01801b060 comp rx=0 tx=0).stop 2024-09-17T03:26:10.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.963+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 msgr2=0x7fc0300cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:10.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.963+0000 7fc035600640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 0x7fc0300cdb30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc02000e7f0 tx=0x7fc02000ecb0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 shutdown_connections 2024-09-17T03:26:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc00c073b30 0x7fc00c076040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0301527b0 0x7fc0300aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc0300b5c70 0x7fc030142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 --2- 172.21.15.46:0/2470298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0300b4de0 0x7fc0300cdb30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 >> 172.21.15.46:0/2470298 conn(0x7fc0300b2040 msgr2=0x7fc0300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:10.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 shutdown_connections 2024-09-17T03:26:10.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:10.967+0000 7fc035600640 1 -- 172.21.15.46:0/2470298 wait complete. 2024-09-17T03:26:11.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 -- 172.21.15.46:0/1659951472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2388155a00 msgr2=0x7f238815f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 --2- 172.21.15.46:0/1659951472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2388155a00 0x7f238815f070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2378009840 tx=0x7f237801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 -- 172.21.15.46:0/1659951472 shutdown_connections 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 --2- 172.21.15.46:0/1659951472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 0x7f23880b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 --2- 172.21.15.46:0/1659951472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2388155a00 0x7f238815f070 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 -- 172.21.15.46:0/1659951472 >> 172.21.15.46:0/1659951472 conn(0x7f238806cdf0 msgr2=0x7f238806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 -- 172.21.15.46:0/1659951472 shutdown_connections 2024-09-17T03:26:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 -- 172.21.15.46:0/1659951472 wait complete. 2024-09-17T03:26:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 Processor -- start 2024-09-17T03:26:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.167+0000 7f238ca00640 1 -- start start 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23880b2300 0x7f23880ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 0x7f23880ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 0x7f23880cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23881687b0 con 0x7f2388155a00 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2388168630 con 0x7f23880b2300 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2388168930 con 0x7f238815f5b0 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 0x7f23880ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 0x7f23880ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46884/0 (socket says 172.21.15.46:46884) 2024-09-17T03:26:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 -- 172.21.15.46:0/2781093866 learned_addr learned my addr 172.21.15.46:0/2781093866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f2386800640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23880b2300 0x7f23880ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 -- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 msgr2=0x7f23880cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f2387200640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 0x7f23880cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 0x7f23880cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 -- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23880b2300 msgr2=0x7f23880ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23880b2300 0x7f23880ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 -- 172.21.15.46:0/2781093866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23881f8870 con 0x7f2388155a00 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f2386800640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23880b2300 0x7f23880ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f2387200640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 0x7f23880cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237de00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 0x7f23880ccf60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f237000e7d0 tx=0x7f237000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237000ccc0 con 0x7f2388155a00 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2370010430 con 0x7f2388155a00 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23881fbf80 con 0x7f2388155a00 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370010860 con 0x7f2388155a00 2024-09-17T03:26:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881fc540 con 0x7f2388155a00 2024-09-17T03:26:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.171+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23880cf840 con 0x7f2388155a00 2024-09-17T03:26:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.175+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2370010a00 con 0x7f2388155a00 2024-09-17T03:26:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.175+0000 7f237f400640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2364073b00 0x7f2364076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.175+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7f2370093500 con 0x7f2388155a00 2024-09-17T03:26:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.175+0000 7f2386800640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2364073b00 0x7f2364076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.175+0000 7f2386800640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2364073b00 0x7f2364076010 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23780af000 tx=0x7f237801b030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.179+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2370060000 con 0x7f2388155a00 2024-09-17T03:26:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.355+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f238815f070 con 0x7f2388155a00 2024-09-17T03:26:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f237f400640 1 -- 172.21.15.46:0/2781093866 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f237005f900 con 0x7f2388155a00 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2364073b00 msgr2=0x7f2364076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2364073b00 0x7f2364076010 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23780af000 tx=0x7f237801b030 comp rx=0 tx=0).stop 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 msgr2=0x7f23880ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 0x7f23880ccf60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f237000e7d0 tx=0x7f237000ec90 comp rx=0 tx=0).stop 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 shutdown_connections 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2364073b00 0x7f2364076010 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f238815f5b0 0x7f23880cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2388155a00 0x7f23880ccf60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 --2- 172.21.15.46:0/2781093866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23880b2300 0x7f23880ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 >> 172.21.15.46:0/2781093866 conn(0x7f238806cdf0 msgr2=0x7f23880737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:11.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 shutdown_connections 2024-09-17T03:26:11.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.359+0000 7f238ca00640 1 -- 172.21.15.46:0/2781093866 wait complete. 2024-09-17T03:26:11.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-17T03:26:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 -- 172.21.15.46:0/2239381305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 msgr2=0x7fc684074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 --2- 172.21.15.46:0/2239381305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc684074db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc6840726f0 tx=0x7fc67801c8c0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 -- 172.21.15.46:0/2239381305 shutdown_connections 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 --2- 172.21.15.46:0/2239381305 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc684160330 0x7fc684160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 --2- 172.21.15.46:0/2239381305 >> v2:172.21.15.99:3300/0 conn(0x7fc6840cefa0 0x7fc6840cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 --2- 172.21.15.46:0/2239381305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc684074db0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 -- 172.21.15.46:0/2239381305 >> 172.21.15.46:0/2239381305 conn(0x7fc6840b2040 msgr2=0x7fc6840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 -- 172.21.15.46:0/2239381305 shutdown_connections 2024-09-17T03:26:11.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 -- 172.21.15.46:0/2239381305 wait complete. 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.543+0000 7fc68c600640 1 Processor -- start 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- start start 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc6841fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6840cefa0 0x7fc6840aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc684160330 0x7fc6840af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6840b3ee0 con 0x7fc6840749a0 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6840b3d60 con 0x7fc684160330 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc6840b4060 con 0x7fc6840cefa0 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc6841fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc6841fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46906/0 (socket says 172.21.15.46:46906) 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 -- 172.21.15.46:0/4136065870 learned_addr learned my addr 172.21.15.46:0/4136065870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689200640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6840cefa0 0x7fc6840aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68a600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc684160330 0x7fc6840af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 -- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6840cefa0 msgr2=0x7fc6840aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6840cefa0 0x7fc6840aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 -- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc684160330 msgr2=0x7fc6840af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc684160330 0x7fc6840af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 -- 172.21.15.46:0/4136065870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6840ccf80 con 0x7fc6840749a0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc689c00640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc6841fc910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc6780b0040 tx=0x7fc678005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780b2040 con 0x7fc6840749a0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc678005320 con 0x7fc6840749a0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780046d0 con 0x7fc6840749a0 2024-09-17T03:26:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6840cd260 con 0x7fc6840749a0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6840cd820 con 0x7fc6840749a0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.547+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6840cf3b0 con 0x7fc6840749a0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.551+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fc6780bd060 con 0x7fc6840749a0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.551+0000 7fc681600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc664073b30 0x7fc664076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.551+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7fc67812c750 con 0x7fc6840749a0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.551+0000 7fc689200640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc664073b30 0x7fc664076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.551+0000 7fc689200640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc664073b30 0x7fc664076040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6840cef70 tx=0x7fc6700091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.555+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc6780164b0 con 0x7fc6840749a0 2024-09-17T03:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:11.791+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7fc684145490 con 0x7fc6840749a0 2024-09-17T03:26:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.439+0000 7fc681600640 1 -- 172.21.15.46:0/4136065870 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v245) ==== 166+0+0 (secure 0 0 0) 0x7fc6780f9250 con 0x7fc6840749a0 2024-09-17T03:26:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc664073b30 msgr2=0x7fc664076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc664073b30 0x7fc664076040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6840cef70 tx=0x7fc6700091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 msgr2=0x7fc6841fc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc6841fc910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc6780b0040 tx=0x7fc678005de0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 shutdown_connections 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc664073b30 0x7fc664076040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc684160330 0x7fc6840af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6840cefa0 0x7fc6840aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 --2- 172.21.15.46:0/4136065870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6840749a0 0x7fc6841fc910 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 >> 172.21.15.46:0/4136065870 conn(0x7fc6840b2040 msgr2=0x7fc6840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 shutdown_connections 2024-09-17T03:26:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.443+0000 7fc68c600640 1 -- 172.21.15.46:0/4136065870 wait complete. 2024-09-17T03:26:12.469 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-17T03:26:12.469 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-17T03:26:12.469 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:26:12.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:26:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 -- 172.21.15.46:0/1661240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b4de0 msgr2=0x7f6ba40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 --2- 172.21.15.46:0/1661240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b4de0 0x7f6ba40b51f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009c80 tx=0x7f6b9c01cd70 comp rx=0 tx=0).stop 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 -- 172.21.15.46:0/1661240 shutdown_connections 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 --2- 172.21.15.46:0/1661240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b4de0 0x7f6ba40b51f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 --2- 172.21.15.46:0/1661240 >> v2:172.21.15.99:3300/0 conn(0x7f6ba40b5c70 0x7f6ba415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 -- 172.21.15.46:0/1661240 >> 172.21.15.46:0/1661240 conn(0x7f6ba40b2040 msgr2=0x7f6ba40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 -- 172.21.15.46:0/1661240 shutdown_connections 2024-09-17T03:26:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.631+0000 7f6baa800640 1 -- 172.21.15.46:0/1661240 wait complete. 2024-09-17T03:26:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 Processor -- start 2024-09-17T03:26:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 -- start start 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ba40b4de0 0x7f6ba40af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b5c70 0x7f6ba414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 0x7f6ba4150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41689d0 con 0x7f6ba41527b0 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4168850 con 0x7f6ba40b4de0 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba4168b50 con 0x7f6ba40b5c70 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 0x7f6ba4150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 0x7f6ba4150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41586/0 (socket says 172.21.15.46:41586) 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 -- 172.21.15.46:0/2800642785 learned_addr learned my addr 172.21.15.46:0/2800642785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3400640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ba40b4de0 0x7f6ba40af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6b9be00640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b5c70 0x7f6ba414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 -- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b5c70 msgr2=0x7f6ba414eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b5c70 0x7f6ba414eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 -- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ba40b4de0 msgr2=0x7f6ba40af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ba40b4de0 0x7f6ba40af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 -- 172.21.15.46:0/2800642785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba41f9f60 con 0x7f6ba41527b0 2024-09-17T03:26:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6ba3e00640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 0x7f6ba4150460 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c01b420 tx=0x7f6b9c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0bb070 con 0x7f6ba41527b0 2024-09-17T03:26:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b9c004b40 con 0x7f6ba41527b0 2024-09-17T03:26:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba41fa240 con 0x7f6ba41527b0 2024-09-17T03:26:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0b2040 con 0x7f6ba41527b0 2024-09-17T03:26:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.635+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41fa800 con 0x7f6ba41527b0 2024-09-17T03:26:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.639+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba40b51f0 con 0x7f6ba41527b0 2024-09-17T03:26:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.639+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6b9c0be070 con 0x7f6ba41527b0 2024-09-17T03:26:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.639+0000 7f6b9b400640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b80073b30 0x7f6b80076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.639+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7f6b9c12c9d0 con 0x7f6ba41527b0 2024-09-17T03:26:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.639+0000 7f6ba3400640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b80073b30 0x7f6b80076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.639+0000 7f6ba3400640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b80073b30 0x7f6b80076040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009600 tx=0x7f6b940091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.643+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b9c12ee20 con 0x7f6ba41527b0 2024-09-17T03:26:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ba40b4290 con 0x7f6ba41527b0 2024-09-17T03:26:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6b9b400640 1 -- 172.21.15.46:0/2800642785 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+13039 (secure 0 0 0) 0x7f6b9c016990 con 0x7f6ba41527b0 2024-09-17T03:26:12.825 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:12.825 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:26:10.963795+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6836/4290314221":"2024-09-18T03:26:08.294526+0000","172.21.15.46:6835/4290314221":"2024-09-18T03:26:08.294526+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:26:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b80073b30 msgr2=0x7f6b80076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b80073b30 0x7f6b80076040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009600 tx=0x7f6b940091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 msgr2=0x7f6ba4150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 0x7f6ba4150460 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c01b420 tx=0x7f6b9c003f40 comp rx=0 tx=0).stop 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 shutdown_connections 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b80073b30 0x7f6b80076040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba41527b0 0x7f6ba4150460 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6ba40b5c70 0x7f6ba414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 --2- 172.21.15.46:0/2800642785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ba40b4de0 0x7f6ba40af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.823+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 >> 172.21.15.46:0/2800642785 conn(0x7f6ba40b2040 msgr2=0x7f6ba4151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.827+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 shutdown_connections 2024-09-17T03:26:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:12.827+0000 7f6baa800640 1 -- 172.21.15.46:0/2800642785 wait complete. 2024-09-17T03:26:12.848 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:26:12.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:26:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f0576400640 1 -- 172.21.15.46:0/443841688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 msgr2=0x7f0554005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f0576400640 1 --2- 172.21.15.46:0/443841688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f0554005160 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ee0 tx=0x7f056801d250 comp rx=0 tx=0).stop 2024-09-17T03:26:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f056fe00640 1 -- 172.21.15.46:0/443841688 reap_dead start 2024-09-17T03:26:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f0576400640 1 -- 172.21.15.46:0/443841688 shutdown_connections 2024-09-17T03:26:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f0576400640 1 -- 172.21.15.46:0/443841688 >> 172.21.15.46:0/443841688 conn(0x7f05700b2040 msgr2=0x7f05700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f0576400640 1 -- 172.21.15.46:0/443841688 shutdown_connections 2024-09-17T03:26:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.007+0000 7f0576400640 1 -- 172.21.15.46:0/443841688 wait complete. 2024-09-17T03:26:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 Processor -- start 2024-09-17T03:26:13.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 -- start start 2024-09-17T03:26:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f05701ea200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05701ea740 0x7f05701f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05701421f0 0x7f05701f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f057015cf30 con 0x7f0554004d50 2024-09-17T03:26:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f057015cdb0 con 0x7f05701421f0 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f0576400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f057015d0b0 con 0x7f05701ea740 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f05701ea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f05701ea200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41614/0 (socket says 172.21.15.46:41614) 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 -- 172.21.15.46:0/222638925 learned_addr learned my addr 172.21.15.46:0/222638925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056fe00640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05701421f0 0x7f05701f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056ea00640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05701ea740 0x7f05701f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 -- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05701ea740 msgr2=0x7f05701f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05701ea740 0x7f05701f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 -- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05701421f0 msgr2=0x7f05701f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05701421f0 0x7f05701f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056f400640 1 -- 172.21.15.46:0/222638925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05700ac8a0 con 0x7f0554004d50 2024-09-17T03:26:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.011+0000 7f056ea00640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05701ea740 0x7f05701f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f056f400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f05701ea200 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0560001a40 tx=0x7f05600127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560013860 con 0x7f0554004d50 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0560013a20 con 0x7f0554004d50 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560010480 con 0x7f0554004d50 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05701f9fc0 con 0x7f0554004d50 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05700ae3c0 con 0x7f0554004d50 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05700ae620 con 0x7f0554004d50 2024-09-17T03:26:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f0560010620 con 0x7f0554004d50 2024-09-17T03:26:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.015+0000 7f0567400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0550073b00 0x7f0550076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.019+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7f0560096820 con 0x7f0554004d50 2024-09-17T03:26:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.019+0000 7f056ea00640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0550073b00 0x7f0550076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.019+0000 7f056ea00640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0550073b00 0x7f0550076010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f05700b6600 tx=0x7f055c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.023+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f05700b5730 con 0x7f0554004d50 2024-09-17T03:26:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.195+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05700768c0 con 0x7f0554004d50 2024-09-17T03:26:13.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0567400640 1 -- 172.21.15.46:0/222638925 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f0560062c40 con 0x7f0554004d50 2024-09-17T03:26:13.200 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:26:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0550073b00 msgr2=0x7f0550076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0550073b00 0x7f0550076010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f05700b6600 tx=0x7f055c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 msgr2=0x7f05701ea200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f05701ea200 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0560001a40 tx=0x7f05600127b0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 shutdown_connections 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0550073b00 0x7f0550076010 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05701421f0 0x7f05701f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05701ea740 0x7f05701f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 --2- 172.21.15.46:0/222638925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0554004d50 0x7f05701ea200 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 >> 172.21.15.46:0/222638925 conn(0x7f05700b2040 msgr2=0x7f0570152d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 shutdown_connections 2024-09-17T03:26:13.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.199+0000 7f0576400640 1 -- 172.21.15.46:0/222638925 wait complete. 2024-09-17T03:26:13.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-17T03:26:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 -- 172.21.15.46:0/1879458956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3780cf900 msgr2=0x7ff37815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 --2- 172.21.15.46:0/1879458956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3780cf900 0x7ff37815e8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff36800a950 tx=0x7ff36801d9d0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 -- 172.21.15.46:0/1879458956 shutdown_connections 2024-09-17T03:26:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 --2- 172.21.15.46:0/1879458956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3780cf900 0x7ff37815e8c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 -- 172.21.15.46:0/1879458956 >> 172.21.15.46:0/1879458956 conn(0x7ff3780b2040 msgr2=0x7ff3780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 -- 172.21.15.46:0/1879458956 shutdown_connections 2024-09-17T03:26:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.379+0000 7ff37d000640 1 -- 172.21.15.46:0/1879458956 wait complete. 2024-09-17T03:26:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 Processor -- start 2024-09-17T03:26:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 -- start start 2024-09-17T03:26:13.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3780cf900 0x7ff37814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff37815e290 0x7ff378152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 0x7ff37813b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff378168a60 con 0x7ff37815ee00 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff378168be0 con 0x7ff37815e290 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff37d000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff378168d60 con 0x7ff3780cf900 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff376800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 0x7ff37813b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff376800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 0x7ff37813b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41624/0 (socket says 172.21.15.46:41624) 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff376800640 1 -- 172.21.15.46:0/1333751163 learned_addr learned my addr 172.21.15.46:0/1333751163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.383+0000 7ff36d400640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff37815e290 0x7ff378152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff375e00640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3780cf900 0x7ff37814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff376800640 1 -- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3780cf900 msgr2=0x7ff37814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff376800640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3780cf900 0x7ff37814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff376800640 1 -- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff37815e290 msgr2=0x7ff378152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:13.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff376800640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff37815e290 0x7ff378152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff376800640 1 -- 172.21.15.46:0/1333751163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3781f4800 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff376800640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 0x7ff37813b690 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff37000c730 tx=0x7ff37000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370003b80 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff370003d20 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37000aab0 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781f4ae0 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3781f50a0 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37815e8c0 con 0x7ff37815ee00 2024-09-17T03:26:13.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.387+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7ff37000ac50 con 0x7ff37815ee00 2024-09-17T03:26:13.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.391+0000 7ff36ea00640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff354073b30 0x7ff354076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.391+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7ff3700926f0 con 0x7ff37815ee00 2024-09-17T03:26:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.391+0000 7ff375e00640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff354073b30 0x7ff354076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:13.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.391+0000 7ff375e00640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff354073b30 0x7ff354076040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3680b0000 tx=0x7ff36801c210 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:13.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.395+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff37815f210 con 0x7ff37815ee00 2024-09-17T03:26:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:13.635+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7ff37815bb60 con 0x7ff37815ee00 2024-09-17T03:26:14.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.447+0000 7ff36ea00640 1 -- 172.21.15.46:0/1333751163 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v247) ==== 170+0+0 (secure 0 0 0) 0x7ff37005eba0 con 0x7ff37815ee00 2024-09-17T03:26:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff354073b30 msgr2=0x7ff354076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff354073b30 0x7ff354076040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3680b0000 tx=0x7ff36801c210 comp rx=0 tx=0).stop 2024-09-17T03:26:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 msgr2=0x7ff37813b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 0x7ff37813b690 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff37000c730 tx=0x7ff37000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 shutdown_connections 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff354073b30 0x7ff354076040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff37815ee00 0x7ff37813b690 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff37815e290 0x7ff378152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 --2- 172.21.15.46:0/1333751163 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff3780cf900 0x7ff37814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 >> 172.21.15.46:0/1333751163 conn(0x7ff3780b2040 msgr2=0x7ff3780cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 shutdown_connections 2024-09-17T03:26:14.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.451+0000 7ff37d000640 1 -- 172.21.15.46:0/1333751163 wait complete. 2024-09-17T03:26:14.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:26:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 -- 172.21.15.46:0/4028227037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a640b54f0 msgr2=0x7f1a6415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4028227037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a640b54f0 0x7f1a6415bb60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009bc0 tx=0x7f1a5c01cae0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 -- 172.21.15.46:0/4028227037 shutdown_connections 2024-09-17T03:26:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4028227037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a640b54f0 0x7f1a6415bb60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 -- 172.21.15.46:0/4028227037 >> 172.21.15.46:0/4028227037 conn(0x7f1a640b2040 msgr2=0x7f1a640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 -- 172.21.15.46:0/4028227037 shutdown_connections 2024-09-17T03:26:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 -- 172.21.15.46:0/4028227037 wait complete. 2024-09-17T03:26:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 Processor -- start 2024-09-17T03:26:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.643+0000 7f1a6c000640 1 -- start start 2024-09-17T03:26:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a640b5c70 0x7f1a641530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:14.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a641527b0 0x7f1a64151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 0x7f1a640ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64169b30 con 0x7f1a64152bc0 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641699b0 con 0x7f1a640b5c70 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1a64169cb0 con 0x7f1a641527b0 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 0x7f1a640ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 0x7f1a640ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41648/0 (socket says 172.21.15.46:41648) 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 -- 172.21.15.46:0/4080891245 learned_addr learned my addr 172.21.15.46:0/4080891245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a68c00640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a641527b0 0x7f1a64151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a69600640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a640b5c70 0x7f1a641530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 -- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a641527b0 msgr2=0x7f1a64151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a641527b0 0x7f1a64151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 -- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a640b5c70 msgr2=0x7f1a641530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a640b5c70 0x7f1a641530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 -- 172.21.15.46:0/4080891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a640c5f60 con 0x7f1a64152bc0 2024-09-17T03:26:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a68c00640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a641527b0 0x7f1a64151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a69600640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a640b5c70 0x7f1a641530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6a000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 0x7f1a640ae910 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c01c550 tx=0x7f1a5c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:14.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0b2040 con 0x7f1a64152bc0 2024-09-17T03:26:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a5c0bb070 con 0x7f1a64152bc0 2024-09-17T03:26:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c004630 con 0x7f1a64152bc0 2024-09-17T03:26:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a64154800 con 0x7f1a64152bc0 2024-09-17T03:26:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a640c59a0 con 0x7f1a64152bc0 2024-09-17T03:26:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.647+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a64152020 con 0x7f1a64152bc0 2024-09-17T03:26:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.651+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f1a5c0c0460 con 0x7f1a64152bc0 2024-09-17T03:26:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.651+0000 7f1a61600640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a44073b00 0x7f1a44076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.651+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7f1a5c12d5e0 con 0x7f1a64152bc0 2024-09-17T03:26:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.651+0000 7f1a69600640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a44073b00 0x7f1a44076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:14.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.651+0000 7f1a69600640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a44073b00 0x7f1a44076010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000f70 tx=0x7f1a58009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.655+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1a5c016790 con 0x7f1a64152bc0 2024-09-17T03:26:14.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.831+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1a6415bb60 con 0x7f1a64152bc0 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.831+0000 7f1a61600640 1 -- 172.21.15.46:0/4080891245 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7f1a5c0fa140 con 0x7f1a64152bc0 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:26:14.834 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:26:14.835 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:26:14.836 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a44073b00 msgr2=0x7f1a44076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a44073b00 0x7f1a44076010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000f70 tx=0x7f1a58009040 comp rx=0 tx=0).stop 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 msgr2=0x7f1a640ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 0x7f1a640ae910 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c01c550 tx=0x7f1a5c005ae0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 shutdown_connections 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1a44073b00 0x7f1a44076010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a64152bc0 0x7f1a640ae910 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1a641527b0 0x7f1a64151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 --2- 172.21.15.46:0/4080891245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a640b5c70 0x7f1a641530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 >> 172.21.15.46:0/4080891245 conn(0x7f1a640b2040 msgr2=0x7f1a6415a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 shutdown_connections 2024-09-17T03:26:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:14.835+0000 7f1a6c000640 1 -- 172.21.15.46:0/4080891245 wait complete. 2024-09-17T03:26:14.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:26:15.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 -- 172.21.15.46:0/790898958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa01527b0 msgr2=0x7f1fa0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 --2- 172.21.15.46:0/790898958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa01527b0 0x7f1fa0074e30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0071cb0 tx=0x7f1f8801c890 comp rx=0 tx=0).stop 2024-09-17T03:26:15.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 -- 172.21.15.46:0/790898958 shutdown_connections 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 --2- 172.21.15.46:0/790898958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa01527b0 0x7f1fa0074e30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 --2- 172.21.15.46:0/790898958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1fa00b4de0 0x7f1fa00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 --2- 172.21.15.46:0/790898958 >> v2:172.21.15.99:3300/0 conn(0x7f1fa00b5c70 0x7f1fa015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 -- 172.21.15.46:0/790898958 >> 172.21.15.46:0/790898958 conn(0x7f1fa00b2040 msgr2=0x7f1fa00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 -- 172.21.15.46:0/790898958 shutdown_connections 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.015+0000 7f1fa6800640 1 -- 172.21.15.46:0/790898958 wait complete. 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 Processor -- start 2024-09-17T03:26:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 -- start start 2024-09-17T03:26:15.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00b4de0 0x7f1fa0144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 0x7f1fa0153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1fa01527b0 0x7f1fa0158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa015f6b0 con 0x7f1fa00b5c70 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa015f530 con 0x7f1fa00b4de0 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1fa6800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa015f830 con 0x7f1fa01527b0 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1f97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 0x7f1fa0153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1f97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 0x7f1fa0153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41666/0 (socket says 172.21.15.46:41666) 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1f97e00640 1 -- 172.21.15.46:0/3065002128 learned_addr learned my addr 172.21.15.46:0/3065002128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.019+0000 7f1f9f400640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00b4de0 0x7f1fa0144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f9fe00640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1fa01527b0 0x7f1fa0158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97e00640 1 -- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1fa01527b0 msgr2=0x7f1fa0158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97e00640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1fa01527b0 0x7f1fa0158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97e00640 1 -- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00b4de0 msgr2=0x7f1fa0144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97e00640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00b4de0 0x7f1fa0144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97e00640 1 -- 172.21.15.46:0/3065002128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa00cce60 con 0x7f1fa00b5c70 2024-09-17T03:26:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f9f400640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00b4de0 0x7f1fa0144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97e00640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 0x7f1fa0153ab0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1f8801af40 tx=0x7f1f88004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f880b2040 con 0x7f1fa00b5c70 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f88005030 con 0x7f1fa00b5c70 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88005460 con 0x7f1fa00b5c70 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa0153ff0 con 0x7f1fa00b5c70 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa01fcdf0 con 0x7f1fa00b5c70 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa015fd70 con 0x7f1fa00b5c70 2024-09-17T03:26:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.023+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f1f880be060 con 0x7f1fa00b5c70 2024-09-17T03:26:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.027+0000 7f1f97400640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f7c073b00 0x7f1f7c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.027+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7f1f8812c140 con 0x7f1fa00b5c70 2024-09-17T03:26:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.027+0000 7f1f9f400640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f7c073b00 0x7f1f7c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.027+0000 7f1f9f400640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f7c073b00 0x7f1f7c076010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000bbd0 tx=0x7f1f9000b5b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:15.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.031+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f880164b0 con 0x7f1fa00b5c70 2024-09-17T03:26:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.203+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1fa00b2730 con 0x7f1fa00b5c70 2024-09-17T03:26:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.207+0000 7f1f97400640 1 -- 172.21.15.46:0/3065002128 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f1f880f8c40 con 0x7f1fa00b5c70 2024-09-17T03:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f7c073b00 msgr2=0x7f1f7c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f7c073b00 0x7f1f7c076010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000bbd0 tx=0x7f1f9000b5b0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 msgr2=0x7f1fa0153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 0x7f1fa0153ab0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1f8801af40 tx=0x7f1f88004040 comp rx=0 tx=0).stop 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 shutdown_connections 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f7c073b00 0x7f1f7c076010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1fa01527b0 0x7f1fa0158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fa00b5c70 0x7f1fa0153ab0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 --2- 172.21.15.46:0/3065002128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa00b4de0 0x7f1fa0144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 >> 172.21.15.46:0/3065002128 conn(0x7f1fa00b2040 msgr2=0x7f1fa00b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 shutdown_connections 2024-09-17T03:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.211+0000 7f1fa6800640 1 -- 172.21.15.46:0/3065002128 wait complete. 2024-09-17T03:26:15.234 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:26:15.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:26:15.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:26:15.386 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:26:15.387 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:15.387 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:15.396 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:26:15.396 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:26:15.396 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:15.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:26:15.405 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:15.406 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:15.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:26:15.447 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:26:15.448 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:15.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:26:15.458 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:26:15.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 -- 172.21.15.46:0/2365596010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc664069500 msgr2=0x7fc6640cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 --2- 172.21.15.46:0/2365596010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc664069500 0x7fc6640cfd50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009900 tx=0x7fc64c01c8b0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 -- 172.21.15.46:0/2365596010 shutdown_connections 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 --2- 172.21.15.46:0/2365596010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc664069500 0x7fc6640cfd50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 -- 172.21.15.46:0/2365596010 >> 172.21.15.46:0/2365596010 conn(0x7fc6640b35f0 msgr2=0x7fc6640b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 -- 172.21.15.46:0/2365596010 shutdown_connections 2024-09-17T03:26:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 -- 172.21.15.46:0/2365596010 wait complete. 2024-09-17T03:26:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 Processor -- start 2024-09-17T03:26:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.647+0000 7fc668e00640 1 -- start start 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6640d0290 0x7fc66414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc664160c20 0x7fc6640b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 0x7fc66415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc66416b380 con 0x7fc6640b50a0 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc66416b200 con 0x7fc664160c20 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc66416b500 con 0x7fc6640d0290 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 0x7fc66415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 0x7fc66415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41674/0 (socket says 172.21.15.46:41674) 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 -- 172.21.15.46:0/3882917584 learned_addr learned my addr 172.21.15.46:0/3882917584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc65be00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc664160c20 0x7fc6640b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc661e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6640d0290 0x7fc66414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 -- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6640d0290 msgr2=0x7fc66414bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6640d0290 0x7fc66414bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 -- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc664160c20 msgr2=0x7fc6640b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc664160c20 0x7fc6640b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 -- 172.21.15.46:0/3882917584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6641fc290 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc662800640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 0x7fc66415b7f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c730 tx=0x7fc65c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c003b80 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc65c003d40 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0052d0 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6641fc570 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641fcb30 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.651+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6640cfd50 con 0x7fc6640b50a0 2024-09-17T03:26:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.655+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fc65c005680 con 0x7fc6640b50a0 2024-09-17T03:26:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.655+0000 7fc65a000640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc640073b00 0x7fc640076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.655+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7fc65c0928c0 con 0x7fc6640b50a0 2024-09-17T03:26:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.655+0000 7fc661e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc640073b00 0x7fc640076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.655+0000 7fc661e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc640073b00 0x7fc640076010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc654007fd0 tx=0x7fc654009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.659+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc65c05ed70 con 0x7fc6640b50a0 2024-09-17T03:26:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.867+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc664168280 con 0x7fc6640b50a0 2024-09-17T03:26:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.867+0000 7fc65a000640 1 -- 172.21.15.46:0/3882917584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3913 (secure 0 0 0) 0x7fc65c0637a0 con 0x7fc6640b50a0 2024-09-17T03:26:15.870 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:15.871 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-17T03:26:09:423350+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":244937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2055000821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2055000821},{"type":"v1","addr":"172.21.15.46:6837","nonce":2055000821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":244943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/96704891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":96704891},{"type":"v1","addr":"172.21.15.99:6834","nonce":96704891}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1568160669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":1568160669},{"type":"v1","addr":"172.21.15.99:6835","nonce":1568160669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":245757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3315961404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3315961404},{"type":"v1","addr":"172.21.15.46:6836","nonce":3315961404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-17T03:26:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc640073b00 msgr2=0x7fc640076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc640073b00 0x7fc640076010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc654007fd0 tx=0x7fc654009040 comp rx=0 tx=0).stop 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 msgr2=0x7fc66415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 0x7fc66415b7f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c730 tx=0x7fc65c00cbf0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 shutdown_connections 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc640073b00 0x7fc640076010 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6640b50a0 0x7fc66415b7f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc664160c20 0x7fc6640b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 --2- 172.21.15.46:0/3882917584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc6640d0290 0x7fc66414bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 >> 172.21.15.46:0/3882917584 conn(0x7fc6640b35f0 msgr2=0x7fc6640b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 shutdown_connections 2024-09-17T03:26:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:15.871+0000 7fc668e00640 1 -- 172.21.15.46:0/3882917584 wait complete. 2024-09-17T03:26:15.895 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:26:15.895 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:26:15.895 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:26:15.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:26:15.895 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:26:15.895 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:26:15.896 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:26:15.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:26:15.896 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:26:15.896 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:26:15.896 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:26:15.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:26:15.896 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:26:15.897 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:26:15.897 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:26:15.897 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:26:15.910 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:26:15.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:26:15.911 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:26:15.911 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:26:15.912 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:26:15.912 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:26:15.919 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:26:15.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:26:15.963 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:26:15.963 DEBUG:teuthology.parallel:result is None 2024-09-17T03:26:15.964 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:26:15.965 DEBUG:teuthology.parallel:result is None 2024-09-17T03:26:15.969 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:26:15.969 DEBUG:teuthology.parallel:result is None 2024-09-17T03:26:15.971 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:26:15.971 DEBUG:teuthology.parallel:result is None 2024-09-17T03:26:15.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:26:16.068 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:26:16.068 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:26:16.069 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:26:16.073 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:26:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 -- 172.21.15.46:0/375483843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 msgr2=0x7f7170004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 --2- 172.21.15.46:0/375483843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f7170004c20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7174001830 tx=0x7f717401eff0 comp rx=0 tx=0).stop 2024-09-17T03:26:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7191200640 1 -- 172.21.15.46:0/375483843 reap_dead start 2024-09-17T03:26:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 -- 172.21.15.46:0/375483843 shutdown_connections 2024-09-17T03:26:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 -- 172.21.15.46:0/375483843 >> 172.21.15.46:0/375483843 conn(0x7f718c0708c0 msgr2=0x7f718c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 -- 172.21.15.46:0/375483843 shutdown_connections 2024-09-17T03:26:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 -- 172.21.15.46:0/375483843 wait complete. 2024-09-17T03:26:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 Processor -- start 2024-09-17T03:26:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.139+0000 7f7193200640 1 -- start start 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f718c148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c1494f0 0x7f718c1f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f718c0b42f0 0x7f718c0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f718c168080 con 0x7f7170004810 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f718c167f00 con 0x7f718c1494f0 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f718c168200 con 0x7f718c0b42f0 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f718c148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f718c148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41744/0 (socket says 172.21.15.46:41744) 2024-09-17T03:26:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 -- 172.21.15.46:0/2987187682 learned_addr learned my addr 172.21.15.46:0/2987187682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7191200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f718c0b42f0 0x7f718c0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7183e00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c1494f0 0x7f718c1f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 -- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f718c0b42f0 msgr2=0x7f718c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f718c0b42f0 0x7f718c0b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 -- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c1494f0 msgr2=0x7f718c1f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c1494f0 0x7f718c1f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 -- 172.21.15.46:0/2987187682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c1f9f30 con 0x7f7170004810 2024-09-17T03:26:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f718be00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f718c148ec0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f717c006630 tx=0x7f717c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c015070 con 0x7f7170004810 2024-09-17T03:26:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f717c011900 con 0x7f7170004810 2024-09-17T03:26:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c0b4c40 con 0x7f7170004810 2024-09-17T03:26:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c011d30 con 0x7f7170004810 2024-09-17T03:26:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1f9c50 con 0x7f7170004810 2024-09-17T03:26:16.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.143+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c159570 con 0x7f7170004810 2024-09-17T03:26:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.147+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f717c018070 con 0x7f7170004810 2024-09-17T03:26:16.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.147+0000 7f7188c00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7168073b00 0x7f7168076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:16.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.147+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4971+0+0 (secure 0 0 0) 0x7f717c093970 con 0x7f7170004810 2024-09-17T03:26:16.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.147+0000 7f7183e00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7168073b00 0x7f7168076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:16.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.147+0000 7f7183e00640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7168073b00 0x7f7168076010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f71700057f0 tx=0x7f717401be30 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.151+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f718c0b4c40 con 0x7f7170004810 2024-09-17T03:26:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:16.339+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f718c0b1c70 con 0x7f7170004810 2024-09-17T03:26:17.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7188c00640 1 -- 172.21.15.46:0/2987187682 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f718c1f9f30 con 0x7f7170004810 2024-09-17T03:26:17.236 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:26:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7168073b00 msgr2=0x7f7168076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7168073b00 0x7f7168076010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f71700057f0 tx=0x7f717401be30 comp rx=0 tx=0).stop 2024-09-17T03:26:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 msgr2=0x7f718c148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f718c148ec0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f717c006630 tx=0x7f717c0039d0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 shutdown_connections 2024-09-17T03:26:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7168073b00 0x7f7168076010 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f718c0b42f0 0x7f718c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f718c1494f0 0x7f718c1f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 --2- 172.21.15.46:0/2987187682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7170004810 0x7f718c148ec0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 >> 172.21.15.46:0/2987187682 conn(0x7f718c0708c0 msgr2=0x7f718c150b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 shutdown_connections 2024-09-17T03:26:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.235+0000 7f7193200640 1 -- 172.21.15.46:0/2987187682 wait complete. 2024-09-17T03:26:17.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 -- 172.21.15.46:0/1906049690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15c5d0 msgr2=0x7f826c073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 --2- 172.21.15.46:0/1906049690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15c5d0 0x7f826c073dc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f826c06ae60 tx=0x7f825c01c890 comp rx=0 tx=0).stop 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 -- 172.21.15.46:0/1906049690 shutdown_connections 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 --2- 172.21.15.46:0/1906049690 >> v2:172.21.15.99:3300/0 conn(0x7f826c074300 0x7f826c15f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 --2- 172.21.15.46:0/1906049690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15c5d0 0x7f826c073dc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 -- 172.21.15.46:0/1906049690 >> 172.21.15.46:0/1906049690 conn(0x7f826c0b2c40 msgr2=0x7f826c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 -- 172.21.15.46:0/1906049690 shutdown_connections 2024-09-17T03:26:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 -- 172.21.15.46:0/1906049690 wait complete. 2024-09-17T03:26:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 Processor -- start 2024-09-17T03:26:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.427+0000 7f8271200640 1 -- start start 2024-09-17T03:26:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 0x7f826c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f826c15c5d0 0x7f826c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f826c15f6b0 0x7f826c072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f826c06f980 con 0x7f826c074300 2024-09-17T03:26:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0b3d50 con 0x7f826c15f6b0 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f826c06fb00 con 0x7f826c15c5d0 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 0x7f826c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 0x7f826c155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41762/0 (socket says 172.21.15.46:41762) 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 -- 172.21.15.46:0/3909568422 learned_addr learned my addr 172.21.15.46:0/3909568422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f826a800640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f826c15f6b0 0x7f826c072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8261400640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f826c15c5d0 0x7f826c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 -- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f826c15c5d0 msgr2=0x7f826c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f826c15c5d0 0x7f826c157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 -- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f826c15f6b0 msgr2=0x7f826c072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f826c15f6b0 0x7f826c072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 -- 172.21.15.46:0/3909568422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c14cf10 con 0x7f826c074300 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8269e00640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 0x7f826c155970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f825c0194a0 tx=0x7f825c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0bc070 con 0x7f826c074300 2024-09-17T03:26:17.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c005320 con 0x7f826c074300 2024-09-17T03:26:17.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0b2040 con 0x7f826c074300 2024-09-17T03:26:17.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c1fa310 con 0x7f826c074300 2024-09-17T03:26:17.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1fa8d0 con 0x7f826c074300 2024-09-17T03:26:17.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.431+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c15f170 con 0x7f826c074300 2024-09-17T03:26:17.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.435+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f825c0046d0 con 0x7f826c074300 2024-09-17T03:26:17.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.435+0000 7f8262a00640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8248073b30 0x7f8248076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.435+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f825c12c710 con 0x7f826c074300 2024-09-17T03:26:17.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.435+0000 7f8261400640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8248073b30 0x7f8248076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.435+0000 7f8261400640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8248073b30 0x7f8248076040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f826c14deb0 tx=0x7f825400b390 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.439+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f825c0164b0 con 0x7f826c074300 2024-09-17T03:26:17.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.675+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f826c154d40 con 0x7f826c074300 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.675+0000 7f8262a00640 1 -- 172.21.15.46:0/3909568422 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f825c0f8bc0 con 0x7f826c074300 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.679 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:26:17.680 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.684 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.686 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:26:17.688 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:26:17.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:26:17.691 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:26:17.692 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:26:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:17.693 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:26:17.693 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8248073b30 msgr2=0x7f8248076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8248073b30 0x7f8248076040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f826c14deb0 tx=0x7f825400b390 comp rx=0 tx=0).stop 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 msgr2=0x7f826c155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 0x7f826c155970 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f825c0194a0 tx=0x7f825c004600 comp rx=0 tx=0).stop 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 shutdown_connections 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8248073b30 0x7f8248076040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f826c15f6b0 0x7f826c072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f826c15c5d0 0x7f826c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 --2- 172.21.15.46:0/3909568422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c074300 0x7f826c155970 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 >> 172.21.15.46:0/3909568422 conn(0x7f826c0b2c40 msgr2=0x7f826c1503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 shutdown_connections 2024-09-17T03:26:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.679+0000 7f8271200640 1 -- 172.21.15.46:0/3909568422 wait complete. 2024-09-17T03:26:17.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:26:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 -- 172.21.15.46:0/3725061338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 msgr2=0x7f4afc0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 --2- 172.21.15.46:0/3725061338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc0b51f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4afc070710 tx=0x7f4af401ccc0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 -- 172.21.15.46:0/3725061338 shutdown_connections 2024-09-17T03:26:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 --2- 172.21.15.46:0/3725061338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc0b51f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 --2- 172.21.15.46:0/3725061338 >> v2:172.21.15.99:3300/0 conn(0x7f4afc1527b0 0x7f4afc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 -- 172.21.15.46:0/3725061338 >> 172.21.15.46:0/3725061338 conn(0x7f4afc0b2040 msgr2=0x7f4afc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 -- 172.21.15.46:0/3725061338 shutdown_connections 2024-09-17T03:26:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 -- 172.21.15.46:0/3725061338 wait complete. 2024-09-17T03:26:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 Processor -- start 2024-09-17T03:26:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.859+0000 7f4b03a00640 1 -- start start 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc1fc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4afc0b5c70 0x7f4afc1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4afc1527b0 0x7f4afc14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc168920 con 0x7f4afc0b4de0 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1687a0 con 0x7f4afc1527b0 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4afc168aa0 con 0x7f4afc0b5c70 2024-09-17T03:26:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc1fc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc1fc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41790/0 (socket says 172.21.15.46:41790) 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 -- 172.21.15.46:0/2494671071 learned_addr learned my addr 172.21.15.46:0/2494671071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4afc1527b0 0x7f4afc14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4afbe00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4afc0b5c70 0x7f4afc1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 -- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4afc0b5c70 msgr2=0x7f4afc1f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4afc0b5c70 0x7f4afc1f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 -- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4afc1527b0 msgr2=0x7f4afc14bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4afc1527b0 0x7f4afc14bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 -- 172.21.15.46:0/2494671071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc1f9ab0 con 0x7f4afc0b4de0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b01000640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc1fc1c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4af000c3c0 tx=0x7f4af000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0017070 con 0x7f4afc0b4de0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af00053c0 con 0x7f4afc0b4de0 2024-09-17T03:26:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc1fcac0 con 0x7f4afc0b4de0 2024-09-17T03:26:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0003860 con 0x7f4afc0b4de0 2024-09-17T03:26:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc1f8d70 con 0x7f4afc0b4de0 2024-09-17T03:26:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.863+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4afc0b51f0 con 0x7f4afc0b4de0 2024-09-17T03:26:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.867+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f4af001a070 con 0x7f4afc0b4de0 2024-09-17T03:26:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.867+0000 7f4af8c00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4adc073b00 0x7f4adc076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.867+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f4af0091ce0 con 0x7f4afc0b4de0 2024-09-17T03:26:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.867+0000 7f4afbe00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4adc073b00 0x7f4adc076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.867+0000 7f4afbe00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4adc073b00 0x7f4adc076010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0cf920 tx=0x7f4ae800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:17.871+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4afc0b5730 con 0x7f4afc0b4de0 2024-09-17T03:26:18.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.043+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4afc0cc710 con 0x7f4afc0b4de0 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4af8c00640 1 -- 172.21.15.46:0/2494671071 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7f4af005e1f0 con 0x7f4afc0b4de0 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:18.048 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4adc073b00 msgr2=0x7f4adc076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4adc073b00 0x7f4adc076010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0cf920 tx=0x7f4ae800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 msgr2=0x7f4afc1fc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc1fc1c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4af000c3c0 tx=0x7f4af000c880 comp rx=0 tx=0).stop 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 shutdown_connections 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4adc073b00 0x7f4adc076010 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4afc1527b0 0x7f4afc14bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4afc0b5c70 0x7f4afc1f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 --2- 172.21.15.46:0/2494671071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc0b4de0 0x7f4afc1fc1c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 >> 172.21.15.46:0/2494671071 conn(0x7f4afc0b2040 msgr2=0x7f4afc1511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:18.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 shutdown_connections 2024-09-17T03:26:18.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.047+0000 7f4b03a00640 1 -- 172.21.15.46:0/2494671071 wait complete. 2024-09-17T03:26:18.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:26:18.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:26:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 -- 172.21.15.46:0/2895272971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 msgr2=0x7f9a180cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 --2- 172.21.15.46:0/2895272971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 0x7f9a180cd970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009840 tx=0x7f9a0401c820 comp rx=0 tx=0).stop 2024-09-17T03:26:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 -- 172.21.15.46:0/2895272971 shutdown_connections 2024-09-17T03:26:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 --2- 172.21.15.46:0/2895272971 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a18163650 0x7f9a18165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 --2- 172.21.15.46:0/2895272971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 0x7f9a180cd970 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 -- 172.21.15.46:0/2895272971 >> 172.21.15.46:0/2895272971 conn(0x7f9a180b2040 msgr2=0x7f9a180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 -- 172.21.15.46:0/2895272971 shutdown_connections 2024-09-17T03:26:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.231+0000 7f9a1f800640 1 -- 172.21.15.46:0/2895272971 wait complete. 2024-09-17T03:26:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.235+0000 7f9a1f800640 1 Processor -- start 2024-09-17T03:26:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.235+0000 7f9a1f800640 1 -- start start 2024-09-17T03:26:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a180ce4c0 0x7f9a1814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 0x7f9a181541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 0x7f9a18157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18168c50 con 0x7f9a18163650 2024-09-17T03:26:18.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18168ad0 con 0x7f9a1815aa50 2024-09-17T03:26:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9a18168dd0 con 0x7f9a180ce4c0 2024-09-17T03:26:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 0x7f9a18157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 0x7f9a18157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41804/0 (socket says 172.21.15.46:41804) 2024-09-17T03:26:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 -- 172.21.15.46:0/107768281 learned_addr learned my addr 172.21.15.46:0/107768281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a17e00640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 0x7f9a181541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1ce00640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a180ce4c0 0x7f9a1814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 -- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a180ce4c0 msgr2=0x7f9a1814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a180ce4c0 0x7f9a1814ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 -- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 msgr2=0x7f9a181541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 0x7f9a181541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 -- 172.21.15.46:0/107768281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a18159730 con 0x7f9a18163650 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1d800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 0x7f9a18157170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000ea80 tx=0x7f9a1000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000cea0 con 0x7f9a18163650 2024-09-17T03:26:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a10004b30 con 0x7f9a18163650 2024-09-17T03:26:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1814efd0 con 0x7f9a18163650 2024-09-17T03:26:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10005020 con 0x7f9a18163650 2024-09-17T03:26:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a18156970 con 0x7f9a18163650 2024-09-17T03:26:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.239+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a18159540 con 0x7f9a18163650 2024-09-17T03:26:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.243+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f9a100051c0 con 0x7f9a18163650 2024-09-17T03:26:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.243+0000 7f9a14c00640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99f8073b00 0x7f99f8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.243+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f9a100936c0 con 0x7f9a18163650 2024-09-17T03:26:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.243+0000 7f9a1ce00640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99f8073b00 0x7f99f8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.243+0000 7f9a1ce00640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99f8073b00 0x7f99f8076010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c007fd0 tx=0x7f9a0c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:18.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.247+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9a1005fa60 con 0x7f9a18163650 2024-09-17T03:26:18.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:18.423+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9a18156bb0 con 0x7f9a18163650 2024-09-17T03:26:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.239+0000 7f9a14c00640 1 -- 172.21.15.46:0/107768281 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f9a10064490 con 0x7f9a18163650 2024-09-17T03:26:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:26:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99f8073b00 msgr2=0x7f99f8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99f8073b00 0x7f99f8076010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c007fd0 tx=0x7f9a0c009040 comp rx=0 tx=0).stop 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 msgr2=0x7f9a18157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 0x7f9a18157170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000ea80 tx=0x7f9a1000ef40 comp rx=0 tx=0).stop 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 shutdown_connections 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99f8073b00 0x7f99f8076010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a18163650 0x7f9a18157170 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a1815aa50 0x7f9a181541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 --2- 172.21.15.46:0/107768281 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9a180ce4c0 0x7f9a1814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 >> 172.21.15.46:0/107768281 conn(0x7f9a180b2040 msgr2=0x7f9a18150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 shutdown_connections 2024-09-17T03:26:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.243+0000 7f9a1f800640 1 -- 172.21.15.46:0/107768281 wait complete. 2024-09-17T03:26:19.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:26:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 -- 172.21.15.46:0/68149608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa544074950 msgr2=0x7fa5440b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 --2- 172.21.15.46:0/68149608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa544074950 0x7fa5440b5260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa544073cf0 tx=0x7fa53c01ccf0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 -- 172.21.15.46:0/68149608 shutdown_connections 2024-09-17T03:26:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 --2- 172.21.15.46:0/68149608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa544074950 0x7fa5440b5260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 --2- 172.21.15.46:0/68149608 >> v2:172.21.15.99:3300/0 conn(0x7fa5441536f0 0x7fa54415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 -- 172.21.15.46:0/68149608 >> 172.21.15.46:0/68149608 conn(0x7fa5440b4560 msgr2=0x7fa5440d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 -- 172.21.15.46:0/68149608 shutdown_connections 2024-09-17T03:26:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.443+0000 7fa54a800640 1 -- 172.21.15.46:0/68149608 wait complete. 2024-09-17T03:26:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 Processor -- start 2024-09-17T03:26:19.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 -- start start 2024-09-17T03:26:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 0x7fa5441491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa544160060 0x7fa54414fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa54414d4a0 0x7fa544150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa544169e10 con 0x7fa5441536f0 2024-09-17T03:26:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa544169c90 con 0x7fa544160060 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa544169f90 con 0x7fa54414d4a0 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 0x7fa5441491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 0x7fa5441491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41812/0 (socket says 172.21.15.46:41812) 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 -- 172.21.15.46:0/2001091208 learned_addr learned my addr 172.21.15.46:0/2001091208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549e00640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa54414d4a0 0x7fa544150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa548a00640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa544160060 0x7fa54414fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 -- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa54414d4a0 msgr2=0x7fa544150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa54414d4a0 0x7fa544150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 -- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa544160060 msgr2=0x7fa54414fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa544160060 0x7fa54414fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 -- 172.21.15.46:0/2001091208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54414c010 con 0x7fa5441536f0 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549e00640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa54414d4a0 0x7fa544150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa548a00640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa544160060 0x7fa54414fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa549400640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 0x7fa5441491b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e7f0 tx=0x7fa53800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53800ccc0 con 0x7fa5441536f0 2024-09-17T03:26:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa53800ce80 con 0x7fa5441536f0 2024-09-17T03:26:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538005280 con 0x7fa5441536f0 2024-09-17T03:26:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.447+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54414c2f0 con 0x7fa5441536f0 2024-09-17T03:26:19.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.451+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5441531c0 con 0x7fa5441536f0 2024-09-17T03:26:19.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.451+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa544149900 con 0x7fa5441536f0 2024-09-17T03:26:19.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.451+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa538005420 con 0x7fa5441536f0 2024-09-17T03:26:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.451+0000 7fa541600640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa524073b30 0x7fa524076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.451+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7fa538093450 con 0x7fa5441536f0 2024-09-17T03:26:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.451+0000 7fa548a00640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa524073b30 0x7fa524076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:19.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.455+0000 7fa548a00640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa524073b30 0x7fa524076040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5441551a0 tx=0x7fa530009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.455+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa53805ff50 con 0x7fa5441536f0 2024-09-17T03:26:19.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:19.631+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa5440b5260 con 0x7fa5441536f0 2024-09-17T03:26:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.247+0000 7fa541600640 1 -- 172.21.15.46:0/2001091208 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7fa53805f850 con 0x7fa5441536f0 2024-09-17T03:26:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:26:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa524073b30 msgr2=0x7fa524076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa524073b30 0x7fa524076040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5441551a0 tx=0x7fa530009040 comp rx=0 tx=0).stop 2024-09-17T03:26:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 msgr2=0x7fa5441491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 0x7fa5441491b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e7f0 tx=0x7fa53800ecb0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 shutdown_connections 2024-09-17T03:26:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa524073b30 0x7fa524076040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa54414d4a0 0x7fa544150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa544160060 0x7fa54414fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 --2- 172.21.15.46:0/2001091208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5441536f0 0x7fa5441491b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 >> 172.21.15.46:0/2001091208 conn(0x7fa5440b4560 msgr2=0x7fa544071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 shutdown_connections 2024-09-17T03:26:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.251+0000 7fa54a800640 1 -- 172.21.15.46:0/2001091208 wait complete. 2024-09-17T03:26:20.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:26:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 -- 172.21.15.46:0/429435661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74074950 msgr2=0x7fcf7415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 --2- 172.21.15.46:0/429435661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74074950 0x7fcf7415cf90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009c80 tx=0x7fcf6c01cbc0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 -- 172.21.15.46:0/429435661 shutdown_connections 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 --2- 172.21.15.46:0/429435661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74074950 0x7fcf7415cf90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 --2- 172.21.15.46:0/429435661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf740b5a80 0x7fcf74074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 -- 172.21.15.46:0/429435661 >> 172.21.15.46:0/429435661 conn(0x7fcf740708c0 msgr2=0x7fcf74070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 -- 172.21.15.46:0/429435661 shutdown_connections 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.459+0000 7fcf7c600640 1 -- 172.21.15.46:0/429435661 wait complete. 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 Processor -- start 2024-09-17T03:26:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 -- start start 2024-09-17T03:26:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf74074950 0x7fcf740b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf740b5a80 0x7fcf740b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 0x7fcf74145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7416a3c0 con 0x7fcf74157030 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7416a240 con 0x7fcf740b5a80 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7416a540 con 0x7fcf74074950 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 0x7fcf74145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 0x7fcf74145c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41818/0 (socket says 172.21.15.46:41818) 2024-09-17T03:26:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 -- 172.21.15.46:0/3704680089 learned_addr learned my addr 172.21.15.46:0/3704680089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf79c00640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf74074950 0x7fcf740b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf79200640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf740b5a80 0x7fcf740b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 -- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf74074950 msgr2=0x7fcf740b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf74074950 0x7fcf740b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 -- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf740b5a80 msgr2=0x7fcf740b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf740b5a80 0x7fcf740b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 -- 172.21.15.46:0/3704680089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf740b3830 con 0x7fcf74157030 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf79200640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf740b5a80 0x7fcf740b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.463+0000 7fcf7a600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 0x7fcf74145c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009bd0 tx=0x7fcf6c009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0bb070 con 0x7fcf74157030 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf6c004d40 con 0x7fcf74157030 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0b2040 con 0x7fcf74157030 2024-09-17T03:26:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf74151fb0 con 0x7fcf74157030 2024-09-17T03:26:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf74145490 con 0x7fcf74157030 2024-09-17T03:26:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7415cf90 con 0x7fcf74157030 2024-09-17T03:26:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fcf6c004260 con 0x7fcf74157030 2024-09-17T03:26:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.467+0000 7fcf71600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.471+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7fcf6c12ce10 con 0x7fcf74157030 2024-09-17T03:26:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.471+0000 7fcf79c00640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.471+0000 7fcf79c00640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098e0 tx=0x7fcf68009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.471+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcf6c0c4020 con 0x7fcf74157030 2024-09-17T03:26:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:20.679+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcf74151d40 con 0x7fcf74157030 2024-09-17T03:26:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.247+0000 7fcf71600640 1 -- 172.21.15.46:0/3704680089 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) ==== 172+0+0 (secure 0 0 0) 0x7fcf74074d60 con 0x7fcf74157030 2024-09-17T03:26:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 8 and data pool 9 2024-09-17T03:26:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 msgr2=0x7fcf54076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098e0 tx=0x7fcf68009060 comp rx=0 tx=0).stop 2024-09-17T03:26:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 msgr2=0x7fcf74145c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 0x7fcf74145c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009bd0 tx=0x7fcf6c009350 comp rx=0 tx=0).stop 2024-09-17T03:26:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 shutdown_connections 2024-09-17T03:26:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf74157030 0x7fcf74145c90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf740b5a80 0x7fcf740b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 --2- 172.21.15.46:0/3704680089 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf74074950 0x7fcf740b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.255+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 >> 172.21.15.46:0/3704680089 conn(0x7fcf740708c0 msgr2=0x7fcf74154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.259+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 shutdown_connections 2024-09-17T03:26:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.259+0000 7fcf7c600640 1 -- 172.21.15.46:0/3704680089 wait complete. 2024-09-17T03:26:21.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74f1a00640 1 -- 172.21.15.46:0/755868999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d0004d50 msgr2=0x7f74d0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74f1a00640 1 --2- 172.21.15.46:0/755868999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d0004d50 0x7f74d0005160 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c610 tx=0x7f74e401fa40 comp rx=0 tx=0).stop 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74eb400640 1 -- 172.21.15.46:0/755868999 reap_dead start 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74f1a00640 1 -- 172.21.15.46:0/755868999 shutdown_connections 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74f1a00640 1 -- 172.21.15.46:0/755868999 >> 172.21.15.46:0/755868999 conn(0x7f74ec0b2040 msgr2=0x7f74ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74f1a00640 1 -- 172.21.15.46:0/755868999 shutdown_connections 2024-09-17T03:26:21.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.471+0000 7f74f1a00640 1 -- 172.21.15.46:0/755868999 wait complete. 2024-09-17T03:26:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 Processor -- start 2024-09-17T03:26:21.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 -- start start 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d0004d50 0x7f74ec0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 0x7f74ec0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ec0af430 0x7f74ec1f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec15cf30 con 0x7f74ec151b80 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec15cdb0 con 0x7f74ec0af430 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74f1a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f74ec15d0b0 con 0x7f74d0004d50 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 0x7f74ec0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 0x7f74ec0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41844/0 (socket says 172.21.15.46:41844) 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 -- 172.21.15.46:0/1065015825 learned_addr learned my addr 172.21.15.46:0/1065015825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74eaa00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d0004d50 0x7f74ec0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74eb400640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ec0af430 0x7f74ec1f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 -- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d0004d50 msgr2=0x7f74ec0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d0004d50 0x7f74ec0ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 -- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ec0af430 msgr2=0x7f74ec1f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ec0af430 0x7f74ec1f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74e2000640 1 -- 172.21.15.46:0/1065015825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec0af840 con 0x7f74ec151b80 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.475+0000 7f74eb400640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ec0af430 0x7f74ec1f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e2000640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 0x7f74ec0aeef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f74d4011ae0 tx=0x7f74d4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004460 con 0x7f74ec151b80 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74d400f9a0 con 0x7f74ec151b80 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d400fdd0 con 0x7f74ec151b80 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec1412d0 con 0x7f74ec151b80 2024-09-17T03:26:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec141890 con 0x7f74ec151b80 2024-09-17T03:26:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ec0cdb30 con 0x7f74ec151b80 2024-09-17T03:26:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f74d4003600 con 0x7f74ec151b80 2024-09-17T03:26:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e3400640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74c8073b00 0x7f74c8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.479+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f74d40936a0 con 0x7f74ec151b80 2024-09-17T03:26:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.483+0000 7f74eaa00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74c8073b00 0x7f74c8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.483+0000 7f74eaa00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74c8073b00 0x7f74c8076010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009c60 tx=0x7f74dc0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.483+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74ec0af840 con 0x7f74ec151b80 2024-09-17T03:26:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.659+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74ec0768c0 con 0x7f74ec151b80 2024-09-17T03:26:21.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74e3400640 1 -- 172.21.15.46:0/1065015825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16702 (secure 0 0 0) 0x7f74d40601a0 con 0x7f74ec151b80 2024-09-17T03:26:21.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:21.664 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:26:20.685478+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:26:18.439366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T03:26:19.646068+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:26:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74c8073b00 msgr2=0x7f74c8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74c8073b00 0x7f74c8076010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009c60 tx=0x7f74dc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 msgr2=0x7f74ec0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 0x7f74ec0aeef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f74d4011ae0 tx=0x7f74d4011ee0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 shutdown_connections 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74c8073b00 0x7f74c8076010 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74ec0af430 0x7f74ec1f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec151b80 0x7f74ec0aeef0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 --2- 172.21.15.46:0/1065015825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d0004d50 0x7f74ec0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 >> 172.21.15.46:0/1065015825 conn(0x7f74ec0b2040 msgr2=0x7f74ec1553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 shutdown_connections 2024-09-17T03:26:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.663+0000 7f74f1a00640 1 -- 172.21.15.46:0/1065015825 wait complete. 2024-09-17T03:26:21.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:26:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 -- 172.21.15.46:0/2337044681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 msgr2=0x7f6e300ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/2337044681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300ce190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009bc0 tx=0x7f6e2801cb10 comp rx=0 tx=0).stop 2024-09-17T03:26:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 -- 172.21.15.46:0/2337044681 shutdown_connections 2024-09-17T03:26:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/2337044681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300ce190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 -- 172.21.15.46:0/2337044681 >> 172.21.15.46:0/2337044681 conn(0x7f6e300b41f0 msgr2=0x7f6e300b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 -- 172.21.15.46:0/2337044681 shutdown_connections 2024-09-17T03:26:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.843+0000 7f6e2fe00640 1 -- 172.21.15.46:0/2337044681 wait complete. 2024-09-17T03:26:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 Processor -- start 2024-09-17T03:26:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 -- start start 2024-09-17T03:26:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e300746e0 0x7f6e3014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 0x7f6e301f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3016b4e0 con 0x7f6e3009aa80 2024-09-17T03:26:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3016b360 con 0x7f6e3015c920 2024-09-17T03:26:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3016b660 con 0x7f6e300746e0 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 0x7f6e301f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 0x7f6e301f9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41850/0 (socket says 172.21.15.46:41850) 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.847+0000 7f6e2f400640 1 -- 172.21.15.46:0/1742647472 learned_addr learned my addr 172.21.15.46:0/1742647472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2ea00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e300746e0 0x7f6e3014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2f400640 1 -- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e300746e0 msgr2=0x7f6e3014fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e27e00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2f400640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e300746e0 0x7f6e3014fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2f400640 1 -- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 msgr2=0x7f6e300c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2f400640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2f400640 1 -- 172.21.15.46:0/1742647472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e30150050 con 0x7f6e3009aa80 2024-09-17T03:26:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2ea00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e300746e0 0x7f6e3014fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e27e00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2f400640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 0x7f6e301f9980 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6e28098500 tx=0x7f6e28005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280b2040 con 0x7f6e3009aa80 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e280bb070 con 0x7f6e3009aa80 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28004570 con 0x7f6e3009aa80 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e301fd350 con 0x7f6e3009aa80 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e300b2cc0 con 0x7f6e3009aa80 2024-09-17T03:26:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e30150220 con 0x7f6e3009aa80 2024-09-17T03:26:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.851+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6e28004710 con 0x7f6e3009aa80 2024-09-17T03:26:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.855+0000 7f6e26a00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e0c073b30 0x7f6e0c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.855+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6e281306a0 con 0x7f6e3009aa80 2024-09-17T03:26:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.855+0000 7f6e2ea00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e0c073b30 0x7f6e0c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.855+0000 7f6e2ea00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e0c073b30 0x7f6e0c076040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e20000f70 tx=0x7f6e20009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:21.855+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6e28016790 con 0x7f6e3009aa80 2024-09-17T03:26:22.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.031+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e30074c30 con 0x7f6e3009aa80 2024-09-17T03:26:22.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e26a00640 1 -- 172.21.15.46:0/1742647472 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16702 (secure 0 0 0) 0x7f6e280fab70 con 0x7f6e3009aa80 2024-09-17T03:26:22.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:22.036 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:26:20.685478+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:26:18.439366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T03:26:19.646068+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:26:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e0c073b30 msgr2=0x7f6e0c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e0c073b30 0x7f6e0c076040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e20000f70 tx=0x7f6e20009040 comp rx=0 tx=0).stop 2024-09-17T03:26:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 msgr2=0x7f6e301f9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 0x7f6e301f9980 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6e28098500 tx=0x7f6e28005ae0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 shutdown_connections 2024-09-17T03:26:22.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e0c073b30 0x7f6e0c076040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3009aa80 0x7f6e301f9980 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e3015c920 0x7f6e300c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 --2- 172.21.15.46:0/1742647472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e300746e0 0x7f6e3014fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 >> 172.21.15.46:0/1742647472 conn(0x7f6e300b41f0 msgr2=0x7f6e300cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 shutdown_connections 2024-09-17T03:26:22.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.035+0000 7f6e2fe00640 1 -- 172.21.15.46:0/1742647472 wait complete. 2024-09-17T03:26:22.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 -- 172.21.15.46:0/1497962208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 msgr2=0x7fa85015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 --2- 172.21.15.46:0/1497962208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 0x7fa85015bb60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa84001c820 comp rx=0 tx=0).stop 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 -- 172.21.15.46:0/1497962208 shutdown_connections 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 --2- 172.21.15.46:0/1497962208 >> v2:172.21.15.99:3300/0 conn(0x7fa8501527b0 0x7fa850074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 --2- 172.21.15.46:0/1497962208 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 0x7fa85015bb60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 -- 172.21.15.46:0/1497962208 >> 172.21.15.46:0/1497962208 conn(0x7fa8500b2040 msgr2=0x7fa8500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 -- 172.21.15.46:0/1497962208 shutdown_connections 2024-09-17T03:26:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.219+0000 7fa855200640 1 -- 172.21.15.46:0/1497962208 wait complete. 2024-09-17T03:26:22.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 Processor -- start 2024-09-17T03:26:22.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 -- start start 2024-09-17T03:26:22.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 0x7fa85014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:22.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 0x7fa85014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:22.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8501527b0 0x7fa8501f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa850168c70 con 0x7fa8500b4de0 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa850168af0 con 0x7fa8501527b0 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa855200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa850168df0 con 0x7fa8500b5c70 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa84de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 0x7fa85014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa84de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 0x7fa85014ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41856/0 (socket says 172.21.15.46:41856) 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa84de00640 1 -- 172.21.15.46:0/501112893 learned_addr learned my addr 172.21.15.46:0/501112893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa847e00640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 0x7fa85014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.223+0000 7fa84e800640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8501527b0 0x7fa8501f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa84de00640 1 -- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 msgr2=0x7fa85014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa84de00640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 0x7fa85014cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa84de00640 1 -- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8501527b0 msgr2=0x7fa8501f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa84de00640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8501527b0 0x7fa8501f9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:22.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa84de00640 1 -- 172.21.15.46:0/501112893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8501f6480 con 0x7fa8500b4de0 2024-09-17T03:26:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa84de00640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 0x7fa85014ca10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa840009850 tx=0x7fa8400a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400b2040 con 0x7fa8500b4de0 2024-09-17T03:26:22.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8400044b0 con 0x7fa8500b4de0 2024-09-17T03:26:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8501f6760 con 0x7fa8500b4de0 2024-09-17T03:26:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840005290 con 0x7fa8500b4de0 2024-09-17T03:26:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501f6d20 con 0x7fa8500b4de0 2024-09-17T03:26:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85015bb60 con 0x7fa8500b4de0 2024-09-17T03:26:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa840005430 con 0x7fa8500b4de0 2024-09-17T03:26:22.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.227+0000 7fa846000640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa82c073b30 0x7fa82c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:22.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.231+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa84012c640 con 0x7fa8500b4de0 2024-09-17T03:26:22.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.231+0000 7fa847e00640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa82c073b30 0x7fa82c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:22.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.231+0000 7fa847e00640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa82c073b30 0x7fa82c076040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa85015b270 tx=0x7fa8380091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:22.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.235+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa850075580 con 0x7fa8500b4de0 2024-09-17T03:26:22.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:22.451+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa85014a180 con 0x7fa8500b4de0 2024-09-17T03:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.279+0000 7fa846000640 1 -- 172.21.15.46:0/501112893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7fa8400164d0 con 0x7fa8500b4de0 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa82c073b30 msgr2=0x7fa82c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa82c073b30 0x7fa82c076040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa85015b270 tx=0x7fa8380091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 msgr2=0x7fa85014ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 0x7fa85014ca10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa840009850 tx=0x7fa8400a5860 comp rx=0 tx=0).stop 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 shutdown_connections 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa82c073b30 0x7fa82c076040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8501527b0 0x7fa8501f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8500b5c70 0x7fa85014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 --2- 172.21.15.46:0/501112893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8500b4de0 0x7fa85014ca10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 >> 172.21.15.46:0/501112893 conn(0x7fa8500b2040 msgr2=0x7fa8501599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 shutdown_connections 2024-09-17T03:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.283+0000 7fa855200640 1 -- 172.21.15.46:0/501112893 wait complete. 2024-09-17T03:26:23.308 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:26:23.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 -- 172.21.15.46:0/2938600280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc06efa0 msgr2=0x7f7cbc06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2938600280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc06f3b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009bc0 tx=0x7f7cb401cb70 comp rx=0 tx=0).stop 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 -- 172.21.15.46:0/2938600280 shutdown_connections 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2938600280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc06f3b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2938600280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cbc0744d0 0x7f7cbc06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 -- 172.21.15.46:0/2938600280 >> 172.21.15.46:0/2938600280 conn(0x7f7cbc0b2c40 msgr2=0x7f7cbc0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 -- 172.21.15.46:0/2938600280 shutdown_connections 2024-09-17T03:26:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 -- 172.21.15.46:0/2938600280 wait complete. 2024-09-17T03:26:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 Processor -- start 2024-09-17T03:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.475+0000 7f7cc1600640 1 -- start start 2024-09-17T03:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cbc15c5d0 0x7f7cbc13d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc1f98f0 0x7f7cbc13d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc168d20 con 0x7f7cbc06efa0 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc0b3d40 con 0x7f7cbc1f98f0 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7cbc168ea0 con 0x7f7cbc15c5d0 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc0b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49930/0 (socket says 172.21.15.46:49930) 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 -- 172.21.15.46:0/2753352573 learned_addr learned my addr 172.21.15.46:0/2753352573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cb3e00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cbc15c5d0 0x7f7cbc13d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbb400640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc1f98f0 0x7f7cbc13d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 -- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cbc15c5d0 msgr2=0x7f7cbc13d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cbc15c5d0 0x7f7cbc13d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 -- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc1f98f0 msgr2=0x7f7cbc13d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc1f98f0 0x7f7cbc13d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 -- 172.21.15.46:0/2753352573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cbc1fa000 con 0x7f7cbc06efa0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cbaa00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc0b0020 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c460 tx=0x7f7cac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0052d0 con 0x7f7cbc06efa0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cac005490 con 0x7f7cbc06efa0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac012890 con 0x7f7cbc06efa0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cbc1fa2e0 con 0x7f7cbc06efa0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cbc074360 con 0x7f7cbc06efa0 2024-09-17T03:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cbc0745c0 con 0x7f7cbc06efa0 2024-09-17T03:26:23.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.479+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f7cac012a30 con 0x7f7cbc06efa0 2024-09-17T03:26:23.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.483+0000 7f7cb2a00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c98073b00 0x7f7c98076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:23.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.483+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7cac092420 con 0x7f7cbc06efa0 2024-09-17T03:26:23.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.483+0000 7f7cb3e00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c98073b00 0x7f7c98076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.483+0000 7f7cb3e00640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c98073b00 0x7f7c98076010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc154080 tx=0x7f7ca4009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:23.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.487+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7cac05e990 con 0x7f7cbc06efa0 2024-09-17T03:26:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:23.695+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7cbc06f3b0 con 0x7f7cbc06efa0 2024-09-17T03:26:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cb2a00640 1 -- 172.21.15.46:0/2753352573 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f7cac0633c0 con 0x7f7cbc06efa0 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c98073b00 msgr2=0x7f7c98076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c98073b00 0x7f7c98076010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc154080 tx=0x7f7ca4009260 comp rx=0 tx=0).stop 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 msgr2=0x7f7cbc0b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc0b0020 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c460 tx=0x7f7cac00c920 comp rx=0 tx=0).stop 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 shutdown_connections 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c98073b00 0x7f7c98076010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cbc1f98f0 0x7f7cbc13d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7cbc15c5d0 0x7f7cbc13d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 --2- 172.21.15.46:0/2753352573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cbc06efa0 0x7f7cbc0b0020 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 >> 172.21.15.46:0/2753352573 conn(0x7f7cbc0b2c40 msgr2=0x7f7cbc075f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 shutdown_connections 2024-09-17T03:26:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.287+0000 7f7cc1600640 1 -- 172.21.15.46:0/2753352573 wait complete. 2024-09-17T03:26:24.316 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:26:24.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 -- 172.21.15.46:0/272417125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 msgr2=0x7f467806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 --2- 172.21.15.46:0/272417125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467806d200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f46780ce170 tx=0x7f466001c830 comp rx=0 tx=0).stop 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 -- 172.21.15.46:0/272417125 shutdown_connections 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 --2- 172.21.15.46:0/272417125 >> v2:172.21.15.99:3300/0 conn(0x7f467806fc90 0x7f46780700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 --2- 172.21.15.46:0/272417125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467806d200 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 -- 172.21.15.46:0/272417125 >> 172.21.15.46:0/272417125 conn(0x7f46780b35f0 msgr2=0x7f46780b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 -- 172.21.15.46:0/272417125 shutdown_connections 2024-09-17T03:26:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 -- 172.21.15.46:0/272417125 wait complete. 2024-09-17T03:26:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 Processor -- start 2024-09-17T03:26:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.479+0000 7f467ec00640 1 -- start start 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467814ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467806fc90 0x7f4678152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4678159130 0x7f4678164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4678168d30 con 0x7f467806cdf0 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4678168bb0 con 0x7f467806fc90 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4678168eb0 con 0x7f4678159130 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467814ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467814ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49952/0 (socket says 172.21.15.46:49952) 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 -- 172.21.15.46:0/1613757338 learned_addr learned my addr 172.21.15.46:0/1613757338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 -- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4678159130 msgr2=0x7f4678164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4678159130 0x7f4678164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 -- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467806fc90 msgr2=0x7f4678152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f466fe00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467806fc90 0x7f4678152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467806fc90 0x7f4678152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 -- 172.21.15.46:0/1613757338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467815d430 con 0x7f467806cdf0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f466fe00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467806fc90 0x7f4678152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4677e00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467814ccb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f466800b410 tx=0x7f466800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668015030 con 0x7f467806cdf0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4668004020 con 0x7f467806cdf0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781515c0 con 0x7f467806cdf0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46680044b0 con 0x7f467806cdf0 2024-09-17T03:26:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4678151b80 con 0x7f467806cdf0 2024-09-17T03:26:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.483+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467806d200 con 0x7f467806cdf0 2024-09-17T03:26:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.487+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f4668018070 con 0x7f467806cdf0 2024-09-17T03:26:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.487+0000 7f4674c00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073b00 0x7f4654076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.487+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f46680926a0 con 0x7f467806cdf0 2024-09-17T03:26:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.487+0000 7f466fe00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073b00 0x7f4654076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.487+0000 7f466fe00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073b00 0x7f4654076010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46600af000 tx=0x7f466001b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.491+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f466805f150 con 0x7f467806cdf0 2024-09-17T03:26:24.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.699+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f467815bd30 con 0x7f467806cdf0 2024-09-17T03:26:24.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f4674c00640 1 -- 172.21.15.46:0/1613757338 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5236 (secure 0 0 0) 0x7f466805ea50 con 0x7f467806cdf0 2024-09-17T03:26:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:24.704 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"btime":"2024-09-17T03:26:24:284029+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:26:20.685438+0000","modified":"2024-09-17T03:26:23.700531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245850":{"gid":245850,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/695855747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":695855747},{"type":"v1","addr":"172.21.15.46:6835","nonce":695855747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245850,"qdb_cluster":[245850]},"id":4}]} 2024-09-17T03:26:24.704 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 35 2024-09-17T03:26:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073b00 msgr2=0x7f4654076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073b00 0x7f4654076010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f46600af000 tx=0x7f466001b070 comp rx=0 tx=0).stop 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 msgr2=0x7f467814ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467814ccb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f466800b410 tx=0x7f466800b8d0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 shutdown_connections 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4654073b00 0x7f4654076010 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4678159130 0x7f4678164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f467806fc90 0x7f4678152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 --2- 172.21.15.46:0/1613757338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467806cdf0 0x7f467814ccb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 >> 172.21.15.46:0/1613757338 conn(0x7f46780b35f0 msgr2=0x7f4678156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 shutdown_connections 2024-09-17T03:26:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.703+0000 7f467ec00640 1 -- 172.21.15.46:0/1613757338 wait complete. 2024-09-17T03:26:24.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:26:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 -- 172.21.15.46:0/2224296454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 msgr2=0x7f80400b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 --2- 172.21.15.46:0/2224296454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f80400b51f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8038009c80 tx=0x7f803801cce0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 -- 172.21.15.46:0/2224296454 shutdown_connections 2024-09-17T03:26:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 --2- 172.21.15.46:0/2224296454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f80400b51f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 --2- 172.21.15.46:0/2224296454 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80401527b0 0x7f804015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 -- 172.21.15.46:0/2224296454 >> 172.21.15.46:0/2224296454 conn(0x7f80400b2040 msgr2=0x7f80400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:24.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 -- 172.21.15.46:0/2224296454 shutdown_connections 2024-09-17T03:26:24.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.883+0000 7f8045e00640 1 -- 172.21.15.46:0/2224296454 wait complete. 2024-09-17T03:26:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 Processor -- start 2024-09-17T03:26:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 -- start start 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f8040157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80400b5c70 0x7f80401515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80401527b0 0x7f80400ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8040168a00 con 0x7f80400b4de0 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8040168880 con 0x7f80401527b0 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8040168b80 con 0x7f80400b5c70 2024-09-17T03:26:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f8040157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f8040157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49972/0 (socket says 172.21.15.46:49972) 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 -- 172.21.15.46:0/581386664 learned_addr learned my addr 172.21.15.46:0/581386664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8037e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80400b5c70 0x7f80401515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803f400640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80401527b0 0x7f80400ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 -- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80400b5c70 msgr2=0x7f80401515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80400b5c70 0x7f80401515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 -- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80401527b0 msgr2=0x7f80400ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80401527b0 0x7f80400ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 -- 172.21.15.46:0/581386664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804015aa20 con 0x7f80400b4de0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f803ea00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f8040157e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f803000c3c0 tx=0x7f803000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030017070 con 0x7f80400b4de0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80300053c0 con 0x7f80400b4de0 2024-09-17T03:26:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030003860 con 0x7f80400b4de0 2024-09-17T03:26:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804015ad00 con 0x7f80400b4de0 2024-09-17T03:26:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.887+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804015b2c0 con 0x7f80400b4de0 2024-09-17T03:26:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.891+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80400b51f0 con 0x7f80400b4de0 2024-09-17T03:26:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.891+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f803001a070 con 0x7f80400b4de0 2024-09-17T03:26:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.891+0000 7f8037400640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f801c073b00 0x7f801c076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.891+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8030091d60 con 0x7f80400b4de0 2024-09-17T03:26:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.891+0000 7f8037e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f801c073b00 0x7f801c076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.891+0000 7f8037e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f801c073b00 0x7f801c076010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8040154d70 tx=0x7f80280091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:24.895+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f80400b5730 con 0x7f80400b4de0 2024-09-17T03:26:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.071+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80400b4980 con 0x7f80400b4de0 2024-09-17T03:26:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.071+0000 7f8037400640 1 -- 172.21.15.46:0/581386664 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7f8030094090 con 0x7f80400b4de0 2024-09-17T03:26:25.073 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:25.074 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":58,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:26:22.266207+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:26:18.439366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T03:26:19.646068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f801c073b00 msgr2=0x7f801c076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f801c073b00 0x7f801c076010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8040154d70 tx=0x7f80280091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 msgr2=0x7f8040157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f8040157e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f803000c3c0 tx=0x7f803000c880 comp rx=0 tx=0).stop 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 shutdown_connections 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f801c073b00 0x7f801c076010 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80401527b0 0x7f80400ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80400b5c70 0x7f80401515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 --2- 172.21.15.46:0/581386664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80400b4de0 0x7f8040157e10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 >> 172.21.15.46:0/581386664 conn(0x7f80400b2040 msgr2=0x7f80400b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 shutdown_connections 2024-09-17T03:26:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.075+0000 7f8045e00640 1 -- 172.21.15.46:0/581386664 wait complete. 2024-09-17T03:26:25.096 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:26:25.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:26:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 -- 172.21.15.46:0/1814626703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac157030 msgr2=0x7f16ac0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 --2- 172.21.15.46:0/1814626703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac157030 0x7f16ac0b5a80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f169c009bc0 tx=0x7f169c01ca30 comp rx=0 tx=0).stop 2024-09-17T03:26:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 -- 172.21.15.46:0/1814626703 shutdown_connections 2024-09-17T03:26:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 --2- 172.21.15.46:0/1814626703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac157030 0x7f16ac0b5a80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 -- 172.21.15.46:0/1814626703 >> 172.21.15.46:0/1814626703 conn(0x7f16ac0708c0 msgr2=0x7f16ac070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 -- 172.21.15.46:0/1814626703 shutdown_connections 2024-09-17T03:26:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 -- 172.21.15.46:0/1814626703 wait complete. 2024-09-17T03:26:25.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 Processor -- start 2024-09-17T03:26:25.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.255+0000 7f16b2c00640 1 -- start start 2024-09-17T03:26:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16ac157030 0x7f16ac06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 0x7f16ac1467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac147010 0x7f16ac1f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac16b880 con 0x7f16ac160d50 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac16b700 con 0x7f16ac147010 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f16ac16ba00 con 0x7f16ac157030 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 0x7f16ac1467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 0x7f16ac1467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49976/0 (socket says 172.21.15.46:49976) 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 -- 172.21.15.46:0/156369515 learned_addr learned my addr 172.21.15.46:0/156369515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16abe00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16ac157030 0x7f16ac06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b0c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac147010 0x7f16ac1f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 -- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16ac157030 msgr2=0x7f16ac06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16ac157030 0x7f16ac06cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 -- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac147010 msgr2=0x7f16ac1f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac147010 0x7f16ac1f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 -- 172.21.15.46:0/156369515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac156160 con 0x7f16ac160d50 2024-09-17T03:26:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b0c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac147010 0x7f16ac1f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a3400640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 0x7f16ac1467b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1694001230 tx=0x7f169400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694016020 con 0x7f16ac160d50 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1694012920 con 0x7f16ac160d50 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694012db0 con 0x7f16ac160d50 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac06d330 con 0x7f16ac160d50 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac0b4890 con 0x7f16ac160d50 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac0d2260 con 0x7f16ac160d50 2024-09-17T03:26:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.259+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f1694019020 con 0x7f16ac160d50 2024-09-17T03:26:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.263+0000 7f16a8c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1688073b00 0x7f1688076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.263+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1694092870 con 0x7f16ac160d50 2024-09-17T03:26:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.263+0000 7f16abe00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1688073b00 0x7f1688076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.263+0000 7f16abe00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1688073b00 0x7f1688076010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f169c0af000 tx=0x7f169c01b270 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.267+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1694020030 con 0x7f16ac160d50 2024-09-17T03:26:25.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.435+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16ac06b540 con 0x7f1688073b00 2024-09-17T03:26:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.443+0000 7f16a8c00640 1 -- 172.21.15.46:0/156369515 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250791 (secure 0 0 0) 0x7f169c0a5770 con 0x7f1688073b00 2024-09-17T03:26:25.447 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:25.449 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:26:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1688073b00 msgr2=0x7f1688076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1688073b00 0x7f1688076010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f169c0af000 tx=0x7f169c01b270 comp rx=0 tx=0).stop 2024-09-17T03:26:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 msgr2=0x7f16ac1467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 0x7f16ac1467b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1694001230 tx=0x7f169400c980 comp rx=0 tx=0).stop 2024-09-17T03:26:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 shutdown_connections 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1688073b00 0x7f1688076010 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac147010 0x7f16ac1f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ac160d50 0x7f16ac1467b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 --2- 172.21.15.46:0/156369515 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16ac157030 0x7f16ac06cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 >> 172.21.15.46:0/156369515 conn(0x7f16ac0708c0 msgr2=0x7f16ac075f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 shutdown_connections 2024-09-17T03:26:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.447+0000 7f16b2c00640 1 -- 172.21.15.46:0/156369515 wait complete. 2024-09-17T03:26:25.469 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-17T03:26:25.303316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386440,"kb_used_data":6420,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754588280,"statfs":{"total":773094113280,"available":772698398720,"internally_reserved":0,"allocated":6574080,"data_stored":3391384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001164"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.290122+0000","last_change":"2024-09-17T03:26:21.271619+0000","last_active":"2024-09-17T03:26:21.290122+0000","last_peered":"2024-09-17T03:26:21.290122+0000","last_clean":"2024-09-17T03:26:21.290122+0000","last_became_active":"2024-09-17T03:26:21.270472+0000","last_became_peered":"2024-09-17T03:26:21.270472+0000","last_unstale":"2024-09-17T03:26:21.290122+0000","last_undegraded":"2024-09-17T03:26:21.290122+0000","last_fullsized":"2024-09-17T03:26:21.290122+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:58.768092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293649+0000","last_change":"2024-09-17T03:26:21.281731+0000","last_active":"2024-09-17T03:26:21.293649+0000","last_peered":"2024-09-17T03:26:21.293649+0000","last_clean":"2024-09-17T03:26:21.293649+0000","last_became_active":"2024-09-17T03:26:21.281402+0000","last_became_peered":"2024-09-17T03:26:21.281402+0000","last_unstale":"2024-09-17T03:26:21.293649+0000","last_undegraded":"2024-09-17T03:26:21.293649+0000","last_fullsized":"2024-09-17T03:26:21.293649+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:18.996659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:21.283316+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:21.282999+0000","last_became_peered":"2024-09-17T03:26:21.282999+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:15.689667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269895+0000","last_change":"2024-09-17T03:26:20.269895+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269895+0000","last_undegraded":"2024-09-17T03:26:20.269895+0000","last_fullsized":"2024-09-17T03:26:20.269895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289760+0000","last_change":"2024-09-17T03:26:21.284187+0000","last_active":"2024-09-17T03:26:21.289760+0000","last_peered":"2024-09-17T03:26:21.289760+0000","last_clean":"2024-09-17T03:26:21.289760+0000","last_became_active":"2024-09-17T03:26:21.283779+0000","last_became_peered":"2024-09-17T03:26:21.283779+0000","last_unstale":"2024-09-17T03:26:21.289760+0000","last_undegraded":"2024-09-17T03:26:21.289760+0000","last_fullsized":"2024-09-17T03:26:21.289760+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:57.934343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300045+0000","last_change":"2024-09-17T03:26:21.296425+0000","last_active":"2024-09-17T03:26:21.300045+0000","last_peered":"2024-09-17T03:26:21.300045+0000","last_clean":"2024-09-17T03:26:21.300045+0000","last_became_active":"2024-09-17T03:26:21.296063+0000","last_became_peered":"2024-09-17T03:26:21.296063+0000","last_unstale":"2024-09-17T03:26:21.300045+0000","last_undegraded":"2024-09-17T03:26:21.300045+0000","last_fullsized":"2024-09-17T03:26:21.300045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:27.881549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284101+0000","last_change":"2024-09-17T03:26:21.281715+0000","last_active":"2024-09-17T03:26:21.284101+0000","last_peered":"2024-09-17T03:26:21.284101+0000","last_clean":"2024-09-17T03:26:21.284101+0000","last_became_active":"2024-09-17T03:26:21.280618+0000","last_became_peered":"2024-09-17T03:26:21.280618+0000","last_unstale":"2024-09-17T03:26:21.284101+0000","last_undegraded":"2024-09-17T03:26:21.284101+0000","last_fullsized":"2024-09-17T03:26:21.284101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:44.647319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285297+0000","last_change":"2024-09-17T03:26:21.277710+0000","last_active":"2024-09-17T03:26:21.285297+0000","last_peered":"2024-09-17T03:26:21.285297+0000","last_clean":"2024-09-17T03:26:21.285297+0000","last_became_active":"2024-09-17T03:26:21.277333+0000","last_became_peered":"2024-09-17T03:26:21.277333+0000","last_unstale":"2024-09-17T03:26:21.285297+0000","last_undegraded":"2024-09-17T03:26:21.285297+0000","last_fullsized":"2024-09-17T03:26:21.285297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:34.691840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291938+0000","last_change":"2024-09-17T03:26:21.290497+0000","last_active":"2024-09-17T03:26:21.291938+0000","last_peered":"2024-09-17T03:26:21.291938+0000","last_clean":"2024-09-17T03:26:21.291938+0000","last_became_active":"2024-09-17T03:26:21.289289+0000","last_became_peered":"2024-09-17T03:26:21.289289+0000","last_unstale":"2024-09-17T03:26:21.291938+0000","last_undegraded":"2024-09-17T03:26:21.291938+0000","last_fullsized":"2024-09-17T03:26:21.291938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:01.560195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263685+0000","last_change":"2024-09-17T03:26:20.263685+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263685+0000","last_undegraded":"2024-09-17T03:26:20.263685+0000","last_fullsized":"2024-09-17T03:26:20.263685+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262752+0000","last_change":"2024-09-17T03:26:20.262752+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262752+0000","last_undegraded":"2024-09-17T03:26:20.262752+0000","last_fullsized":"2024-09-17T03:26:20.262752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269255+0000","last_change":"2024-09-17T03:26:20.269255+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269255+0000","last_undegraded":"2024-09-17T03:26:20.269255+0000","last_fullsized":"2024-09-17T03:26:20.269255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301774+0000","last_change":"2024-09-17T03:26:21.300221+0000","last_active":"2024-09-17T03:26:21.301774+0000","last_peered":"2024-09-17T03:26:21.301774+0000","last_clean":"2024-09-17T03:26:21.301774+0000","last_became_active":"2024-09-17T03:26:21.299864+0000","last_became_peered":"2024-09-17T03:26:21.299864+0000","last_unstale":"2024-09-17T03:26:21.301774+0000","last_undegraded":"2024-09-17T03:26:21.301774+0000","last_fullsized":"2024-09-17T03:26:21.301774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280369+0000","last_change":"2024-09-17T03:26:21.274122+0000","last_active":"2024-09-17T03:26:21.280369+0000","last_peered":"2024-09-17T03:26:21.280369+0000","last_clean":"2024-09-17T03:26:21.280369+0000","last_became_active":"2024-09-17T03:26:21.273777+0000","last_became_peered":"2024-09-17T03:26:21.273777+0000","last_unstale":"2024-09-17T03:26:21.280369+0000","last_undegraded":"2024-09-17T03:26:21.280369+0000","last_fullsized":"2024-09-17T03:26:21.280369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295641+0000","last_change":"2024-09-17T03:26:21.292845+0000","last_active":"2024-09-17T03:26:21.295641+0000","last_peered":"2024-09-17T03:26:21.295641+0000","last_clean":"2024-09-17T03:26:21.295641+0000","last_became_active":"2024-09-17T03:26:21.292432+0000","last_became_peered":"2024-09-17T03:26:21.292432+0000","last_unstale":"2024-09-17T03:26:21.295641+0000","last_undegraded":"2024-09-17T03:26:21.295641+0000","last_fullsized":"2024-09-17T03:26:21.295641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285635+0000","last_change":"2024-09-17T03:26:21.282672+0000","last_active":"2024-09-17T03:26:21.285635+0000","last_peered":"2024-09-17T03:26:21.285635+0000","last_clean":"2024-09-17T03:26:21.285635+0000","last_became_active":"2024-09-17T03:26:21.282359+0000","last_became_peered":"2024-09-17T03:26:21.282359+0000","last_unstale":"2024-09-17T03:26:21.285635+0000","last_undegraded":"2024-09-17T03:26:21.285635+0000","last_fullsized":"2024-09-17T03:26:21.285635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:11.693379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283260+0000","last_change":"2024-09-17T03:26:21.277694+0000","last_active":"2024-09-17T03:26:21.283260+0000","last_peered":"2024-09-17T03:26:21.283260+0000","last_clean":"2024-09-17T03:26:21.283260+0000","last_became_active":"2024-09-17T03:26:21.277341+0000","last_became_peered":"2024-09-17T03:26:21.277341+0000","last_unstale":"2024-09-17T03:26:21.283260+0000","last_undegraded":"2024-09-17T03:26:21.283260+0000","last_fullsized":"2024-09-17T03:26:21.283260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:16.287612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294961+0000","last_change":"2024-09-17T03:26:21.289536+0000","last_active":"2024-09-17T03:26:21.294961+0000","last_peered":"2024-09-17T03:26:21.294961+0000","last_clean":"2024-09-17T03:26:21.294961+0000","last_became_active":"2024-09-17T03:26:21.289230+0000","last_became_peered":"2024-09-17T03:26:21.289230+0000","last_unstale":"2024-09-17T03:26:21.294961+0000","last_undegraded":"2024-09-17T03:26:21.294961+0000","last_fullsized":"2024-09-17T03:26:21.294961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:31.982607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288531+0000","last_change":"2024-09-17T03:26:21.272001+0000","last_active":"2024-09-17T03:26:21.288531+0000","last_peered":"2024-09-17T03:26:21.288531+0000","last_clean":"2024-09-17T03:26:21.288531+0000","last_became_active":"2024-09-17T03:26:21.271673+0000","last_became_peered":"2024-09-17T03:26:21.271673+0000","last_unstale":"2024-09-17T03:26:21.288531+0000","last_undegraded":"2024-09-17T03:26:21.288531+0000","last_fullsized":"2024-09-17T03:26:21.288531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:48.995086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295028+0000","last_change":"2024-09-17T03:26:21.291765+0000","last_active":"2024-09-17T03:26:21.295028+0000","last_peered":"2024-09-17T03:26:21.295028+0000","last_clean":"2024-09-17T03:26:21.295028+0000","last_became_active":"2024-09-17T03:26:21.291430+0000","last_became_peered":"2024-09-17T03:26:21.291430+0000","last_unstale":"2024-09-17T03:26:21.295028+0000","last_undegraded":"2024-09-17T03:26:21.295028+0000","last_fullsized":"2024-09-17T03:26:21.295028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.144097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289278+0000","last_change":"2024-09-17T03:26:21.285773+0000","last_active":"2024-09-17T03:26:21.289278+0000","last_peered":"2024-09-17T03:26:21.289278+0000","last_clean":"2024-09-17T03:26:21.289278+0000","last_became_active":"2024-09-17T03:26:21.285118+0000","last_became_peered":"2024-09-17T03:26:21.285118+0000","last_unstale":"2024-09-17T03:26:21.289278+0000","last_undegraded":"2024-09-17T03:26:21.289278+0000","last_fullsized":"2024-09-17T03:26:21.289278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:50.670027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283248+0000","last_change":"2024-09-17T03:26:21.276516+0000","last_active":"2024-09-17T03:26:21.283248+0000","last_peered":"2024-09-17T03:26:21.283248+0000","last_clean":"2024-09-17T03:26:21.283248+0000","last_became_active":"2024-09-17T03:26:21.275398+0000","last_became_peered":"2024-09-17T03:26:21.275398+0000","last_unstale":"2024-09-17T03:26:21.283248+0000","last_undegraded":"2024-09-17T03:26:21.283248+0000","last_fullsized":"2024-09-17T03:26:21.283248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:22.952091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291438+0000","last_change":"2024-09-17T03:26:21.286047+0000","last_active":"2024-09-17T03:26:21.291438+0000","last_peered":"2024-09-17T03:26:21.291438+0000","last_clean":"2024-09-17T03:26:21.291438+0000","last_became_active":"2024-09-17T03:26:21.284398+0000","last_became_peered":"2024-09-17T03:26:21.284398+0000","last_unstale":"2024-09-17T03:26:21.291438+0000","last_undegraded":"2024-09-17T03:26:21.291438+0000","last_fullsized":"2024-09-17T03:26:21.291438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:38.105159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293886+0000","last_change":"2024-09-17T03:26:21.289586+0000","last_active":"2024-09-17T03:26:21.293886+0000","last_peered":"2024-09-17T03:26:21.293886+0000","last_clean":"2024-09-17T03:26:21.293886+0000","last_became_active":"2024-09-17T03:26:21.289242+0000","last_became_peered":"2024-09-17T03:26:21.289242+0000","last_unstale":"2024-09-17T03:26:21.293886+0000","last_undegraded":"2024-09-17T03:26:21.293886+0000","last_fullsized":"2024-09-17T03:26:21.293886+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:47.275691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293738+0000","last_change":"2024-09-17T03:26:21.287754+0000","last_active":"2024-09-17T03:26:21.293738+0000","last_peered":"2024-09-17T03:26:21.293738+0000","last_clean":"2024-09-17T03:26:21.293738+0000","last_became_active":"2024-09-17T03:26:21.287382+0000","last_became_peered":"2024-09-17T03:26:21.287382+0000","last_unstale":"2024-09-17T03:26:21.293738+0000","last_undegraded":"2024-09-17T03:26:21.293738+0000","last_fullsized":"2024-09-17T03:26:21.293738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:03.289618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275232+0000","last_change":"2024-09-17T03:26:20.275232+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275232+0000","last_undegraded":"2024-09-17T03:26:20.275232+0000","last_fullsized":"2024-09-17T03:26:20.275232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284955+0000","last_change":"2024-09-17T03:26:21.282975+0000","last_active":"2024-09-17T03:26:21.284955+0000","last_peered":"2024-09-17T03:26:21.284955+0000","last_clean":"2024-09-17T03:26:21.284955+0000","last_became_active":"2024-09-17T03:26:21.282116+0000","last_became_peered":"2024-09-17T03:26:21.282116+0000","last_unstale":"2024-09-17T03:26:21.284955+0000","last_undegraded":"2024-09-17T03:26:21.284955+0000","last_fullsized":"2024-09-17T03:26:21.284955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:08.367097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293266+0000","last_change":"2024-09-17T03:26:21.287660+0000","last_active":"2024-09-17T03:26:21.293266+0000","last_peered":"2024-09-17T03:26:21.293266+0000","last_clean":"2024-09-17T03:26:21.293266+0000","last_became_active":"2024-09-17T03:26:21.287329+0000","last_became_peered":"2024-09-17T03:26:21.287329+0000","last_unstale":"2024-09-17T03:26:21.293266+0000","last_undegraded":"2024-09-17T03:26:21.293266+0000","last_fullsized":"2024-09-17T03:26:21.293266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:07.684829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293795+0000","last_change":"2024-09-17T03:26:21.281015+0000","last_active":"2024-09-17T03:26:21.293795+0000","last_peered":"2024-09-17T03:26:21.293795+0000","last_clean":"2024-09-17T03:26:21.293795+0000","last_became_active":"2024-09-17T03:26:21.280690+0000","last_became_peered":"2024-09-17T03:26:21.280690+0000","last_unstale":"2024-09-17T03:26:21.293795+0000","last_undegraded":"2024-09-17T03:26:21.293795+0000","last_fullsized":"2024-09-17T03:26:21.293795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:27.474853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288915+0000","last_change":"2024-09-17T03:26:21.282370+0000","last_active":"2024-09-17T03:26:21.288915+0000","last_peered":"2024-09-17T03:26:21.288915+0000","last_clean":"2024-09-17T03:26:21.288915+0000","last_became_active":"2024-09-17T03:26:21.282053+0000","last_became_peered":"2024-09-17T03:26:21.282053+0000","last_unstale":"2024-09-17T03:26:21.288915+0000","last_undegraded":"2024-09-17T03:26:21.288915+0000","last_fullsized":"2024-09-17T03:26:21.288915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:27.209668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295471+0000","last_change":"2024-09-17T03:26:21.293642+0000","last_active":"2024-09-17T03:26:21.295471+0000","last_peered":"2024-09-17T03:26:21.295471+0000","last_clean":"2024-09-17T03:26:21.295471+0000","last_became_active":"2024-09-17T03:26:21.289857+0000","last_became_peered":"2024-09-17T03:26:21.289857+0000","last_unstale":"2024-09-17T03:26:21.295471+0000","last_undegraded":"2024-09-17T03:26:21.295471+0000","last_fullsized":"2024-09-17T03:26:21.295471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:56.082574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281534+0000","last_change":"2024-09-17T03:26:21.278899+0000","last_active":"2024-09-17T03:26:21.281534+0000","last_peered":"2024-09-17T03:26:21.281534+0000","last_clean":"2024-09-17T03:26:21.281534+0000","last_became_active":"2024-09-17T03:26:21.278562+0000","last_became_peered":"2024-09-17T03:26:21.278562+0000","last_unstale":"2024-09-17T03:26:21.281534+0000","last_undegraded":"2024-09-17T03:26:21.281534+0000","last_fullsized":"2024-09-17T03:26:21.281534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:06.391945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281831+0000","last_change":"2024-09-17T03:26:21.279072+0000","last_active":"2024-09-17T03:26:21.281831+0000","last_peered":"2024-09-17T03:26:21.281831+0000","last_clean":"2024-09-17T03:26:21.281831+0000","last_became_active":"2024-09-17T03:26:21.278730+0000","last_became_peered":"2024-09-17T03:26:21.278730+0000","last_unstale":"2024-09-17T03:26:21.281831+0000","last_undegraded":"2024-09-17T03:26:21.281831+0000","last_fullsized":"2024-09-17T03:26:21.281831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:31.274794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300099+0000","last_change":"2024-09-17T03:26:20.283656+0000","last_active":"2024-09-17T03:26:21.300099+0000","last_peered":"2024-09-17T03:26:21.300099+0000","last_clean":"2024-09-17T03:26:21.300099+0000","last_became_active":"2024-09-17T03:26:20.283331+0000","last_became_peered":"2024-09-17T03:26:20.283331+0000","last_unstale":"2024-09-17T03:26:21.300099+0000","last_undegraded":"2024-09-17T03:26:21.300099+0000","last_fullsized":"2024-09-17T03:26:21.300099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:03.041707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263563+0000","last_change":"2024-09-17T03:26:20.263563+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263563+0000","last_undegraded":"2024-09-17T03:26:20.263563+0000","last_fullsized":"2024-09-17T03:26:20.263563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:20.266917+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:20.266370+0000","last_became_peered":"2024-09-17T03:26:20.266370+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:57.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300820+0000","last_change":"2024-09-17T03:26:21.299198+0000","last_active":"2024-09-17T03:26:21.300820+0000","last_peered":"2024-09-17T03:26:21.300820+0000","last_clean":"2024-09-17T03:26:21.300820+0000","last_became_active":"2024-09-17T03:26:21.298717+0000","last_became_peered":"2024-09-17T03:26:21.298717+0000","last_unstale":"2024-09-17T03:26:21.300820+0000","last_undegraded":"2024-09-17T03:26:21.300820+0000","last_fullsized":"2024-09-17T03:26:21.300820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:48.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301538+0000","last_change":"2024-09-17T03:26:20.286649+0000","last_active":"2024-09-17T03:26:21.301538+0000","last_peered":"2024-09-17T03:26:21.301538+0000","last_clean":"2024-09-17T03:26:21.301538+0000","last_became_active":"2024-09-17T03:26:20.285886+0000","last_became_peered":"2024-09-17T03:26:20.285886+0000","last_unstale":"2024-09-17T03:26:21.301538+0000","last_undegraded":"2024-09-17T03:26:21.301538+0000","last_fullsized":"2024-09-17T03:26:21.301538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:46.887782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300191+0000","last_change":"2024-09-17T03:26:21.289923+0000","last_active":"2024-09-17T03:26:21.300191+0000","last_peered":"2024-09-17T03:26:21.300191+0000","last_clean":"2024-09-17T03:26:21.300191+0000","last_became_active":"2024-09-17T03:26:21.289556+0000","last_became_peered":"2024-09-17T03:26:21.289556+0000","last_unstale":"2024-09-17T03:26:21.300191+0000","last_undegraded":"2024-09-17T03:26:21.300191+0000","last_fullsized":"2024-09-17T03:26:21.300191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:41.180633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286893+0000","last_change":"2024-09-17T03:26:20.283187+0000","last_active":"2024-09-17T03:26:20.286893+0000","last_peered":"2024-09-17T03:26:20.286893+0000","last_clean":"2024-09-17T03:26:20.286893+0000","last_became_active":"2024-09-17T03:26:20.282354+0000","last_became_peered":"2024-09-17T03:26:20.282354+0000","last_unstale":"2024-09-17T03:26:20.286893+0000","last_undegraded":"2024-09-17T03:26:20.286893+0000","last_fullsized":"2024-09-17T03:26:20.286893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:57.123281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293825+0000","last_change":"2024-09-17T03:26:21.287275+0000","last_active":"2024-09-17T03:26:21.293825+0000","last_peered":"2024-09-17T03:26:21.293825+0000","last_clean":"2024-09-17T03:26:21.293825+0000","last_became_active":"2024-09-17T03:26:21.286995+0000","last_became_peered":"2024-09-17T03:26:21.286995+0000","last_unstale":"2024-09-17T03:26:21.293825+0000","last_undegraded":"2024-09-17T03:26:21.293825+0000","last_fullsized":"2024-09-17T03:26:21.293825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:58.245607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.298865+0000","last_change":"2024-09-17T03:26:20.283433+0000","last_active":"2024-09-17T03:26:21.298865+0000","last_peered":"2024-09-17T03:26:21.298865+0000","last_clean":"2024-09-17T03:26:21.298865+0000","last_became_active":"2024-09-17T03:26:20.277797+0000","last_became_peered":"2024-09-17T03:26:20.277797+0000","last_unstale":"2024-09-17T03:26:21.298865+0000","last_undegraded":"2024-09-17T03:26:21.298865+0000","last_fullsized":"2024-09-17T03:26:21.298865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294978+0000","last_change":"2024-09-17T03:26:21.293180+0000","last_active":"2024-09-17T03:26:21.294978+0000","last_peered":"2024-09-17T03:26:21.294978+0000","last_clean":"2024-09-17T03:26:21.294978+0000","last_became_active":"2024-09-17T03:26:21.292882+0000","last_became_peered":"2024-09-17T03:26:21.292882+0000","last_unstale":"2024-09-17T03:26:21.294978+0000","last_undegraded":"2024-09-17T03:26:21.294978+0000","last_fullsized":"2024-09-17T03:26:21.294978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:16.723527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301141+0000","last_change":"2024-09-17T03:26:20.279365+0000","last_active":"2024-09-17T03:26:21.301141+0000","last_peered":"2024-09-17T03:26:21.301141+0000","last_clean":"2024-09-17T03:26:21.301141+0000","last_became_active":"2024-09-17T03:26:20.278923+0000","last_became_peered":"2024-09-17T03:26:20.278923+0000","last_unstale":"2024-09-17T03:26:21.301141+0000","last_undegraded":"2024-09-17T03:26:21.301141+0000","last_fullsized":"2024-09-17T03:26:21.301141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:14.310020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286603+0000","last_change":"2024-09-17T03:26:21.285430+0000","last_active":"2024-09-17T03:26:21.286603+0000","last_peered":"2024-09-17T03:26:21.286603+0000","last_clean":"2024-09-17T03:26:21.286603+0000","last_became_active":"2024-09-17T03:26:21.285111+0000","last_became_peered":"2024-09-17T03:26:21.285111+0000","last_unstale":"2024-09-17T03:26:21.286603+0000","last_undegraded":"2024-09-17T03:26:21.286603+0000","last_fullsized":"2024-09-17T03:26:21.286603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:47.441700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284199+0000","last_change":"2024-09-17T03:26:20.279560+0000","last_active":"2024-09-17T03:26:20.284199+0000","last_peered":"2024-09-17T03:26:20.284199+0000","last_clean":"2024-09-17T03:26:20.284199+0000","last_became_active":"2024-09-17T03:26:20.278746+0000","last_became_peered":"2024-09-17T03:26:20.278746+0000","last_unstale":"2024-09-17T03:26:20.284199+0000","last_undegraded":"2024-09-17T03:26:20.284199+0000","last_fullsized":"2024-09-17T03:26:20.284199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:10.047880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286823+0000","last_change":"2024-09-17T03:26:21.279707+0000","last_active":"2024-09-17T03:26:21.286823+0000","last_peered":"2024-09-17T03:26:21.286823+0000","last_clean":"2024-09-17T03:26:21.286823+0000","last_became_active":"2024-09-17T03:26:21.279020+0000","last_became_peered":"2024-09-17T03:26:21.279020+0000","last_unstale":"2024-09-17T03:26:21.286823+0000","last_undegraded":"2024-09-17T03:26:21.286823+0000","last_fullsized":"2024-09-17T03:26:21.286823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:48.268297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264232+0000","last_change":"2024-09-17T03:26:20.283353+0000","last_active":"2024-09-17T03:26:21.264232+0000","last_peered":"2024-09-17T03:26:21.264232+0000","last_clean":"2024-09-17T03:26:21.264232+0000","last_became_active":"2024-09-17T03:26:20.283049+0000","last_became_peered":"2024-09-17T03:26:20.283049+0000","last_unstale":"2024-09-17T03:26:21.264232+0000","last_undegraded":"2024-09-17T03:26:21.264232+0000","last_fullsized":"2024-09-17T03:26:21.264232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291964+0000","last_change":"2024-09-17T03:26:21.290216+0000","last_active":"2024-09-17T03:26:21.291964+0000","last_peered":"2024-09-17T03:26:21.291964+0000","last_clean":"2024-09-17T03:26:21.291964+0000","last_became_active":"2024-09-17T03:26:21.288236+0000","last_became_peered":"2024-09-17T03:26:21.288236+0000","last_unstale":"2024-09-17T03:26:21.291964+0000","last_undegraded":"2024-09-17T03:26:21.291964+0000","last_fullsized":"2024-09-17T03:26:21.291964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:34.999880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269902+0000","last_change":"2024-09-17T03:26:20.289405+0000","last_active":"2024-09-17T03:26:21.269902+0000","last_peered":"2024-09-17T03:26:21.269902+0000","last_clean":"2024-09-17T03:26:21.269902+0000","last_became_active":"2024-09-17T03:26:20.288979+0000","last_became_peered":"2024-09-17T03:26:20.288979+0000","last_unstale":"2024-09-17T03:26:21.269902+0000","last_undegraded":"2024-09-17T03:26:21.269902+0000","last_fullsized":"2024-09-17T03:26:21.269902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:10.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.303142+0000","last_change":"2024-09-17T03:26:21.301608+0000","last_active":"2024-09-17T03:26:21.303142+0000","last_peered":"2024-09-17T03:26:21.303142+0000","last_clean":"2024-09-17T03:26:21.303142+0000","last_became_active":"2024-09-17T03:26:21.301203+0000","last_became_peered":"2024-09-17T03:26:21.301203+0000","last_unstale":"2024-09-17T03:26:21.303142+0000","last_undegraded":"2024-09-17T03:26:21.303142+0000","last_fullsized":"2024-09-17T03:26:21.303142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:23.395294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.271410+0000","last_change":"2024-09-17T03:26:20.263118+0000","last_active":"2024-09-17T03:26:21.271410+0000","last_peered":"2024-09-17T03:26:21.271410+0000","last_clean":"2024-09-17T03:26:21.271410+0000","last_became_active":"2024-09-17T03:26:20.262773+0000","last_became_peered":"2024-09-17T03:26:20.262773+0000","last_unstale":"2024-09-17T03:26:21.271410+0000","last_undegraded":"2024-09-17T03:26:21.271410+0000","last_fullsized":"2024-09-17T03:26:21.271410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:02.639519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263081+0000","last_change":"2024-09-17T03:26:20.263081+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263081+0000","last_undegraded":"2024-09-17T03:26:20.263081+0000","last_fullsized":"2024-09-17T03:26:20.263081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.306381+0000","last_change":"2024-09-17T03:26:20.277795+0000","last_active":"2024-09-17T03:26:21.306381+0000","last_peered":"2024-09-17T03:26:21.306381+0000","last_clean":"2024-09-17T03:26:21.306381+0000","last_became_active":"2024-09-17T03:26:20.277456+0000","last_became_peered":"2024-09-17T03:26:20.277456+0000","last_unstale":"2024-09-17T03:26:21.306381+0000","last_undegraded":"2024-09-17T03:26:21.306381+0000","last_fullsized":"2024-09-17T03:26:21.306381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:43.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297424+0000","last_change":"2024-09-17T03:26:21.292382+0000","last_active":"2024-09-17T03:26:21.297424+0000","last_peered":"2024-09-17T03:26:21.297424+0000","last_clean":"2024-09-17T03:26:21.297424+0000","last_became_active":"2024-09-17T03:26:21.292043+0000","last_became_peered":"2024-09-17T03:26:21.292043+0000","last_unstale":"2024-09-17T03:26:21.297424+0000","last_undegraded":"2024-09-17T03:26:21.297424+0000","last_fullsized":"2024-09-17T03:26:21.297424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:02.648204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.308666+0000","last_change":"2024-09-17T03:26:20.289201+0000","last_active":"2024-09-17T03:26:21.308666+0000","last_peered":"2024-09-17T03:26:21.308666+0000","last_clean":"2024-09-17T03:26:21.308666+0000","last_became_active":"2024-09-17T03:26:20.288427+0000","last_became_peered":"2024-09-17T03:26:20.288427+0000","last_unstale":"2024-09-17T03:26:21.308666+0000","last_undegraded":"2024-09-17T03:26:21.308666+0000","last_fullsized":"2024-09-17T03:26:21.308666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:46.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275142+0000","last_change":"2024-09-17T03:26:20.275142+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275142+0000","last_undegraded":"2024-09-17T03:26:20.275142+0000","last_fullsized":"2024-09-17T03:26:20.275142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.292629+0000","last_change":"2024-09-17T03:26:20.265758+0000","last_active":"2024-09-17T03:26:21.292629+0000","last_peered":"2024-09-17T03:26:21.292629+0000","last_clean":"2024-09-17T03:26:21.292629+0000","last_became_active":"2024-09-17T03:26:20.262782+0000","last_became_peered":"2024-09-17T03:26:20.262782+0000","last_unstale":"2024-09-17T03:26:21.292629+0000","last_undegraded":"2024-09-17T03:26:21.292629+0000","last_fullsized":"2024-09-17T03:26:21.292629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:10.171064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283171+0000","last_change":"2024-09-17T03:26:21.278170+0000","last_active":"2024-09-17T03:26:21.283171+0000","last_peered":"2024-09-17T03:26:21.283171+0000","last_clean":"2024-09-17T03:26:21.283171+0000","last_became_active":"2024-09-17T03:26:21.277329+0000","last_became_peered":"2024-09-17T03:26:21.277329+0000","last_unstale":"2024-09-17T03:26:21.283171+0000","last_undegraded":"2024-09-17T03:26:21.283171+0000","last_fullsized":"2024-09-17T03:26:21.283171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:32.373126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267858+0000","last_change":"2024-09-17T03:26:20.269475+0000","last_active":"2024-09-17T03:26:21.267858+0000","last_peered":"2024-09-17T03:26:21.267858+0000","last_clean":"2024-09-17T03:26:21.267858+0000","last_became_active":"2024-09-17T03:26:20.269101+0000","last_became_peered":"2024-09-17T03:26:20.269101+0000","last_unstale":"2024-09-17T03:26:21.267858+0000","last_undegraded":"2024-09-17T03:26:21.267858+0000","last_fullsized":"2024-09-17T03:26:21.267858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283965+0000","last_change":"2024-09-17T03:26:21.281823+0000","last_active":"2024-09-17T03:26:21.283965+0000","last_peered":"2024-09-17T03:26:21.283965+0000","last_clean":"2024-09-17T03:26:21.283965+0000","last_became_active":"2024-09-17T03:26:21.280688+0000","last_became_peered":"2024-09-17T03:26:21.280688+0000","last_unstale":"2024-09-17T03:26:21.283965+0000","last_undegraded":"2024-09-17T03:26:21.283965+0000","last_fullsized":"2024-09-17T03:26:21.283965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:36.912044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284389+0000","last_change":"2024-09-17T03:26:20.281345+0000","last_active":"2024-09-17T03:26:20.284389+0000","last_peered":"2024-09-17T03:26:20.284389+0000","last_clean":"2024-09-17T03:26:20.284389+0000","last_became_active":"2024-09-17T03:26:20.280261+0000","last_became_peered":"2024-09-17T03:26:20.280261+0000","last_unstale":"2024-09-17T03:26:20.284389+0000","last_undegraded":"2024-09-17T03:26:20.284389+0000","last_fullsized":"2024-09-17T03:26:20.284389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:47.948863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294667+0000","last_change":"2024-09-17T03:26:21.289462+0000","last_active":"2024-09-17T03:26:21.294667+0000","last_peered":"2024-09-17T03:26:21.294667+0000","last_clean":"2024-09-17T03:26:21.294667+0000","last_became_active":"2024-09-17T03:26:21.285434+0000","last_became_peered":"2024-09-17T03:26:21.285434+0000","last_unstale":"2024-09-17T03:26:21.294667+0000","last_undegraded":"2024-09-17T03:26:21.294667+0000","last_fullsized":"2024-09-17T03:26:21.294667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:24.253438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301604+0000","last_change":"2024-09-17T03:26:20.275945+0000","last_active":"2024-09-17T03:26:21.301604+0000","last_peered":"2024-09-17T03:26:21.301604+0000","last_clean":"2024-09-17T03:26:21.301604+0000","last_became_active":"2024-09-17T03:26:20.275123+0000","last_became_peered":"2024-09-17T03:26:20.275123+0000","last_unstale":"2024-09-17T03:26:21.301604+0000","last_undegraded":"2024-09-17T03:26:21.301604+0000","last_fullsized":"2024-09-17T03:26:21.301604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:02.236108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291615+0000","last_change":"2024-09-17T03:26:21.284952+0000","last_active":"2024-09-17T03:26:21.291615+0000","last_peered":"2024-09-17T03:26:21.291615+0000","last_clean":"2024-09-17T03:26:21.291615+0000","last_became_active":"2024-09-17T03:26:21.284572+0000","last_became_peered":"2024-09-17T03:26:21.284572+0000","last_unstale":"2024-09-17T03:26:21.291615+0000","last_undegraded":"2024-09-17T03:26:21.291615+0000","last_fullsized":"2024-09-17T03:26:21.291615+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:11.920676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295381+0000","last_change":"2024-09-17T03:26:20.269212+0000","last_active":"2024-09-17T03:26:21.295381+0000","last_peered":"2024-09-17T03:26:21.295381+0000","last_clean":"2024-09-17T03:26:21.295381+0000","last_became_active":"2024-09-17T03:26:20.268883+0000","last_became_peered":"2024-09-17T03:26:20.268883+0000","last_unstale":"2024-09-17T03:26:21.295381+0000","last_undegraded":"2024-09-17T03:26:21.295381+0000","last_fullsized":"2024-09-17T03:26:21.295381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:41:56.450980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274114+0000","last_change":"2024-09-17T03:26:20.274114+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274114+0000","last_undegraded":"2024-09-17T03:26:20.274114+0000","last_fullsized":"2024-09-17T03:26:20.274114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300538+0000","last_change":"2024-09-17T03:26:20.279146+0000","last_active":"2024-09-17T03:26:21.300538+0000","last_peered":"2024-09-17T03:26:21.300538+0000","last_clean":"2024-09-17T03:26:21.300538+0000","last_became_active":"2024-09-17T03:26:20.278888+0000","last_became_peered":"2024-09-17T03:26:20.278888+0000","last_unstale":"2024-09-17T03:26:21.300538+0000","last_undegraded":"2024-09-17T03:26:21.300538+0000","last_fullsized":"2024-09-17T03:26:21.300538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:40.150476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286599+0000","last_change":"2024-09-17T03:26:21.278295+0000","last_active":"2024-09-17T03:26:21.286599+0000","last_peered":"2024-09-17T03:26:21.286599+0000","last_clean":"2024-09-17T03:26:21.286599+0000","last_became_active":"2024-09-17T03:26:21.277728+0000","last_became_peered":"2024-09-17T03:26:21.277728+0000","last_unstale":"2024-09-17T03:26:21.286599+0000","last_undegraded":"2024-09-17T03:26:21.286599+0000","last_fullsized":"2024-09-17T03:26:21.286599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:38.173310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277895+0000","last_change":"2024-09-17T03:26:20.272850+0000","last_active":"2024-09-17T03:26:20.277895+0000","last_peered":"2024-09-17T03:26:20.277895+0000","last_clean":"2024-09-17T03:26:20.277895+0000","last_became_active":"2024-09-17T03:26:20.272036+0000","last_became_peered":"2024-09-17T03:26:20.272036+0000","last_unstale":"2024-09-17T03:26:20.277895+0000","last_undegraded":"2024-09-17T03:26:20.277895+0000","last_fullsized":"2024-09-17T03:26:20.277895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:25.846783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297474+0000","last_change":"2024-09-17T03:26:21.294723+0000","last_active":"2024-09-17T03:26:21.297474+0000","last_peered":"2024-09-17T03:26:21.297474+0000","last_clean":"2024-09-17T03:26:21.297474+0000","last_became_active":"2024-09-17T03:26:21.294375+0000","last_became_peered":"2024-09-17T03:26:21.294375+0000","last_unstale":"2024-09-17T03:26:21.297474+0000","last_undegraded":"2024-09-17T03:26:21.297474+0000","last_fullsized":"2024-09-17T03:26:21.297474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:22.471193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.275453+0000","last_change":"2024-09-17T03:26:20.281457+0000","last_active":"2024-09-17T03:26:21.275453+0000","last_peered":"2024-09-17T03:26:21.275453+0000","last_clean":"2024-09-17T03:26:21.275453+0000","last_became_active":"2024-09-17T03:26:20.281078+0000","last_became_peered":"2024-09-17T03:26:20.281078+0000","last_unstale":"2024-09-17T03:26:21.275453+0000","last_undegraded":"2024-09-17T03:26:21.275453+0000","last_fullsized":"2024-09-17T03:26:21.275453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:22.720948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.272277+0000","last_change":"2024-09-17T03:26:20.272277+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.272277+0000","last_undegraded":"2024-09-17T03:26:20.272277+0000","last_fullsized":"2024-09-17T03:26:20.272277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.296826+0000","last_change":"2024-09-17T03:26:20.280006+0000","last_active":"2024-09-17T03:26:21.296826+0000","last_peered":"2024-09-17T03:26:21.296826+0000","last_clean":"2024-09-17T03:26:21.296826+0000","last_became_active":"2024-09-17T03:26:20.279611+0000","last_became_peered":"2024-09-17T03:26:20.279611+0000","last_unstale":"2024-09-17T03:26:21.296826+0000","last_undegraded":"2024-09-17T03:26:21.296826+0000","last_fullsized":"2024-09-17T03:26:21.296826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:29.773567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289195+0000","last_change":"2024-09-17T03:26:21.284722+0000","last_active":"2024-09-17T03:26:21.289195+0000","last_peered":"2024-09-17T03:26:21.289195+0000","last_clean":"2024-09-17T03:26:21.289195+0000","last_became_active":"2024-09-17T03:26:21.284399+0000","last_became_peered":"2024-09-17T03:26:21.284399+0000","last_unstale":"2024-09-17T03:26:21.289195+0000","last_undegraded":"2024-09-17T03:26:21.289195+0000","last_fullsized":"2024-09-17T03:26:21.289195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:55.320737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277267+0000","last_change":"2024-09-17T03:26:20.274933+0000","last_active":"2024-09-17T03:26:20.277267+0000","last_peered":"2024-09-17T03:26:20.277267+0000","last_clean":"2024-09-17T03:26:20.277267+0000","last_became_active":"2024-09-17T03:26:20.274536+0000","last_became_peered":"2024-09-17T03:26:20.274536+0000","last_unstale":"2024-09-17T03:26:20.277267+0000","last_undegraded":"2024-09-17T03:26:20.277267+0000","last_fullsized":"2024-09-17T03:26:20.277267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:23.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280742+0000","last_change":"2024-09-17T03:26:21.277208+0000","last_active":"2024-09-17T03:26:21.280742+0000","last_peered":"2024-09-17T03:26:21.280742+0000","last_clean":"2024-09-17T03:26:21.280742+0000","last_became_active":"2024-09-17T03:26:21.276812+0000","last_became_peered":"2024-09-17T03:26:21.276812+0000","last_unstale":"2024-09-17T03:26:21.280742+0000","last_undegraded":"2024-09-17T03:26:21.280742+0000","last_fullsized":"2024-09-17T03:26:21.280742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:12.678429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284384+0000","last_change":"2024-09-17T03:26:20.281254+0000","last_active":"2024-09-17T03:26:20.284384+0000","last_peered":"2024-09-17T03:26:20.284384+0000","last_clean":"2024-09-17T03:26:20.284384+0000","last_became_active":"2024-09-17T03:26:20.280858+0000","last_became_peered":"2024-09-17T03:26:20.280858+0000","last_unstale":"2024-09-17T03:26:20.284384+0000","last_undegraded":"2024-09-17T03:26:20.284384+0000","last_fullsized":"2024-09-17T03:26:20.284384+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:22.288031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274734+0000","last_change":"2024-09-17T03:26:20.274734+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274734+0000","last_undegraded":"2024-09-17T03:26:20.274734+0000","last_fullsized":"2024-09-17T03:26:20.274734+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277412+0000","last_change":"2024-09-17T03:26:20.272578+0000","last_active":"2024-09-17T03:26:20.277412+0000","last_peered":"2024-09-17T03:26:20.277412+0000","last_clean":"2024-09-17T03:26:20.277412+0000","last_became_active":"2024-09-17T03:26:20.272215+0000","last_became_peered":"2024-09-17T03:26:20.272215+0000","last_unstale":"2024-09-17T03:26:20.277412+0000","last_undegraded":"2024-09-17T03:26:20.277412+0000","last_fullsized":"2024-09-17T03:26:20.277412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:56.751763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.273561+0000","last_change":"2024-09-17T03:26:20.273561+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.273561+0000","last_undegraded":"2024-09-17T03:26:20.273561+0000","last_fullsized":"2024-09-17T03:26:20.273561+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.276867+0000","last_change":"2024-09-17T03:26:20.281152+0000","last_active":"2024-09-17T03:26:21.276867+0000","last_peered":"2024-09-17T03:26:21.276867+0000","last_clean":"2024-09-17T03:26:21.276867+0000","last_became_active":"2024-09-17T03:26:20.280782+0000","last_became_peered":"2024-09-17T03:26:20.280782+0000","last_unstale":"2024-09-17T03:26:21.276867+0000","last_undegraded":"2024-09-17T03:26:21.276867+0000","last_fullsized":"2024-09-17T03:26:21.276867+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:08.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262545+0000","last_change":"2024-09-17T03:26:20.262545+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262545+0000","last_undegraded":"2024-09-17T03:26:20.262545+0000","last_fullsized":"2024-09-17T03:26:20.262545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267915+0000","last_change":"2024-09-17T03:26:20.277793+0000","last_active":"2024-09-17T03:26:21.267915+0000","last_peered":"2024-09-17T03:26:21.267915+0000","last_clean":"2024-09-17T03:26:21.267915+0000","last_became_active":"2024-09-17T03:26:20.276428+0000","last_became_peered":"2024-09-17T03:26:20.276428+0000","last_unstale":"2024-09-17T03:26:21.267915+0000","last_undegraded":"2024-09-17T03:26:21.267915+0000","last_fullsized":"2024-09-17T03:26:21.267915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289751+0000","last_change":"2024-09-17T03:26:21.283319+0000","last_active":"2024-09-17T03:26:21.289751+0000","last_peered":"2024-09-17T03:26:21.289751+0000","last_clean":"2024-09-17T03:26:21.289751+0000","last_became_active":"2024-09-17T03:26:21.282574+0000","last_became_peered":"2024-09-17T03:26:21.282574+0000","last_unstale":"2024-09-17T03:26:21.289751+0000","last_undegraded":"2024-09-17T03:26:21.289751+0000","last_fullsized":"2024-09-17T03:26:21.289751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:54.060063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286863+0000","last_change":"2024-09-17T03:26:20.282834+0000","last_active":"2024-09-17T03:26:20.286863+0000","last_peered":"2024-09-17T03:26:20.286863+0000","last_clean":"2024-09-17T03:26:20.286863+0000","last_became_active":"2024-09-17T03:26:20.279265+0000","last_became_peered":"2024-09-17T03:26:20.279265+0000","last_unstale":"2024-09-17T03:26:20.286863+0000","last_undegraded":"2024-09-17T03:26:20.286863+0000","last_fullsized":"2024-09-17T03:26:20.286863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.512156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293855+0000","last_change":"2024-09-17T03:26:21.290929+0000","last_active":"2024-09-17T03:26:21.293855+0000","last_peered":"2024-09-17T03:26:21.293855+0000","last_clean":"2024-09-17T03:26:21.293855+0000","last_became_active":"2024-09-17T03:26:21.290598+0000","last_became_peered":"2024-09-17T03:26:21.290598+0000","last_unstale":"2024-09-17T03:26:21.293855+0000","last_undegraded":"2024-09-17T03:26:21.293855+0000","last_fullsized":"2024-09-17T03:26:21.293855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:48.278348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277591+0000","last_change":"2024-09-17T03:26:20.275141+0000","last_active":"2024-09-17T03:26:20.277591+0000","last_peered":"2024-09-17T03:26:20.277591+0000","last_clean":"2024-09-17T03:26:20.277591+0000","last_became_active":"2024-09-17T03:26:20.274824+0000","last_became_peered":"2024-09-17T03:26:20.274824+0000","last_unstale":"2024-09-17T03:26:20.277591+0000","last_undegraded":"2024-09-17T03:26:20.277591+0000","last_fullsized":"2024-09-17T03:26:20.277591+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:00.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289251+0000","last_change":"2024-09-17T03:26:21.285419+0000","last_active":"2024-09-17T03:26:21.289251+0000","last_peered":"2024-09-17T03:26:21.289251+0000","last_clean":"2024-09-17T03:26:21.289251+0000","last_became_active":"2024-09-17T03:26:21.284707+0000","last_became_peered":"2024-09-17T03:26:21.284707+0000","last_unstale":"2024-09-17T03:26:21.289251+0000","last_undegraded":"2024-09-17T03:26:21.289251+0000","last_fullsized":"2024-09-17T03:26:21.289251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:07:50.249027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264570+0000","last_change":"2024-09-17T03:26:20.288747+0000","last_active":"2024-09-17T03:26:21.264570+0000","last_peered":"2024-09-17T03:26:21.264570+0000","last_clean":"2024-09-17T03:26:21.264570+0000","last_became_active":"2024-09-17T03:26:20.288347+0000","last_became_peered":"2024-09-17T03:26:20.288347+0000","last_unstale":"2024-09-17T03:26:21.264570+0000","last_undegraded":"2024-09-17T03:26:21.264570+0000","last_fullsized":"2024-09-17T03:26:21.264570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:46.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.261173+0000","last_change":"2024-09-17T03:26:20.261173+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.261173+0000","last_undegraded":"2024-09-17T03:26:20.261173+0000","last_fullsized":"2024-09-17T03:26:20.261173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.287028+0000","last_change":"2024-09-17T03:26:20.286225+0000","last_active":"2024-09-17T03:26:20.287028+0000","last_peered":"2024-09-17T03:26:20.287028+0000","last_clean":"2024-09-17T03:26:20.287028+0000","last_became_active":"2024-09-17T03:26:20.285812+0000","last_became_peered":"2024-09-17T03:26:20.285812+0000","last_unstale":"2024-09-17T03:26:20.287028+0000","last_undegraded":"2024-09-17T03:26:20.287028+0000","last_fullsized":"2024-09-17T03:26:20.287028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:14.007459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286857+0000","last_change":"2024-09-17T03:26:21.280085+0000","last_active":"2024-09-17T03:26:21.286857+0000","last_peered":"2024-09-17T03:26:21.286857+0000","last_clean":"2024-09-17T03:26:21.286857+0000","last_became_active":"2024-09-17T03:26:21.279401+0000","last_became_peered":"2024-09-17T03:26:21.279401+0000","last_unstale":"2024-09-17T03:26:21.286857+0000","last_undegraded":"2024-09-17T03:26:21.286857+0000","last_fullsized":"2024-09-17T03:26:21.286857+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.738915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.274511+0000","last_change":"2024-09-17T03:26:20.271261+0000","last_active":"2024-09-17T03:26:21.274511+0000","last_peered":"2024-09-17T03:26:21.274511+0000","last_clean":"2024-09-17T03:26:21.274511+0000","last_became_active":"2024-09-17T03:26:20.270867+0000","last_became_peered":"2024-09-17T03:26:20.270867+0000","last_unstale":"2024-09-17T03:26:21.274511+0000","last_undegraded":"2024-09-17T03:26:21.274511+0000","last_fullsized":"2024-09-17T03:26:21.274511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:19.783352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295189+0000","last_change":"2024-09-17T03:26:21.287202+0000","last_active":"2024-09-17T03:26:21.295189+0000","last_peered":"2024-09-17T03:26:21.295189+0000","last_clean":"2024-09-17T03:26:21.295189+0000","last_became_active":"2024-09-17T03:26:21.286886+0000","last_became_peered":"2024-09-17T03:26:21.286886+0000","last_unstale":"2024-09-17T03:26:21.295189+0000","last_undegraded":"2024-09-17T03:26:21.295189+0000","last_fullsized":"2024-09-17T03:26:21.295189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:51.956495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267707+0000","last_change":"2024-09-17T03:26:20.272318+0000","last_active":"2024-09-17T03:26:21.267707+0000","last_peered":"2024-09-17T03:26:21.267707+0000","last_clean":"2024-09-17T03:26:21.267707+0000","last_became_active":"2024-09-17T03:26:20.270871+0000","last_became_peered":"2024-09-17T03:26:20.270871+0000","last_unstale":"2024-09-17T03:26:21.267707+0000","last_undegraded":"2024-09-17T03:26:21.267707+0000","last_fullsized":"2024-09-17T03:26:21.267707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:56.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269003+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:26:21.269003+0000","last_peered":"2024-09-17T03:26:21.269003+0000","last_clean":"2024-09-17T03:26:21.269003+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:26:21.269003+0000","last_undegraded":"2024-09-17T03:26:21.269003+0000","last_fullsized":"2024-09-17T03:26:21.269003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58804,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313036,"statfs":{"total":96636764160,"available":96576548864,"internally_reserved":0,"allocated":1277952,"data_stored":870018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509508,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45740,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326100,"statfs":{"total":96636764160,"available":96589926400,"internally_reserved":0,"allocated":675840,"data_stored":279490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45748,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326092,"statfs":{"total":96636764160,"available":96589918208,"internally_reserved":0,"allocated":688128,"data_stored":281102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49908,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321932,"statfs":{"total":96636764160,"available":96585658368,"internally_reserved":0,"allocated":684032,"data_stored":279546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45728,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326112,"statfs":{"total":96636764160,"available":96589938688,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50468,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321372,"statfs":{"total":96636764160,"available":96585084928,"internally_reserved":0,"allocated":1257472,"data_stored":863083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:26:25.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:26:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f9134a00640 1 -- 172.21.15.46:0/2469282258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 msgr2=0x7f9114004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f9134a00640 1 --2- 172.21.15.46:0/2469282258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f9114004810 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9118006f70 tx=0x7f911801cec0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f912e800640 1 -- 172.21.15.46:0/2469282258 reap_dead start 2024-09-17T03:26:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f9134a00640 1 -- 172.21.15.46:0/2469282258 shutdown_connections 2024-09-17T03:26:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f9134a00640 1 -- 172.21.15.46:0/2469282258 >> 172.21.15.46:0/2469282258 conn(0x7f91300b2040 msgr2=0x7f91300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f9134a00640 1 -- 172.21.15.46:0/2469282258 shutdown_connections 2024-09-17T03:26:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.667+0000 7f9134a00640 1 -- 172.21.15.46:0/2469282258 wait complete. 2024-09-17T03:26:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 Processor -- start 2024-09-17T03:26:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 -- start start 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f91300ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91301638c0 0x7f91300afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9130159d00 0x7f91300b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9130165c40 con 0x7f91140041d0 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9130165ac0 con 0x7f91301638c0 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9130165dc0 con 0x7f9130159d00 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f91300ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f91300ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49984/0 (socket says 172.21.15.46:49984) 2024-09-17T03:26:25.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 -- 172.21.15.46:0/1412767047 learned_addr learned my addr 172.21.15.46:0/1412767047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9127e00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91301638c0 0x7f91300afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912e800640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9130159d00 0x7f91300b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 -- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9130159d00 msgr2=0x7f91300b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9130159d00 0x7f91300b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 -- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91301638c0 msgr2=0x7f91300afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91301638c0 0x7f91300afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 -- 172.21.15.46:0/1412767047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91300b1610 con 0x7f91140041d0 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9127e00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91301638c0 0x7f91300afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912e800640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9130159d00 0x7f91300b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f912de00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f91300ce330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9120005ed0 tx=0x7f9120011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120005320 con 0x7f91140041d0 2024-09-17T03:26:25.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9120018020 con 0x7f91140041d0 2024-09-17T03:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120003860 con 0x7f91140041d0 2024-09-17T03:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91300af370 con 0x7f91140041d0 2024-09-17T03:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.671+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91300af930 con 0x7f91140041d0 2024-09-17T03:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.675+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91300ce870 con 0x7f91140041d0 2024-09-17T03:26:25.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.675+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f9120003a00 con 0x7f91140041d0 2024-09-17T03:26:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.675+0000 7f9126000640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f910c073b30 0x7f910c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.675+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9120096d00 con 0x7f91140041d0 2024-09-17T03:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.675+0000 7f9127e00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f910c073b30 0x7f910c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.679+0000 7f9127e00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f910c073b30 0x7f910c076040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91180ac000 tx=0x7f9118019d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.679+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f91300ce870 con 0x7f91140041d0 2024-09-17T03:26:25.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.851+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9130163ce0 con 0x7f910c073b30 2024-09-17T03:26:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.855+0000 7f9126000640 1 -- 172.21.15.46:0/1412767047 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250791 (secure 0 0 0) 0x7f9130163ce0 con 0x7f910c073b30 2024-09-17T03:26:25.859 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:25.860 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:26:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f910c073b30 msgr2=0x7f910c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f910c073b30 0x7f910c076040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91180ac000 tx=0x7f9118019d00 comp rx=0 tx=0).stop 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 msgr2=0x7f91300ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f91300ce330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9120005ed0 tx=0x7f9120011740 comp rx=0 tx=0).stop 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 shutdown_connections 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f910c073b30 0x7f910c076040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9130159d00 0x7f91300b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91301638c0 0x7f91300afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 --2- 172.21.15.46:0/1412767047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91140041d0 0x7f91300ce330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 >> 172.21.15.46:0/1412767047 conn(0x7f91300b2040 msgr2=0x7f913015a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 shutdown_connections 2024-09-17T03:26:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:25.859+0000 7f9134a00640 1 -- 172.21.15.46:0/1412767047 wait complete. 2024-09-17T03:26:25.881 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-17T03:26:25.303316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386440,"kb_used_data":6420,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754588280,"statfs":{"total":773094113280,"available":772698398720,"internally_reserved":0,"allocated":6574080,"data_stored":3391384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001164"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.290122+0000","last_change":"2024-09-17T03:26:21.271619+0000","last_active":"2024-09-17T03:26:21.290122+0000","last_peered":"2024-09-17T03:26:21.290122+0000","last_clean":"2024-09-17T03:26:21.290122+0000","last_became_active":"2024-09-17T03:26:21.270472+0000","last_became_peered":"2024-09-17T03:26:21.270472+0000","last_unstale":"2024-09-17T03:26:21.290122+0000","last_undegraded":"2024-09-17T03:26:21.290122+0000","last_fullsized":"2024-09-17T03:26:21.290122+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:58.768092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293649+0000","last_change":"2024-09-17T03:26:21.281731+0000","last_active":"2024-09-17T03:26:21.293649+0000","last_peered":"2024-09-17T03:26:21.293649+0000","last_clean":"2024-09-17T03:26:21.293649+0000","last_became_active":"2024-09-17T03:26:21.281402+0000","last_became_peered":"2024-09-17T03:26:21.281402+0000","last_unstale":"2024-09-17T03:26:21.293649+0000","last_undegraded":"2024-09-17T03:26:21.293649+0000","last_fullsized":"2024-09-17T03:26:21.293649+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:18.996659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:21.283316+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:21.282999+0000","last_became_peered":"2024-09-17T03:26:21.282999+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:15.689667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269895+0000","last_change":"2024-09-17T03:26:20.269895+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269895+0000","last_undegraded":"2024-09-17T03:26:20.269895+0000","last_fullsized":"2024-09-17T03:26:20.269895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289760+0000","last_change":"2024-09-17T03:26:21.284187+0000","last_active":"2024-09-17T03:26:21.289760+0000","last_peered":"2024-09-17T03:26:21.289760+0000","last_clean":"2024-09-17T03:26:21.289760+0000","last_became_active":"2024-09-17T03:26:21.283779+0000","last_became_peered":"2024-09-17T03:26:21.283779+0000","last_unstale":"2024-09-17T03:26:21.289760+0000","last_undegraded":"2024-09-17T03:26:21.289760+0000","last_fullsized":"2024-09-17T03:26:21.289760+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:57.934343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300045+0000","last_change":"2024-09-17T03:26:21.296425+0000","last_active":"2024-09-17T03:26:21.300045+0000","last_peered":"2024-09-17T03:26:21.300045+0000","last_clean":"2024-09-17T03:26:21.300045+0000","last_became_active":"2024-09-17T03:26:21.296063+0000","last_became_peered":"2024-09-17T03:26:21.296063+0000","last_unstale":"2024-09-17T03:26:21.300045+0000","last_undegraded":"2024-09-17T03:26:21.300045+0000","last_fullsized":"2024-09-17T03:26:21.300045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:27.881549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284101+0000","last_change":"2024-09-17T03:26:21.281715+0000","last_active":"2024-09-17T03:26:21.284101+0000","last_peered":"2024-09-17T03:26:21.284101+0000","last_clean":"2024-09-17T03:26:21.284101+0000","last_became_active":"2024-09-17T03:26:21.280618+0000","last_became_peered":"2024-09-17T03:26:21.280618+0000","last_unstale":"2024-09-17T03:26:21.284101+0000","last_undegraded":"2024-09-17T03:26:21.284101+0000","last_fullsized":"2024-09-17T03:26:21.284101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:44.647319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285297+0000","last_change":"2024-09-17T03:26:21.277710+0000","last_active":"2024-09-17T03:26:21.285297+0000","last_peered":"2024-09-17T03:26:21.285297+0000","last_clean":"2024-09-17T03:26:21.285297+0000","last_became_active":"2024-09-17T03:26:21.277333+0000","last_became_peered":"2024-09-17T03:26:21.277333+0000","last_unstale":"2024-09-17T03:26:21.285297+0000","last_undegraded":"2024-09-17T03:26:21.285297+0000","last_fullsized":"2024-09-17T03:26:21.285297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:34.691840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291938+0000","last_change":"2024-09-17T03:26:21.290497+0000","last_active":"2024-09-17T03:26:21.291938+0000","last_peered":"2024-09-17T03:26:21.291938+0000","last_clean":"2024-09-17T03:26:21.291938+0000","last_became_active":"2024-09-17T03:26:21.289289+0000","last_became_peered":"2024-09-17T03:26:21.289289+0000","last_unstale":"2024-09-17T03:26:21.291938+0000","last_undegraded":"2024-09-17T03:26:21.291938+0000","last_fullsized":"2024-09-17T03:26:21.291938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:01.560195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263685+0000","last_change":"2024-09-17T03:26:20.263685+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263685+0000","last_undegraded":"2024-09-17T03:26:20.263685+0000","last_fullsized":"2024-09-17T03:26:20.263685+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262752+0000","last_change":"2024-09-17T03:26:20.262752+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262752+0000","last_undegraded":"2024-09-17T03:26:20.262752+0000","last_fullsized":"2024-09-17T03:26:20.262752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269255+0000","last_change":"2024-09-17T03:26:20.269255+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269255+0000","last_undegraded":"2024-09-17T03:26:20.269255+0000","last_fullsized":"2024-09-17T03:26:20.269255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301774+0000","last_change":"2024-09-17T03:26:21.300221+0000","last_active":"2024-09-17T03:26:21.301774+0000","last_peered":"2024-09-17T03:26:21.301774+0000","last_clean":"2024-09-17T03:26:21.301774+0000","last_became_active":"2024-09-17T03:26:21.299864+0000","last_became_peered":"2024-09-17T03:26:21.299864+0000","last_unstale":"2024-09-17T03:26:21.301774+0000","last_undegraded":"2024-09-17T03:26:21.301774+0000","last_fullsized":"2024-09-17T03:26:21.301774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280369+0000","last_change":"2024-09-17T03:26:21.274122+0000","last_active":"2024-09-17T03:26:21.280369+0000","last_peered":"2024-09-17T03:26:21.280369+0000","last_clean":"2024-09-17T03:26:21.280369+0000","last_became_active":"2024-09-17T03:26:21.273777+0000","last_became_peered":"2024-09-17T03:26:21.273777+0000","last_unstale":"2024-09-17T03:26:21.280369+0000","last_undegraded":"2024-09-17T03:26:21.280369+0000","last_fullsized":"2024-09-17T03:26:21.280369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295641+0000","last_change":"2024-09-17T03:26:21.292845+0000","last_active":"2024-09-17T03:26:21.295641+0000","last_peered":"2024-09-17T03:26:21.295641+0000","last_clean":"2024-09-17T03:26:21.295641+0000","last_became_active":"2024-09-17T03:26:21.292432+0000","last_became_peered":"2024-09-17T03:26:21.292432+0000","last_unstale":"2024-09-17T03:26:21.295641+0000","last_undegraded":"2024-09-17T03:26:21.295641+0000","last_fullsized":"2024-09-17T03:26:21.295641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285635+0000","last_change":"2024-09-17T03:26:21.282672+0000","last_active":"2024-09-17T03:26:21.285635+0000","last_peered":"2024-09-17T03:26:21.285635+0000","last_clean":"2024-09-17T03:26:21.285635+0000","last_became_active":"2024-09-17T03:26:21.282359+0000","last_became_peered":"2024-09-17T03:26:21.282359+0000","last_unstale":"2024-09-17T03:26:21.285635+0000","last_undegraded":"2024-09-17T03:26:21.285635+0000","last_fullsized":"2024-09-17T03:26:21.285635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:11.693379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283260+0000","last_change":"2024-09-17T03:26:21.277694+0000","last_active":"2024-09-17T03:26:21.283260+0000","last_peered":"2024-09-17T03:26:21.283260+0000","last_clean":"2024-09-17T03:26:21.283260+0000","last_became_active":"2024-09-17T03:26:21.277341+0000","last_became_peered":"2024-09-17T03:26:21.277341+0000","last_unstale":"2024-09-17T03:26:21.283260+0000","last_undegraded":"2024-09-17T03:26:21.283260+0000","last_fullsized":"2024-09-17T03:26:21.283260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:16.287612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294961+0000","last_change":"2024-09-17T03:26:21.289536+0000","last_active":"2024-09-17T03:26:21.294961+0000","last_peered":"2024-09-17T03:26:21.294961+0000","last_clean":"2024-09-17T03:26:21.294961+0000","last_became_active":"2024-09-17T03:26:21.289230+0000","last_became_peered":"2024-09-17T03:26:21.289230+0000","last_unstale":"2024-09-17T03:26:21.294961+0000","last_undegraded":"2024-09-17T03:26:21.294961+0000","last_fullsized":"2024-09-17T03:26:21.294961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:31.982607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288531+0000","last_change":"2024-09-17T03:26:21.272001+0000","last_active":"2024-09-17T03:26:21.288531+0000","last_peered":"2024-09-17T03:26:21.288531+0000","last_clean":"2024-09-17T03:26:21.288531+0000","last_became_active":"2024-09-17T03:26:21.271673+0000","last_became_peered":"2024-09-17T03:26:21.271673+0000","last_unstale":"2024-09-17T03:26:21.288531+0000","last_undegraded":"2024-09-17T03:26:21.288531+0000","last_fullsized":"2024-09-17T03:26:21.288531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:48.995086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295028+0000","last_change":"2024-09-17T03:26:21.291765+0000","last_active":"2024-09-17T03:26:21.295028+0000","last_peered":"2024-09-17T03:26:21.295028+0000","last_clean":"2024-09-17T03:26:21.295028+0000","last_became_active":"2024-09-17T03:26:21.291430+0000","last_became_peered":"2024-09-17T03:26:21.291430+0000","last_unstale":"2024-09-17T03:26:21.295028+0000","last_undegraded":"2024-09-17T03:26:21.295028+0000","last_fullsized":"2024-09-17T03:26:21.295028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.144097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289278+0000","last_change":"2024-09-17T03:26:21.285773+0000","last_active":"2024-09-17T03:26:21.289278+0000","last_peered":"2024-09-17T03:26:21.289278+0000","last_clean":"2024-09-17T03:26:21.289278+0000","last_became_active":"2024-09-17T03:26:21.285118+0000","last_became_peered":"2024-09-17T03:26:21.285118+0000","last_unstale":"2024-09-17T03:26:21.289278+0000","last_undegraded":"2024-09-17T03:26:21.289278+0000","last_fullsized":"2024-09-17T03:26:21.289278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:50.670027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283248+0000","last_change":"2024-09-17T03:26:21.276516+0000","last_active":"2024-09-17T03:26:21.283248+0000","last_peered":"2024-09-17T03:26:21.283248+0000","last_clean":"2024-09-17T03:26:21.283248+0000","last_became_active":"2024-09-17T03:26:21.275398+0000","last_became_peered":"2024-09-17T03:26:21.275398+0000","last_unstale":"2024-09-17T03:26:21.283248+0000","last_undegraded":"2024-09-17T03:26:21.283248+0000","last_fullsized":"2024-09-17T03:26:21.283248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:22.952091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291438+0000","last_change":"2024-09-17T03:26:21.286047+0000","last_active":"2024-09-17T03:26:21.291438+0000","last_peered":"2024-09-17T03:26:21.291438+0000","last_clean":"2024-09-17T03:26:21.291438+0000","last_became_active":"2024-09-17T03:26:21.284398+0000","last_became_peered":"2024-09-17T03:26:21.284398+0000","last_unstale":"2024-09-17T03:26:21.291438+0000","last_undegraded":"2024-09-17T03:26:21.291438+0000","last_fullsized":"2024-09-17T03:26:21.291438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:38.105159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293886+0000","last_change":"2024-09-17T03:26:21.289586+0000","last_active":"2024-09-17T03:26:21.293886+0000","last_peered":"2024-09-17T03:26:21.293886+0000","last_clean":"2024-09-17T03:26:21.293886+0000","last_became_active":"2024-09-17T03:26:21.289242+0000","last_became_peered":"2024-09-17T03:26:21.289242+0000","last_unstale":"2024-09-17T03:26:21.293886+0000","last_undegraded":"2024-09-17T03:26:21.293886+0000","last_fullsized":"2024-09-17T03:26:21.293886+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:47.275691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293738+0000","last_change":"2024-09-17T03:26:21.287754+0000","last_active":"2024-09-17T03:26:21.293738+0000","last_peered":"2024-09-17T03:26:21.293738+0000","last_clean":"2024-09-17T03:26:21.293738+0000","last_became_active":"2024-09-17T03:26:21.287382+0000","last_became_peered":"2024-09-17T03:26:21.287382+0000","last_unstale":"2024-09-17T03:26:21.293738+0000","last_undegraded":"2024-09-17T03:26:21.293738+0000","last_fullsized":"2024-09-17T03:26:21.293738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:03.289618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275232+0000","last_change":"2024-09-17T03:26:20.275232+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275232+0000","last_undegraded":"2024-09-17T03:26:20.275232+0000","last_fullsized":"2024-09-17T03:26:20.275232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284955+0000","last_change":"2024-09-17T03:26:21.282975+0000","last_active":"2024-09-17T03:26:21.284955+0000","last_peered":"2024-09-17T03:26:21.284955+0000","last_clean":"2024-09-17T03:26:21.284955+0000","last_became_active":"2024-09-17T03:26:21.282116+0000","last_became_peered":"2024-09-17T03:26:21.282116+0000","last_unstale":"2024-09-17T03:26:21.284955+0000","last_undegraded":"2024-09-17T03:26:21.284955+0000","last_fullsized":"2024-09-17T03:26:21.284955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:08.367097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293266+0000","last_change":"2024-09-17T03:26:21.287660+0000","last_active":"2024-09-17T03:26:21.293266+0000","last_peered":"2024-09-17T03:26:21.293266+0000","last_clean":"2024-09-17T03:26:21.293266+0000","last_became_active":"2024-09-17T03:26:21.287329+0000","last_became_peered":"2024-09-17T03:26:21.287329+0000","last_unstale":"2024-09-17T03:26:21.293266+0000","last_undegraded":"2024-09-17T03:26:21.293266+0000","last_fullsized":"2024-09-17T03:26:21.293266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:07.684829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293795+0000","last_change":"2024-09-17T03:26:21.281015+0000","last_active":"2024-09-17T03:26:21.293795+0000","last_peered":"2024-09-17T03:26:21.293795+0000","last_clean":"2024-09-17T03:26:21.293795+0000","last_became_active":"2024-09-17T03:26:21.280690+0000","last_became_peered":"2024-09-17T03:26:21.280690+0000","last_unstale":"2024-09-17T03:26:21.293795+0000","last_undegraded":"2024-09-17T03:26:21.293795+0000","last_fullsized":"2024-09-17T03:26:21.293795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:27.474853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288915+0000","last_change":"2024-09-17T03:26:21.282370+0000","last_active":"2024-09-17T03:26:21.288915+0000","last_peered":"2024-09-17T03:26:21.288915+0000","last_clean":"2024-09-17T03:26:21.288915+0000","last_became_active":"2024-09-17T03:26:21.282053+0000","last_became_peered":"2024-09-17T03:26:21.282053+0000","last_unstale":"2024-09-17T03:26:21.288915+0000","last_undegraded":"2024-09-17T03:26:21.288915+0000","last_fullsized":"2024-09-17T03:26:21.288915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:27.209668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295471+0000","last_change":"2024-09-17T03:26:21.293642+0000","last_active":"2024-09-17T03:26:21.295471+0000","last_peered":"2024-09-17T03:26:21.295471+0000","last_clean":"2024-09-17T03:26:21.295471+0000","last_became_active":"2024-09-17T03:26:21.289857+0000","last_became_peered":"2024-09-17T03:26:21.289857+0000","last_unstale":"2024-09-17T03:26:21.295471+0000","last_undegraded":"2024-09-17T03:26:21.295471+0000","last_fullsized":"2024-09-17T03:26:21.295471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:56.082574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281534+0000","last_change":"2024-09-17T03:26:21.278899+0000","last_active":"2024-09-17T03:26:21.281534+0000","last_peered":"2024-09-17T03:26:21.281534+0000","last_clean":"2024-09-17T03:26:21.281534+0000","last_became_active":"2024-09-17T03:26:21.278562+0000","last_became_peered":"2024-09-17T03:26:21.278562+0000","last_unstale":"2024-09-17T03:26:21.281534+0000","last_undegraded":"2024-09-17T03:26:21.281534+0000","last_fullsized":"2024-09-17T03:26:21.281534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:06.391945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281831+0000","last_change":"2024-09-17T03:26:21.279072+0000","last_active":"2024-09-17T03:26:21.281831+0000","last_peered":"2024-09-17T03:26:21.281831+0000","last_clean":"2024-09-17T03:26:21.281831+0000","last_became_active":"2024-09-17T03:26:21.278730+0000","last_became_peered":"2024-09-17T03:26:21.278730+0000","last_unstale":"2024-09-17T03:26:21.281831+0000","last_undegraded":"2024-09-17T03:26:21.281831+0000","last_fullsized":"2024-09-17T03:26:21.281831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:31.274794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300099+0000","last_change":"2024-09-17T03:26:20.283656+0000","last_active":"2024-09-17T03:26:21.300099+0000","last_peered":"2024-09-17T03:26:21.300099+0000","last_clean":"2024-09-17T03:26:21.300099+0000","last_became_active":"2024-09-17T03:26:20.283331+0000","last_became_peered":"2024-09-17T03:26:20.283331+0000","last_unstale":"2024-09-17T03:26:21.300099+0000","last_undegraded":"2024-09-17T03:26:21.300099+0000","last_fullsized":"2024-09-17T03:26:21.300099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:03.041707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263563+0000","last_change":"2024-09-17T03:26:20.263563+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263563+0000","last_undegraded":"2024-09-17T03:26:20.263563+0000","last_fullsized":"2024-09-17T03:26:20.263563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:20.266917+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:20.266370+0000","last_became_peered":"2024-09-17T03:26:20.266370+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:57.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300820+0000","last_change":"2024-09-17T03:26:21.299198+0000","last_active":"2024-09-17T03:26:21.300820+0000","last_peered":"2024-09-17T03:26:21.300820+0000","last_clean":"2024-09-17T03:26:21.300820+0000","last_became_active":"2024-09-17T03:26:21.298717+0000","last_became_peered":"2024-09-17T03:26:21.298717+0000","last_unstale":"2024-09-17T03:26:21.300820+0000","last_undegraded":"2024-09-17T03:26:21.300820+0000","last_fullsized":"2024-09-17T03:26:21.300820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:48.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301538+0000","last_change":"2024-09-17T03:26:20.286649+0000","last_active":"2024-09-17T03:26:21.301538+0000","last_peered":"2024-09-17T03:26:21.301538+0000","last_clean":"2024-09-17T03:26:21.301538+0000","last_became_active":"2024-09-17T03:26:20.285886+0000","last_became_peered":"2024-09-17T03:26:20.285886+0000","last_unstale":"2024-09-17T03:26:21.301538+0000","last_undegraded":"2024-09-17T03:26:21.301538+0000","last_fullsized":"2024-09-17T03:26:21.301538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:46.887782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300191+0000","last_change":"2024-09-17T03:26:21.289923+0000","last_active":"2024-09-17T03:26:21.300191+0000","last_peered":"2024-09-17T03:26:21.300191+0000","last_clean":"2024-09-17T03:26:21.300191+0000","last_became_active":"2024-09-17T03:26:21.289556+0000","last_became_peered":"2024-09-17T03:26:21.289556+0000","last_unstale":"2024-09-17T03:26:21.300191+0000","last_undegraded":"2024-09-17T03:26:21.300191+0000","last_fullsized":"2024-09-17T03:26:21.300191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:41.180633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286893+0000","last_change":"2024-09-17T03:26:20.283187+0000","last_active":"2024-09-17T03:26:20.286893+0000","last_peered":"2024-09-17T03:26:20.286893+0000","last_clean":"2024-09-17T03:26:20.286893+0000","last_became_active":"2024-09-17T03:26:20.282354+0000","last_became_peered":"2024-09-17T03:26:20.282354+0000","last_unstale":"2024-09-17T03:26:20.286893+0000","last_undegraded":"2024-09-17T03:26:20.286893+0000","last_fullsized":"2024-09-17T03:26:20.286893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:57.123281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293825+0000","last_change":"2024-09-17T03:26:21.287275+0000","last_active":"2024-09-17T03:26:21.293825+0000","last_peered":"2024-09-17T03:26:21.293825+0000","last_clean":"2024-09-17T03:26:21.293825+0000","last_became_active":"2024-09-17T03:26:21.286995+0000","last_became_peered":"2024-09-17T03:26:21.286995+0000","last_unstale":"2024-09-17T03:26:21.293825+0000","last_undegraded":"2024-09-17T03:26:21.293825+0000","last_fullsized":"2024-09-17T03:26:21.293825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:58.245607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.298865+0000","last_change":"2024-09-17T03:26:20.283433+0000","last_active":"2024-09-17T03:26:21.298865+0000","last_peered":"2024-09-17T03:26:21.298865+0000","last_clean":"2024-09-17T03:26:21.298865+0000","last_became_active":"2024-09-17T03:26:20.277797+0000","last_became_peered":"2024-09-17T03:26:20.277797+0000","last_unstale":"2024-09-17T03:26:21.298865+0000","last_undegraded":"2024-09-17T03:26:21.298865+0000","last_fullsized":"2024-09-17T03:26:21.298865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294978+0000","last_change":"2024-09-17T03:26:21.293180+0000","last_active":"2024-09-17T03:26:21.294978+0000","last_peered":"2024-09-17T03:26:21.294978+0000","last_clean":"2024-09-17T03:26:21.294978+0000","last_became_active":"2024-09-17T03:26:21.292882+0000","last_became_peered":"2024-09-17T03:26:21.292882+0000","last_unstale":"2024-09-17T03:26:21.294978+0000","last_undegraded":"2024-09-17T03:26:21.294978+0000","last_fullsized":"2024-09-17T03:26:21.294978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:16.723527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301141+0000","last_change":"2024-09-17T03:26:20.279365+0000","last_active":"2024-09-17T03:26:21.301141+0000","last_peered":"2024-09-17T03:26:21.301141+0000","last_clean":"2024-09-17T03:26:21.301141+0000","last_became_active":"2024-09-17T03:26:20.278923+0000","last_became_peered":"2024-09-17T03:26:20.278923+0000","last_unstale":"2024-09-17T03:26:21.301141+0000","last_undegraded":"2024-09-17T03:26:21.301141+0000","last_fullsized":"2024-09-17T03:26:21.301141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:14.310020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286603+0000","last_change":"2024-09-17T03:26:21.285430+0000","last_active":"2024-09-17T03:26:21.286603+0000","last_peered":"2024-09-17T03:26:21.286603+0000","last_clean":"2024-09-17T03:26:21.286603+0000","last_became_active":"2024-09-17T03:26:21.285111+0000","last_became_peered":"2024-09-17T03:26:21.285111+0000","last_unstale":"2024-09-17T03:26:21.286603+0000","last_undegraded":"2024-09-17T03:26:21.286603+0000","last_fullsized":"2024-09-17T03:26:21.286603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:47.441700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284199+0000","last_change":"2024-09-17T03:26:20.279560+0000","last_active":"2024-09-17T03:26:20.284199+0000","last_peered":"2024-09-17T03:26:20.284199+0000","last_clean":"2024-09-17T03:26:20.284199+0000","last_became_active":"2024-09-17T03:26:20.278746+0000","last_became_peered":"2024-09-17T03:26:20.278746+0000","last_unstale":"2024-09-17T03:26:20.284199+0000","last_undegraded":"2024-09-17T03:26:20.284199+0000","last_fullsized":"2024-09-17T03:26:20.284199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:10.047880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286823+0000","last_change":"2024-09-17T03:26:21.279707+0000","last_active":"2024-09-17T03:26:21.286823+0000","last_peered":"2024-09-17T03:26:21.286823+0000","last_clean":"2024-09-17T03:26:21.286823+0000","last_became_active":"2024-09-17T03:26:21.279020+0000","last_became_peered":"2024-09-17T03:26:21.279020+0000","last_unstale":"2024-09-17T03:26:21.286823+0000","last_undegraded":"2024-09-17T03:26:21.286823+0000","last_fullsized":"2024-09-17T03:26:21.286823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:48.268297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264232+0000","last_change":"2024-09-17T03:26:20.283353+0000","last_active":"2024-09-17T03:26:21.264232+0000","last_peered":"2024-09-17T03:26:21.264232+0000","last_clean":"2024-09-17T03:26:21.264232+0000","last_became_active":"2024-09-17T03:26:20.283049+0000","last_became_peered":"2024-09-17T03:26:20.283049+0000","last_unstale":"2024-09-17T03:26:21.264232+0000","last_undegraded":"2024-09-17T03:26:21.264232+0000","last_fullsized":"2024-09-17T03:26:21.264232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291964+0000","last_change":"2024-09-17T03:26:21.290216+0000","last_active":"2024-09-17T03:26:21.291964+0000","last_peered":"2024-09-17T03:26:21.291964+0000","last_clean":"2024-09-17T03:26:21.291964+0000","last_became_active":"2024-09-17T03:26:21.288236+0000","last_became_peered":"2024-09-17T03:26:21.288236+0000","last_unstale":"2024-09-17T03:26:21.291964+0000","last_undegraded":"2024-09-17T03:26:21.291964+0000","last_fullsized":"2024-09-17T03:26:21.291964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:34.999880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269902+0000","last_change":"2024-09-17T03:26:20.289405+0000","last_active":"2024-09-17T03:26:21.269902+0000","last_peered":"2024-09-17T03:26:21.269902+0000","last_clean":"2024-09-17T03:26:21.269902+0000","last_became_active":"2024-09-17T03:26:20.288979+0000","last_became_peered":"2024-09-17T03:26:20.288979+0000","last_unstale":"2024-09-17T03:26:21.269902+0000","last_undegraded":"2024-09-17T03:26:21.269902+0000","last_fullsized":"2024-09-17T03:26:21.269902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:10.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.303142+0000","last_change":"2024-09-17T03:26:21.301608+0000","last_active":"2024-09-17T03:26:21.303142+0000","last_peered":"2024-09-17T03:26:21.303142+0000","last_clean":"2024-09-17T03:26:21.303142+0000","last_became_active":"2024-09-17T03:26:21.301203+0000","last_became_peered":"2024-09-17T03:26:21.301203+0000","last_unstale":"2024-09-17T03:26:21.303142+0000","last_undegraded":"2024-09-17T03:26:21.303142+0000","last_fullsized":"2024-09-17T03:26:21.303142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:23.395294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.271410+0000","last_change":"2024-09-17T03:26:20.263118+0000","last_active":"2024-09-17T03:26:21.271410+0000","last_peered":"2024-09-17T03:26:21.271410+0000","last_clean":"2024-09-17T03:26:21.271410+0000","last_became_active":"2024-09-17T03:26:20.262773+0000","last_became_peered":"2024-09-17T03:26:20.262773+0000","last_unstale":"2024-09-17T03:26:21.271410+0000","last_undegraded":"2024-09-17T03:26:21.271410+0000","last_fullsized":"2024-09-17T03:26:21.271410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:02.639519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263081+0000","last_change":"2024-09-17T03:26:20.263081+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263081+0000","last_undegraded":"2024-09-17T03:26:20.263081+0000","last_fullsized":"2024-09-17T03:26:20.263081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.306381+0000","last_change":"2024-09-17T03:26:20.277795+0000","last_active":"2024-09-17T03:26:21.306381+0000","last_peered":"2024-09-17T03:26:21.306381+0000","last_clean":"2024-09-17T03:26:21.306381+0000","last_became_active":"2024-09-17T03:26:20.277456+0000","last_became_peered":"2024-09-17T03:26:20.277456+0000","last_unstale":"2024-09-17T03:26:21.306381+0000","last_undegraded":"2024-09-17T03:26:21.306381+0000","last_fullsized":"2024-09-17T03:26:21.306381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:43.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297424+0000","last_change":"2024-09-17T03:26:21.292382+0000","last_active":"2024-09-17T03:26:21.297424+0000","last_peered":"2024-09-17T03:26:21.297424+0000","last_clean":"2024-09-17T03:26:21.297424+0000","last_became_active":"2024-09-17T03:26:21.292043+0000","last_became_peered":"2024-09-17T03:26:21.292043+0000","last_unstale":"2024-09-17T03:26:21.297424+0000","last_undegraded":"2024-09-17T03:26:21.297424+0000","last_fullsized":"2024-09-17T03:26:21.297424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:02.648204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.308666+0000","last_change":"2024-09-17T03:26:20.289201+0000","last_active":"2024-09-17T03:26:21.308666+0000","last_peered":"2024-09-17T03:26:21.308666+0000","last_clean":"2024-09-17T03:26:21.308666+0000","last_became_active":"2024-09-17T03:26:20.288427+0000","last_became_peered":"2024-09-17T03:26:20.288427+0000","last_unstale":"2024-09-17T03:26:21.308666+0000","last_undegraded":"2024-09-17T03:26:21.308666+0000","last_fullsized":"2024-09-17T03:26:21.308666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:46.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275142+0000","last_change":"2024-09-17T03:26:20.275142+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275142+0000","last_undegraded":"2024-09-17T03:26:20.275142+0000","last_fullsized":"2024-09-17T03:26:20.275142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.292629+0000","last_change":"2024-09-17T03:26:20.265758+0000","last_active":"2024-09-17T03:26:21.292629+0000","last_peered":"2024-09-17T03:26:21.292629+0000","last_clean":"2024-09-17T03:26:21.292629+0000","last_became_active":"2024-09-17T03:26:20.262782+0000","last_became_peered":"2024-09-17T03:26:20.262782+0000","last_unstale":"2024-09-17T03:26:21.292629+0000","last_undegraded":"2024-09-17T03:26:21.292629+0000","last_fullsized":"2024-09-17T03:26:21.292629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:10.171064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283171+0000","last_change":"2024-09-17T03:26:21.278170+0000","last_active":"2024-09-17T03:26:21.283171+0000","last_peered":"2024-09-17T03:26:21.283171+0000","last_clean":"2024-09-17T03:26:21.283171+0000","last_became_active":"2024-09-17T03:26:21.277329+0000","last_became_peered":"2024-09-17T03:26:21.277329+0000","last_unstale":"2024-09-17T03:26:21.283171+0000","last_undegraded":"2024-09-17T03:26:21.283171+0000","last_fullsized":"2024-09-17T03:26:21.283171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:32.373126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267858+0000","last_change":"2024-09-17T03:26:20.269475+0000","last_active":"2024-09-17T03:26:21.267858+0000","last_peered":"2024-09-17T03:26:21.267858+0000","last_clean":"2024-09-17T03:26:21.267858+0000","last_became_active":"2024-09-17T03:26:20.269101+0000","last_became_peered":"2024-09-17T03:26:20.269101+0000","last_unstale":"2024-09-17T03:26:21.267858+0000","last_undegraded":"2024-09-17T03:26:21.267858+0000","last_fullsized":"2024-09-17T03:26:21.267858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283965+0000","last_change":"2024-09-17T03:26:21.281823+0000","last_active":"2024-09-17T03:26:21.283965+0000","last_peered":"2024-09-17T03:26:21.283965+0000","last_clean":"2024-09-17T03:26:21.283965+0000","last_became_active":"2024-09-17T03:26:21.280688+0000","last_became_peered":"2024-09-17T03:26:21.280688+0000","last_unstale":"2024-09-17T03:26:21.283965+0000","last_undegraded":"2024-09-17T03:26:21.283965+0000","last_fullsized":"2024-09-17T03:26:21.283965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:36.912044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284389+0000","last_change":"2024-09-17T03:26:20.281345+0000","last_active":"2024-09-17T03:26:20.284389+0000","last_peered":"2024-09-17T03:26:20.284389+0000","last_clean":"2024-09-17T03:26:20.284389+0000","last_became_active":"2024-09-17T03:26:20.280261+0000","last_became_peered":"2024-09-17T03:26:20.280261+0000","last_unstale":"2024-09-17T03:26:20.284389+0000","last_undegraded":"2024-09-17T03:26:20.284389+0000","last_fullsized":"2024-09-17T03:26:20.284389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:47.948863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294667+0000","last_change":"2024-09-17T03:26:21.289462+0000","last_active":"2024-09-17T03:26:21.294667+0000","last_peered":"2024-09-17T03:26:21.294667+0000","last_clean":"2024-09-17T03:26:21.294667+0000","last_became_active":"2024-09-17T03:26:21.285434+0000","last_became_peered":"2024-09-17T03:26:21.285434+0000","last_unstale":"2024-09-17T03:26:21.294667+0000","last_undegraded":"2024-09-17T03:26:21.294667+0000","last_fullsized":"2024-09-17T03:26:21.294667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:24.253438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301604+0000","last_change":"2024-09-17T03:26:20.275945+0000","last_active":"2024-09-17T03:26:21.301604+0000","last_peered":"2024-09-17T03:26:21.301604+0000","last_clean":"2024-09-17T03:26:21.301604+0000","last_became_active":"2024-09-17T03:26:20.275123+0000","last_became_peered":"2024-09-17T03:26:20.275123+0000","last_unstale":"2024-09-17T03:26:21.301604+0000","last_undegraded":"2024-09-17T03:26:21.301604+0000","last_fullsized":"2024-09-17T03:26:21.301604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:02.236108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291615+0000","last_change":"2024-09-17T03:26:21.284952+0000","last_active":"2024-09-17T03:26:21.291615+0000","last_peered":"2024-09-17T03:26:21.291615+0000","last_clean":"2024-09-17T03:26:21.291615+0000","last_became_active":"2024-09-17T03:26:21.284572+0000","last_became_peered":"2024-09-17T03:26:21.284572+0000","last_unstale":"2024-09-17T03:26:21.291615+0000","last_undegraded":"2024-09-17T03:26:21.291615+0000","last_fullsized":"2024-09-17T03:26:21.291615+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:11.920676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295381+0000","last_change":"2024-09-17T03:26:20.269212+0000","last_active":"2024-09-17T03:26:21.295381+0000","last_peered":"2024-09-17T03:26:21.295381+0000","last_clean":"2024-09-17T03:26:21.295381+0000","last_became_active":"2024-09-17T03:26:20.268883+0000","last_became_peered":"2024-09-17T03:26:20.268883+0000","last_unstale":"2024-09-17T03:26:21.295381+0000","last_undegraded":"2024-09-17T03:26:21.295381+0000","last_fullsized":"2024-09-17T03:26:21.295381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:41:56.450980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274114+0000","last_change":"2024-09-17T03:26:20.274114+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274114+0000","last_undegraded":"2024-09-17T03:26:20.274114+0000","last_fullsized":"2024-09-17T03:26:20.274114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300538+0000","last_change":"2024-09-17T03:26:20.279146+0000","last_active":"2024-09-17T03:26:21.300538+0000","last_peered":"2024-09-17T03:26:21.300538+0000","last_clean":"2024-09-17T03:26:21.300538+0000","last_became_active":"2024-09-17T03:26:20.278888+0000","last_became_peered":"2024-09-17T03:26:20.278888+0000","last_unstale":"2024-09-17T03:26:21.300538+0000","last_undegraded":"2024-09-17T03:26:21.300538+0000","last_fullsized":"2024-09-17T03:26:21.300538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:40.150476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286599+0000","last_change":"2024-09-17T03:26:21.278295+0000","last_active":"2024-09-17T03:26:21.286599+0000","last_peered":"2024-09-17T03:26:21.286599+0000","last_clean":"2024-09-17T03:26:21.286599+0000","last_became_active":"2024-09-17T03:26:21.277728+0000","last_became_peered":"2024-09-17T03:26:21.277728+0000","last_unstale":"2024-09-17T03:26:21.286599+0000","last_undegraded":"2024-09-17T03:26:21.286599+0000","last_fullsized":"2024-09-17T03:26:21.286599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:38.173310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277895+0000","last_change":"2024-09-17T03:26:20.272850+0000","last_active":"2024-09-17T03:26:20.277895+0000","last_peered":"2024-09-17T03:26:20.277895+0000","last_clean":"2024-09-17T03:26:20.277895+0000","last_became_active":"2024-09-17T03:26:20.272036+0000","last_became_peered":"2024-09-17T03:26:20.272036+0000","last_unstale":"2024-09-17T03:26:20.277895+0000","last_undegraded":"2024-09-17T03:26:20.277895+0000","last_fullsized":"2024-09-17T03:26:20.277895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:25.846783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297474+0000","last_change":"2024-09-17T03:26:21.294723+0000","last_active":"2024-09-17T03:26:21.297474+0000","last_peered":"2024-09-17T03:26:21.297474+0000","last_clean":"2024-09-17T03:26:21.297474+0000","last_became_active":"2024-09-17T03:26:21.294375+0000","last_became_peered":"2024-09-17T03:26:21.294375+0000","last_unstale":"2024-09-17T03:26:21.297474+0000","last_undegraded":"2024-09-17T03:26:21.297474+0000","last_fullsized":"2024-09-17T03:26:21.297474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:22.471193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.275453+0000","last_change":"2024-09-17T03:26:20.281457+0000","last_active":"2024-09-17T03:26:21.275453+0000","last_peered":"2024-09-17T03:26:21.275453+0000","last_clean":"2024-09-17T03:26:21.275453+0000","last_became_active":"2024-09-17T03:26:20.281078+0000","last_became_peered":"2024-09-17T03:26:20.281078+0000","last_unstale":"2024-09-17T03:26:21.275453+0000","last_undegraded":"2024-09-17T03:26:21.275453+0000","last_fullsized":"2024-09-17T03:26:21.275453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:22.720948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.272277+0000","last_change":"2024-09-17T03:26:20.272277+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.272277+0000","last_undegraded":"2024-09-17T03:26:20.272277+0000","last_fullsized":"2024-09-17T03:26:20.272277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.296826+0000","last_change":"2024-09-17T03:26:20.280006+0000","last_active":"2024-09-17T03:26:21.296826+0000","last_peered":"2024-09-17T03:26:21.296826+0000","last_clean":"2024-09-17T03:26:21.296826+0000","last_became_active":"2024-09-17T03:26:20.279611+0000","last_became_peered":"2024-09-17T03:26:20.279611+0000","last_unstale":"2024-09-17T03:26:21.296826+0000","last_undegraded":"2024-09-17T03:26:21.296826+0000","last_fullsized":"2024-09-17T03:26:21.296826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:29.773567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289195+0000","last_change":"2024-09-17T03:26:21.284722+0000","last_active":"2024-09-17T03:26:21.289195+0000","last_peered":"2024-09-17T03:26:21.289195+0000","last_clean":"2024-09-17T03:26:21.289195+0000","last_became_active":"2024-09-17T03:26:21.284399+0000","last_became_peered":"2024-09-17T03:26:21.284399+0000","last_unstale":"2024-09-17T03:26:21.289195+0000","last_undegraded":"2024-09-17T03:26:21.289195+0000","last_fullsized":"2024-09-17T03:26:21.289195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:55.320737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277267+0000","last_change":"2024-09-17T03:26:20.274933+0000","last_active":"2024-09-17T03:26:20.277267+0000","last_peered":"2024-09-17T03:26:20.277267+0000","last_clean":"2024-09-17T03:26:20.277267+0000","last_became_active":"2024-09-17T03:26:20.274536+0000","last_became_peered":"2024-09-17T03:26:20.274536+0000","last_unstale":"2024-09-17T03:26:20.277267+0000","last_undegraded":"2024-09-17T03:26:20.277267+0000","last_fullsized":"2024-09-17T03:26:20.277267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:23.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280742+0000","last_change":"2024-09-17T03:26:21.277208+0000","last_active":"2024-09-17T03:26:21.280742+0000","last_peered":"2024-09-17T03:26:21.280742+0000","last_clean":"2024-09-17T03:26:21.280742+0000","last_became_active":"2024-09-17T03:26:21.276812+0000","last_became_peered":"2024-09-17T03:26:21.276812+0000","last_unstale":"2024-09-17T03:26:21.280742+0000","last_undegraded":"2024-09-17T03:26:21.280742+0000","last_fullsized":"2024-09-17T03:26:21.280742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:12.678429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284384+0000","last_change":"2024-09-17T03:26:20.281254+0000","last_active":"2024-09-17T03:26:20.284384+0000","last_peered":"2024-09-17T03:26:20.284384+0000","last_clean":"2024-09-17T03:26:20.284384+0000","last_became_active":"2024-09-17T03:26:20.280858+0000","last_became_peered":"2024-09-17T03:26:20.280858+0000","last_unstale":"2024-09-17T03:26:20.284384+0000","last_undegraded":"2024-09-17T03:26:20.284384+0000","last_fullsized":"2024-09-17T03:26:20.284384+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:22.288031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274734+0000","last_change":"2024-09-17T03:26:20.274734+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274734+0000","last_undegraded":"2024-09-17T03:26:20.274734+0000","last_fullsized":"2024-09-17T03:26:20.274734+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277412+0000","last_change":"2024-09-17T03:26:20.272578+0000","last_active":"2024-09-17T03:26:20.277412+0000","last_peered":"2024-09-17T03:26:20.277412+0000","last_clean":"2024-09-17T03:26:20.277412+0000","last_became_active":"2024-09-17T03:26:20.272215+0000","last_became_peered":"2024-09-17T03:26:20.272215+0000","last_unstale":"2024-09-17T03:26:20.277412+0000","last_undegraded":"2024-09-17T03:26:20.277412+0000","last_fullsized":"2024-09-17T03:26:20.277412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:56.751763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.273561+0000","last_change":"2024-09-17T03:26:20.273561+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.273561+0000","last_undegraded":"2024-09-17T03:26:20.273561+0000","last_fullsized":"2024-09-17T03:26:20.273561+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.276867+0000","last_change":"2024-09-17T03:26:20.281152+0000","last_active":"2024-09-17T03:26:21.276867+0000","last_peered":"2024-09-17T03:26:21.276867+0000","last_clean":"2024-09-17T03:26:21.276867+0000","last_became_active":"2024-09-17T03:26:20.280782+0000","last_became_peered":"2024-09-17T03:26:20.280782+0000","last_unstale":"2024-09-17T03:26:21.276867+0000","last_undegraded":"2024-09-17T03:26:21.276867+0000","last_fullsized":"2024-09-17T03:26:21.276867+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:08.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262545+0000","last_change":"2024-09-17T03:26:20.262545+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262545+0000","last_undegraded":"2024-09-17T03:26:20.262545+0000","last_fullsized":"2024-09-17T03:26:20.262545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267915+0000","last_change":"2024-09-17T03:26:20.277793+0000","last_active":"2024-09-17T03:26:21.267915+0000","last_peered":"2024-09-17T03:26:21.267915+0000","last_clean":"2024-09-17T03:26:21.267915+0000","last_became_active":"2024-09-17T03:26:20.276428+0000","last_became_peered":"2024-09-17T03:26:20.276428+0000","last_unstale":"2024-09-17T03:26:21.267915+0000","last_undegraded":"2024-09-17T03:26:21.267915+0000","last_fullsized":"2024-09-17T03:26:21.267915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289751+0000","last_change":"2024-09-17T03:26:21.283319+0000","last_active":"2024-09-17T03:26:21.289751+0000","last_peered":"2024-09-17T03:26:21.289751+0000","last_clean":"2024-09-17T03:26:21.289751+0000","last_became_active":"2024-09-17T03:26:21.282574+0000","last_became_peered":"2024-09-17T03:26:21.282574+0000","last_unstale":"2024-09-17T03:26:21.289751+0000","last_undegraded":"2024-09-17T03:26:21.289751+0000","last_fullsized":"2024-09-17T03:26:21.289751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:54.060063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286863+0000","last_change":"2024-09-17T03:26:20.282834+0000","last_active":"2024-09-17T03:26:20.286863+0000","last_peered":"2024-09-17T03:26:20.286863+0000","last_clean":"2024-09-17T03:26:20.286863+0000","last_became_active":"2024-09-17T03:26:20.279265+0000","last_became_peered":"2024-09-17T03:26:20.279265+0000","last_unstale":"2024-09-17T03:26:20.286863+0000","last_undegraded":"2024-09-17T03:26:20.286863+0000","last_fullsized":"2024-09-17T03:26:20.286863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.512156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293855+0000","last_change":"2024-09-17T03:26:21.290929+0000","last_active":"2024-09-17T03:26:21.293855+0000","last_peered":"2024-09-17T03:26:21.293855+0000","last_clean":"2024-09-17T03:26:21.293855+0000","last_became_active":"2024-09-17T03:26:21.290598+0000","last_became_peered":"2024-09-17T03:26:21.290598+0000","last_unstale":"2024-09-17T03:26:21.293855+0000","last_undegraded":"2024-09-17T03:26:21.293855+0000","last_fullsized":"2024-09-17T03:26:21.293855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:48.278348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277591+0000","last_change":"2024-09-17T03:26:20.275141+0000","last_active":"2024-09-17T03:26:20.277591+0000","last_peered":"2024-09-17T03:26:20.277591+0000","last_clean":"2024-09-17T03:26:20.277591+0000","last_became_active":"2024-09-17T03:26:20.274824+0000","last_became_peered":"2024-09-17T03:26:20.274824+0000","last_unstale":"2024-09-17T03:26:20.277591+0000","last_undegraded":"2024-09-17T03:26:20.277591+0000","last_fullsized":"2024-09-17T03:26:20.277591+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:00.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289251+0000","last_change":"2024-09-17T03:26:21.285419+0000","last_active":"2024-09-17T03:26:21.289251+0000","last_peered":"2024-09-17T03:26:21.289251+0000","last_clean":"2024-09-17T03:26:21.289251+0000","last_became_active":"2024-09-17T03:26:21.284707+0000","last_became_peered":"2024-09-17T03:26:21.284707+0000","last_unstale":"2024-09-17T03:26:21.289251+0000","last_undegraded":"2024-09-17T03:26:21.289251+0000","last_fullsized":"2024-09-17T03:26:21.289251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:07:50.249027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264570+0000","last_change":"2024-09-17T03:26:20.288747+0000","last_active":"2024-09-17T03:26:21.264570+0000","last_peered":"2024-09-17T03:26:21.264570+0000","last_clean":"2024-09-17T03:26:21.264570+0000","last_became_active":"2024-09-17T03:26:20.288347+0000","last_became_peered":"2024-09-17T03:26:20.288347+0000","last_unstale":"2024-09-17T03:26:21.264570+0000","last_undegraded":"2024-09-17T03:26:21.264570+0000","last_fullsized":"2024-09-17T03:26:21.264570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:46.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.261173+0000","last_change":"2024-09-17T03:26:20.261173+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.261173+0000","last_undegraded":"2024-09-17T03:26:20.261173+0000","last_fullsized":"2024-09-17T03:26:20.261173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.287028+0000","last_change":"2024-09-17T03:26:20.286225+0000","last_active":"2024-09-17T03:26:20.287028+0000","last_peered":"2024-09-17T03:26:20.287028+0000","last_clean":"2024-09-17T03:26:20.287028+0000","last_became_active":"2024-09-17T03:26:20.285812+0000","last_became_peered":"2024-09-17T03:26:20.285812+0000","last_unstale":"2024-09-17T03:26:20.287028+0000","last_undegraded":"2024-09-17T03:26:20.287028+0000","last_fullsized":"2024-09-17T03:26:20.287028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:14.007459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286857+0000","last_change":"2024-09-17T03:26:21.280085+0000","last_active":"2024-09-17T03:26:21.286857+0000","last_peered":"2024-09-17T03:26:21.286857+0000","last_clean":"2024-09-17T03:26:21.286857+0000","last_became_active":"2024-09-17T03:26:21.279401+0000","last_became_peered":"2024-09-17T03:26:21.279401+0000","last_unstale":"2024-09-17T03:26:21.286857+0000","last_undegraded":"2024-09-17T03:26:21.286857+0000","last_fullsized":"2024-09-17T03:26:21.286857+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.738915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.274511+0000","last_change":"2024-09-17T03:26:20.271261+0000","last_active":"2024-09-17T03:26:21.274511+0000","last_peered":"2024-09-17T03:26:21.274511+0000","last_clean":"2024-09-17T03:26:21.274511+0000","last_became_active":"2024-09-17T03:26:20.270867+0000","last_became_peered":"2024-09-17T03:26:20.270867+0000","last_unstale":"2024-09-17T03:26:21.274511+0000","last_undegraded":"2024-09-17T03:26:21.274511+0000","last_fullsized":"2024-09-17T03:26:21.274511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:19.783352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295189+0000","last_change":"2024-09-17T03:26:21.287202+0000","last_active":"2024-09-17T03:26:21.295189+0000","last_peered":"2024-09-17T03:26:21.295189+0000","last_clean":"2024-09-17T03:26:21.295189+0000","last_became_active":"2024-09-17T03:26:21.286886+0000","last_became_peered":"2024-09-17T03:26:21.286886+0000","last_unstale":"2024-09-17T03:26:21.295189+0000","last_undegraded":"2024-09-17T03:26:21.295189+0000","last_fullsized":"2024-09-17T03:26:21.295189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:51.956495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267707+0000","last_change":"2024-09-17T03:26:20.272318+0000","last_active":"2024-09-17T03:26:21.267707+0000","last_peered":"2024-09-17T03:26:21.267707+0000","last_clean":"2024-09-17T03:26:21.267707+0000","last_became_active":"2024-09-17T03:26:20.270871+0000","last_became_peered":"2024-09-17T03:26:20.270871+0000","last_unstale":"2024-09-17T03:26:21.267707+0000","last_undegraded":"2024-09-17T03:26:21.267707+0000","last_fullsized":"2024-09-17T03:26:21.267707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:56.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269003+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:26:21.269003+0000","last_peered":"2024-09-17T03:26:21.269003+0000","last_clean":"2024-09-17T03:26:21.269003+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:26:21.269003+0000","last_undegraded":"2024-09-17T03:26:21.269003+0000","last_fullsized":"2024-09-17T03:26:21.269003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58804,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313036,"statfs":{"total":96636764160,"available":96576548864,"internally_reserved":0,"allocated":1277952,"data_stored":870018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509508,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45740,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326100,"statfs":{"total":96636764160,"available":96589926400,"internally_reserved":0,"allocated":675840,"data_stored":279490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45748,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326092,"statfs":{"total":96636764160,"available":96589918208,"internally_reserved":0,"allocated":688128,"data_stored":281102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49908,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321932,"statfs":{"total":96636764160,"available":96585658368,"internally_reserved":0,"allocated":684032,"data_stored":279546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45728,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326112,"statfs":{"total":96636764160,"available":96589938688,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50468,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321372,"statfs":{"total":96636764160,"available":96585084928,"internally_reserved":0,"allocated":1257472,"data_stored":863083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:26:25.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 -- 172.21.15.46:0/1168740470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60074950 msgr2=0x7fad600b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 --2- 172.21.15.46:0/1168740470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60074950 0x7fad600b5260 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fad60073cf0 tx=0x7fad5801ccf0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 -- 172.21.15.46:0/1168740470 shutdown_connections 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 --2- 172.21.15.46:0/1168740470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60074950 0x7fad600b5260 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 --2- 172.21.15.46:0/1168740470 >> v2:172.21.15.99:3300/0 conn(0x7fad601536f0 0x7fad6015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 -- 172.21.15.46:0/1168740470 >> 172.21.15.46:0/1168740470 conn(0x7fad600b4560 msgr2=0x7fad600d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 -- 172.21.15.46:0/1168740470 shutdown_connections 2024-09-17T03:26:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.083+0000 7fad67000640 1 -- 172.21.15.46:0/1168740470 wait complete. 2024-09-17T03:26:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 Processor -- start 2024-09-17T03:26:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 -- start start 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad601536f0 0x7fad6014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 0x7fad6015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad6014ba20 0x7fad6015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad60169e10 con 0x7fad60160060 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fad60169c90 con 0x7fad601536f0 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fad60169f90 con 0x7fad6014ba20 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 0x7fad6015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 0x7fad6015b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50004/0 (socket says 172.21.15.46:50004) 2024-09-17T03:26:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 -- 172.21.15.46:0/1901626667 learned_addr learned my addr 172.21.15.46:0/1901626667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65c00640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad601536f0 0x7fad6014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad66600640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad6014ba20 0x7fad6015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 -- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad6014ba20 msgr2=0x7fad6015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad6014ba20 0x7fad6015b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 -- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad601536f0 msgr2=0x7fad6014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad601536f0 0x7fad6014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 -- 172.21.15.46:0/1901626667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6014b810 con 0x7fad60160060 2024-09-17T03:26:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65c00640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad601536f0 0x7fad6014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad65200640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 0x7fad6015b4b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00c3c0 tx=0x7fad4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c017070 con 0x7fad60160060 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad4c0053c0 con 0x7fad60160060 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad601fbea0 con 0x7fad60160060 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c003860 con 0x7fad60160060 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.087+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad601fc460 con 0x7fad60160060 2024-09-17T03:26:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.091+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad600b5260 con 0x7fad60160060 2024-09-17T03:26:26.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.091+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fad4c01a070 con 0x7fad60160060 2024-09-17T03:26:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.091+0000 7fad5d600640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad40073b00 0x7fad40076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.091+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fad4c091ce0 con 0x7fad60160060 2024-09-17T03:26:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.091+0000 7fad65c00640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad40073b00 0x7fad40076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.091+0000 7fad65c00640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad40073b00 0x7fad40076010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad5400bbd0 tx=0x7fad5400b5b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.095+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad6015d9c0 con 0x7fad60160060 2024-09-17T03:26:26.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.267+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad6006b540 con 0x7fad40073b00 2024-09-17T03:26:26.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.271+0000 7fad5d600640 1 -- 172.21.15.46:0/1901626667 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250791 (secure 0 0 0) 0x7fad5400b7c0 con 0x7fad40073b00 2024-09-17T03:26:26.276 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:26:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad40073b00 msgr2=0x7fad40076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad40073b00 0x7fad40076010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad5400bbd0 tx=0x7fad5400b5b0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 msgr2=0x7fad6015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 0x7fad6015b4b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00c3c0 tx=0x7fad4c00c880 comp rx=0 tx=0).stop 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 shutdown_connections 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad40073b00 0x7fad40076010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad6014ba20 0x7fad6015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad60160060 0x7fad6015b4b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 --2- 172.21.15.46:0/1901626667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad601536f0 0x7fad6014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 >> 172.21.15.46:0/1901626667 conn(0x7fad600b4560 msgr2=0x7fad60072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 shutdown_connections 2024-09-17T03:26:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.275+0000 7fad67000640 1 -- 172.21.15.46:0/1901626667 wait complete. 2024-09-17T03:26:26.299 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-17T03:26:25.303316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386440,"kb_used_data":6420,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754588280,"statfs":{"total":773094113280,"available":772698398720,"internally_reserved":0,"allocated":6574080,"data_stored":3391384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001164"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.290122+0000","last_change":"2024-09-17T03:26:21.271619+0000","last_active":"2024-09-17T03:26:21.290122+0000","last_peered":"2024-09-17T03:26:21.290122+0000","last_clean":"2024-09-17T03:26:21.290122+0000","last_became_active":"2024-09-17T03:26:21.270472+0000","last_became_peered":"2024-09-17T03:26:21.270472+0000","last_unstale":"2024-09-17T03:26:21.290122+0000","last_undegraded":"2024-09-17T03:26:21.290122+0000","last_fullsized":"2024-09-17T03:26:21.290122+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:58.768092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293649+0000","last_change":"2024-09-17T03:26:21.281731+0000","last_active":"2024-09-17T03:26:21.293649+0000","last_peered":"2024-09-17T03:26:21.293649+0000","last_clean":"2024-09-17T03:26:21.293649+0000","last_became_active":"2024-09-17T03:26:21.281402+0000","last_became_peered":"2024-09-17T03:26:21.281402+0000","last_unstale":"2024-09-17T03:26:21.293649+0000","last_undegraded":"2024-09-17T03:26:21.293649+0000","last_fullsized":"2024-09-17T03:26:21.293649+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:18.996659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:21.283316+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:21.282999+0000","last_became_peered":"2024-09-17T03:26:21.282999+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:15.689667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269895+0000","last_change":"2024-09-17T03:26:20.269895+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269895+0000","last_undegraded":"2024-09-17T03:26:20.269895+0000","last_fullsized":"2024-09-17T03:26:20.269895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289760+0000","last_change":"2024-09-17T03:26:21.284187+0000","last_active":"2024-09-17T03:26:21.289760+0000","last_peered":"2024-09-17T03:26:21.289760+0000","last_clean":"2024-09-17T03:26:21.289760+0000","last_became_active":"2024-09-17T03:26:21.283779+0000","last_became_peered":"2024-09-17T03:26:21.283779+0000","last_unstale":"2024-09-17T03:26:21.289760+0000","last_undegraded":"2024-09-17T03:26:21.289760+0000","last_fullsized":"2024-09-17T03:26:21.289760+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:57.934343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300045+0000","last_change":"2024-09-17T03:26:21.296425+0000","last_active":"2024-09-17T03:26:21.300045+0000","last_peered":"2024-09-17T03:26:21.300045+0000","last_clean":"2024-09-17T03:26:21.300045+0000","last_became_active":"2024-09-17T03:26:21.296063+0000","last_became_peered":"2024-09-17T03:26:21.296063+0000","last_unstale":"2024-09-17T03:26:21.300045+0000","last_undegraded":"2024-09-17T03:26:21.300045+0000","last_fullsized":"2024-09-17T03:26:21.300045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:27.881549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284101+0000","last_change":"2024-09-17T03:26:21.281715+0000","last_active":"2024-09-17T03:26:21.284101+0000","last_peered":"2024-09-17T03:26:21.284101+0000","last_clean":"2024-09-17T03:26:21.284101+0000","last_became_active":"2024-09-17T03:26:21.280618+0000","last_became_peered":"2024-09-17T03:26:21.280618+0000","last_unstale":"2024-09-17T03:26:21.284101+0000","last_undegraded":"2024-09-17T03:26:21.284101+0000","last_fullsized":"2024-09-17T03:26:21.284101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:44.647319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285297+0000","last_change":"2024-09-17T03:26:21.277710+0000","last_active":"2024-09-17T03:26:21.285297+0000","last_peered":"2024-09-17T03:26:21.285297+0000","last_clean":"2024-09-17T03:26:21.285297+0000","last_became_active":"2024-09-17T03:26:21.277333+0000","last_became_peered":"2024-09-17T03:26:21.277333+0000","last_unstale":"2024-09-17T03:26:21.285297+0000","last_undegraded":"2024-09-17T03:26:21.285297+0000","last_fullsized":"2024-09-17T03:26:21.285297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:34.691840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291938+0000","last_change":"2024-09-17T03:26:21.290497+0000","last_active":"2024-09-17T03:26:21.291938+0000","last_peered":"2024-09-17T03:26:21.291938+0000","last_clean":"2024-09-17T03:26:21.291938+0000","last_became_active":"2024-09-17T03:26:21.289289+0000","last_became_peered":"2024-09-17T03:26:21.289289+0000","last_unstale":"2024-09-17T03:26:21.291938+0000","last_undegraded":"2024-09-17T03:26:21.291938+0000","last_fullsized":"2024-09-17T03:26:21.291938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:01.560195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263685+0000","last_change":"2024-09-17T03:26:20.263685+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263685+0000","last_undegraded":"2024-09-17T03:26:20.263685+0000","last_fullsized":"2024-09-17T03:26:20.263685+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262752+0000","last_change":"2024-09-17T03:26:20.262752+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262752+0000","last_undegraded":"2024-09-17T03:26:20.262752+0000","last_fullsized":"2024-09-17T03:26:20.262752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269255+0000","last_change":"2024-09-17T03:26:20.269255+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269255+0000","last_undegraded":"2024-09-17T03:26:20.269255+0000","last_fullsized":"2024-09-17T03:26:20.269255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301774+0000","last_change":"2024-09-17T03:26:21.300221+0000","last_active":"2024-09-17T03:26:21.301774+0000","last_peered":"2024-09-17T03:26:21.301774+0000","last_clean":"2024-09-17T03:26:21.301774+0000","last_became_active":"2024-09-17T03:26:21.299864+0000","last_became_peered":"2024-09-17T03:26:21.299864+0000","last_unstale":"2024-09-17T03:26:21.301774+0000","last_undegraded":"2024-09-17T03:26:21.301774+0000","last_fullsized":"2024-09-17T03:26:21.301774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280369+0000","last_change":"2024-09-17T03:26:21.274122+0000","last_active":"2024-09-17T03:26:21.280369+0000","last_peered":"2024-09-17T03:26:21.280369+0000","last_clean":"2024-09-17T03:26:21.280369+0000","last_became_active":"2024-09-17T03:26:21.273777+0000","last_became_peered":"2024-09-17T03:26:21.273777+0000","last_unstale":"2024-09-17T03:26:21.280369+0000","last_undegraded":"2024-09-17T03:26:21.280369+0000","last_fullsized":"2024-09-17T03:26:21.280369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295641+0000","last_change":"2024-09-17T03:26:21.292845+0000","last_active":"2024-09-17T03:26:21.295641+0000","last_peered":"2024-09-17T03:26:21.295641+0000","last_clean":"2024-09-17T03:26:21.295641+0000","last_became_active":"2024-09-17T03:26:21.292432+0000","last_became_peered":"2024-09-17T03:26:21.292432+0000","last_unstale":"2024-09-17T03:26:21.295641+0000","last_undegraded":"2024-09-17T03:26:21.295641+0000","last_fullsized":"2024-09-17T03:26:21.295641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285635+0000","last_change":"2024-09-17T03:26:21.282672+0000","last_active":"2024-09-17T03:26:21.285635+0000","last_peered":"2024-09-17T03:26:21.285635+0000","last_clean":"2024-09-17T03:26:21.285635+0000","last_became_active":"2024-09-17T03:26:21.282359+0000","last_became_peered":"2024-09-17T03:26:21.282359+0000","last_unstale":"2024-09-17T03:26:21.285635+0000","last_undegraded":"2024-09-17T03:26:21.285635+0000","last_fullsized":"2024-09-17T03:26:21.285635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:11.693379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283260+0000","last_change":"2024-09-17T03:26:21.277694+0000","last_active":"2024-09-17T03:26:21.283260+0000","last_peered":"2024-09-17T03:26:21.283260+0000","last_clean":"2024-09-17T03:26:21.283260+0000","last_became_active":"2024-09-17T03:26:21.277341+0000","last_became_peered":"2024-09-17T03:26:21.277341+0000","last_unstale":"2024-09-17T03:26:21.283260+0000","last_undegraded":"2024-09-17T03:26:21.283260+0000","last_fullsized":"2024-09-17T03:26:21.283260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:16.287612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294961+0000","last_change":"2024-09-17T03:26:21.289536+0000","last_active":"2024-09-17T03:26:21.294961+0000","last_peered":"2024-09-17T03:26:21.294961+0000","last_clean":"2024-09-17T03:26:21.294961+0000","last_became_active":"2024-09-17T03:26:21.289230+0000","last_became_peered":"2024-09-17T03:26:21.289230+0000","last_unstale":"2024-09-17T03:26:21.294961+0000","last_undegraded":"2024-09-17T03:26:21.294961+0000","last_fullsized":"2024-09-17T03:26:21.294961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:31.982607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288531+0000","last_change":"2024-09-17T03:26:21.272001+0000","last_active":"2024-09-17T03:26:21.288531+0000","last_peered":"2024-09-17T03:26:21.288531+0000","last_clean":"2024-09-17T03:26:21.288531+0000","last_became_active":"2024-09-17T03:26:21.271673+0000","last_became_peered":"2024-09-17T03:26:21.271673+0000","last_unstale":"2024-09-17T03:26:21.288531+0000","last_undegraded":"2024-09-17T03:26:21.288531+0000","last_fullsized":"2024-09-17T03:26:21.288531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:48.995086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295028+0000","last_change":"2024-09-17T03:26:21.291765+0000","last_active":"2024-09-17T03:26:21.295028+0000","last_peered":"2024-09-17T03:26:21.295028+0000","last_clean":"2024-09-17T03:26:21.295028+0000","last_became_active":"2024-09-17T03:26:21.291430+0000","last_became_peered":"2024-09-17T03:26:21.291430+0000","last_unstale":"2024-09-17T03:26:21.295028+0000","last_undegraded":"2024-09-17T03:26:21.295028+0000","last_fullsized":"2024-09-17T03:26:21.295028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.144097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289278+0000","last_change":"2024-09-17T03:26:21.285773+0000","last_active":"2024-09-17T03:26:21.289278+0000","last_peered":"2024-09-17T03:26:21.289278+0000","last_clean":"2024-09-17T03:26:21.289278+0000","last_became_active":"2024-09-17T03:26:21.285118+0000","last_became_peered":"2024-09-17T03:26:21.285118+0000","last_unstale":"2024-09-17T03:26:21.289278+0000","last_undegraded":"2024-09-17T03:26:21.289278+0000","last_fullsized":"2024-09-17T03:26:21.289278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:50.670027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283248+0000","last_change":"2024-09-17T03:26:21.276516+0000","last_active":"2024-09-17T03:26:21.283248+0000","last_peered":"2024-09-17T03:26:21.283248+0000","last_clean":"2024-09-17T03:26:21.283248+0000","last_became_active":"2024-09-17T03:26:21.275398+0000","last_became_peered":"2024-09-17T03:26:21.275398+0000","last_unstale":"2024-09-17T03:26:21.283248+0000","last_undegraded":"2024-09-17T03:26:21.283248+0000","last_fullsized":"2024-09-17T03:26:21.283248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:22.952091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291438+0000","last_change":"2024-09-17T03:26:21.286047+0000","last_active":"2024-09-17T03:26:21.291438+0000","last_peered":"2024-09-17T03:26:21.291438+0000","last_clean":"2024-09-17T03:26:21.291438+0000","last_became_active":"2024-09-17T03:26:21.284398+0000","last_became_peered":"2024-09-17T03:26:21.284398+0000","last_unstale":"2024-09-17T03:26:21.291438+0000","last_undegraded":"2024-09-17T03:26:21.291438+0000","last_fullsized":"2024-09-17T03:26:21.291438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:38.105159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293886+0000","last_change":"2024-09-17T03:26:21.289586+0000","last_active":"2024-09-17T03:26:21.293886+0000","last_peered":"2024-09-17T03:26:21.293886+0000","last_clean":"2024-09-17T03:26:21.293886+0000","last_became_active":"2024-09-17T03:26:21.289242+0000","last_became_peered":"2024-09-17T03:26:21.289242+0000","last_unstale":"2024-09-17T03:26:21.293886+0000","last_undegraded":"2024-09-17T03:26:21.293886+0000","last_fullsized":"2024-09-17T03:26:21.293886+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:47.275691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293738+0000","last_change":"2024-09-17T03:26:21.287754+0000","last_active":"2024-09-17T03:26:21.293738+0000","last_peered":"2024-09-17T03:26:21.293738+0000","last_clean":"2024-09-17T03:26:21.293738+0000","last_became_active":"2024-09-17T03:26:21.287382+0000","last_became_peered":"2024-09-17T03:26:21.287382+0000","last_unstale":"2024-09-17T03:26:21.293738+0000","last_undegraded":"2024-09-17T03:26:21.293738+0000","last_fullsized":"2024-09-17T03:26:21.293738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:03.289618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275232+0000","last_change":"2024-09-17T03:26:20.275232+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275232+0000","last_undegraded":"2024-09-17T03:26:20.275232+0000","last_fullsized":"2024-09-17T03:26:20.275232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284955+0000","last_change":"2024-09-17T03:26:21.282975+0000","last_active":"2024-09-17T03:26:21.284955+0000","last_peered":"2024-09-17T03:26:21.284955+0000","last_clean":"2024-09-17T03:26:21.284955+0000","last_became_active":"2024-09-17T03:26:21.282116+0000","last_became_peered":"2024-09-17T03:26:21.282116+0000","last_unstale":"2024-09-17T03:26:21.284955+0000","last_undegraded":"2024-09-17T03:26:21.284955+0000","last_fullsized":"2024-09-17T03:26:21.284955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:08.367097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293266+0000","last_change":"2024-09-17T03:26:21.287660+0000","last_active":"2024-09-17T03:26:21.293266+0000","last_peered":"2024-09-17T03:26:21.293266+0000","last_clean":"2024-09-17T03:26:21.293266+0000","last_became_active":"2024-09-17T03:26:21.287329+0000","last_became_peered":"2024-09-17T03:26:21.287329+0000","last_unstale":"2024-09-17T03:26:21.293266+0000","last_undegraded":"2024-09-17T03:26:21.293266+0000","last_fullsized":"2024-09-17T03:26:21.293266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:07.684829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293795+0000","last_change":"2024-09-17T03:26:21.281015+0000","last_active":"2024-09-17T03:26:21.293795+0000","last_peered":"2024-09-17T03:26:21.293795+0000","last_clean":"2024-09-17T03:26:21.293795+0000","last_became_active":"2024-09-17T03:26:21.280690+0000","last_became_peered":"2024-09-17T03:26:21.280690+0000","last_unstale":"2024-09-17T03:26:21.293795+0000","last_undegraded":"2024-09-17T03:26:21.293795+0000","last_fullsized":"2024-09-17T03:26:21.293795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:27.474853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288915+0000","last_change":"2024-09-17T03:26:21.282370+0000","last_active":"2024-09-17T03:26:21.288915+0000","last_peered":"2024-09-17T03:26:21.288915+0000","last_clean":"2024-09-17T03:26:21.288915+0000","last_became_active":"2024-09-17T03:26:21.282053+0000","last_became_peered":"2024-09-17T03:26:21.282053+0000","last_unstale":"2024-09-17T03:26:21.288915+0000","last_undegraded":"2024-09-17T03:26:21.288915+0000","last_fullsized":"2024-09-17T03:26:21.288915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:27.209668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295471+0000","last_change":"2024-09-17T03:26:21.293642+0000","last_active":"2024-09-17T03:26:21.295471+0000","last_peered":"2024-09-17T03:26:21.295471+0000","last_clean":"2024-09-17T03:26:21.295471+0000","last_became_active":"2024-09-17T03:26:21.289857+0000","last_became_peered":"2024-09-17T03:26:21.289857+0000","last_unstale":"2024-09-17T03:26:21.295471+0000","last_undegraded":"2024-09-17T03:26:21.295471+0000","last_fullsized":"2024-09-17T03:26:21.295471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:56.082574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281534+0000","last_change":"2024-09-17T03:26:21.278899+0000","last_active":"2024-09-17T03:26:21.281534+0000","last_peered":"2024-09-17T03:26:21.281534+0000","last_clean":"2024-09-17T03:26:21.281534+0000","last_became_active":"2024-09-17T03:26:21.278562+0000","last_became_peered":"2024-09-17T03:26:21.278562+0000","last_unstale":"2024-09-17T03:26:21.281534+0000","last_undegraded":"2024-09-17T03:26:21.281534+0000","last_fullsized":"2024-09-17T03:26:21.281534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:06.391945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281831+0000","last_change":"2024-09-17T03:26:21.279072+0000","last_active":"2024-09-17T03:26:21.281831+0000","last_peered":"2024-09-17T03:26:21.281831+0000","last_clean":"2024-09-17T03:26:21.281831+0000","last_became_active":"2024-09-17T03:26:21.278730+0000","last_became_peered":"2024-09-17T03:26:21.278730+0000","last_unstale":"2024-09-17T03:26:21.281831+0000","last_undegraded":"2024-09-17T03:26:21.281831+0000","last_fullsized":"2024-09-17T03:26:21.281831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:31.274794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300099+0000","last_change":"2024-09-17T03:26:20.283656+0000","last_active":"2024-09-17T03:26:21.300099+0000","last_peered":"2024-09-17T03:26:21.300099+0000","last_clean":"2024-09-17T03:26:21.300099+0000","last_became_active":"2024-09-17T03:26:20.283331+0000","last_became_peered":"2024-09-17T03:26:20.283331+0000","last_unstale":"2024-09-17T03:26:21.300099+0000","last_undegraded":"2024-09-17T03:26:21.300099+0000","last_fullsized":"2024-09-17T03:26:21.300099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:03.041707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263563+0000","last_change":"2024-09-17T03:26:20.263563+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263563+0000","last_undegraded":"2024-09-17T03:26:20.263563+0000","last_fullsized":"2024-09-17T03:26:20.263563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:20.266917+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:20.266370+0000","last_became_peered":"2024-09-17T03:26:20.266370+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:57.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300820+0000","last_change":"2024-09-17T03:26:21.299198+0000","last_active":"2024-09-17T03:26:21.300820+0000","last_peered":"2024-09-17T03:26:21.300820+0000","last_clean":"2024-09-17T03:26:21.300820+0000","last_became_active":"2024-09-17T03:26:21.298717+0000","last_became_peered":"2024-09-17T03:26:21.298717+0000","last_unstale":"2024-09-17T03:26:21.300820+0000","last_undegraded":"2024-09-17T03:26:21.300820+0000","last_fullsized":"2024-09-17T03:26:21.300820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:48.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301538+0000","last_change":"2024-09-17T03:26:20.286649+0000","last_active":"2024-09-17T03:26:21.301538+0000","last_peered":"2024-09-17T03:26:21.301538+0000","last_clean":"2024-09-17T03:26:21.301538+0000","last_became_active":"2024-09-17T03:26:20.285886+0000","last_became_peered":"2024-09-17T03:26:20.285886+0000","last_unstale":"2024-09-17T03:26:21.301538+0000","last_undegraded":"2024-09-17T03:26:21.301538+0000","last_fullsized":"2024-09-17T03:26:21.301538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:46.887782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300191+0000","last_change":"2024-09-17T03:26:21.289923+0000","last_active":"2024-09-17T03:26:21.300191+0000","last_peered":"2024-09-17T03:26:21.300191+0000","last_clean":"2024-09-17T03:26:21.300191+0000","last_became_active":"2024-09-17T03:26:21.289556+0000","last_became_peered":"2024-09-17T03:26:21.289556+0000","last_unstale":"2024-09-17T03:26:21.300191+0000","last_undegraded":"2024-09-17T03:26:21.300191+0000","last_fullsized":"2024-09-17T03:26:21.300191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:41.180633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286893+0000","last_change":"2024-09-17T03:26:20.283187+0000","last_active":"2024-09-17T03:26:20.286893+0000","last_peered":"2024-09-17T03:26:20.286893+0000","last_clean":"2024-09-17T03:26:20.286893+0000","last_became_active":"2024-09-17T03:26:20.282354+0000","last_became_peered":"2024-09-17T03:26:20.282354+0000","last_unstale":"2024-09-17T03:26:20.286893+0000","last_undegraded":"2024-09-17T03:26:20.286893+0000","last_fullsized":"2024-09-17T03:26:20.286893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:57.123281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293825+0000","last_change":"2024-09-17T03:26:21.287275+0000","last_active":"2024-09-17T03:26:21.293825+0000","last_peered":"2024-09-17T03:26:21.293825+0000","last_clean":"2024-09-17T03:26:21.293825+0000","last_became_active":"2024-09-17T03:26:21.286995+0000","last_became_peered":"2024-09-17T03:26:21.286995+0000","last_unstale":"2024-09-17T03:26:21.293825+0000","last_undegraded":"2024-09-17T03:26:21.293825+0000","last_fullsized":"2024-09-17T03:26:21.293825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:58.245607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.298865+0000","last_change":"2024-09-17T03:26:20.283433+0000","last_active":"2024-09-17T03:26:21.298865+0000","last_peered":"2024-09-17T03:26:21.298865+0000","last_clean":"2024-09-17T03:26:21.298865+0000","last_became_active":"2024-09-17T03:26:20.277797+0000","last_became_peered":"2024-09-17T03:26:20.277797+0000","last_unstale":"2024-09-17T03:26:21.298865+0000","last_undegraded":"2024-09-17T03:26:21.298865+0000","last_fullsized":"2024-09-17T03:26:21.298865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294978+0000","last_change":"2024-09-17T03:26:21.293180+0000","last_active":"2024-09-17T03:26:21.294978+0000","last_peered":"2024-09-17T03:26:21.294978+0000","last_clean":"2024-09-17T03:26:21.294978+0000","last_became_active":"2024-09-17T03:26:21.292882+0000","last_became_peered":"2024-09-17T03:26:21.292882+0000","last_unstale":"2024-09-17T03:26:21.294978+0000","last_undegraded":"2024-09-17T03:26:21.294978+0000","last_fullsized":"2024-09-17T03:26:21.294978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:16.723527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301141+0000","last_change":"2024-09-17T03:26:20.279365+0000","last_active":"2024-09-17T03:26:21.301141+0000","last_peered":"2024-09-17T03:26:21.301141+0000","last_clean":"2024-09-17T03:26:21.301141+0000","last_became_active":"2024-09-17T03:26:20.278923+0000","last_became_peered":"2024-09-17T03:26:20.278923+0000","last_unstale":"2024-09-17T03:26:21.301141+0000","last_undegraded":"2024-09-17T03:26:21.301141+0000","last_fullsized":"2024-09-17T03:26:21.301141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:14.310020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286603+0000","last_change":"2024-09-17T03:26:21.285430+0000","last_active":"2024-09-17T03:26:21.286603+0000","last_peered":"2024-09-17T03:26:21.286603+0000","last_clean":"2024-09-17T03:26:21.286603+0000","last_became_active":"2024-09-17T03:26:21.285111+0000","last_became_peered":"2024-09-17T03:26:21.285111+0000","last_unstale":"2024-09-17T03:26:21.286603+0000","last_undegraded":"2024-09-17T03:26:21.286603+0000","last_fullsized":"2024-09-17T03:26:21.286603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:47.441700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284199+0000","last_change":"2024-09-17T03:26:20.279560+0000","last_active":"2024-09-17T03:26:20.284199+0000","last_peered":"2024-09-17T03:26:20.284199+0000","last_clean":"2024-09-17T03:26:20.284199+0000","last_became_active":"2024-09-17T03:26:20.278746+0000","last_became_peered":"2024-09-17T03:26:20.278746+0000","last_unstale":"2024-09-17T03:26:20.284199+0000","last_undegraded":"2024-09-17T03:26:20.284199+0000","last_fullsized":"2024-09-17T03:26:20.284199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:10.047880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286823+0000","last_change":"2024-09-17T03:26:21.279707+0000","last_active":"2024-09-17T03:26:21.286823+0000","last_peered":"2024-09-17T03:26:21.286823+0000","last_clean":"2024-09-17T03:26:21.286823+0000","last_became_active":"2024-09-17T03:26:21.279020+0000","last_became_peered":"2024-09-17T03:26:21.279020+0000","last_unstale":"2024-09-17T03:26:21.286823+0000","last_undegraded":"2024-09-17T03:26:21.286823+0000","last_fullsized":"2024-09-17T03:26:21.286823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:48.268297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264232+0000","last_change":"2024-09-17T03:26:20.283353+0000","last_active":"2024-09-17T03:26:21.264232+0000","last_peered":"2024-09-17T03:26:21.264232+0000","last_clean":"2024-09-17T03:26:21.264232+0000","last_became_active":"2024-09-17T03:26:20.283049+0000","last_became_peered":"2024-09-17T03:26:20.283049+0000","last_unstale":"2024-09-17T03:26:21.264232+0000","last_undegraded":"2024-09-17T03:26:21.264232+0000","last_fullsized":"2024-09-17T03:26:21.264232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291964+0000","last_change":"2024-09-17T03:26:21.290216+0000","last_active":"2024-09-17T03:26:21.291964+0000","last_peered":"2024-09-17T03:26:21.291964+0000","last_clean":"2024-09-17T03:26:21.291964+0000","last_became_active":"2024-09-17T03:26:21.288236+0000","last_became_peered":"2024-09-17T03:26:21.288236+0000","last_unstale":"2024-09-17T03:26:21.291964+0000","last_undegraded":"2024-09-17T03:26:21.291964+0000","last_fullsized":"2024-09-17T03:26:21.291964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:34.999880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269902+0000","last_change":"2024-09-17T03:26:20.289405+0000","last_active":"2024-09-17T03:26:21.269902+0000","last_peered":"2024-09-17T03:26:21.269902+0000","last_clean":"2024-09-17T03:26:21.269902+0000","last_became_active":"2024-09-17T03:26:20.288979+0000","last_became_peered":"2024-09-17T03:26:20.288979+0000","last_unstale":"2024-09-17T03:26:21.269902+0000","last_undegraded":"2024-09-17T03:26:21.269902+0000","last_fullsized":"2024-09-17T03:26:21.269902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:10.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.303142+0000","last_change":"2024-09-17T03:26:21.301608+0000","last_active":"2024-09-17T03:26:21.303142+0000","last_peered":"2024-09-17T03:26:21.303142+0000","last_clean":"2024-09-17T03:26:21.303142+0000","last_became_active":"2024-09-17T03:26:21.301203+0000","last_became_peered":"2024-09-17T03:26:21.301203+0000","last_unstale":"2024-09-17T03:26:21.303142+0000","last_undegraded":"2024-09-17T03:26:21.303142+0000","last_fullsized":"2024-09-17T03:26:21.303142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:23.395294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.271410+0000","last_change":"2024-09-17T03:26:20.263118+0000","last_active":"2024-09-17T03:26:21.271410+0000","last_peered":"2024-09-17T03:26:21.271410+0000","last_clean":"2024-09-17T03:26:21.271410+0000","last_became_active":"2024-09-17T03:26:20.262773+0000","last_became_peered":"2024-09-17T03:26:20.262773+0000","last_unstale":"2024-09-17T03:26:21.271410+0000","last_undegraded":"2024-09-17T03:26:21.271410+0000","last_fullsized":"2024-09-17T03:26:21.271410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:02.639519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263081+0000","last_change":"2024-09-17T03:26:20.263081+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263081+0000","last_undegraded":"2024-09-17T03:26:20.263081+0000","last_fullsized":"2024-09-17T03:26:20.263081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.306381+0000","last_change":"2024-09-17T03:26:20.277795+0000","last_active":"2024-09-17T03:26:21.306381+0000","last_peered":"2024-09-17T03:26:21.306381+0000","last_clean":"2024-09-17T03:26:21.306381+0000","last_became_active":"2024-09-17T03:26:20.277456+0000","last_became_peered":"2024-09-17T03:26:20.277456+0000","last_unstale":"2024-09-17T03:26:21.306381+0000","last_undegraded":"2024-09-17T03:26:21.306381+0000","last_fullsized":"2024-09-17T03:26:21.306381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:43.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297424+0000","last_change":"2024-09-17T03:26:21.292382+0000","last_active":"2024-09-17T03:26:21.297424+0000","last_peered":"2024-09-17T03:26:21.297424+0000","last_clean":"2024-09-17T03:26:21.297424+0000","last_became_active":"2024-09-17T03:26:21.292043+0000","last_became_peered":"2024-09-17T03:26:21.292043+0000","last_unstale":"2024-09-17T03:26:21.297424+0000","last_undegraded":"2024-09-17T03:26:21.297424+0000","last_fullsized":"2024-09-17T03:26:21.297424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:02.648204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.308666+0000","last_change":"2024-09-17T03:26:20.289201+0000","last_active":"2024-09-17T03:26:21.308666+0000","last_peered":"2024-09-17T03:26:21.308666+0000","last_clean":"2024-09-17T03:26:21.308666+0000","last_became_active":"2024-09-17T03:26:20.288427+0000","last_became_peered":"2024-09-17T03:26:20.288427+0000","last_unstale":"2024-09-17T03:26:21.308666+0000","last_undegraded":"2024-09-17T03:26:21.308666+0000","last_fullsized":"2024-09-17T03:26:21.308666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:46.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275142+0000","last_change":"2024-09-17T03:26:20.275142+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275142+0000","last_undegraded":"2024-09-17T03:26:20.275142+0000","last_fullsized":"2024-09-17T03:26:20.275142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.292629+0000","last_change":"2024-09-17T03:26:20.265758+0000","last_active":"2024-09-17T03:26:21.292629+0000","last_peered":"2024-09-17T03:26:21.292629+0000","last_clean":"2024-09-17T03:26:21.292629+0000","last_became_active":"2024-09-17T03:26:20.262782+0000","last_became_peered":"2024-09-17T03:26:20.262782+0000","last_unstale":"2024-09-17T03:26:21.292629+0000","last_undegraded":"2024-09-17T03:26:21.292629+0000","last_fullsized":"2024-09-17T03:26:21.292629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:10.171064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283171+0000","last_change":"2024-09-17T03:26:21.278170+0000","last_active":"2024-09-17T03:26:21.283171+0000","last_peered":"2024-09-17T03:26:21.283171+0000","last_clean":"2024-09-17T03:26:21.283171+0000","last_became_active":"2024-09-17T03:26:21.277329+0000","last_became_peered":"2024-09-17T03:26:21.277329+0000","last_unstale":"2024-09-17T03:26:21.283171+0000","last_undegraded":"2024-09-17T03:26:21.283171+0000","last_fullsized":"2024-09-17T03:26:21.283171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:32.373126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267858+0000","last_change":"2024-09-17T03:26:20.269475+0000","last_active":"2024-09-17T03:26:21.267858+0000","last_peered":"2024-09-17T03:26:21.267858+0000","last_clean":"2024-09-17T03:26:21.267858+0000","last_became_active":"2024-09-17T03:26:20.269101+0000","last_became_peered":"2024-09-17T03:26:20.269101+0000","last_unstale":"2024-09-17T03:26:21.267858+0000","last_undegraded":"2024-09-17T03:26:21.267858+0000","last_fullsized":"2024-09-17T03:26:21.267858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283965+0000","last_change":"2024-09-17T03:26:21.281823+0000","last_active":"2024-09-17T03:26:21.283965+0000","last_peered":"2024-09-17T03:26:21.283965+0000","last_clean":"2024-09-17T03:26:21.283965+0000","last_became_active":"2024-09-17T03:26:21.280688+0000","last_became_peered":"2024-09-17T03:26:21.280688+0000","last_unstale":"2024-09-17T03:26:21.283965+0000","last_undegraded":"2024-09-17T03:26:21.283965+0000","last_fullsized":"2024-09-17T03:26:21.283965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:36.912044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284389+0000","last_change":"2024-09-17T03:26:20.281345+0000","last_active":"2024-09-17T03:26:20.284389+0000","last_peered":"2024-09-17T03:26:20.284389+0000","last_clean":"2024-09-17T03:26:20.284389+0000","last_became_active":"2024-09-17T03:26:20.280261+0000","last_became_peered":"2024-09-17T03:26:20.280261+0000","last_unstale":"2024-09-17T03:26:20.284389+0000","last_undegraded":"2024-09-17T03:26:20.284389+0000","last_fullsized":"2024-09-17T03:26:20.284389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:47.948863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294667+0000","last_change":"2024-09-17T03:26:21.289462+0000","last_active":"2024-09-17T03:26:21.294667+0000","last_peered":"2024-09-17T03:26:21.294667+0000","last_clean":"2024-09-17T03:26:21.294667+0000","last_became_active":"2024-09-17T03:26:21.285434+0000","last_became_peered":"2024-09-17T03:26:21.285434+0000","last_unstale":"2024-09-17T03:26:21.294667+0000","last_undegraded":"2024-09-17T03:26:21.294667+0000","last_fullsized":"2024-09-17T03:26:21.294667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:24.253438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301604+0000","last_change":"2024-09-17T03:26:20.275945+0000","last_active":"2024-09-17T03:26:21.301604+0000","last_peered":"2024-09-17T03:26:21.301604+0000","last_clean":"2024-09-17T03:26:21.301604+0000","last_became_active":"2024-09-17T03:26:20.275123+0000","last_became_peered":"2024-09-17T03:26:20.275123+0000","last_unstale":"2024-09-17T03:26:21.301604+0000","last_undegraded":"2024-09-17T03:26:21.301604+0000","last_fullsized":"2024-09-17T03:26:21.301604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:02.236108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291615+0000","last_change":"2024-09-17T03:26:21.284952+0000","last_active":"2024-09-17T03:26:21.291615+0000","last_peered":"2024-09-17T03:26:21.291615+0000","last_clean":"2024-09-17T03:26:21.291615+0000","last_became_active":"2024-09-17T03:26:21.284572+0000","last_became_peered":"2024-09-17T03:26:21.284572+0000","last_unstale":"2024-09-17T03:26:21.291615+0000","last_undegraded":"2024-09-17T03:26:21.291615+0000","last_fullsized":"2024-09-17T03:26:21.291615+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:11.920676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295381+0000","last_change":"2024-09-17T03:26:20.269212+0000","last_active":"2024-09-17T03:26:21.295381+0000","last_peered":"2024-09-17T03:26:21.295381+0000","last_clean":"2024-09-17T03:26:21.295381+0000","last_became_active":"2024-09-17T03:26:20.268883+0000","last_became_peered":"2024-09-17T03:26:20.268883+0000","last_unstale":"2024-09-17T03:26:21.295381+0000","last_undegraded":"2024-09-17T03:26:21.295381+0000","last_fullsized":"2024-09-17T03:26:21.295381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:41:56.450980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274114+0000","last_change":"2024-09-17T03:26:20.274114+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274114+0000","last_undegraded":"2024-09-17T03:26:20.274114+0000","last_fullsized":"2024-09-17T03:26:20.274114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300538+0000","last_change":"2024-09-17T03:26:20.279146+0000","last_active":"2024-09-17T03:26:21.300538+0000","last_peered":"2024-09-17T03:26:21.300538+0000","last_clean":"2024-09-17T03:26:21.300538+0000","last_became_active":"2024-09-17T03:26:20.278888+0000","last_became_peered":"2024-09-17T03:26:20.278888+0000","last_unstale":"2024-09-17T03:26:21.300538+0000","last_undegraded":"2024-09-17T03:26:21.300538+0000","last_fullsized":"2024-09-17T03:26:21.300538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:40.150476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286599+0000","last_change":"2024-09-17T03:26:21.278295+0000","last_active":"2024-09-17T03:26:21.286599+0000","last_peered":"2024-09-17T03:26:21.286599+0000","last_clean":"2024-09-17T03:26:21.286599+0000","last_became_active":"2024-09-17T03:26:21.277728+0000","last_became_peered":"2024-09-17T03:26:21.277728+0000","last_unstale":"2024-09-17T03:26:21.286599+0000","last_undegraded":"2024-09-17T03:26:21.286599+0000","last_fullsized":"2024-09-17T03:26:21.286599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:38.173310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277895+0000","last_change":"2024-09-17T03:26:20.272850+0000","last_active":"2024-09-17T03:26:20.277895+0000","last_peered":"2024-09-17T03:26:20.277895+0000","last_clean":"2024-09-17T03:26:20.277895+0000","last_became_active":"2024-09-17T03:26:20.272036+0000","last_became_peered":"2024-09-17T03:26:20.272036+0000","last_unstale":"2024-09-17T03:26:20.277895+0000","last_undegraded":"2024-09-17T03:26:20.277895+0000","last_fullsized":"2024-09-17T03:26:20.277895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:25.846783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297474+0000","last_change":"2024-09-17T03:26:21.294723+0000","last_active":"2024-09-17T03:26:21.297474+0000","last_peered":"2024-09-17T03:26:21.297474+0000","last_clean":"2024-09-17T03:26:21.297474+0000","last_became_active":"2024-09-17T03:26:21.294375+0000","last_became_peered":"2024-09-17T03:26:21.294375+0000","last_unstale":"2024-09-17T03:26:21.297474+0000","last_undegraded":"2024-09-17T03:26:21.297474+0000","last_fullsized":"2024-09-17T03:26:21.297474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:22.471193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.275453+0000","last_change":"2024-09-17T03:26:20.281457+0000","last_active":"2024-09-17T03:26:21.275453+0000","last_peered":"2024-09-17T03:26:21.275453+0000","last_clean":"2024-09-17T03:26:21.275453+0000","last_became_active":"2024-09-17T03:26:20.281078+0000","last_became_peered":"2024-09-17T03:26:20.281078+0000","last_unstale":"2024-09-17T03:26:21.275453+0000","last_undegraded":"2024-09-17T03:26:21.275453+0000","last_fullsized":"2024-09-17T03:26:21.275453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:22.720948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.272277+0000","last_change":"2024-09-17T03:26:20.272277+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.272277+0000","last_undegraded":"2024-09-17T03:26:20.272277+0000","last_fullsized":"2024-09-17T03:26:20.272277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.296826+0000","last_change":"2024-09-17T03:26:20.280006+0000","last_active":"2024-09-17T03:26:21.296826+0000","last_peered":"2024-09-17T03:26:21.296826+0000","last_clean":"2024-09-17T03:26:21.296826+0000","last_became_active":"2024-09-17T03:26:20.279611+0000","last_became_peered":"2024-09-17T03:26:20.279611+0000","last_unstale":"2024-09-17T03:26:21.296826+0000","last_undegraded":"2024-09-17T03:26:21.296826+0000","last_fullsized":"2024-09-17T03:26:21.296826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:29.773567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289195+0000","last_change":"2024-09-17T03:26:21.284722+0000","last_active":"2024-09-17T03:26:21.289195+0000","last_peered":"2024-09-17T03:26:21.289195+0000","last_clean":"2024-09-17T03:26:21.289195+0000","last_became_active":"2024-09-17T03:26:21.284399+0000","last_became_peered":"2024-09-17T03:26:21.284399+0000","last_unstale":"2024-09-17T03:26:21.289195+0000","last_undegraded":"2024-09-17T03:26:21.289195+0000","last_fullsized":"2024-09-17T03:26:21.289195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:55.320737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277267+0000","last_change":"2024-09-17T03:26:20.274933+0000","last_active":"2024-09-17T03:26:20.277267+0000","last_peered":"2024-09-17T03:26:20.277267+0000","last_clean":"2024-09-17T03:26:20.277267+0000","last_became_active":"2024-09-17T03:26:20.274536+0000","last_became_peered":"2024-09-17T03:26:20.274536+0000","last_unstale":"2024-09-17T03:26:20.277267+0000","last_undegraded":"2024-09-17T03:26:20.277267+0000","last_fullsized":"2024-09-17T03:26:20.277267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:23.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280742+0000","last_change":"2024-09-17T03:26:21.277208+0000","last_active":"2024-09-17T03:26:21.280742+0000","last_peered":"2024-09-17T03:26:21.280742+0000","last_clean":"2024-09-17T03:26:21.280742+0000","last_became_active":"2024-09-17T03:26:21.276812+0000","last_became_peered":"2024-09-17T03:26:21.276812+0000","last_unstale":"2024-09-17T03:26:21.280742+0000","last_undegraded":"2024-09-17T03:26:21.280742+0000","last_fullsized":"2024-09-17T03:26:21.280742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:12.678429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284384+0000","last_change":"2024-09-17T03:26:20.281254+0000","last_active":"2024-09-17T03:26:20.284384+0000","last_peered":"2024-09-17T03:26:20.284384+0000","last_clean":"2024-09-17T03:26:20.284384+0000","last_became_active":"2024-09-17T03:26:20.280858+0000","last_became_peered":"2024-09-17T03:26:20.280858+0000","last_unstale":"2024-09-17T03:26:20.284384+0000","last_undegraded":"2024-09-17T03:26:20.284384+0000","last_fullsized":"2024-09-17T03:26:20.284384+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:22.288031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274734+0000","last_change":"2024-09-17T03:26:20.274734+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274734+0000","last_undegraded":"2024-09-17T03:26:20.274734+0000","last_fullsized":"2024-09-17T03:26:20.274734+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277412+0000","last_change":"2024-09-17T03:26:20.272578+0000","last_active":"2024-09-17T03:26:20.277412+0000","last_peered":"2024-09-17T03:26:20.277412+0000","last_clean":"2024-09-17T03:26:20.277412+0000","last_became_active":"2024-09-17T03:26:20.272215+0000","last_became_peered":"2024-09-17T03:26:20.272215+0000","last_unstale":"2024-09-17T03:26:20.277412+0000","last_undegraded":"2024-09-17T03:26:20.277412+0000","last_fullsized":"2024-09-17T03:26:20.277412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:56.751763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.273561+0000","last_change":"2024-09-17T03:26:20.273561+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.273561+0000","last_undegraded":"2024-09-17T03:26:20.273561+0000","last_fullsized":"2024-09-17T03:26:20.273561+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.276867+0000","last_change":"2024-09-17T03:26:20.281152+0000","last_active":"2024-09-17T03:26:21.276867+0000","last_peered":"2024-09-17T03:26:21.276867+0000","last_clean":"2024-09-17T03:26:21.276867+0000","last_became_active":"2024-09-17T03:26:20.280782+0000","last_became_peered":"2024-09-17T03:26:20.280782+0000","last_unstale":"2024-09-17T03:26:21.276867+0000","last_undegraded":"2024-09-17T03:26:21.276867+0000","last_fullsized":"2024-09-17T03:26:21.276867+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:08.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262545+0000","last_change":"2024-09-17T03:26:20.262545+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262545+0000","last_undegraded":"2024-09-17T03:26:20.262545+0000","last_fullsized":"2024-09-17T03:26:20.262545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267915+0000","last_change":"2024-09-17T03:26:20.277793+0000","last_active":"2024-09-17T03:26:21.267915+0000","last_peered":"2024-09-17T03:26:21.267915+0000","last_clean":"2024-09-17T03:26:21.267915+0000","last_became_active":"2024-09-17T03:26:20.276428+0000","last_became_peered":"2024-09-17T03:26:20.276428+0000","last_unstale":"2024-09-17T03:26:21.267915+0000","last_undegraded":"2024-09-17T03:26:21.267915+0000","last_fullsized":"2024-09-17T03:26:21.267915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289751+0000","last_change":"2024-09-17T03:26:21.283319+0000","last_active":"2024-09-17T03:26:21.289751+0000","last_peered":"2024-09-17T03:26:21.289751+0000","last_clean":"2024-09-17T03:26:21.289751+0000","last_became_active":"2024-09-17T03:26:21.282574+0000","last_became_peered":"2024-09-17T03:26:21.282574+0000","last_unstale":"2024-09-17T03:26:21.289751+0000","last_undegraded":"2024-09-17T03:26:21.289751+0000","last_fullsized":"2024-09-17T03:26:21.289751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:54.060063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286863+0000","last_change":"2024-09-17T03:26:20.282834+0000","last_active":"2024-09-17T03:26:20.286863+0000","last_peered":"2024-09-17T03:26:20.286863+0000","last_clean":"2024-09-17T03:26:20.286863+0000","last_became_active":"2024-09-17T03:26:20.279265+0000","last_became_peered":"2024-09-17T03:26:20.279265+0000","last_unstale":"2024-09-17T03:26:20.286863+0000","last_undegraded":"2024-09-17T03:26:20.286863+0000","last_fullsized":"2024-09-17T03:26:20.286863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.512156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293855+0000","last_change":"2024-09-17T03:26:21.290929+0000","last_active":"2024-09-17T03:26:21.293855+0000","last_peered":"2024-09-17T03:26:21.293855+0000","last_clean":"2024-09-17T03:26:21.293855+0000","last_became_active":"2024-09-17T03:26:21.290598+0000","last_became_peered":"2024-09-17T03:26:21.290598+0000","last_unstale":"2024-09-17T03:26:21.293855+0000","last_undegraded":"2024-09-17T03:26:21.293855+0000","last_fullsized":"2024-09-17T03:26:21.293855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:48.278348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277591+0000","last_change":"2024-09-17T03:26:20.275141+0000","last_active":"2024-09-17T03:26:20.277591+0000","last_peered":"2024-09-17T03:26:20.277591+0000","last_clean":"2024-09-17T03:26:20.277591+0000","last_became_active":"2024-09-17T03:26:20.274824+0000","last_became_peered":"2024-09-17T03:26:20.274824+0000","last_unstale":"2024-09-17T03:26:20.277591+0000","last_undegraded":"2024-09-17T03:26:20.277591+0000","last_fullsized":"2024-09-17T03:26:20.277591+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:00.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289251+0000","last_change":"2024-09-17T03:26:21.285419+0000","last_active":"2024-09-17T03:26:21.289251+0000","last_peered":"2024-09-17T03:26:21.289251+0000","last_clean":"2024-09-17T03:26:21.289251+0000","last_became_active":"2024-09-17T03:26:21.284707+0000","last_became_peered":"2024-09-17T03:26:21.284707+0000","last_unstale":"2024-09-17T03:26:21.289251+0000","last_undegraded":"2024-09-17T03:26:21.289251+0000","last_fullsized":"2024-09-17T03:26:21.289251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:07:50.249027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264570+0000","last_change":"2024-09-17T03:26:20.288747+0000","last_active":"2024-09-17T03:26:21.264570+0000","last_peered":"2024-09-17T03:26:21.264570+0000","last_clean":"2024-09-17T03:26:21.264570+0000","last_became_active":"2024-09-17T03:26:20.288347+0000","last_became_peered":"2024-09-17T03:26:20.288347+0000","last_unstale":"2024-09-17T03:26:21.264570+0000","last_undegraded":"2024-09-17T03:26:21.264570+0000","last_fullsized":"2024-09-17T03:26:21.264570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:46.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.261173+0000","last_change":"2024-09-17T03:26:20.261173+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.261173+0000","last_undegraded":"2024-09-17T03:26:20.261173+0000","last_fullsized":"2024-09-17T03:26:20.261173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.287028+0000","last_change":"2024-09-17T03:26:20.286225+0000","last_active":"2024-09-17T03:26:20.287028+0000","last_peered":"2024-09-17T03:26:20.287028+0000","last_clean":"2024-09-17T03:26:20.287028+0000","last_became_active":"2024-09-17T03:26:20.285812+0000","last_became_peered":"2024-09-17T03:26:20.285812+0000","last_unstale":"2024-09-17T03:26:20.287028+0000","last_undegraded":"2024-09-17T03:26:20.287028+0000","last_fullsized":"2024-09-17T03:26:20.287028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:14.007459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286857+0000","last_change":"2024-09-17T03:26:21.280085+0000","last_active":"2024-09-17T03:26:21.286857+0000","last_peered":"2024-09-17T03:26:21.286857+0000","last_clean":"2024-09-17T03:26:21.286857+0000","last_became_active":"2024-09-17T03:26:21.279401+0000","last_became_peered":"2024-09-17T03:26:21.279401+0000","last_unstale":"2024-09-17T03:26:21.286857+0000","last_undegraded":"2024-09-17T03:26:21.286857+0000","last_fullsized":"2024-09-17T03:26:21.286857+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.738915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.274511+0000","last_change":"2024-09-17T03:26:20.271261+0000","last_active":"2024-09-17T03:26:21.274511+0000","last_peered":"2024-09-17T03:26:21.274511+0000","last_clean":"2024-09-17T03:26:21.274511+0000","last_became_active":"2024-09-17T03:26:20.270867+0000","last_became_peered":"2024-09-17T03:26:20.270867+0000","last_unstale":"2024-09-17T03:26:21.274511+0000","last_undegraded":"2024-09-17T03:26:21.274511+0000","last_fullsized":"2024-09-17T03:26:21.274511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:19.783352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295189+0000","last_change":"2024-09-17T03:26:21.287202+0000","last_active":"2024-09-17T03:26:21.295189+0000","last_peered":"2024-09-17T03:26:21.295189+0000","last_clean":"2024-09-17T03:26:21.295189+0000","last_became_active":"2024-09-17T03:26:21.286886+0000","last_became_peered":"2024-09-17T03:26:21.286886+0000","last_unstale":"2024-09-17T03:26:21.295189+0000","last_undegraded":"2024-09-17T03:26:21.295189+0000","last_fullsized":"2024-09-17T03:26:21.295189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:51.956495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267707+0000","last_change":"2024-09-17T03:26:20.272318+0000","last_active":"2024-09-17T03:26:21.267707+0000","last_peered":"2024-09-17T03:26:21.267707+0000","last_clean":"2024-09-17T03:26:21.267707+0000","last_became_active":"2024-09-17T03:26:20.270871+0000","last_became_peered":"2024-09-17T03:26:20.270871+0000","last_unstale":"2024-09-17T03:26:21.267707+0000","last_undegraded":"2024-09-17T03:26:21.267707+0000","last_fullsized":"2024-09-17T03:26:21.267707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:56.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269003+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:26:21.269003+0000","last_peered":"2024-09-17T03:26:21.269003+0000","last_clean":"2024-09-17T03:26:21.269003+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:26:21.269003+0000","last_undegraded":"2024-09-17T03:26:21.269003+0000","last_fullsized":"2024-09-17T03:26:21.269003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58804,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313036,"statfs":{"total":96636764160,"available":96576548864,"internally_reserved":0,"allocated":1277952,"data_stored":870018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509508,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45740,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326100,"statfs":{"total":96636764160,"available":96589926400,"internally_reserved":0,"allocated":675840,"data_stored":279490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45748,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326092,"statfs":{"total":96636764160,"available":96589918208,"internally_reserved":0,"allocated":688128,"data_stored":281102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49908,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321932,"statfs":{"total":96636764160,"available":96585658368,"internally_reserved":0,"allocated":684032,"data_stored":279546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45728,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326112,"statfs":{"total":96636764160,"available":96589938688,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50468,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321372,"statfs":{"total":96636764160,"available":96585084928,"internally_reserved":0,"allocated":1257472,"data_stored":863083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:26:26.303 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3c is not active+clean 2024-09-17T03:26:26.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.269895+0000', 'last_change': '2024-09-17T03:26:20.269895+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.269895+0000', 'last_undegraded': '2024-09-17T03:26:20.269895+0000', 'last_fullsized': '2024-09-17T03:26:20.269895+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.303 INFO:tasks.ceph.ceph_manager.ceph:PG 9.36 is not active+clean 2024-09-17T03:26:26.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.263685+0000', 'last_change': '2024-09-17T03:26:20.263685+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.263685+0000', 'last_undegraded': '2024-09-17T03:26:20.263685+0000', 'last_fullsized': '2024-09-17T03:26:20.263685+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.35 is not active+clean 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.262752+0000', 'last_change': '2024-09-17T03:26:20.262752+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.262752+0000', 'last_undegraded': '2024-09-17T03:26:20.262752+0000', 'last_fullsized': '2024-09-17T03:26:20.262752+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.269255+0000', 'last_change': '2024-09-17T03:26:20.269255+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.269255+0000', 'last_undegraded': '2024-09-17T03:26:20.269255+0000', 'last_fullsized': '2024-09-17T03:26:20.269255+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.26 is not active+clean 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.275232+0000', 'last_change': '2024-09-17T03:26:20.275232+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.275232+0000', 'last_undegraded': '2024-09-17T03:26:20.275232+0000', 'last_fullsized': '2024-09-17T03:26:20.275232+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.263563+0000', 'last_change': '2024-09-17T03:26:20.263563+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.263563+0000', 'last_undegraded': '2024-09-17T03:26:20.263563+0000', 'last_fullsized': '2024-09-17T03:26:20.263563+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-17T03:26:26.304 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.263081+0000', 'last_change': '2024-09-17T03:26:20.263081+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.263081+0000', 'last_undegraded': '2024-09-17T03:26:20.263081+0000', 'last_fullsized': '2024-09-17T03:26:20.263081+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.275142+0000', 'last_change': '2024-09-17T03:26:20.275142+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.275142+0000', 'last_undegraded': '2024-09-17T03:26:20.275142+0000', 'last_fullsized': '2024-09-17T03:26:20.275142+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.274114+0000', 'last_change': '2024-09-17T03:26:20.274114+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.274114+0000', 'last_undegraded': '2024-09-17T03:26:20.274114+0000', 'last_fullsized': '2024-09-17T03:26:20.274114+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.272277+0000', 'last_change': '2024-09-17T03:26:20.272277+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.272277+0000', 'last_undegraded': '2024-09-17T03:26:20.272277+0000', 'last_fullsized': '2024-09-17T03:26:20.272277+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.274734+0000', 'last_change': '2024-09-17T03:26:20.274734+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.274734+0000', 'last_undegraded': '2024-09-17T03:26:20.274734+0000', 'last_fullsized': '2024-09-17T03:26:20.274734+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-17T03:26:26.305 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.273561+0000', 'last_change': '2024-09-17T03:26:20.273561+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.273561+0000', 'last_undegraded': '2024-09-17T03:26:20.273561+0000', 'last_fullsized': '2024-09-17T03:26:20.273561+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:26:26.306 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-17T03:26:26.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.262545+0000', 'last_change': '2024-09-17T03:26:20.262545+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.262545+0000', 'last_undegraded': '2024-09-17T03:26:20.262545+0000', 'last_fullsized': '2024-09-17T03:26:20.262545+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:26:26.306 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-17T03:26:26.306 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:26:20.261173+0000', 'last_change': '2024-09-17T03:26:20.261173+0000', 'last_active': '2024-09-17T03:26:20.242858+0000', 'last_peered': '2024-09-17T03:26:20.242858+0000', 'last_clean': '2024-09-17T03:26:20.242858+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:26:20.261173+0000', 'last_undegraded': '2024-09-17T03:26:20.261173+0000', 'last_fullsized': '2024-09-17T03:26:20.261173+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'last_clean_scrub_stamp': '2024-09-17T03:26:20.242858+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:26:26.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:26:26.464 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-09-17T03:26:26.463+0000 7fa34d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:26:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 -- 172.21.15.46:0/520505830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c1527b0 msgr2=0x7f2a8c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 --2- 172.21.15.46:0/520505830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c1527b0 0x7f2a8c074e30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c071cb0 tx=0x7f2a7401c800 comp rx=0 tx=0).stop 2024-09-17T03:26:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 -- 172.21.15.46:0/520505830 shutdown_connections 2024-09-17T03:26:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 --2- 172.21.15.46:0/520505830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c1527b0 0x7f2a8c074e30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 --2- 172.21.15.46:0/520505830 >> v2:172.21.15.99:3300/0 conn(0x7f2a8c0b5c70 0x7f2a8c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 -- 172.21.15.46:0/520505830 >> 172.21.15.46:0/520505830 conn(0x7f2a8c0b2040 msgr2=0x7f2a8c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:26.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 -- 172.21.15.46:0/520505830 shutdown_connections 2024-09-17T03:26:26.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 -- 172.21.15.46:0/520505830 wait complete. 2024-09-17T03:26:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 Processor -- start 2024-09-17T03:26:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.503+0000 7f2a91000640 1 -- start start 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 0x7f2a8c1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a8c0b5c70 0x7f2a8c1fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c1527b0 0x7f2a8c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c168b20 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1689a0 con 0x7f2a8c1527b0 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8c168ca0 con 0x7f2a8c0b5c70 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 0x7f2a8c1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 0x7f2a8c1498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50022/0 (socket says 172.21.15.46:50022) 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 -- 172.21.15.46:0/3491674990 learned_addr learned my addr 172.21.15.46:0/3491674990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a8a800640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c1527b0 0x7f2a8c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a83e00640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a8c0b5c70 0x7f2a8c1fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 -- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a8c0b5c70 msgr2=0x7f2a8c1fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a8c0b5c70 0x7f2a8c1fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 -- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c1527b0 msgr2=0x7f2a8c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c1527b0 0x7f2a8c15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 -- 172.21.15.46:0/3491674990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c1f6c40 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a89e00640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 0x7f2a8c1498b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d750 tx=0x7f2a7c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c017070 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a7c009d20 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c005280 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c1f6f20 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1f74e0 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.507+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c15bb60 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.511+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2a7c005420 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.511+0000 7f2a82000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a68073b00 0x7f2a68076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.511+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c092c60 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.511+0000 7f2a83e00640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a68073b00 0x7f2a68076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.511+0000 7f2a83e00640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a68073b00 0x7f2a68076010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2a740af000 tx=0x7f2a7401b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.515+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2a7c00de60 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.747+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2a8c0d0070 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.747+0000 7f2a82000640 1 -- 172.21.15.46:0/3491674990 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1143 (secure 0 0 0) 0x7f2a7c05f730 con 0x7f2a8c0b4de0 2024-09-17T03:26:26.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:26.751 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":344,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1726543255,"num_in_osds":8,"osd_in_since":1726543245,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592708,"bytes_used":395714560,"bytes_avail":772698398720,"bytes_total":773094113280,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":35,"btime":"2024-09-17T03:26:24:284029+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":245850}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-17T03:25:09.264615+0000","services":{}},"progress_events":{}} 2024-09-17T03:26:26.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a68073b00 msgr2=0x7f2a68076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a68073b00 0x7f2a68076010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2a740af000 tx=0x7f2a7401b040 comp rx=0 tx=0).stop 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 msgr2=0x7f2a8c1498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 0x7f2a8c1498b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d750 tx=0x7f2a7c00dc10 comp rx=0 tx=0).stop 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 shutdown_connections 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a68073b00 0x7f2a68076010 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c1527b0 0x7f2a8c15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a8c0b5c70 0x7f2a8c1fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 --2- 172.21.15.46:0/3491674990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a8c0b4de0 0x7f2a8c1498b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 >> 172.21.15.46:0/3491674990 conn(0x7f2a8c0b2040 msgr2=0x7f2a8c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 shutdown_connections 2024-09-17T03:26:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.751+0000 7f2a91000640 1 -- 172.21.15.46:0/3491674990 wait complete. 2024-09-17T03:26:26.774 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:26:26.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:26:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 -- 172.21.15.46:0/1545584059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 msgr2=0x7fb31c0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 --2- 172.21.15.46:0/1545584059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c0741a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb314009bf0 tx=0x7fb31401cb50 comp rx=0 tx=0).stop 2024-09-17T03:26:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 -- 172.21.15.46:0/1545584059 shutdown_connections 2024-09-17T03:26:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 --2- 172.21.15.46:0/1545584059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c0741a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 -- 172.21.15.46:0/1545584059 >> 172.21.15.46:0/1545584059 conn(0x7fb31c0b2040 msgr2=0x7fb31c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 -- 172.21.15.46:0/1545584059 shutdown_connections 2024-09-17T03:26:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 -- 172.21.15.46:0/1545584059 wait complete. 2024-09-17T03:26:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 Processor -- start 2024-09-17T03:26:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.931+0000 7fb322e00640 1 -- start start 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c072100 0x7fb31c145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 0x7fb31c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c169e70 con 0x7fb31c146ce0 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c169cf0 con 0x7fb31c072100 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c169ff0 con 0x7fb31c155b50 2024-09-17T03:26:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 0x7fb31c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 0x7fb31c153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50026/0 (socket says 172.21.15.46:50026) 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 -- 172.21.15.46:0/2795439028 learned_addr learned my addr 172.21.15.46:0/2795439028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb31be00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c072100 0x7fb31c145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb31b400640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 -- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 msgr2=0x7fb31c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 -- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c072100 msgr2=0x7fb31c145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c072100 0x7fb31c145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 -- 172.21.15.46:0/2795439028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0cedf0 con 0x7fb31c146ce0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb31b400640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb320e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 0x7fb31c153df0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb31401b3d0 tx=0x7fb314005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3140b2040 con 0x7fb31c146ce0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb314004040 con 0x7fb31c146ce0 2024-09-17T03:26:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1f86a0 con 0x7fb31c146ce0 2024-09-17T03:26:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3140043f0 con 0x7fb31c146ce0 2024-09-17T03:26:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1f8c60 con 0x7fb31c146ce0 2024-09-17T03:26:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.935+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c0b4de0 con 0x7fb31c146ce0 2024-09-17T03:26:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.939+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fb314004590 con 0x7fb31c146ce0 2024-09-17T03:26:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.939+0000 7fb313e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2fc073b00 0x7fb2fc076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.939+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb31412dc60 con 0x7fb31c146ce0 2024-09-17T03:26:26.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.939+0000 7fb31be00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2fc073b00 0x7fb2fc076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:26.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.939+0000 7fb31be00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2fc073b00 0x7fb2fc076010 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb30c007fd0 tx=0x7fb30c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:26.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:26.943+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb3140167d0 con 0x7fb31c146ce0 2024-09-17T03:26:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.127+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb31c06b540 con 0x7fb2fc073b00 2024-09-17T03:26:27.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.131+0000 7fb313e00640 1 -- 172.21.15.46:0/2795439028 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250791 (secure 0 0 0) 0x7fb30c001480 con 0x7fb2fc073b00 2024-09-17T03:26:27.135 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2fc073b00 msgr2=0x7fb2fc076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2fc073b00 0x7fb2fc076010 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb30c007fd0 tx=0x7fb30c009040 comp rx=0 tx=0).stop 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 msgr2=0x7fb31c153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 0x7fb31c153df0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb31401b3d0 tx=0x7fb314005a60 comp rx=0 tx=0).stop 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 shutdown_connections 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2fc073b00 0x7fb2fc076010 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31c146ce0 0x7fb31c153df0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb31c155b50 0x7fb31c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:27.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 --2- 172.21.15.46:0/2795439028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb31c072100 0x7fb31c145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:27.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 >> 172.21.15.46:0/2795439028 conn(0x7fb31c0b2040 msgr2=0x7fb31c073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:27.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 shutdown_connections 2024-09-17T03:26:27.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:27.135+0000 7fb322e00640 1 -- 172.21.15.46:0/2795439028 wait complete. 2024-09-17T03:26:27.156 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-17T03:26:25.303316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386440,"kb_used_data":6420,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754588280,"statfs":{"total":773094113280,"available":772698398720,"internally_reserved":0,"allocated":6574080,"data_stored":3391384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001164"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.290122+0000","last_change":"2024-09-17T03:26:21.271619+0000","last_active":"2024-09-17T03:26:21.290122+0000","last_peered":"2024-09-17T03:26:21.290122+0000","last_clean":"2024-09-17T03:26:21.290122+0000","last_became_active":"2024-09-17T03:26:21.270472+0000","last_became_peered":"2024-09-17T03:26:21.270472+0000","last_unstale":"2024-09-17T03:26:21.290122+0000","last_undegraded":"2024-09-17T03:26:21.290122+0000","last_fullsized":"2024-09-17T03:26:21.290122+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:58.768092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293649+0000","last_change":"2024-09-17T03:26:21.281731+0000","last_active":"2024-09-17T03:26:21.293649+0000","last_peered":"2024-09-17T03:26:21.293649+0000","last_clean":"2024-09-17T03:26:21.293649+0000","last_became_active":"2024-09-17T03:26:21.281402+0000","last_became_peered":"2024-09-17T03:26:21.281402+0000","last_unstale":"2024-09-17T03:26:21.293649+0000","last_undegraded":"2024-09-17T03:26:21.293649+0000","last_fullsized":"2024-09-17T03:26:21.293649+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:18.996659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:21.283316+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:21.282999+0000","last_became_peered":"2024-09-17T03:26:21.282999+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:15.689667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269895+0000","last_change":"2024-09-17T03:26:20.269895+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269895+0000","last_undegraded":"2024-09-17T03:26:20.269895+0000","last_fullsized":"2024-09-17T03:26:20.269895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289760+0000","last_change":"2024-09-17T03:26:21.284187+0000","last_active":"2024-09-17T03:26:21.289760+0000","last_peered":"2024-09-17T03:26:21.289760+0000","last_clean":"2024-09-17T03:26:21.289760+0000","last_became_active":"2024-09-17T03:26:21.283779+0000","last_became_peered":"2024-09-17T03:26:21.283779+0000","last_unstale":"2024-09-17T03:26:21.289760+0000","last_undegraded":"2024-09-17T03:26:21.289760+0000","last_fullsized":"2024-09-17T03:26:21.289760+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:57.934343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300045+0000","last_change":"2024-09-17T03:26:21.296425+0000","last_active":"2024-09-17T03:26:21.300045+0000","last_peered":"2024-09-17T03:26:21.300045+0000","last_clean":"2024-09-17T03:26:21.300045+0000","last_became_active":"2024-09-17T03:26:21.296063+0000","last_became_peered":"2024-09-17T03:26:21.296063+0000","last_unstale":"2024-09-17T03:26:21.300045+0000","last_undegraded":"2024-09-17T03:26:21.300045+0000","last_fullsized":"2024-09-17T03:26:21.300045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:27.881549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284101+0000","last_change":"2024-09-17T03:26:21.281715+0000","last_active":"2024-09-17T03:26:21.284101+0000","last_peered":"2024-09-17T03:26:21.284101+0000","last_clean":"2024-09-17T03:26:21.284101+0000","last_became_active":"2024-09-17T03:26:21.280618+0000","last_became_peered":"2024-09-17T03:26:21.280618+0000","last_unstale":"2024-09-17T03:26:21.284101+0000","last_undegraded":"2024-09-17T03:26:21.284101+0000","last_fullsized":"2024-09-17T03:26:21.284101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:44.647319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285297+0000","last_change":"2024-09-17T03:26:21.277710+0000","last_active":"2024-09-17T03:26:21.285297+0000","last_peered":"2024-09-17T03:26:21.285297+0000","last_clean":"2024-09-17T03:26:21.285297+0000","last_became_active":"2024-09-17T03:26:21.277333+0000","last_became_peered":"2024-09-17T03:26:21.277333+0000","last_unstale":"2024-09-17T03:26:21.285297+0000","last_undegraded":"2024-09-17T03:26:21.285297+0000","last_fullsized":"2024-09-17T03:26:21.285297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:34.691840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291938+0000","last_change":"2024-09-17T03:26:21.290497+0000","last_active":"2024-09-17T03:26:21.291938+0000","last_peered":"2024-09-17T03:26:21.291938+0000","last_clean":"2024-09-17T03:26:21.291938+0000","last_became_active":"2024-09-17T03:26:21.289289+0000","last_became_peered":"2024-09-17T03:26:21.289289+0000","last_unstale":"2024-09-17T03:26:21.291938+0000","last_undegraded":"2024-09-17T03:26:21.291938+0000","last_fullsized":"2024-09-17T03:26:21.291938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:01.560195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263685+0000","last_change":"2024-09-17T03:26:20.263685+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263685+0000","last_undegraded":"2024-09-17T03:26:20.263685+0000","last_fullsized":"2024-09-17T03:26:20.263685+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262752+0000","last_change":"2024-09-17T03:26:20.262752+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262752+0000","last_undegraded":"2024-09-17T03:26:20.262752+0000","last_fullsized":"2024-09-17T03:26:20.262752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.269255+0000","last_change":"2024-09-17T03:26:20.269255+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.269255+0000","last_undegraded":"2024-09-17T03:26:20.269255+0000","last_fullsized":"2024-09-17T03:26:20.269255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301774+0000","last_change":"2024-09-17T03:26:21.300221+0000","last_active":"2024-09-17T03:26:21.301774+0000","last_peered":"2024-09-17T03:26:21.301774+0000","last_clean":"2024-09-17T03:26:21.301774+0000","last_became_active":"2024-09-17T03:26:21.299864+0000","last_became_peered":"2024-09-17T03:26:21.299864+0000","last_unstale":"2024-09-17T03:26:21.301774+0000","last_undegraded":"2024-09-17T03:26:21.301774+0000","last_fullsized":"2024-09-17T03:26:21.301774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280369+0000","last_change":"2024-09-17T03:26:21.274122+0000","last_active":"2024-09-17T03:26:21.280369+0000","last_peered":"2024-09-17T03:26:21.280369+0000","last_clean":"2024-09-17T03:26:21.280369+0000","last_became_active":"2024-09-17T03:26:21.273777+0000","last_became_peered":"2024-09-17T03:26:21.273777+0000","last_unstale":"2024-09-17T03:26:21.280369+0000","last_undegraded":"2024-09-17T03:26:21.280369+0000","last_fullsized":"2024-09-17T03:26:21.280369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:02.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295641+0000","last_change":"2024-09-17T03:26:21.292845+0000","last_active":"2024-09-17T03:26:21.295641+0000","last_peered":"2024-09-17T03:26:21.295641+0000","last_clean":"2024-09-17T03:26:21.295641+0000","last_became_active":"2024-09-17T03:26:21.292432+0000","last_became_peered":"2024-09-17T03:26:21.292432+0000","last_unstale":"2024-09-17T03:26:21.295641+0000","last_undegraded":"2024-09-17T03:26:21.295641+0000","last_fullsized":"2024-09-17T03:26:21.295641+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:19.913825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.285635+0000","last_change":"2024-09-17T03:26:21.282672+0000","last_active":"2024-09-17T03:26:21.285635+0000","last_peered":"2024-09-17T03:26:21.285635+0000","last_clean":"2024-09-17T03:26:21.285635+0000","last_became_active":"2024-09-17T03:26:21.282359+0000","last_became_peered":"2024-09-17T03:26:21.282359+0000","last_unstale":"2024-09-17T03:26:21.285635+0000","last_undegraded":"2024-09-17T03:26:21.285635+0000","last_fullsized":"2024-09-17T03:26:21.285635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:11.693379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283260+0000","last_change":"2024-09-17T03:26:21.277694+0000","last_active":"2024-09-17T03:26:21.283260+0000","last_peered":"2024-09-17T03:26:21.283260+0000","last_clean":"2024-09-17T03:26:21.283260+0000","last_became_active":"2024-09-17T03:26:21.277341+0000","last_became_peered":"2024-09-17T03:26:21.277341+0000","last_unstale":"2024-09-17T03:26:21.283260+0000","last_undegraded":"2024-09-17T03:26:21.283260+0000","last_fullsized":"2024-09-17T03:26:21.283260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:16.287612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294961+0000","last_change":"2024-09-17T03:26:21.289536+0000","last_active":"2024-09-17T03:26:21.294961+0000","last_peered":"2024-09-17T03:26:21.294961+0000","last_clean":"2024-09-17T03:26:21.294961+0000","last_became_active":"2024-09-17T03:26:21.289230+0000","last_became_peered":"2024-09-17T03:26:21.289230+0000","last_unstale":"2024-09-17T03:26:21.294961+0000","last_undegraded":"2024-09-17T03:26:21.294961+0000","last_fullsized":"2024-09-17T03:26:21.294961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:31.982607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288531+0000","last_change":"2024-09-17T03:26:21.272001+0000","last_active":"2024-09-17T03:26:21.288531+0000","last_peered":"2024-09-17T03:26:21.288531+0000","last_clean":"2024-09-17T03:26:21.288531+0000","last_became_active":"2024-09-17T03:26:21.271673+0000","last_became_peered":"2024-09-17T03:26:21.271673+0000","last_unstale":"2024-09-17T03:26:21.288531+0000","last_undegraded":"2024-09-17T03:26:21.288531+0000","last_fullsized":"2024-09-17T03:26:21.288531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:48.995086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295028+0000","last_change":"2024-09-17T03:26:21.291765+0000","last_active":"2024-09-17T03:26:21.295028+0000","last_peered":"2024-09-17T03:26:21.295028+0000","last_clean":"2024-09-17T03:26:21.295028+0000","last_became_active":"2024-09-17T03:26:21.291430+0000","last_became_peered":"2024-09-17T03:26:21.291430+0000","last_unstale":"2024-09-17T03:26:21.295028+0000","last_undegraded":"2024-09-17T03:26:21.295028+0000","last_fullsized":"2024-09-17T03:26:21.295028+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.144097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289278+0000","last_change":"2024-09-17T03:26:21.285773+0000","last_active":"2024-09-17T03:26:21.289278+0000","last_peered":"2024-09-17T03:26:21.289278+0000","last_clean":"2024-09-17T03:26:21.289278+0000","last_became_active":"2024-09-17T03:26:21.285118+0000","last_became_peered":"2024-09-17T03:26:21.285118+0000","last_unstale":"2024-09-17T03:26:21.289278+0000","last_undegraded":"2024-09-17T03:26:21.289278+0000","last_fullsized":"2024-09-17T03:26:21.289278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:50.670027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283248+0000","last_change":"2024-09-17T03:26:21.276516+0000","last_active":"2024-09-17T03:26:21.283248+0000","last_peered":"2024-09-17T03:26:21.283248+0000","last_clean":"2024-09-17T03:26:21.283248+0000","last_became_active":"2024-09-17T03:26:21.275398+0000","last_became_peered":"2024-09-17T03:26:21.275398+0000","last_unstale":"2024-09-17T03:26:21.283248+0000","last_undegraded":"2024-09-17T03:26:21.283248+0000","last_fullsized":"2024-09-17T03:26:21.283248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:22.952091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291438+0000","last_change":"2024-09-17T03:26:21.286047+0000","last_active":"2024-09-17T03:26:21.291438+0000","last_peered":"2024-09-17T03:26:21.291438+0000","last_clean":"2024-09-17T03:26:21.291438+0000","last_became_active":"2024-09-17T03:26:21.284398+0000","last_became_peered":"2024-09-17T03:26:21.284398+0000","last_unstale":"2024-09-17T03:26:21.291438+0000","last_undegraded":"2024-09-17T03:26:21.291438+0000","last_fullsized":"2024-09-17T03:26:21.291438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:38.105159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293886+0000","last_change":"2024-09-17T03:26:21.289586+0000","last_active":"2024-09-17T03:26:21.293886+0000","last_peered":"2024-09-17T03:26:21.293886+0000","last_clean":"2024-09-17T03:26:21.293886+0000","last_became_active":"2024-09-17T03:26:21.289242+0000","last_became_peered":"2024-09-17T03:26:21.289242+0000","last_unstale":"2024-09-17T03:26:21.293886+0000","last_undegraded":"2024-09-17T03:26:21.293886+0000","last_fullsized":"2024-09-17T03:26:21.293886+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:47.275691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293738+0000","last_change":"2024-09-17T03:26:21.287754+0000","last_active":"2024-09-17T03:26:21.293738+0000","last_peered":"2024-09-17T03:26:21.293738+0000","last_clean":"2024-09-17T03:26:21.293738+0000","last_became_active":"2024-09-17T03:26:21.287382+0000","last_became_peered":"2024-09-17T03:26:21.287382+0000","last_unstale":"2024-09-17T03:26:21.293738+0000","last_undegraded":"2024-09-17T03:26:21.293738+0000","last_fullsized":"2024-09-17T03:26:21.293738+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:03.289618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275232+0000","last_change":"2024-09-17T03:26:20.275232+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275232+0000","last_undegraded":"2024-09-17T03:26:20.275232+0000","last_fullsized":"2024-09-17T03:26:20.275232+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.284955+0000","last_change":"2024-09-17T03:26:21.282975+0000","last_active":"2024-09-17T03:26:21.284955+0000","last_peered":"2024-09-17T03:26:21.284955+0000","last_clean":"2024-09-17T03:26:21.284955+0000","last_became_active":"2024-09-17T03:26:21.282116+0000","last_became_peered":"2024-09-17T03:26:21.282116+0000","last_unstale":"2024-09-17T03:26:21.284955+0000","last_undegraded":"2024-09-17T03:26:21.284955+0000","last_fullsized":"2024-09-17T03:26:21.284955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:08.367097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293266+0000","last_change":"2024-09-17T03:26:21.287660+0000","last_active":"2024-09-17T03:26:21.293266+0000","last_peered":"2024-09-17T03:26:21.293266+0000","last_clean":"2024-09-17T03:26:21.293266+0000","last_became_active":"2024-09-17T03:26:21.287329+0000","last_became_peered":"2024-09-17T03:26:21.287329+0000","last_unstale":"2024-09-17T03:26:21.293266+0000","last_undegraded":"2024-09-17T03:26:21.293266+0000","last_fullsized":"2024-09-17T03:26:21.293266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:07.684829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293795+0000","last_change":"2024-09-17T03:26:21.281015+0000","last_active":"2024-09-17T03:26:21.293795+0000","last_peered":"2024-09-17T03:26:21.293795+0000","last_clean":"2024-09-17T03:26:21.293795+0000","last_became_active":"2024-09-17T03:26:21.280690+0000","last_became_peered":"2024-09-17T03:26:21.280690+0000","last_unstale":"2024-09-17T03:26:21.293795+0000","last_undegraded":"2024-09-17T03:26:21.293795+0000","last_fullsized":"2024-09-17T03:26:21.293795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:27.474853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.288915+0000","last_change":"2024-09-17T03:26:21.282370+0000","last_active":"2024-09-17T03:26:21.288915+0000","last_peered":"2024-09-17T03:26:21.288915+0000","last_clean":"2024-09-17T03:26:21.288915+0000","last_became_active":"2024-09-17T03:26:21.282053+0000","last_became_peered":"2024-09-17T03:26:21.282053+0000","last_unstale":"2024-09-17T03:26:21.288915+0000","last_undegraded":"2024-09-17T03:26:21.288915+0000","last_fullsized":"2024-09-17T03:26:21.288915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:27.209668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295471+0000","last_change":"2024-09-17T03:26:21.293642+0000","last_active":"2024-09-17T03:26:21.295471+0000","last_peered":"2024-09-17T03:26:21.295471+0000","last_clean":"2024-09-17T03:26:21.295471+0000","last_became_active":"2024-09-17T03:26:21.289857+0000","last_became_peered":"2024-09-17T03:26:21.289857+0000","last_unstale":"2024-09-17T03:26:21.295471+0000","last_undegraded":"2024-09-17T03:26:21.295471+0000","last_fullsized":"2024-09-17T03:26:21.295471+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:56.082574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281534+0000","last_change":"2024-09-17T03:26:21.278899+0000","last_active":"2024-09-17T03:26:21.281534+0000","last_peered":"2024-09-17T03:26:21.281534+0000","last_clean":"2024-09-17T03:26:21.281534+0000","last_became_active":"2024-09-17T03:26:21.278562+0000","last_became_peered":"2024-09-17T03:26:21.278562+0000","last_unstale":"2024-09-17T03:26:21.281534+0000","last_undegraded":"2024-09-17T03:26:21.281534+0000","last_fullsized":"2024-09-17T03:26:21.281534+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:06.391945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.281831+0000","last_change":"2024-09-17T03:26:21.279072+0000","last_active":"2024-09-17T03:26:21.281831+0000","last_peered":"2024-09-17T03:26:21.281831+0000","last_clean":"2024-09-17T03:26:21.281831+0000","last_became_active":"2024-09-17T03:26:21.278730+0000","last_became_peered":"2024-09-17T03:26:21.278730+0000","last_unstale":"2024-09-17T03:26:21.281831+0000","last_undegraded":"2024-09-17T03:26:21.281831+0000","last_fullsized":"2024-09-17T03:26:21.281831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:31.274794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300099+0000","last_change":"2024-09-17T03:26:20.283656+0000","last_active":"2024-09-17T03:26:21.300099+0000","last_peered":"2024-09-17T03:26:21.300099+0000","last_clean":"2024-09-17T03:26:21.300099+0000","last_became_active":"2024-09-17T03:26:20.283331+0000","last_became_peered":"2024-09-17T03:26:20.283331+0000","last_unstale":"2024-09-17T03:26:21.300099+0000","last_undegraded":"2024-09-17T03:26:21.300099+0000","last_fullsized":"2024-09-17T03:26:21.300099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:03.041707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263563+0000","last_change":"2024-09-17T03:26:20.263563+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263563+0000","last_undegraded":"2024-09-17T03:26:20.263563+0000","last_fullsized":"2024-09-17T03:26:20.263563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289686+0000","last_change":"2024-09-17T03:26:20.266917+0000","last_active":"2024-09-17T03:26:21.289686+0000","last_peered":"2024-09-17T03:26:21.289686+0000","last_clean":"2024-09-17T03:26:21.289686+0000","last_became_active":"2024-09-17T03:26:20.266370+0000","last_became_peered":"2024-09-17T03:26:20.266370+0000","last_unstale":"2024-09-17T03:26:21.289686+0000","last_undegraded":"2024-09-17T03:26:21.289686+0000","last_fullsized":"2024-09-17T03:26:21.289686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:57.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300820+0000","last_change":"2024-09-17T03:26:21.299198+0000","last_active":"2024-09-17T03:26:21.300820+0000","last_peered":"2024-09-17T03:26:21.300820+0000","last_clean":"2024-09-17T03:26:21.300820+0000","last_became_active":"2024-09-17T03:26:21.298717+0000","last_became_peered":"2024-09-17T03:26:21.298717+0000","last_unstale":"2024-09-17T03:26:21.300820+0000","last_undegraded":"2024-09-17T03:26:21.300820+0000","last_fullsized":"2024-09-17T03:26:21.300820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:48.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301538+0000","last_change":"2024-09-17T03:26:20.286649+0000","last_active":"2024-09-17T03:26:21.301538+0000","last_peered":"2024-09-17T03:26:21.301538+0000","last_clean":"2024-09-17T03:26:21.301538+0000","last_became_active":"2024-09-17T03:26:20.285886+0000","last_became_peered":"2024-09-17T03:26:20.285886+0000","last_unstale":"2024-09-17T03:26:21.301538+0000","last_undegraded":"2024-09-17T03:26:21.301538+0000","last_fullsized":"2024-09-17T03:26:21.301538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:46.887782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300191+0000","last_change":"2024-09-17T03:26:21.289923+0000","last_active":"2024-09-17T03:26:21.300191+0000","last_peered":"2024-09-17T03:26:21.300191+0000","last_clean":"2024-09-17T03:26:21.300191+0000","last_became_active":"2024-09-17T03:26:21.289556+0000","last_became_peered":"2024-09-17T03:26:21.289556+0000","last_unstale":"2024-09-17T03:26:21.300191+0000","last_undegraded":"2024-09-17T03:26:21.300191+0000","last_fullsized":"2024-09-17T03:26:21.300191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:41.180633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286893+0000","last_change":"2024-09-17T03:26:20.283187+0000","last_active":"2024-09-17T03:26:20.286893+0000","last_peered":"2024-09-17T03:26:20.286893+0000","last_clean":"2024-09-17T03:26:20.286893+0000","last_became_active":"2024-09-17T03:26:20.282354+0000","last_became_peered":"2024-09-17T03:26:20.282354+0000","last_unstale":"2024-09-17T03:26:20.286893+0000","last_undegraded":"2024-09-17T03:26:20.286893+0000","last_fullsized":"2024-09-17T03:26:20.286893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:57.123281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293825+0000","last_change":"2024-09-17T03:26:21.287275+0000","last_active":"2024-09-17T03:26:21.293825+0000","last_peered":"2024-09-17T03:26:21.293825+0000","last_clean":"2024-09-17T03:26:21.293825+0000","last_became_active":"2024-09-17T03:26:21.286995+0000","last_became_peered":"2024-09-17T03:26:21.286995+0000","last_unstale":"2024-09-17T03:26:21.293825+0000","last_undegraded":"2024-09-17T03:26:21.293825+0000","last_fullsized":"2024-09-17T03:26:21.293825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:58.245607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.298865+0000","last_change":"2024-09-17T03:26:20.283433+0000","last_active":"2024-09-17T03:26:21.298865+0000","last_peered":"2024-09-17T03:26:21.298865+0000","last_clean":"2024-09-17T03:26:21.298865+0000","last_became_active":"2024-09-17T03:26:20.277797+0000","last_became_peered":"2024-09-17T03:26:20.277797+0000","last_unstale":"2024-09-17T03:26:21.298865+0000","last_undegraded":"2024-09-17T03:26:21.298865+0000","last_fullsized":"2024-09-17T03:26:21.298865+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294978+0000","last_change":"2024-09-17T03:26:21.293180+0000","last_active":"2024-09-17T03:26:21.294978+0000","last_peered":"2024-09-17T03:26:21.294978+0000","last_clean":"2024-09-17T03:26:21.294978+0000","last_became_active":"2024-09-17T03:26:21.292882+0000","last_became_peered":"2024-09-17T03:26:21.292882+0000","last_unstale":"2024-09-17T03:26:21.294978+0000","last_undegraded":"2024-09-17T03:26:21.294978+0000","last_fullsized":"2024-09-17T03:26:21.294978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:16.723527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301141+0000","last_change":"2024-09-17T03:26:20.279365+0000","last_active":"2024-09-17T03:26:21.301141+0000","last_peered":"2024-09-17T03:26:21.301141+0000","last_clean":"2024-09-17T03:26:21.301141+0000","last_became_active":"2024-09-17T03:26:20.278923+0000","last_became_peered":"2024-09-17T03:26:20.278923+0000","last_unstale":"2024-09-17T03:26:21.301141+0000","last_undegraded":"2024-09-17T03:26:21.301141+0000","last_fullsized":"2024-09-17T03:26:21.301141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:14.310020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286603+0000","last_change":"2024-09-17T03:26:21.285430+0000","last_active":"2024-09-17T03:26:21.286603+0000","last_peered":"2024-09-17T03:26:21.286603+0000","last_clean":"2024-09-17T03:26:21.286603+0000","last_became_active":"2024-09-17T03:26:21.285111+0000","last_became_peered":"2024-09-17T03:26:21.285111+0000","last_unstale":"2024-09-17T03:26:21.286603+0000","last_undegraded":"2024-09-17T03:26:21.286603+0000","last_fullsized":"2024-09-17T03:26:21.286603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:47.441700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284199+0000","last_change":"2024-09-17T03:26:20.279560+0000","last_active":"2024-09-17T03:26:20.284199+0000","last_peered":"2024-09-17T03:26:20.284199+0000","last_clean":"2024-09-17T03:26:20.284199+0000","last_became_active":"2024-09-17T03:26:20.278746+0000","last_became_peered":"2024-09-17T03:26:20.278746+0000","last_unstale":"2024-09-17T03:26:20.284199+0000","last_undegraded":"2024-09-17T03:26:20.284199+0000","last_fullsized":"2024-09-17T03:26:20.284199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:10.047880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286823+0000","last_change":"2024-09-17T03:26:21.279707+0000","last_active":"2024-09-17T03:26:21.286823+0000","last_peered":"2024-09-17T03:26:21.286823+0000","last_clean":"2024-09-17T03:26:21.286823+0000","last_became_active":"2024-09-17T03:26:21.279020+0000","last_became_peered":"2024-09-17T03:26:21.279020+0000","last_unstale":"2024-09-17T03:26:21.286823+0000","last_undegraded":"2024-09-17T03:26:21.286823+0000","last_fullsized":"2024-09-17T03:26:21.286823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:48.268297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264232+0000","last_change":"2024-09-17T03:26:20.283353+0000","last_active":"2024-09-17T03:26:21.264232+0000","last_peered":"2024-09-17T03:26:21.264232+0000","last_clean":"2024-09-17T03:26:21.264232+0000","last_became_active":"2024-09-17T03:26:20.283049+0000","last_became_peered":"2024-09-17T03:26:20.283049+0000","last_unstale":"2024-09-17T03:26:21.264232+0000","last_undegraded":"2024-09-17T03:26:21.264232+0000","last_fullsized":"2024-09-17T03:26:21.264232+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291964+0000","last_change":"2024-09-17T03:26:21.290216+0000","last_active":"2024-09-17T03:26:21.291964+0000","last_peered":"2024-09-17T03:26:21.291964+0000","last_clean":"2024-09-17T03:26:21.291964+0000","last_became_active":"2024-09-17T03:26:21.288236+0000","last_became_peered":"2024-09-17T03:26:21.288236+0000","last_unstale":"2024-09-17T03:26:21.291964+0000","last_undegraded":"2024-09-17T03:26:21.291964+0000","last_fullsized":"2024-09-17T03:26:21.291964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:34.999880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269902+0000","last_change":"2024-09-17T03:26:20.289405+0000","last_active":"2024-09-17T03:26:21.269902+0000","last_peered":"2024-09-17T03:26:21.269902+0000","last_clean":"2024-09-17T03:26:21.269902+0000","last_became_active":"2024-09-17T03:26:20.288979+0000","last_became_peered":"2024-09-17T03:26:20.288979+0000","last_unstale":"2024-09-17T03:26:21.269902+0000","last_undegraded":"2024-09-17T03:26:21.269902+0000","last_fullsized":"2024-09-17T03:26:21.269902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:10.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.303142+0000","last_change":"2024-09-17T03:26:21.301608+0000","last_active":"2024-09-17T03:26:21.303142+0000","last_peered":"2024-09-17T03:26:21.303142+0000","last_clean":"2024-09-17T03:26:21.303142+0000","last_became_active":"2024-09-17T03:26:21.301203+0000","last_became_peered":"2024-09-17T03:26:21.301203+0000","last_unstale":"2024-09-17T03:26:21.303142+0000","last_undegraded":"2024-09-17T03:26:21.303142+0000","last_fullsized":"2024-09-17T03:26:21.303142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:23.395294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.271410+0000","last_change":"2024-09-17T03:26:20.263118+0000","last_active":"2024-09-17T03:26:21.271410+0000","last_peered":"2024-09-17T03:26:21.271410+0000","last_clean":"2024-09-17T03:26:21.271410+0000","last_became_active":"2024-09-17T03:26:20.262773+0000","last_became_peered":"2024-09-17T03:26:20.262773+0000","last_unstale":"2024-09-17T03:26:21.271410+0000","last_undegraded":"2024-09-17T03:26:21.271410+0000","last_fullsized":"2024-09-17T03:26:21.271410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:02.639519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.263081+0000","last_change":"2024-09-17T03:26:20.263081+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.263081+0000","last_undegraded":"2024-09-17T03:26:20.263081+0000","last_fullsized":"2024-09-17T03:26:20.263081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.306381+0000","last_change":"2024-09-17T03:26:20.277795+0000","last_active":"2024-09-17T03:26:21.306381+0000","last_peered":"2024-09-17T03:26:21.306381+0000","last_clean":"2024-09-17T03:26:21.306381+0000","last_became_active":"2024-09-17T03:26:20.277456+0000","last_became_peered":"2024-09-17T03:26:20.277456+0000","last_unstale":"2024-09-17T03:26:21.306381+0000","last_undegraded":"2024-09-17T03:26:21.306381+0000","last_fullsized":"2024-09-17T03:26:21.306381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:43.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297424+0000","last_change":"2024-09-17T03:26:21.292382+0000","last_active":"2024-09-17T03:26:21.297424+0000","last_peered":"2024-09-17T03:26:21.297424+0000","last_clean":"2024-09-17T03:26:21.297424+0000","last_became_active":"2024-09-17T03:26:21.292043+0000","last_became_peered":"2024-09-17T03:26:21.292043+0000","last_unstale":"2024-09-17T03:26:21.297424+0000","last_undegraded":"2024-09-17T03:26:21.297424+0000","last_fullsized":"2024-09-17T03:26:21.297424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:02.648204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.308666+0000","last_change":"2024-09-17T03:26:20.289201+0000","last_active":"2024-09-17T03:26:21.308666+0000","last_peered":"2024-09-17T03:26:21.308666+0000","last_clean":"2024-09-17T03:26:21.308666+0000","last_became_active":"2024-09-17T03:26:20.288427+0000","last_became_peered":"2024-09-17T03:26:20.288427+0000","last_unstale":"2024-09-17T03:26:21.308666+0000","last_undegraded":"2024-09-17T03:26:21.308666+0000","last_fullsized":"2024-09-17T03:26:21.308666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:46.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.275142+0000","last_change":"2024-09-17T03:26:20.275142+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.275142+0000","last_undegraded":"2024-09-17T03:26:20.275142+0000","last_fullsized":"2024-09-17T03:26:20.275142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.292629+0000","last_change":"2024-09-17T03:26:20.265758+0000","last_active":"2024-09-17T03:26:21.292629+0000","last_peered":"2024-09-17T03:26:21.292629+0000","last_clean":"2024-09-17T03:26:21.292629+0000","last_became_active":"2024-09-17T03:26:20.262782+0000","last_became_peered":"2024-09-17T03:26:20.262782+0000","last_unstale":"2024-09-17T03:26:21.292629+0000","last_undegraded":"2024-09-17T03:26:21.292629+0000","last_fullsized":"2024-09-17T03:26:21.292629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:10.171064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283171+0000","last_change":"2024-09-17T03:26:21.278170+0000","last_active":"2024-09-17T03:26:21.283171+0000","last_peered":"2024-09-17T03:26:21.283171+0000","last_clean":"2024-09-17T03:26:21.283171+0000","last_became_active":"2024-09-17T03:26:21.277329+0000","last_became_peered":"2024-09-17T03:26:21.277329+0000","last_unstale":"2024-09-17T03:26:21.283171+0000","last_undegraded":"2024-09-17T03:26:21.283171+0000","last_fullsized":"2024-09-17T03:26:21.283171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:32.373126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267858+0000","last_change":"2024-09-17T03:26:20.269475+0000","last_active":"2024-09-17T03:26:21.267858+0000","last_peered":"2024-09-17T03:26:21.267858+0000","last_clean":"2024-09-17T03:26:21.267858+0000","last_became_active":"2024-09-17T03:26:20.269101+0000","last_became_peered":"2024-09-17T03:26:20.269101+0000","last_unstale":"2024-09-17T03:26:21.267858+0000","last_undegraded":"2024-09-17T03:26:21.267858+0000","last_fullsized":"2024-09-17T03:26:21.267858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.283965+0000","last_change":"2024-09-17T03:26:21.281823+0000","last_active":"2024-09-17T03:26:21.283965+0000","last_peered":"2024-09-17T03:26:21.283965+0000","last_clean":"2024-09-17T03:26:21.283965+0000","last_became_active":"2024-09-17T03:26:21.280688+0000","last_became_peered":"2024-09-17T03:26:21.280688+0000","last_unstale":"2024-09-17T03:26:21.283965+0000","last_undegraded":"2024-09-17T03:26:21.283965+0000","last_fullsized":"2024-09-17T03:26:21.283965+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:36.912044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284389+0000","last_change":"2024-09-17T03:26:20.281345+0000","last_active":"2024-09-17T03:26:20.284389+0000","last_peered":"2024-09-17T03:26:20.284389+0000","last_clean":"2024-09-17T03:26:20.284389+0000","last_became_active":"2024-09-17T03:26:20.280261+0000","last_became_peered":"2024-09-17T03:26:20.280261+0000","last_unstale":"2024-09-17T03:26:20.284389+0000","last_undegraded":"2024-09-17T03:26:20.284389+0000","last_fullsized":"2024-09-17T03:26:20.284389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:47.948863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.294667+0000","last_change":"2024-09-17T03:26:21.289462+0000","last_active":"2024-09-17T03:26:21.294667+0000","last_peered":"2024-09-17T03:26:21.294667+0000","last_clean":"2024-09-17T03:26:21.294667+0000","last_became_active":"2024-09-17T03:26:21.285434+0000","last_became_peered":"2024-09-17T03:26:21.285434+0000","last_unstale":"2024-09-17T03:26:21.294667+0000","last_undegraded":"2024-09-17T03:26:21.294667+0000","last_fullsized":"2024-09-17T03:26:21.294667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:24.253438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.301604+0000","last_change":"2024-09-17T03:26:20.275945+0000","last_active":"2024-09-17T03:26:21.301604+0000","last_peered":"2024-09-17T03:26:21.301604+0000","last_clean":"2024-09-17T03:26:21.301604+0000","last_became_active":"2024-09-17T03:26:20.275123+0000","last_became_peered":"2024-09-17T03:26:20.275123+0000","last_unstale":"2024-09-17T03:26:21.301604+0000","last_undegraded":"2024-09-17T03:26:21.301604+0000","last_fullsized":"2024-09-17T03:26:21.301604+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:02.236108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.291615+0000","last_change":"2024-09-17T03:26:21.284952+0000","last_active":"2024-09-17T03:26:21.291615+0000","last_peered":"2024-09-17T03:26:21.291615+0000","last_clean":"2024-09-17T03:26:21.291615+0000","last_became_active":"2024-09-17T03:26:21.284572+0000","last_became_peered":"2024-09-17T03:26:21.284572+0000","last_unstale":"2024-09-17T03:26:21.291615+0000","last_undegraded":"2024-09-17T03:26:21.291615+0000","last_fullsized":"2024-09-17T03:26:21.291615+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:11.920676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295381+0000","last_change":"2024-09-17T03:26:20.269212+0000","last_active":"2024-09-17T03:26:21.295381+0000","last_peered":"2024-09-17T03:26:21.295381+0000","last_clean":"2024-09-17T03:26:21.295381+0000","last_became_active":"2024-09-17T03:26:20.268883+0000","last_became_peered":"2024-09-17T03:26:20.268883+0000","last_unstale":"2024-09-17T03:26:21.295381+0000","last_undegraded":"2024-09-17T03:26:21.295381+0000","last_fullsized":"2024-09-17T03:26:21.295381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:41:56.450980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274114+0000","last_change":"2024-09-17T03:26:20.274114+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274114+0000","last_undegraded":"2024-09-17T03:26:20.274114+0000","last_fullsized":"2024-09-17T03:26:20.274114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.300538+0000","last_change":"2024-09-17T03:26:20.279146+0000","last_active":"2024-09-17T03:26:21.300538+0000","last_peered":"2024-09-17T03:26:21.300538+0000","last_clean":"2024-09-17T03:26:21.300538+0000","last_became_active":"2024-09-17T03:26:20.278888+0000","last_became_peered":"2024-09-17T03:26:20.278888+0000","last_unstale":"2024-09-17T03:26:21.300538+0000","last_undegraded":"2024-09-17T03:26:21.300538+0000","last_fullsized":"2024-09-17T03:26:21.300538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:40.150476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286599+0000","last_change":"2024-09-17T03:26:21.278295+0000","last_active":"2024-09-17T03:26:21.286599+0000","last_peered":"2024-09-17T03:26:21.286599+0000","last_clean":"2024-09-17T03:26:21.286599+0000","last_became_active":"2024-09-17T03:26:21.277728+0000","last_became_peered":"2024-09-17T03:26:21.277728+0000","last_unstale":"2024-09-17T03:26:21.286599+0000","last_undegraded":"2024-09-17T03:26:21.286599+0000","last_fullsized":"2024-09-17T03:26:21.286599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:38.173310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277895+0000","last_change":"2024-09-17T03:26:20.272850+0000","last_active":"2024-09-17T03:26:20.277895+0000","last_peered":"2024-09-17T03:26:20.277895+0000","last_clean":"2024-09-17T03:26:20.277895+0000","last_became_active":"2024-09-17T03:26:20.272036+0000","last_became_peered":"2024-09-17T03:26:20.272036+0000","last_unstale":"2024-09-17T03:26:20.277895+0000","last_undegraded":"2024-09-17T03:26:20.277895+0000","last_fullsized":"2024-09-17T03:26:20.277895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:25.846783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.297474+0000","last_change":"2024-09-17T03:26:21.294723+0000","last_active":"2024-09-17T03:26:21.297474+0000","last_peered":"2024-09-17T03:26:21.297474+0000","last_clean":"2024-09-17T03:26:21.297474+0000","last_became_active":"2024-09-17T03:26:21.294375+0000","last_became_peered":"2024-09-17T03:26:21.294375+0000","last_unstale":"2024-09-17T03:26:21.297474+0000","last_undegraded":"2024-09-17T03:26:21.297474+0000","last_fullsized":"2024-09-17T03:26:21.297474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:22.471193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.275453+0000","last_change":"2024-09-17T03:26:20.281457+0000","last_active":"2024-09-17T03:26:21.275453+0000","last_peered":"2024-09-17T03:26:21.275453+0000","last_clean":"2024-09-17T03:26:21.275453+0000","last_became_active":"2024-09-17T03:26:20.281078+0000","last_became_peered":"2024-09-17T03:26:20.281078+0000","last_unstale":"2024-09-17T03:26:21.275453+0000","last_undegraded":"2024-09-17T03:26:21.275453+0000","last_fullsized":"2024-09-17T03:26:21.275453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:22.720948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.272277+0000","last_change":"2024-09-17T03:26:20.272277+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.272277+0000","last_undegraded":"2024-09-17T03:26:20.272277+0000","last_fullsized":"2024-09-17T03:26:20.272277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.296826+0000","last_change":"2024-09-17T03:26:20.280006+0000","last_active":"2024-09-17T03:26:21.296826+0000","last_peered":"2024-09-17T03:26:21.296826+0000","last_clean":"2024-09-17T03:26:21.296826+0000","last_became_active":"2024-09-17T03:26:20.279611+0000","last_became_peered":"2024-09-17T03:26:20.279611+0000","last_unstale":"2024-09-17T03:26:21.296826+0000","last_undegraded":"2024-09-17T03:26:21.296826+0000","last_fullsized":"2024-09-17T03:26:21.296826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:29.773567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289195+0000","last_change":"2024-09-17T03:26:21.284722+0000","last_active":"2024-09-17T03:26:21.289195+0000","last_peered":"2024-09-17T03:26:21.289195+0000","last_clean":"2024-09-17T03:26:21.289195+0000","last_became_active":"2024-09-17T03:26:21.284399+0000","last_became_peered":"2024-09-17T03:26:21.284399+0000","last_unstale":"2024-09-17T03:26:21.289195+0000","last_undegraded":"2024-09-17T03:26:21.289195+0000","last_fullsized":"2024-09-17T03:26:21.289195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:55.320737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277267+0000","last_change":"2024-09-17T03:26:20.274933+0000","last_active":"2024-09-17T03:26:20.277267+0000","last_peered":"2024-09-17T03:26:20.277267+0000","last_clean":"2024-09-17T03:26:20.277267+0000","last_became_active":"2024-09-17T03:26:20.274536+0000","last_became_peered":"2024-09-17T03:26:20.274536+0000","last_unstale":"2024-09-17T03:26:20.277267+0000","last_undegraded":"2024-09-17T03:26:20.277267+0000","last_fullsized":"2024-09-17T03:26:20.277267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:23.329231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.280742+0000","last_change":"2024-09-17T03:26:21.277208+0000","last_active":"2024-09-17T03:26:21.280742+0000","last_peered":"2024-09-17T03:26:21.280742+0000","last_clean":"2024-09-17T03:26:21.280742+0000","last_became_active":"2024-09-17T03:26:21.276812+0000","last_became_peered":"2024-09-17T03:26:21.276812+0000","last_unstale":"2024-09-17T03:26:21.280742+0000","last_undegraded":"2024-09-17T03:26:21.280742+0000","last_fullsized":"2024-09-17T03:26:21.280742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:12.678429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.284384+0000","last_change":"2024-09-17T03:26:20.281254+0000","last_active":"2024-09-17T03:26:20.284384+0000","last_peered":"2024-09-17T03:26:20.284384+0000","last_clean":"2024-09-17T03:26:20.284384+0000","last_became_active":"2024-09-17T03:26:20.280858+0000","last_became_peered":"2024-09-17T03:26:20.280858+0000","last_unstale":"2024-09-17T03:26:20.284384+0000","last_undegraded":"2024-09-17T03:26:20.284384+0000","last_fullsized":"2024-09-17T03:26:20.284384+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:22.288031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.274734+0000","last_change":"2024-09-17T03:26:20.274734+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.274734+0000","last_undegraded":"2024-09-17T03:26:20.274734+0000","last_fullsized":"2024-09-17T03:26:20.274734+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277412+0000","last_change":"2024-09-17T03:26:20.272578+0000","last_active":"2024-09-17T03:26:20.277412+0000","last_peered":"2024-09-17T03:26:20.277412+0000","last_clean":"2024-09-17T03:26:20.277412+0000","last_became_active":"2024-09-17T03:26:20.272215+0000","last_became_peered":"2024-09-17T03:26:20.272215+0000","last_unstale":"2024-09-17T03:26:20.277412+0000","last_undegraded":"2024-09-17T03:26:20.277412+0000","last_fullsized":"2024-09-17T03:26:20.277412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:56.751763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.273561+0000","last_change":"2024-09-17T03:26:20.273561+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.273561+0000","last_undegraded":"2024-09-17T03:26:20.273561+0000","last_fullsized":"2024-09-17T03:26:20.273561+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.276867+0000","last_change":"2024-09-17T03:26:20.281152+0000","last_active":"2024-09-17T03:26:21.276867+0000","last_peered":"2024-09-17T03:26:21.276867+0000","last_clean":"2024-09-17T03:26:21.276867+0000","last_became_active":"2024-09-17T03:26:20.280782+0000","last_became_peered":"2024-09-17T03:26:20.280782+0000","last_unstale":"2024-09-17T03:26:21.276867+0000","last_undegraded":"2024-09-17T03:26:21.276867+0000","last_fullsized":"2024-09-17T03:26:21.276867+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:08.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.262545+0000","last_change":"2024-09-17T03:26:20.262545+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.262545+0000","last_undegraded":"2024-09-17T03:26:20.262545+0000","last_fullsized":"2024-09-17T03:26:20.262545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267915+0000","last_change":"2024-09-17T03:26:20.277793+0000","last_active":"2024-09-17T03:26:21.267915+0000","last_peered":"2024-09-17T03:26:21.267915+0000","last_clean":"2024-09-17T03:26:21.267915+0000","last_became_active":"2024-09-17T03:26:20.276428+0000","last_became_peered":"2024-09-17T03:26:20.276428+0000","last_unstale":"2024-09-17T03:26:21.267915+0000","last_undegraded":"2024-09-17T03:26:21.267915+0000","last_fullsized":"2024-09-17T03:26:21.267915+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289751+0000","last_change":"2024-09-17T03:26:21.283319+0000","last_active":"2024-09-17T03:26:21.289751+0000","last_peered":"2024-09-17T03:26:21.289751+0000","last_clean":"2024-09-17T03:26:21.289751+0000","last_became_active":"2024-09-17T03:26:21.282574+0000","last_became_peered":"2024-09-17T03:26:21.282574+0000","last_unstale":"2024-09-17T03:26:21.289751+0000","last_undegraded":"2024-09-17T03:26:21.289751+0000","last_fullsized":"2024-09-17T03:26:21.289751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:54.060063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.286863+0000","last_change":"2024-09-17T03:26:20.282834+0000","last_active":"2024-09-17T03:26:20.286863+0000","last_peered":"2024-09-17T03:26:20.286863+0000","last_clean":"2024-09-17T03:26:20.286863+0000","last_became_active":"2024-09-17T03:26:20.279265+0000","last_became_peered":"2024-09-17T03:26:20.279265+0000","last_unstale":"2024-09-17T03:26:20.286863+0000","last_undegraded":"2024-09-17T03:26:20.286863+0000","last_fullsized":"2024-09-17T03:26:20.286863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.512156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.293855+0000","last_change":"2024-09-17T03:26:21.290929+0000","last_active":"2024-09-17T03:26:21.293855+0000","last_peered":"2024-09-17T03:26:21.293855+0000","last_clean":"2024-09-17T03:26:21.293855+0000","last_became_active":"2024-09-17T03:26:21.290598+0000","last_became_peered":"2024-09-17T03:26:21.290598+0000","last_unstale":"2024-09-17T03:26:21.293855+0000","last_undegraded":"2024-09-17T03:26:21.293855+0000","last_fullsized":"2024-09-17T03:26:21.293855+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:48.278348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.277591+0000","last_change":"2024-09-17T03:26:20.275141+0000","last_active":"2024-09-17T03:26:20.277591+0000","last_peered":"2024-09-17T03:26:20.277591+0000","last_clean":"2024-09-17T03:26:20.277591+0000","last_became_active":"2024-09-17T03:26:20.274824+0000","last_became_peered":"2024-09-17T03:26:20.274824+0000","last_unstale":"2024-09-17T03:26:20.277591+0000","last_undegraded":"2024-09-17T03:26:20.277591+0000","last_fullsized":"2024-09-17T03:26:20.277591+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:00.828792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.289251+0000","last_change":"2024-09-17T03:26:21.285419+0000","last_active":"2024-09-17T03:26:21.289251+0000","last_peered":"2024-09-17T03:26:21.289251+0000","last_clean":"2024-09-17T03:26:21.289251+0000","last_became_active":"2024-09-17T03:26:21.284707+0000","last_became_peered":"2024-09-17T03:26:21.284707+0000","last_unstale":"2024-09-17T03:26:21.289251+0000","last_undegraded":"2024-09-17T03:26:21.289251+0000","last_fullsized":"2024-09-17T03:26:21.289251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:07:50.249027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.264570+0000","last_change":"2024-09-17T03:26:20.288747+0000","last_active":"2024-09-17T03:26:21.264570+0000","last_peered":"2024-09-17T03:26:21.264570+0000","last_clean":"2024-09-17T03:26:21.264570+0000","last_became_active":"2024-09-17T03:26:20.288347+0000","last_became_peered":"2024-09-17T03:26:20.288347+0000","last_unstale":"2024-09-17T03:26:21.264570+0000","last_undegraded":"2024-09-17T03:26:21.264570+0000","last_fullsized":"2024-09-17T03:26:21.264570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:46.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-17T03:26:20.261173+0000","last_change":"2024-09-17T03:26:20.261173+0000","last_active":"2024-09-17T03:26:20.242858+0000","last_peered":"2024-09-17T03:26:20.242858+0000","last_clean":"2024-09-17T03:26:20.242858+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:26:20.261173+0000","last_undegraded":"2024-09-17T03:26:20.261173+0000","last_fullsized":"2024-09-17T03:26:20.261173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-17T03:26:20.287028+0000","last_change":"2024-09-17T03:26:20.286225+0000","last_active":"2024-09-17T03:26:20.287028+0000","last_peered":"2024-09-17T03:26:20.287028+0000","last_clean":"2024-09-17T03:26:20.287028+0000","last_became_active":"2024-09-17T03:26:20.285812+0000","last_became_peered":"2024-09-17T03:26:20.285812+0000","last_unstale":"2024-09-17T03:26:20.287028+0000","last_undegraded":"2024-09-17T03:26:20.287028+0000","last_fullsized":"2024-09-17T03:26:20.287028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:14.007459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.286857+0000","last_change":"2024-09-17T03:26:21.280085+0000","last_active":"2024-09-17T03:26:21.286857+0000","last_peered":"2024-09-17T03:26:21.286857+0000","last_clean":"2024-09-17T03:26:21.286857+0000","last_became_active":"2024-09-17T03:26:21.279401+0000","last_became_peered":"2024-09-17T03:26:21.279401+0000","last_unstale":"2024-09-17T03:26:21.286857+0000","last_undegraded":"2024-09-17T03:26:21.286857+0000","last_fullsized":"2024-09-17T03:26:21.286857+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.738915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.274511+0000","last_change":"2024-09-17T03:26:20.271261+0000","last_active":"2024-09-17T03:26:21.274511+0000","last_peered":"2024-09-17T03:26:21.274511+0000","last_clean":"2024-09-17T03:26:21.274511+0000","last_became_active":"2024-09-17T03:26:20.270867+0000","last_became_peered":"2024-09-17T03:26:20.270867+0000","last_unstale":"2024-09-17T03:26:21.274511+0000","last_undegraded":"2024-09-17T03:26:21.274511+0000","last_fullsized":"2024-09-17T03:26:21.274511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:19.783352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.295189+0000","last_change":"2024-09-17T03:26:21.287202+0000","last_active":"2024-09-17T03:26:21.295189+0000","last_peered":"2024-09-17T03:26:21.295189+0000","last_clean":"2024-09-17T03:26:21.295189+0000","last_became_active":"2024-09-17T03:26:21.286886+0000","last_became_peered":"2024-09-17T03:26:21.286886+0000","last_unstale":"2024-09-17T03:26:21.295189+0000","last_undegraded":"2024-09-17T03:26:21.295189+0000","last_fullsized":"2024-09-17T03:26:21.295189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:51.956495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.267707+0000","last_change":"2024-09-17T03:26:20.272318+0000","last_active":"2024-09-17T03:26:21.267707+0000","last_peered":"2024-09-17T03:26:21.267707+0000","last_clean":"2024-09-17T03:26:21.267707+0000","last_became_active":"2024-09-17T03:26:20.270871+0000","last_became_peered":"2024-09-17T03:26:20.270871+0000","last_unstale":"2024-09-17T03:26:21.267707+0000","last_undegraded":"2024-09-17T03:26:21.267707+0000","last_fullsized":"2024-09-17T03:26:21.267707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:56.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T03:26:21.269003+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:26:21.269003+0000","last_peered":"2024-09-17T03:26:21.269003+0000","last_clean":"2024-09-17T03:26:21.269003+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:26:21.269003+0000","last_undegraded":"2024-09-17T03:26:21.269003+0000","last_fullsized":"2024-09-17T03:26:21.269003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58804,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313036,"statfs":{"total":96636764160,"available":96576548864,"internally_reserved":0,"allocated":1277952,"data_stored":870018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509508,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45740,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326100,"statfs":{"total":96636764160,"available":96589926400,"internally_reserved":0,"allocated":675840,"data_stored":279490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45748,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326092,"statfs":{"total":96636764160,"available":96589918208,"internally_reserved":0,"allocated":688128,"data_stored":281102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49908,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321932,"statfs":{"total":96636764160,"available":96585658368,"internally_reserved":0,"allocated":684032,"data_stored":279546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45728,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326112,"statfs":{"total":96636764160,"available":96589938688,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50468,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321372,"statfs":{"total":96636764160,"available":96585084928,"internally_reserved":0,"allocated":1257472,"data_stored":863083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":663552,"data_stored":272715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:26:29.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:26:29.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:26:30.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 -- 172.21.15.46:0/3766540391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c1527b0 msgr2=0x7f993c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 --2- 172.21.15.46:0/3766540391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c1527b0 0x7f993c15bb60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9924009900 tx=0x7f992401c6d0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 -- 172.21.15.46:0/3766540391 shutdown_connections 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 --2- 172.21.15.46:0/3766540391 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 0x7f993c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 --2- 172.21.15.46:0/3766540391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c1527b0 0x7f993c15bb60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 -- 172.21.15.46:0/3766540391 >> 172.21.15.46:0/3766540391 conn(0x7f993c0b2040 msgr2=0x7f993c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 -- 172.21.15.46:0/3766540391 shutdown_connections 2024-09-17T03:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.331+0000 7f9941600640 1 -- 172.21.15.46:0/3766540391 wait complete. 2024-09-17T03:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 Processor -- start 2024-09-17T03:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 -- start start 2024-09-17T03:26:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 0x7f993c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 0x7f993c145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f993c1527b0 0x7f993c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f993c168a20 con 0x7f993c0b5c70 2024-09-17T03:26:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1688a0 con 0x7f993c1527b0 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f993c168ba0 con 0x7f993c0b4de0 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 0x7f993c145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 0x7f993c145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50040/0 (socket says 172.21.15.46:50040) 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 -- 172.21.15.46:0/1363432145 learned_addr learned my addr 172.21.15.46:0/1363432145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f993aa00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 0x7f993c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f993b400640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f993c1527b0 0x7f993c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 -- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 msgr2=0x7f993c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 0x7f993c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 -- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f993c1527b0 msgr2=0x7f993c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f993c1527b0 0x7f993c159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 -- 172.21.15.46:0/1363432145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c15aa90 con 0x7f993c0b5c70 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f993aa00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 0x7f993c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9933e00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 0x7f993c145b30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9924009850 tx=0x7f99240a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240ba020 con 0x7f993c0b5c70 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9924004430 con 0x7f993c0b5c70 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240b2040 con 0x7f993c0b5c70 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c0ce3c0 con 0x7f993c0b5c70 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.335+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c0ce980 con 0x7f993c0b5c70 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.339+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c0af5d0 con 0x7f993c0b5c70 2024-09-17T03:26:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.339+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f99240bd070 con 0x7f993c0b5c70 2024-09-17T03:26:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.339+0000 7f9932a00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9918073b00 0x7f9918076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.339+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f992412c270 con 0x7f993c0b5c70 2024-09-17T03:26:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.339+0000 7f993aa00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9918073b00 0x7f9918076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.339+0000 7f993aa00640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9918073b00 0x7f9918076010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098b0 tx=0x7f992c008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.343+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f993c15aa90 con 0x7f993c0b5c70 2024-09-17T03:26:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.519+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f993c06b540 con 0x7f9918073b00 2024-09-17T03:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.523+0000 7f9932a00640 1 -- 172.21.15.46:0/1363432145 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251908 (secure 0 0 0) 0x7f992c0094a0 con 0x7f9918073b00 2024-09-17T03:26:30.528 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:30.529 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9918073b00 msgr2=0x7f9918076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9918073b00 0x7f9918076010 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098b0 tx=0x7f992c008040 comp rx=0 tx=0).stop 2024-09-17T03:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 msgr2=0x7f993c145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 0x7f993c145b30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9924009850 tx=0x7f99240a5620 comp rx=0 tx=0).stop 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 shutdown_connections 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9918073b00 0x7f9918076010 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f993c1527b0 0x7f993c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f993c0b5c70 0x7f993c145b30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 --2- 172.21.15.46:0/1363432145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f993c0b4de0 0x7f993c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 >> 172.21.15.46:0/1363432145 conn(0x7f993c0b2040 msgr2=0x7f993c154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 shutdown_connections 2024-09-17T03:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.527+0000 7f9941600640 1 -- 172.21.15.46:0/1363432145 wait complete. 2024-09-17T03:26:30.550 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-17T03:26:29.304266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386636,"kb_used_data":6600,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754588084,"statfs":{"total":773094113280,"available":772698198016,"internally_reserved":0,"allocated":6758400,"data_stored":3471876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.050312"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.371223+0000","last_change":"2024-09-17T03:26:21.271619+0000","last_active":"2024-09-17T03:26:22.371223+0000","last_peered":"2024-09-17T03:26:22.371223+0000","last_clean":"2024-09-17T03:26:22.371223+0000","last_became_active":"2024-09-17T03:26:21.270472+0000","last_became_peered":"2024-09-17T03:26:21.270472+0000","last_unstale":"2024-09-17T03:26:22.371223+0000","last_undegraded":"2024-09-17T03:26:22.371223+0000","last_fullsized":"2024-09-17T03:26:22.371223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:18.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280233+0000","last_change":"2024-09-17T03:26:21.281731+0000","last_active":"2024-09-17T03:26:22.280233+0000","last_peered":"2024-09-17T03:26:22.280233+0000","last_clean":"2024-09-17T03:26:22.280233+0000","last_became_active":"2024-09-17T03:26:21.281402+0000","last_became_peered":"2024-09-17T03:26:21.281402+0000","last_unstale":"2024-09-17T03:26:22.280233+0000","last_undegraded":"2024-09-17T03:26:22.280233+0000","last_fullsized":"2024-09-17T03:26:22.280233+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:16.952858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581480+0000","last_change":"2024-09-17T03:26:21.283316+0000","last_active":"2024-09-17T03:26:22.581480+0000","last_peered":"2024-09-17T03:26:22.581480+0000","last_clean":"2024-09-17T03:26:22.581480+0000","last_became_active":"2024-09-17T03:26:21.282999+0000","last_became_peered":"2024-09-17T03:26:21.282999+0000","last_unstale":"2024-09-17T03:26:22.581480+0000","last_undegraded":"2024-09-17T03:26:22.581480+0000","last_fullsized":"2024-09-17T03:26:22.581480+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:11.693379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582572+0000","last_change":"2024-09-17T03:26:21.280152+0000","last_active":"2024-09-17T03:26:22.582572+0000","last_peered":"2024-09-17T03:26:22.582572+0000","last_clean":"2024-09-17T03:26:22.582572+0000","last_became_active":"2024-09-17T03:26:21.279716+0000","last_became_peered":"2024-09-17T03:26:21.279716+0000","last_unstale":"2024-09-17T03:26:22.582572+0000","last_undegraded":"2024-09-17T03:26:22.582572+0000","last_fullsized":"2024-09-17T03:26:22.582572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:15.070983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.370954+0000","last_change":"2024-09-17T03:26:21.284187+0000","last_active":"2024-09-17T03:26:22.370954+0000","last_peered":"2024-09-17T03:26:22.370954+0000","last_clean":"2024-09-17T03:26:22.370954+0000","last_became_active":"2024-09-17T03:26:21.283779+0000","last_became_peered":"2024-09-17T03:26:21.283779+0000","last_unstale":"2024-09-17T03:26:22.370954+0000","last_undegraded":"2024-09-17T03:26:22.370954+0000","last_fullsized":"2024-09-17T03:26:22.370954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:47.091690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575202+0000","last_change":"2024-09-17T03:26:21.296425+0000","last_active":"2024-09-17T03:26:22.575202+0000","last_peered":"2024-09-17T03:26:22.575202+0000","last_clean":"2024-09-17T03:26:22.575202+0000","last_became_active":"2024-09-17T03:26:21.296063+0000","last_became_peered":"2024-09-17T03:26:21.296063+0000","last_unstale":"2024-09-17T03:26:22.575202+0000","last_undegraded":"2024-09-17T03:26:22.575202+0000","last_fullsized":"2024-09-17T03:26:22.575202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:58.692959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.284627+0000","last_change":"2024-09-17T03:26:21.281715+0000","last_active":"2024-09-17T03:26:22.284627+0000","last_peered":"2024-09-17T03:26:22.284627+0000","last_clean":"2024-09-17T03:26:22.284627+0000","last_became_active":"2024-09-17T03:26:21.280618+0000","last_became_peered":"2024-09-17T03:26:21.280618+0000","last_unstale":"2024-09-17T03:26:22.284627+0000","last_undegraded":"2024-09-17T03:26:22.284627+0000","last_fullsized":"2024-09-17T03:26:22.284627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:11.578388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279884+0000","last_change":"2024-09-17T03:26:21.277710+0000","last_active":"2024-09-17T03:26:22.279884+0000","last_peered":"2024-09-17T03:26:22.279884+0000","last_clean":"2024-09-17T03:26:22.279884+0000","last_became_active":"2024-09-17T03:26:21.277333+0000","last_became_peered":"2024-09-17T03:26:21.277333+0000","last_unstale":"2024-09-17T03:26:22.279884+0000","last_undegraded":"2024-09-17T03:26:22.279884+0000","last_fullsized":"2024-09-17T03:26:22.279884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:45.165886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.285050+0000","last_change":"2024-09-17T03:26:21.290497+0000","last_active":"2024-09-17T03:26:22.285050+0000","last_peered":"2024-09-17T03:26:22.285050+0000","last_clean":"2024-09-17T03:26:22.285050+0000","last_became_active":"2024-09-17T03:26:21.289289+0000","last_became_peered":"2024-09-17T03:26:21.289289+0000","last_unstale":"2024-09-17T03:26:22.285050+0000","last_undegraded":"2024-09-17T03:26:22.285050+0000","last_fullsized":"2024-09-17T03:26:22.285050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:27.209668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.277145+0000","last_change":"2024-09-17T03:26:21.280576+0000","last_active":"2024-09-17T03:26:22.277145+0000","last_peered":"2024-09-17T03:26:22.277145+0000","last_clean":"2024-09-17T03:26:22.277145+0000","last_became_active":"2024-09-17T03:26:21.280140+0000","last_became_peered":"2024-09-17T03:26:21.280140+0000","last_unstale":"2024-09-17T03:26:22.277145+0000","last_undegraded":"2024-09-17T03:26:22.277145+0000","last_fullsized":"2024-09-17T03:26:22.277145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:38.539682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.277120+0000","last_change":"2024-09-17T03:26:21.298246+0000","last_active":"2024-09-17T03:26:22.277120+0000","last_peered":"2024-09-17T03:26:22.277120+0000","last_clean":"2024-09-17T03:26:22.277120+0000","last_became_active":"2024-09-17T03:26:21.297872+0000","last_became_peered":"2024-09-17T03:26:21.297872+0000","last_unstale":"2024-09-17T03:26:22.277120+0000","last_undegraded":"2024-09-17T03:26:22.277120+0000","last_fullsized":"2024-09-17T03:26:22.277120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:34.600999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582416+0000","last_change":"2024-09-17T03:26:21.286735+0000","last_active":"2024-09-17T03:26:22.582416+0000","last_peered":"2024-09-17T03:26:22.582416+0000","last_clean":"2024-09-17T03:26:22.582416+0000","last_became_active":"2024-09-17T03:26:21.283746+0000","last_became_peered":"2024-09-17T03:26:21.283746+0000","last_unstale":"2024-09-17T03:26:22.582416+0000","last_undegraded":"2024-09-17T03:26:22.582416+0000","last_fullsized":"2024-09-17T03:26:22.582416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:47:46.299653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.334007+0000","last_change":"2024-09-17T03:26:21.300221+0000","last_active":"2024-09-17T03:26:22.334007+0000","last_peered":"2024-09-17T03:26:22.334007+0000","last_clean":"2024-09-17T03:26:22.334007+0000","last_became_active":"2024-09-17T03:26:21.299864+0000","last_became_peered":"2024-09-17T03:26:21.299864+0000","last_unstale":"2024-09-17T03:26:22.334007+0000","last_undegraded":"2024-09-17T03:26:22.334007+0000","last_fullsized":"2024-09-17T03:26:22.334007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:11.984599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.284834+0000","last_change":"2024-09-17T03:26:21.274122+0000","last_active":"2024-09-17T03:26:22.284834+0000","last_peered":"2024-09-17T03:26:22.284834+0000","last_clean":"2024-09-17T03:26:22.284834+0000","last_became_active":"2024-09-17T03:26:21.273777+0000","last_became_peered":"2024-09-17T03:26:21.273777+0000","last_unstale":"2024-09-17T03:26:22.284834+0000","last_undegraded":"2024-09-17T03:26:22.284834+0000","last_fullsized":"2024-09-17T03:26:22.284834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:44.896419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.284982+0000","last_change":"2024-09-17T03:26:21.292845+0000","last_active":"2024-09-17T03:26:22.284982+0000","last_peered":"2024-09-17T03:26:22.284982+0000","last_clean":"2024-09-17T03:26:22.284982+0000","last_became_active":"2024-09-17T03:26:21.292432+0000","last_became_peered":"2024-09-17T03:26:21.292432+0000","last_unstale":"2024-09-17T03:26:22.284982+0000","last_undegraded":"2024-09-17T03:26:22.284982+0000","last_fullsized":"2024-09-17T03:26:22.284982+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:32.373126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575125+0000","last_change":"2024-09-17T03:26:21.282672+0000","last_active":"2024-09-17T03:26:22.575125+0000","last_peered":"2024-09-17T03:26:22.575125+0000","last_clean":"2024-09-17T03:26:22.575125+0000","last_became_active":"2024-09-17T03:26:21.282359+0000","last_became_peered":"2024-09-17T03:26:21.282359+0000","last_unstale":"2024-09-17T03:26:22.575125+0000","last_undegraded":"2024-09-17T03:26:22.575125+0000","last_fullsized":"2024-09-17T03:26:22.575125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:24.797352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280467+0000","last_change":"2024-09-17T03:26:21.277694+0000","last_active":"2024-09-17T03:26:22.280467+0000","last_peered":"2024-09-17T03:26:22.280467+0000","last_clean":"2024-09-17T03:26:22.280467+0000","last_became_active":"2024-09-17T03:26:21.277341+0000","last_became_peered":"2024-09-17T03:26:21.277341+0000","last_unstale":"2024-09-17T03:26:22.280467+0000","last_undegraded":"2024-09-17T03:26:22.280467+0000","last_fullsized":"2024-09-17T03:26:22.280467+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:52.596688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581820+0000","last_change":"2024-09-17T03:26:21.289536+0000","last_active":"2024-09-17T03:26:22.581820+0000","last_peered":"2024-09-17T03:26:22.581820+0000","last_clean":"2024-09-17T03:26:22.581820+0000","last_became_active":"2024-09-17T03:26:21.289230+0000","last_became_peered":"2024-09-17T03:26:21.289230+0000","last_unstale":"2024-09-17T03:26:22.581820+0000","last_undegraded":"2024-09-17T03:26:22.581820+0000","last_fullsized":"2024-09-17T03:26:22.581820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:27.881549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.371068+0000","last_change":"2024-09-17T03:26:21.272001+0000","last_active":"2024-09-17T03:26:22.371068+0000","last_peered":"2024-09-17T03:26:22.371068+0000","last_clean":"2024-09-17T03:26:22.371068+0000","last_became_active":"2024-09-17T03:26:21.271673+0000","last_became_peered":"2024-09-17T03:26:21.271673+0000","last_unstale":"2024-09-17T03:26:22.371068+0000","last_undegraded":"2024-09-17T03:26:22.371068+0000","last_fullsized":"2024-09-17T03:26:22.371068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:37.818418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279895+0000","last_change":"2024-09-17T03:26:21.291765+0000","last_active":"2024-09-17T03:26:22.279895+0000","last_peered":"2024-09-17T03:26:22.279895+0000","last_clean":"2024-09-17T03:26:22.279895+0000","last_became_active":"2024-09-17T03:26:21.291430+0000","last_became_peered":"2024-09-17T03:26:21.291430+0000","last_unstale":"2024-09-17T03:26:22.279895+0000","last_undegraded":"2024-09-17T03:26:22.279895+0000","last_fullsized":"2024-09-17T03:26:22.279895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:58.174321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581207+0000","last_change":"2024-09-17T03:26:21.285773+0000","last_active":"2024-09-17T03:26:22.581207+0000","last_peered":"2024-09-17T03:26:22.581207+0000","last_clean":"2024-09-17T03:26:22.581207+0000","last_became_active":"2024-09-17T03:26:21.285118+0000","last_became_peered":"2024-09-17T03:26:21.285118+0000","last_unstale":"2024-09-17T03:26:22.581207+0000","last_undegraded":"2024-09-17T03:26:22.581207+0000","last_fullsized":"2024-09-17T03:26:22.581207+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:08.367097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279603+0000","last_change":"2024-09-17T03:26:21.276516+0000","last_active":"2024-09-17T03:26:22.279603+0000","last_peered":"2024-09-17T03:26:22.279603+0000","last_clean":"2024-09-17T03:26:22.279603+0000","last_became_active":"2024-09-17T03:26:21.275398+0000","last_became_peered":"2024-09-17T03:26:21.275398+0000","last_unstale":"2024-09-17T03:26:22.279603+0000","last_undegraded":"2024-09-17T03:26:22.279603+0000","last_fullsized":"2024-09-17T03:26:22.279603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:27.257636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581036+0000","last_change":"2024-09-17T03:26:21.286047+0000","last_active":"2024-09-17T03:26:22.581036+0000","last_peered":"2024-09-17T03:26:22.581036+0000","last_clean":"2024-09-17T03:26:22.581036+0000","last_became_active":"2024-09-17T03:26:21.284398+0000","last_became_peered":"2024-09-17T03:26:21.284398+0000","last_unstale":"2024-09-17T03:26:22.581036+0000","last_undegraded":"2024-09-17T03:26:22.581036+0000","last_fullsized":"2024-09-17T03:26:22.581036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:06.391945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.333901+0000","last_change":"2024-09-17T03:26:21.289586+0000","last_active":"2024-09-17T03:26:22.333901+0000","last_peered":"2024-09-17T03:26:22.333901+0000","last_clean":"2024-09-17T03:26:22.333901+0000","last_became_active":"2024-09-17T03:26:21.289242+0000","last_became_peered":"2024-09-17T03:26:21.289242+0000","last_unstale":"2024-09-17T03:26:22.333901+0000","last_undegraded":"2024-09-17T03:26:22.333901+0000","last_fullsized":"2024-09-17T03:26:22.333901+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:37.143275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.371482+0000","last_change":"2024-09-17T03:26:21.287754+0000","last_active":"2024-09-17T03:26:22.371482+0000","last_peered":"2024-09-17T03:26:22.371482+0000","last_clean":"2024-09-17T03:26:22.371482+0000","last_became_active":"2024-09-17T03:26:21.287382+0000","last_became_peered":"2024-09-17T03:26:21.287382+0000","last_unstale":"2024-09-17T03:26:22.371482+0000","last_undegraded":"2024-09-17T03:26:22.371482+0000","last_fullsized":"2024-09-17T03:26:22.371482+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:39.242139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582096+0000","last_change":"2024-09-17T03:26:21.284316+0000","last_active":"2024-09-17T03:26:22.582096+0000","last_peered":"2024-09-17T03:26:22.582096+0000","last_clean":"2024-09-17T03:26:22.582096+0000","last_became_active":"2024-09-17T03:26:21.283985+0000","last_became_peered":"2024-09-17T03:26:21.283985+0000","last_unstale":"2024-09-17T03:26:22.582096+0000","last_undegraded":"2024-09-17T03:26:22.582096+0000","last_fullsized":"2024-09-17T03:26:22.582096+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:17.680277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575599+0000","last_change":"2024-09-17T03:26:21.282975+0000","last_active":"2024-09-17T03:26:22.575599+0000","last_peered":"2024-09-17T03:26:22.575599+0000","last_clean":"2024-09-17T03:26:22.575599+0000","last_became_active":"2024-09-17T03:26:21.282116+0000","last_became_peered":"2024-09-17T03:26:21.282116+0000","last_unstale":"2024-09-17T03:26:22.575599+0000","last_undegraded":"2024-09-17T03:26:22.575599+0000","last_fullsized":"2024-09-17T03:26:22.575599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:06.795106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.371260+0000","last_change":"2024-09-17T03:26:21.287660+0000","last_active":"2024-09-17T03:26:22.371260+0000","last_peered":"2024-09-17T03:26:22.371260+0000","last_clean":"2024-09-17T03:26:22.371260+0000","last_became_active":"2024-09-17T03:26:21.287329+0000","last_became_peered":"2024-09-17T03:26:21.287329+0000","last_unstale":"2024-09-17T03:26:22.371260+0000","last_undegraded":"2024-09-17T03:26:22.371260+0000","last_fullsized":"2024-09-17T03:26:22.371260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:00.850321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280051+0000","last_change":"2024-09-17T03:26:21.281015+0000","last_active":"2024-09-17T03:26:22.280051+0000","last_peered":"2024-09-17T03:26:22.280051+0000","last_clean":"2024-09-17T03:26:22.280051+0000","last_became_active":"2024-09-17T03:26:21.280690+0000","last_became_peered":"2024-09-17T03:26:21.280690+0000","last_unstale":"2024-09-17T03:26:22.280051+0000","last_undegraded":"2024-09-17T03:26:22.280051+0000","last_fullsized":"2024-09-17T03:26:22.280051+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:52.397861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581041+0000","last_change":"2024-09-17T03:26:21.282370+0000","last_active":"2024-09-17T03:26:22.581041+0000","last_peered":"2024-09-17T03:26:22.581041+0000","last_clean":"2024-09-17T03:26:22.581041+0000","last_became_active":"2024-09-17T03:26:21.282053+0000","last_became_peered":"2024-09-17T03:26:21.282053+0000","last_unstale":"2024-09-17T03:26:22.581041+0000","last_undegraded":"2024-09-17T03:26:22.581041+0000","last_fullsized":"2024-09-17T03:26:22.581041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:48.024867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.370881+0000","last_change":"2024-09-17T03:26:21.293642+0000","last_active":"2024-09-17T03:26:22.370881+0000","last_peered":"2024-09-17T03:26:22.370881+0000","last_clean":"2024-09-17T03:26:22.370881+0000","last_became_active":"2024-09-17T03:26:21.289857+0000","last_became_peered":"2024-09-17T03:26:21.289857+0000","last_unstale":"2024-09-17T03:26:22.370881+0000","last_undegraded":"2024-09-17T03:26:22.370881+0000","last_fullsized":"2024-09-17T03:26:22.370881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:22.392069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575174+0000","last_change":"2024-09-17T03:26:21.278899+0000","last_active":"2024-09-17T03:26:22.575174+0000","last_peered":"2024-09-17T03:26:22.575174+0000","last_clean":"2024-09-17T03:26:22.575174+0000","last_became_active":"2024-09-17T03:26:21.278562+0000","last_became_peered":"2024-09-17T03:26:21.278562+0000","last_unstale":"2024-09-17T03:26:22.575174+0000","last_undegraded":"2024-09-17T03:26:22.575174+0000","last_fullsized":"2024-09-17T03:26:22.575174+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:34.691840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575668+0000","last_change":"2024-09-17T03:26:21.279072+0000","last_active":"2024-09-17T03:26:22.575668+0000","last_peered":"2024-09-17T03:26:22.575668+0000","last_clean":"2024-09-17T03:26:22.575668+0000","last_became_active":"2024-09-17T03:26:21.278730+0000","last_became_peered":"2024-09-17T03:26:21.278730+0000","last_unstale":"2024-09-17T03:26:22.575668+0000","last_undegraded":"2024-09-17T03:26:22.575668+0000","last_fullsized":"2024-09-17T03:26:22.575668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:00.154108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":19,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575901+0000","last_change":"2024-09-17T03:26:20.283656+0000","last_active":"2024-09-17T03:26:22.575901+0000","last_peered":"2024-09-17T03:26:22.575901+0000","last_clean":"2024-09-17T03:26:22.575901+0000","last_became_active":"2024-09-17T03:26:20.283331+0000","last_became_peered":"2024-09-17T03:26:20.283331+0000","last_unstale":"2024-09-17T03:26:22.575901+0000","last_undegraded":"2024-09-17T03:26:22.575901+0000","last_fullsized":"2024-09-17T03:26:22.575901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:03.041707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.277382+0000","last_change":"2024-09-17T03:26:21.283530+0000","last_active":"2024-09-17T03:26:22.277382+0000","last_peered":"2024-09-17T03:26:22.277382+0000","last_clean":"2024-09-17T03:26:22.277382+0000","last_became_active":"2024-09-17T03:26:21.283144+0000","last_became_peered":"2024-09-17T03:26:21.283144+0000","last_unstale":"2024-09-17T03:26:22.277382+0000","last_undegraded":"2024-09-17T03:26:22.277382+0000","last_fullsized":"2024-09-17T03:26:22.277382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:23.291431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280208+0000","last_change":"2024-09-17T03:26:20.266917+0000","last_active":"2024-09-17T03:26:22.280208+0000","last_peered":"2024-09-17T03:26:22.280208+0000","last_clean":"2024-09-17T03:26:22.280208+0000","last_became_active":"2024-09-17T03:26:20.266370+0000","last_became_peered":"2024-09-17T03:26:20.266370+0000","last_unstale":"2024-09-17T03:26:22.280208+0000","last_undegraded":"2024-09-17T03:26:22.280208+0000","last_fullsized":"2024-09-17T03:26:22.280208+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:57.609874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575487+0000","last_change":"2024-09-17T03:26:21.299198+0000","last_active":"2024-09-17T03:26:22.575487+0000","last_peered":"2024-09-17T03:26:22.575487+0000","last_clean":"2024-09-17T03:26:22.575487+0000","last_became_active":"2024-09-17T03:26:21.298717+0000","last_became_peered":"2024-09-17T03:26:21.298717+0000","last_unstale":"2024-09-17T03:26:22.575487+0000","last_undegraded":"2024-09-17T03:26:22.575487+0000","last_fullsized":"2024-09-17T03:26:22.575487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:41.325766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575791+0000","last_change":"2024-09-17T03:26:20.286649+0000","last_active":"2024-09-17T03:26:22.575791+0000","last_peered":"2024-09-17T03:26:22.575791+0000","last_clean":"2024-09-17T03:26:22.575791+0000","last_became_active":"2024-09-17T03:26:20.285886+0000","last_became_peered":"2024-09-17T03:26:20.285886+0000","last_unstale":"2024-09-17T03:26:22.575791+0000","last_undegraded":"2024-09-17T03:26:22.575791+0000","last_fullsized":"2024-09-17T03:26:22.575791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:36:46.887782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575611+0000","last_change":"2024-09-17T03:26:21.289923+0000","last_active":"2024-09-17T03:26:22.575611+0000","last_peered":"2024-09-17T03:26:22.575611+0000","last_clean":"2024-09-17T03:26:22.575611+0000","last_became_active":"2024-09-17T03:26:21.289556+0000","last_became_peered":"2024-09-17T03:26:21.289556+0000","last_unstale":"2024-09-17T03:26:22.575611+0000","last_undegraded":"2024-09-17T03:26:22.575611+0000","last_fullsized":"2024-09-17T03:26:22.575611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:59.520924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582765+0000","last_change":"2024-09-17T03:26:20.283187+0000","last_active":"2024-09-17T03:26:22.582765+0000","last_peered":"2024-09-17T03:26:22.582765+0000","last_clean":"2024-09-17T03:26:22.582765+0000","last_became_active":"2024-09-17T03:26:20.282354+0000","last_became_peered":"2024-09-17T03:26:20.282354+0000","last_unstale":"2024-09-17T03:26:22.582765+0000","last_undegraded":"2024-09-17T03:26:22.582765+0000","last_fullsized":"2024-09-17T03:26:22.582765+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:41.618275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279904+0000","last_change":"2024-09-17T03:26:21.287275+0000","last_active":"2024-09-17T03:26:22.279904+0000","last_peered":"2024-09-17T03:26:22.279904+0000","last_clean":"2024-09-17T03:26:22.279904+0000","last_became_active":"2024-09-17T03:26:21.286995+0000","last_became_peered":"2024-09-17T03:26:21.286995+0000","last_unstale":"2024-09-17T03:26:22.279904+0000","last_undegraded":"2024-09-17T03:26:22.279904+0000","last_fullsized":"2024-09-17T03:26:22.279904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:24.082977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.284857+0000","last_change":"2024-09-17T03:26:20.283433+0000","last_active":"2024-09-17T03:26:22.284857+0000","last_peered":"2024-09-17T03:26:22.284857+0000","last_clean":"2024-09-17T03:26:22.284857+0000","last_became_active":"2024-09-17T03:26:20.277797+0000","last_became_peered":"2024-09-17T03:26:20.277797+0000","last_unstale":"2024-09-17T03:26:22.284857+0000","last_undegraded":"2024-09-17T03:26:22.284857+0000","last_fullsized":"2024-09-17T03:26:22.284857+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279809+0000","last_change":"2024-09-17T03:26:21.293180+0000","last_active":"2024-09-17T03:26:22.279809+0000","last_peered":"2024-09-17T03:26:22.279809+0000","last_clean":"2024-09-17T03:26:22.279809+0000","last_became_active":"2024-09-17T03:26:21.292882+0000","last_became_peered":"2024-09-17T03:26:21.292882+0000","last_unstale":"2024-09-17T03:26:22.279809+0000","last_undegraded":"2024-09-17T03:26:22.279809+0000","last_fullsized":"2024-09-17T03:26:22.279809+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:30.711551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575217+0000","last_change":"2024-09-17T03:26:20.279365+0000","last_active":"2024-09-17T03:26:22.575217+0000","last_peered":"2024-09-17T03:26:22.575217+0000","last_clean":"2024-09-17T03:26:22.575217+0000","last_became_active":"2024-09-17T03:26:20.278923+0000","last_became_peered":"2024-09-17T03:26:20.278923+0000","last_unstale":"2024-09-17T03:26:22.575217+0000","last_undegraded":"2024-09-17T03:26:22.575217+0000","last_fullsized":"2024-09-17T03:26:22.575217+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:14.310020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.333764+0000","last_change":"2024-09-17T03:26:21.285430+0000","last_active":"2024-09-17T03:26:22.333764+0000","last_peered":"2024-09-17T03:26:22.333764+0000","last_clean":"2024-09-17T03:26:22.333764+0000","last_became_active":"2024-09-17T03:26:21.285111+0000","last_became_peered":"2024-09-17T03:26:21.285111+0000","last_unstale":"2024-09-17T03:26:22.333764+0000","last_undegraded":"2024-09-17T03:26:22.333764+0000","last_fullsized":"2024-09-17T03:26:22.333764+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:30.028880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581559+0000","last_change":"2024-09-17T03:26:20.279560+0000","last_active":"2024-09-17T03:26:22.581559+0000","last_peered":"2024-09-17T03:26:22.581559+0000","last_clean":"2024-09-17T03:26:22.581559+0000","last_became_active":"2024-09-17T03:26:20.278746+0000","last_became_peered":"2024-09-17T03:26:20.278746+0000","last_unstale":"2024-09-17T03:26:22.581559+0000","last_undegraded":"2024-09-17T03:26:22.581559+0000","last_fullsized":"2024-09-17T03:26:22.581559+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:02.254459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279663+0000","last_change":"2024-09-17T03:26:21.279707+0000","last_active":"2024-09-17T03:26:22.279663+0000","last_peered":"2024-09-17T03:26:22.279663+0000","last_clean":"2024-09-17T03:26:22.279663+0000","last_became_active":"2024-09-17T03:26:21.279020+0000","last_became_peered":"2024-09-17T03:26:21.279020+0000","last_unstale":"2024-09-17T03:26:22.279663+0000","last_undegraded":"2024-09-17T03:26:22.279663+0000","last_fullsized":"2024-09-17T03:26:22.279663+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:51.703543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.284581+0000","last_change":"2024-09-17T03:26:20.283353+0000","last_active":"2024-09-17T03:26:22.284581+0000","last_peered":"2024-09-17T03:26:22.284581+0000","last_clean":"2024-09-17T03:26:22.284581+0000","last_became_active":"2024-09-17T03:26:20.283049+0000","last_became_peered":"2024-09-17T03:26:20.283049+0000","last_unstale":"2024-09-17T03:26:22.284581+0000","last_undegraded":"2024-09-17T03:26:22.284581+0000","last_fullsized":"2024-09-17T03:26:22.284581+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.285271+0000","last_change":"2024-09-17T03:26:21.290216+0000","last_active":"2024-09-17T03:26:22.285271+0000","last_peered":"2024-09-17T03:26:22.285271+0000","last_clean":"2024-09-17T03:26:22.285271+0000","last_became_active":"2024-09-17T03:26:21.288236+0000","last_became_peered":"2024-09-17T03:26:21.288236+0000","last_unstale":"2024-09-17T03:26:22.285271+0000","last_undegraded":"2024-09-17T03:26:22.285271+0000","last_fullsized":"2024-09-17T03:26:22.285271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:15.689667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582453+0000","last_change":"2024-09-17T03:26:20.289405+0000","last_active":"2024-09-17T03:26:22.582453+0000","last_peered":"2024-09-17T03:26:22.582453+0000","last_clean":"2024-09-17T03:26:22.582453+0000","last_became_active":"2024-09-17T03:26:20.288979+0000","last_became_peered":"2024-09-17T03:26:20.288979+0000","last_unstale":"2024-09-17T03:26:22.582453+0000","last_undegraded":"2024-09-17T03:26:22.582453+0000","last_fullsized":"2024-09-17T03:26:22.582453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:10.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582389+0000","last_change":"2024-09-17T03:26:21.301608+0000","last_active":"2024-09-17T03:26:22.582389+0000","last_peered":"2024-09-17T03:26:22.582389+0000","last_clean":"2024-09-17T03:26:22.582389+0000","last_became_active":"2024-09-17T03:26:21.301203+0000","last_became_peered":"2024-09-17T03:26:21.301203+0000","last_unstale":"2024-09-17T03:26:22.582389+0000","last_undegraded":"2024-09-17T03:26:22.582389+0000","last_fullsized":"2024-09-17T03:26:22.582389+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:34.691840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280311+0000","last_change":"2024-09-17T03:26:20.263118+0000","last_active":"2024-09-17T03:26:22.280311+0000","last_peered":"2024-09-17T03:26:22.280311+0000","last_clean":"2024-09-17T03:26:22.280311+0000","last_became_active":"2024-09-17T03:26:20.262773+0000","last_became_peered":"2024-09-17T03:26:20.262773+0000","last_unstale":"2024-09-17T03:26:22.280311+0000","last_undegraded":"2024-09-17T03:26:22.280311+0000","last_fullsized":"2024-09-17T03:26:22.280311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:02.639519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.276802+0000","last_change":"2024-09-17T03:26:21.283319+0000","last_active":"2024-09-17T03:26:22.276802+0000","last_peered":"2024-09-17T03:26:22.276802+0000","last_clean":"2024-09-17T03:26:22.276802+0000","last_became_active":"2024-09-17T03:26:21.282594+0000","last_became_peered":"2024-09-17T03:26:21.282594+0000","last_unstale":"2024-09-17T03:26:22.276802+0000","last_undegraded":"2024-09-17T03:26:22.276802+0000","last_fullsized":"2024-09-17T03:26:22.276802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:22.760410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582360+0000","last_change":"2024-09-17T03:26:20.277795+0000","last_active":"2024-09-17T03:26:22.582360+0000","last_peered":"2024-09-17T03:26:22.582360+0000","last_clean":"2024-09-17T03:26:22.582360+0000","last_became_active":"2024-09-17T03:26:20.277456+0000","last_became_peered":"2024-09-17T03:26:20.277456+0000","last_unstale":"2024-09-17T03:26:22.582360+0000","last_undegraded":"2024-09-17T03:26:22.582360+0000","last_fullsized":"2024-09-17T03:26:22.582360+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:43.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581595+0000","last_change":"2024-09-17T03:26:21.292382+0000","last_active":"2024-09-17T03:26:22.581595+0000","last_peered":"2024-09-17T03:26:22.581595+0000","last_clean":"2024-09-17T03:26:22.581595+0000","last_became_active":"2024-09-17T03:26:21.292043+0000","last_became_peered":"2024-09-17T03:26:21.292043+0000","last_unstale":"2024-09-17T03:26:22.581595+0000","last_undegraded":"2024-09-17T03:26:22.581595+0000","last_fullsized":"2024-09-17T03:26:22.581595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:55.320737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.285228+0000","last_change":"2024-09-17T03:26:20.289201+0000","last_active":"2024-09-17T03:26:22.285228+0000","last_peered":"2024-09-17T03:26:22.285228+0000","last_clean":"2024-09-17T03:26:22.285228+0000","last_became_active":"2024-09-17T03:26:20.288427+0000","last_became_peered":"2024-09-17T03:26:20.288427+0000","last_unstale":"2024-09-17T03:26:22.285228+0000","last_undegraded":"2024-09-17T03:26:22.285228+0000","last_fullsized":"2024-09-17T03:26:22.285228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:46.272291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581979+0000","last_change":"2024-09-17T03:26:21.302309+0000","last_active":"2024-09-17T03:26:22.581979+0000","last_peered":"2024-09-17T03:26:22.581979+0000","last_clean":"2024-09-17T03:26:22.581979+0000","last_became_active":"2024-09-17T03:26:21.301672+0000","last_became_peered":"2024-09-17T03:26:21.301672+0000","last_unstale":"2024-09-17T03:26:22.581979+0000","last_undegraded":"2024-09-17T03:26:22.581979+0000","last_fullsized":"2024-09-17T03:26:22.581979+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:52.166965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279944+0000","last_change":"2024-09-17T03:26:20.265758+0000","last_active":"2024-09-17T03:26:22.279944+0000","last_peered":"2024-09-17T03:26:22.279944+0000","last_clean":"2024-09-17T03:26:22.279944+0000","last_became_active":"2024-09-17T03:26:20.262782+0000","last_became_peered":"2024-09-17T03:26:20.262782+0000","last_unstale":"2024-09-17T03:26:22.279944+0000","last_undegraded":"2024-09-17T03:26:22.279944+0000","last_fullsized":"2024-09-17T03:26:22.279944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:10.171064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581312+0000","last_change":"2024-09-17T03:26:21.278170+0000","last_active":"2024-09-17T03:26:22.581312+0000","last_peered":"2024-09-17T03:26:22.581312+0000","last_clean":"2024-09-17T03:26:22.581312+0000","last_became_active":"2024-09-17T03:26:21.277329+0000","last_became_peered":"2024-09-17T03:26:21.277329+0000","last_unstale":"2024-09-17T03:26:22.581312+0000","last_undegraded":"2024-09-17T03:26:22.581312+0000","last_fullsized":"2024-09-17T03:26:22.581312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:12.678429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.333632+0000","last_change":"2024-09-17T03:26:20.269475+0000","last_active":"2024-09-17T03:26:22.333632+0000","last_peered":"2024-09-17T03:26:22.333632+0000","last_clean":"2024-09-17T03:26:22.333632+0000","last_became_active":"2024-09-17T03:26:20.269101+0000","last_became_peered":"2024-09-17T03:26:20.269101+0000","last_unstale":"2024-09-17T03:26:22.333632+0000","last_undegraded":"2024-09-17T03:26:22.333632+0000","last_fullsized":"2024-09-17T03:26:22.333632+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:28.751421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.284594+0000","last_change":"2024-09-17T03:26:21.281823+0000","last_active":"2024-09-17T03:26:22.284594+0000","last_peered":"2024-09-17T03:26:22.284594+0000","last_clean":"2024-09-17T03:26:22.284594+0000","last_became_active":"2024-09-17T03:26:21.280688+0000","last_became_peered":"2024-09-17T03:26:21.280688+0000","last_unstale":"2024-09-17T03:26:22.284594+0000","last_undegraded":"2024-09-17T03:26:22.284594+0000","last_fullsized":"2024-09-17T03:26:22.284594+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:03.239508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581815+0000","last_change":"2024-09-17T03:26:20.281345+0000","last_active":"2024-09-17T03:26:22.581815+0000","last_peered":"2024-09-17T03:26:22.581815+0000","last_clean":"2024-09-17T03:26:22.581815+0000","last_became_active":"2024-09-17T03:26:20.280261+0000","last_became_peered":"2024-09-17T03:26:20.280261+0000","last_unstale":"2024-09-17T03:26:22.581815+0000","last_undegraded":"2024-09-17T03:26:22.581815+0000","last_fullsized":"2024-09-17T03:26:22.581815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:53.284121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581938+0000","last_change":"2024-09-17T03:26:21.289462+0000","last_active":"2024-09-17T03:26:22.581938+0000","last_peered":"2024-09-17T03:26:22.581938+0000","last_clean":"2024-09-17T03:26:22.581938+0000","last_became_active":"2024-09-17T03:26:21.285434+0000","last_became_peered":"2024-09-17T03:26:21.285434+0000","last_unstale":"2024-09-17T03:26:22.581938+0000","last_undegraded":"2024-09-17T03:26:22.581938+0000","last_fullsized":"2024-09-17T03:26:22.581938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:48.639268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":23,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.580909+0000","last_change":"2024-09-17T03:26:20.275945+0000","last_active":"2024-09-17T03:26:22.580909+0000","last_peered":"2024-09-17T03:26:22.580909+0000","last_clean":"2024-09-17T03:26:22.580909+0000","last_became_active":"2024-09-17T03:26:20.275123+0000","last_became_peered":"2024-09-17T03:26:20.275123+0000","last_unstale":"2024-09-17T03:26:22.580909+0000","last_undegraded":"2024-09-17T03:26:22.580909+0000","last_fullsized":"2024-09-17T03:26:22.580909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:02.236108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582107+0000","last_change":"2024-09-17T03:26:21.284952+0000","last_active":"2024-09-17T03:26:22.582107+0000","last_peered":"2024-09-17T03:26:22.582107+0000","last_clean":"2024-09-17T03:26:22.582107+0000","last_became_active":"2024-09-17T03:26:21.284572+0000","last_became_peered":"2024-09-17T03:26:21.284572+0000","last_unstale":"2024-09-17T03:26:22.582107+0000","last_undegraded":"2024-09-17T03:26:22.582107+0000","last_fullsized":"2024-09-17T03:26:22.582107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:24.797352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280456+0000","last_change":"2024-09-17T03:26:20.269212+0000","last_active":"2024-09-17T03:26:22.280456+0000","last_peered":"2024-09-17T03:26:22.280456+0000","last_clean":"2024-09-17T03:26:22.280456+0000","last_became_active":"2024-09-17T03:26:20.268883+0000","last_became_peered":"2024-09-17T03:26:20.268883+0000","last_unstale":"2024-09-17T03:26:22.280456+0000","last_undegraded":"2024-09-17T03:26:22.280456+0000","last_fullsized":"2024-09-17T03:26:22.280456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:41:56.450980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582259+0000","last_change":"2024-09-17T03:26:21.301528+0000","last_active":"2024-09-17T03:26:22.582259+0000","last_peered":"2024-09-17T03:26:22.582259+0000","last_clean":"2024-09-17T03:26:22.582259+0000","last_became_active":"2024-09-17T03:26:21.301110+0000","last_became_peered":"2024-09-17T03:26:21.301110+0000","last_unstale":"2024-09-17T03:26:22.582259+0000","last_undegraded":"2024-09-17T03:26:22.582259+0000","last_fullsized":"2024-09-17T03:26:22.582259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:48.608160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575742+0000","last_change":"2024-09-17T03:26:20.279146+0000","last_active":"2024-09-17T03:26:22.575742+0000","last_peered":"2024-09-17T03:26:22.575742+0000","last_clean":"2024-09-17T03:26:22.575742+0000","last_became_active":"2024-09-17T03:26:20.278888+0000","last_became_peered":"2024-09-17T03:26:20.278888+0000","last_unstale":"2024-09-17T03:26:22.575742+0000","last_undegraded":"2024-09-17T03:26:22.575742+0000","last_fullsized":"2024-09-17T03:26:22.575742+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:40.150476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280158+0000","last_change":"2024-09-17T03:26:21.278295+0000","last_active":"2024-09-17T03:26:22.280158+0000","last_peered":"2024-09-17T03:26:22.280158+0000","last_clean":"2024-09-17T03:26:22.280158+0000","last_became_active":"2024-09-17T03:26:21.277728+0000","last_became_peered":"2024-09-17T03:26:21.277728+0000","last_unstale":"2024-09-17T03:26:22.280158+0000","last_undegraded":"2024-09-17T03:26:22.280158+0000","last_fullsized":"2024-09-17T03:26:22.280158+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:20.893100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582375+0000","last_change":"2024-09-17T03:26:20.272850+0000","last_active":"2024-09-17T03:26:22.582375+0000","last_peered":"2024-09-17T03:26:22.582375+0000","last_clean":"2024-09-17T03:26:22.582375+0000","last_became_active":"2024-09-17T03:26:20.272036+0000","last_became_peered":"2024-09-17T03:26:20.272036+0000","last_unstale":"2024-09-17T03:26:22.582375+0000","last_undegraded":"2024-09-17T03:26:22.582375+0000","last_fullsized":"2024-09-17T03:26:22.582375+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:39.476815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581756+0000","last_change":"2024-09-17T03:26:21.294723+0000","last_active":"2024-09-17T03:26:22.581756+0000","last_peered":"2024-09-17T03:26:22.581756+0000","last_clean":"2024-09-17T03:26:22.581756+0000","last_became_active":"2024-09-17T03:26:21.294375+0000","last_became_peered":"2024-09-17T03:26:21.294375+0000","last_unstale":"2024-09-17T03:26:22.581756+0000","last_undegraded":"2024-09-17T03:26:22.581756+0000","last_fullsized":"2024-09-17T03:26:22.581756+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:41.180633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575587+0000","last_change":"2024-09-17T03:26:20.281457+0000","last_active":"2024-09-17T03:26:22.575587+0000","last_peered":"2024-09-17T03:26:22.575587+0000","last_clean":"2024-09-17T03:26:22.575587+0000","last_became_active":"2024-09-17T03:26:20.281078+0000","last_became_peered":"2024-09-17T03:26:20.281078+0000","last_unstale":"2024-09-17T03:26:22.575587+0000","last_undegraded":"2024-09-17T03:26:22.575587+0000","last_fullsized":"2024-09-17T03:26:22.575587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:22.720948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582639+0000","last_change":"2024-09-17T03:26:21.302012+0000","last_active":"2024-09-17T03:26:22.582639+0000","last_peered":"2024-09-17T03:26:22.582639+0000","last_clean":"2024-09-17T03:26:22.582639+0000","last_became_active":"2024-09-17T03:26:21.301171+0000","last_became_peered":"2024-09-17T03:26:21.301171+0000","last_unstale":"2024-09-17T03:26:22.582639+0000","last_undegraded":"2024-09-17T03:26:22.582639+0000","last_fullsized":"2024-09-17T03:26:22.582639+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:58.402868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280182+0000","last_change":"2024-09-17T03:26:20.280006+0000","last_active":"2024-09-17T03:26:22.280182+0000","last_peered":"2024-09-17T03:26:22.280182+0000","last_clean":"2024-09-17T03:26:22.280182+0000","last_became_active":"2024-09-17T03:26:20.279611+0000","last_became_peered":"2024-09-17T03:26:20.279611+0000","last_unstale":"2024-09-17T03:26:22.280182+0000","last_undegraded":"2024-09-17T03:26:22.280182+0000","last_fullsized":"2024-09-17T03:26:22.280182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:29.773567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575449+0000","last_change":"2024-09-17T03:26:21.284722+0000","last_active":"2024-09-17T03:26:22.575449+0000","last_peered":"2024-09-17T03:26:22.575449+0000","last_clean":"2024-09-17T03:26:22.575449+0000","last_became_active":"2024-09-17T03:26:21.284399+0000","last_became_peered":"2024-09-17T03:26:21.284399+0000","last_unstale":"2024-09-17T03:26:22.575449+0000","last_undegraded":"2024-09-17T03:26:22.575449+0000","last_fullsized":"2024-09-17T03:26:22.575449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.738915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.276843+0000","last_change":"2024-09-17T03:26:20.274933+0000","last_active":"2024-09-17T03:26:22.276843+0000","last_peered":"2024-09-17T03:26:22.276843+0000","last_clean":"2024-09-17T03:26:22.276843+0000","last_became_active":"2024-09-17T03:26:20.274536+0000","last_became_peered":"2024-09-17T03:26:20.274536+0000","last_unstale":"2024-09-17T03:26:22.276843+0000","last_undegraded":"2024-09-17T03:26:22.276843+0000","last_fullsized":"2024-09-17T03:26:22.276843+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:56.607939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575209+0000","last_change":"2024-09-17T03:26:21.277208+0000","last_active":"2024-09-17T03:26:22.575209+0000","last_peered":"2024-09-17T03:26:22.575209+0000","last_clean":"2024-09-17T03:26:22.575209+0000","last_became_active":"2024-09-17T03:26:21.276812+0000","last_became_peered":"2024-09-17T03:26:21.276812+0000","last_unstale":"2024-09-17T03:26:22.575209+0000","last_undegraded":"2024-09-17T03:26:22.575209+0000","last_fullsized":"2024-09-17T03:26:22.575209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:38.173310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581570+0000","last_change":"2024-09-17T03:26:20.281254+0000","last_active":"2024-09-17T03:26:22.581570+0000","last_peered":"2024-09-17T03:26:22.581570+0000","last_clean":"2024-09-17T03:26:22.581570+0000","last_became_active":"2024-09-17T03:26:20.280858+0000","last_became_peered":"2024-09-17T03:26:20.280858+0000","last_unstale":"2024-09-17T03:26:22.581570+0000","last_undegraded":"2024-09-17T03:26:22.581570+0000","last_fullsized":"2024-09-17T03:26:22.581570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:26.185156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582253+0000","last_change":"2024-09-17T03:26:21.280065+0000","last_active":"2024-09-17T03:26:22.582253+0000","last_peered":"2024-09-17T03:26:22.582253+0000","last_clean":"2024-09-17T03:26:22.582253+0000","last_became_active":"2024-09-17T03:26:21.279670+0000","last_became_peered":"2024-09-17T03:26:21.279670+0000","last_unstale":"2024-09-17T03:26:22.582253+0000","last_undegraded":"2024-09-17T03:26:22.582253+0000","last_fullsized":"2024-09-17T03:26:22.582253+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:09.173168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.277357+0000","last_change":"2024-09-17T03:26:20.272578+0000","last_active":"2024-09-17T03:26:22.277357+0000","last_peered":"2024-09-17T03:26:22.277357+0000","last_clean":"2024-09-17T03:26:22.277357+0000","last_became_active":"2024-09-17T03:26:20.272215+0000","last_became_peered":"2024-09-17T03:26:20.272215+0000","last_unstale":"2024-09-17T03:26:22.277357+0000","last_undegraded":"2024-09-17T03:26:22.277357+0000","last_fullsized":"2024-09-17T03:26:22.277357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:20.231120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582465+0000","last_change":"2024-09-17T03:26:21.285235+0000","last_active":"2024-09-17T03:26:22.582465+0000","last_peered":"2024-09-17T03:26:22.582465+0000","last_clean":"2024-09-17T03:26:22.582465+0000","last_became_active":"2024-09-17T03:26:21.283526+0000","last_became_peered":"2024-09-17T03:26:21.283526+0000","last_unstale":"2024-09-17T03:26:22.582465+0000","last_undegraded":"2024-09-17T03:26:22.582465+0000","last_fullsized":"2024-09-17T03:26:22.582465+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:51.012220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575862+0000","last_change":"2024-09-17T03:26:20.281152+0000","last_active":"2024-09-17T03:26:22.575862+0000","last_peered":"2024-09-17T03:26:22.575862+0000","last_clean":"2024-09-17T03:26:22.575862+0000","last_became_active":"2024-09-17T03:26:20.280782+0000","last_became_peered":"2024-09-17T03:26:20.280782+0000","last_unstale":"2024-09-17T03:26:22.575862+0000","last_undegraded":"2024-09-17T03:26:22.575862+0000","last_fullsized":"2024-09-17T03:26:22.575862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:08.814817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.276979+0000","last_change":"2024-09-17T03:26:21.282941+0000","last_active":"2024-09-17T03:26:22.276979+0000","last_peered":"2024-09-17T03:26:22.276979+0000","last_clean":"2024-09-17T03:26:22.276979+0000","last_became_active":"2024-09-17T03:26:21.282247+0000","last_became_peered":"2024-09-17T03:26:21.282247+0000","last_unstale":"2024-09-17T03:26:22.276979+0000","last_undegraded":"2024-09-17T03:26:22.276979+0000","last_fullsized":"2024-09-17T03:26:22.276979+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:11.051280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.333837+0000","last_change":"2024-09-17T03:26:20.277793+0000","last_active":"2024-09-17T03:26:22.333837+0000","last_peered":"2024-09-17T03:26:22.333837+0000","last_clean":"2024-09-17T03:26:22.333837+0000","last_became_active":"2024-09-17T03:26:20.276428+0000","last_became_peered":"2024-09-17T03:26:20.276428+0000","last_unstale":"2024-09-17T03:26:22.333837+0000","last_undegraded":"2024-09-17T03:26:22.333837+0000","last_fullsized":"2024-09-17T03:26:22.333837+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:18.967736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.575821+0000","last_change":"2024-09-17T03:26:21.283319+0000","last_active":"2024-09-17T03:26:22.575821+0000","last_peered":"2024-09-17T03:26:22.575821+0000","last_clean":"2024-09-17T03:26:22.575821+0000","last_became_active":"2024-09-17T03:26:21.282574+0000","last_became_peered":"2024-09-17T03:26:21.282574+0000","last_unstale":"2024-09-17T03:26:22.575821+0000","last_undegraded":"2024-09-17T03:26:22.575821+0000","last_fullsized":"2024-09-17T03:26:22.575821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:30.326912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.582806+0000","last_change":"2024-09-17T03:26:20.282834+0000","last_active":"2024-09-17T03:26:22.582806+0000","last_peered":"2024-09-17T03:26:22.582806+0000","last_clean":"2024-09-17T03:26:22.582806+0000","last_became_active":"2024-09-17T03:26:20.279265+0000","last_became_peered":"2024-09-17T03:26:20.279265+0000","last_unstale":"2024-09-17T03:26:22.582806+0000","last_undegraded":"2024-09-17T03:26:22.582806+0000","last_fullsized":"2024-09-17T03:26:22.582806+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:29.854324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.280058+0000","last_change":"2024-09-17T03:26:21.290929+0000","last_active":"2024-09-17T03:26:22.280058+0000","last_peered":"2024-09-17T03:26:22.280058+0000","last_clean":"2024-09-17T03:26:22.280058+0000","last_became_active":"2024-09-17T03:26:21.290598+0000","last_became_peered":"2024-09-17T03:26:21.290598+0000","last_unstale":"2024-09-17T03:26:22.280058+0000","last_undegraded":"2024-09-17T03:26:22.280058+0000","last_fullsized":"2024-09-17T03:26:22.280058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:49.179482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.276667+0000","last_change":"2024-09-17T03:26:20.275141+0000","last_active":"2024-09-17T03:26:22.276667+0000","last_peered":"2024-09-17T03:26:22.276667+0000","last_clean":"2024-09-17T03:26:22.276667+0000","last_became_active":"2024-09-17T03:26:20.274824+0000","last_became_peered":"2024-09-17T03:26:20.274824+0000","last_unstale":"2024-09-17T03:26:22.276667+0000","last_undegraded":"2024-09-17T03:26:22.276667+0000","last_fullsized":"2024-09-17T03:26:22.276667+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:16.341319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.581595+0000","last_change":"2024-09-17T03:26:21.285419+0000","last_active":"2024-09-17T03:26:22.581595+0000","last_peered":"2024-09-17T03:26:22.581595+0000","last_clean":"2024-09-17T03:26:22.581595+0000","last_became_active":"2024-09-17T03:26:21.284707+0000","last_became_peered":"2024-09-17T03:26:21.284707+0000","last_unstale":"2024-09-17T03:26:22.581595+0000","last_undegraded":"2024-09-17T03:26:22.581595+0000","last_fullsized":"2024-09-17T03:26:22.581595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:54.060063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.285029+0000","last_change":"2024-09-17T03:26:20.288747+0000","last_active":"2024-09-17T03:26:22.285029+0000","last_peered":"2024-09-17T03:26:22.285029+0000","last_clean":"2024-09-17T03:26:22.285029+0000","last_became_active":"2024-09-17T03:26:20.288347+0000","last_became_peered":"2024-09-17T03:26:20.288347+0000","last_unstale":"2024-09-17T03:26:22.285029+0000","last_undegraded":"2024-09-17T03:26:22.285029+0000","last_fullsized":"2024-09-17T03:26:22.285029+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:46.438301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.276597+0000","last_change":"2024-09-17T03:26:21.286603+0000","last_active":"2024-09-17T03:26:22.276597+0000","last_peered":"2024-09-17T03:26:22.276597+0000","last_clean":"2024-09-17T03:26:22.276597+0000","last_became_active":"2024-09-17T03:26:21.286189+0000","last_became_peered":"2024-09-17T03:26:21.286189+0000","last_unstale":"2024-09-17T03:26:22.276597+0000","last_undegraded":"2024-09-17T03:26:22.276597+0000","last_fullsized":"2024-09-17T03:26:22.276597+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:22.623106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.276648+0000","last_change":"2024-09-17T03:26:20.286225+0000","last_active":"2024-09-17T03:26:22.276648+0000","last_peered":"2024-09-17T03:26:22.276648+0000","last_clean":"2024-09-17T03:26:22.276648+0000","last_became_active":"2024-09-17T03:26:20.285812+0000","last_became_peered":"2024-09-17T03:26:20.285812+0000","last_unstale":"2024-09-17T03:26:22.276648+0000","last_undegraded":"2024-09-17T03:26:22.276648+0000","last_fullsized":"2024-09-17T03:26:22.276648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:11.341615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279689+0000","last_change":"2024-09-17T03:26:21.280085+0000","last_active":"2024-09-17T03:26:22.279689+0000","last_peered":"2024-09-17T03:26:22.279689+0000","last_clean":"2024-09-17T03:26:22.279689+0000","last_became_active":"2024-09-17T03:26:21.279401+0000","last_became_peered":"2024-09-17T03:26:21.279401+0000","last_unstale":"2024-09-17T03:26:22.279689+0000","last_undegraded":"2024-09-17T03:26:22.279689+0000","last_fullsized":"2024-09-17T03:26:22.279689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:51.698781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279805+0000","last_change":"2024-09-17T03:26:20.271261+0000","last_active":"2024-09-17T03:26:22.279805+0000","last_peered":"2024-09-17T03:26:22.279805+0000","last_clean":"2024-09-17T03:26:22.279805+0000","last_became_active":"2024-09-17T03:26:20.270867+0000","last_became_peered":"2024-09-17T03:26:20.270867+0000","last_unstale":"2024-09-17T03:26:22.279805+0000","last_undegraded":"2024-09-17T03:26:22.279805+0000","last_fullsized":"2024-09-17T03:26:22.279805+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:19.783352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.279598+0000","last_change":"2024-09-17T03:26:21.287202+0000","last_active":"2024-09-17T03:26:22.279598+0000","last_peered":"2024-09-17T03:26:22.279598+0000","last_clean":"2024-09-17T03:26:22.279598+0000","last_became_active":"2024-09-17T03:26:21.286886+0000","last_became_peered":"2024-09-17T03:26:21.286886+0000","last_unstale":"2024-09-17T03:26:22.279598+0000","last_undegraded":"2024-09-17T03:26:22.279598+0000","last_fullsized":"2024-09-17T03:26:22.279598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:20.242858+0000","last_clean_scrub_stamp":"2024-09-17T03:26:20.242858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:48.994290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.333664+0000","last_change":"2024-09-17T03:26:20.272318+0000","last_active":"2024-09-17T03:26:22.333664+0000","last_peered":"2024-09-17T03:26:22.333664+0000","last_clean":"2024-09-17T03:26:22.333664+0000","last_became_active":"2024-09-17T03:26:20.270871+0000","last_became_peered":"2024-09-17T03:26:20.270871+0000","last_unstale":"2024-09-17T03:26:22.333664+0000","last_undegraded":"2024-09-17T03:26:22.333664+0000","last_fullsized":"2024-09-17T03:26:22.333664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:26:19.239458+0000","last_clean_scrub_stamp":"2024-09-17T03:26:19.239458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:56.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T03:26:22.580920+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:26:22.580920+0000","last_peered":"2024-09-17T03:26:22.580920+0000","last_clean":"2024-09-17T03:26:22.580920+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:26:22.580920+0000","last_undegraded":"2024-09-17T03:26:22.580920+0000","last_fullsized":"2024-09-17T03:26:22.580920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509509,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58820,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313020,"statfs":{"total":96636764160,"available":96576532480,"internally_reserved":0,"allocated":1290240,"data_stored":876318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509509,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45756,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326084,"statfs":{"total":96636764160,"available":96589910016,"internally_reserved":0,"allocated":688128,"data_stored":285790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45764,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326076,"statfs":{"total":96636764160,"available":96589901824,"internally_reserved":0,"allocated":700416,"data_stored":287402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49924,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321916,"statfs":{"total":96636764160,"available":96585641984,"internally_reserved":0,"allocated":696320,"data_stored":285846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45756,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326084,"statfs":{"total":96636764160,"available":96589910016,"internally_reserved":0,"allocated":696320,"data_stored":285846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50508,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321332,"statfs":{"total":96636764160,"available":96585043968,"internally_reserved":0,"allocated":1298432,"data_stored":878352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":286442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49920,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321920,"statfs":{"total":96636764160,"available":96585646080,"internally_reserved":0,"allocated":692224,"data_stored":285880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:26:30.554 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:26:30.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:26:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 -- 172.21.15.46:0/3719356758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 msgr2=0x7f15ec0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 --2- 172.21.15.46:0/3719356758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 0x7f15ec0b51f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009bc0 tx=0x7f15e401cae0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 -- 172.21.15.46:0/3719356758 shutdown_connections 2024-09-17T03:26:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 --2- 172.21.15.46:0/3719356758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 0x7f15ec0b51f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 --2- 172.21.15.46:0/3719356758 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15ec0b5c70 0x7f15ec15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 -- 172.21.15.46:0/3719356758 >> 172.21.15.46:0/3719356758 conn(0x7f15ec0b2040 msgr2=0x7f15ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 -- 172.21.15.46:0/3719356758 shutdown_connections 2024-09-17T03:26:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.759+0000 7f15f2400640 1 -- 172.21.15.46:0/3719356758 wait complete. 2024-09-17T03:26:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 Processor -- start 2024-09-17T03:26:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 -- start start 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 0x7f15ec0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 0x7f15ec14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15ec1527b0 0x7f15ec1fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec168ab0 con 0x7f15ec0b5c70 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec168930 con 0x7f15ec0b4de0 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec168c30 con 0x7f15ec1527b0 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 0x7f15ec14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 0x7f15ec14ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50042/0 (socket says 172.21.15.46:50042) 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 -- 172.21.15.46:0/968946135 learned_addr learned my addr 172.21.15.46:0/968946135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15ebe00640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15ec1527b0 0x7f15ec1fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15eb400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 0x7f15ec0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 -- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15ec1527b0 msgr2=0x7f15ec1fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15ec1527b0 0x7f15ec1fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 -- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 msgr2=0x7f15ec0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:30.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 0x7f15ec0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:30.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 -- 172.21.15.46:0/968946135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec0cefc0 con 0x7f15ec0b5c70 2024-09-17T03:26:30.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3e00640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 0x7f15ec14ee50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f15d400c760 tx=0x7f15d400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4006aa0 con 0x7f15ec0b5c70 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d4006c40 con 0x7f15ec0b5c70 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec0ce720 con 0x7f15ec0b5c70 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4007290 con 0x7f15ec0b5c70 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.763+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec0cece0 con 0x7f15ec0b5c70 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.767+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec0b51f0 con 0x7f15ec0b5c70 2024-09-17T03:26:30.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.767+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f15d4007430 con 0x7f15ec0b5c70 2024-09-17T03:26:30.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.767+0000 7f15e3400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15c8073b00 0x7f15c8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:30.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.767+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f15d40921c0 con 0x7f15ec0b5c70 2024-09-17T03:26:30.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.767+0000 7f15eb400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15c8073b00 0x7f15c8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:30.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.767+0000 7f15eb400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15c8073b00 0x7f15c8076010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009b40 tx=0x7f15dc0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:30.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:30.771+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f15d4020020 con 0x7f15ec0b5c70 2024-09-17T03:26:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.007+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f15ec0b62d0 con 0x7f15ec0b5c70 2024-09-17T03:26:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15e3400640 1 -- 172.21.15.46:0/968946135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7f15ec0b5730 con 0x7f15ec0b5c70 2024-09-17T03:26:31.013 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:26:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:26:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:26:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:26:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:26:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15c8073b00 msgr2=0x7f15c8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15c8073b00 0x7f15c8076010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009b40 tx=0x7f15dc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 msgr2=0x7f15ec14ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 0x7f15ec14ee50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f15d400c760 tx=0x7f15d400cc20 comp rx=0 tx=0).stop 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 shutdown_connections 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15c8073b00 0x7f15c8076010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15ec1527b0 0x7f15ec1fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15ec0b5c70 0x7f15ec14ee50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 --2- 172.21.15.46:0/968946135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15ec0b4de0 0x7f15ec0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 >> 172.21.15.46:0/968946135 conn(0x7f15ec0b2040 msgr2=0x7f15ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 shutdown_connections 2024-09-17T03:26:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.011+0000 7f15f2400640 1 -- 172.21.15.46:0/968946135 wait complete. 2024-09-17T03:26:31.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:26:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 -- 172.21.15.46:0/34786557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82a4004810 msgr2=0x7f82a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 --2- 172.21.15.46:0/34786557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82a4004810 0x7f82a4004c20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009d10 tx=0x7f82a801d470 comp rx=0 tx=0).stop 2024-09-17T03:26:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82bf400640 1 -- 172.21.15.46:0/34786557 reap_dead start 2024-09-17T03:26:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 -- 172.21.15.46:0/34786557 shutdown_connections 2024-09-17T03:26:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 -- 172.21.15.46:0/34786557 >> 172.21.15.46:0/34786557 conn(0x7f82c00b2040 msgr2=0x7f82c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 -- 172.21.15.46:0/34786557 shutdown_connections 2024-09-17T03:26:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 -- 172.21.15.46:0/34786557 wait complete. 2024-09-17T03:26:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.195+0000 7f82c5800640 1 Processor -- start 2024-09-17T03:26:31.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 -- start start 2024-09-17T03:26:31.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a4004810 0x7f82c00af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 0x7f82c00abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82c00ac3b0 0x7f82c00ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0160fd0 con 0x7f82c0159e70 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0160e50 con 0x7f82a4004810 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f82c0161150 con 0x7f82c00ac3b0 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 0x7f82c00abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 0x7f82c00abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50064/0 (socket says 172.21.15.46:50064) 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 -- 172.21.15.46:0/2311570636 learned_addr learned my addr 172.21.15.46:0/2311570636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82bea00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a4004810 0x7f82c00af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82bf400640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82c00ac3b0 0x7f82c00ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 -- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82c00ac3b0 msgr2=0x7f82c00ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82c00ac3b0 0x7f82c00ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 -- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a4004810 msgr2=0x7f82c00af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a4004810 0x7f82c00af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 -- 172.21.15.46:0/2311570636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c00acd00 con 0x7f82c0159e70 2024-09-17T03:26:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b7e00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 0x7f82c00abe70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f82a801d2a0 tx=0x7f82a80098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8005230 con 0x7f82c0159e70 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a80b9030 con 0x7f82c0159e70 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c00aadd0 con 0x7f82c0159e70 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a80046d0 con 0x7f82c0159e70 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.199+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c00ab390 con 0x7f82c0159e70 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.203+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c00750c0 con 0x7f82c0159e70 2024-09-17T03:26:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.203+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f82a8004870 con 0x7f82c0159e70 2024-09-17T03:26:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.203+0000 7f82b6a00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f829c073b30 0x7f829c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.203+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f82a81306a0 con 0x7f82c0159e70 2024-09-17T03:26:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.203+0000 7f82bea00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f829c073b30 0x7f829c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.203+0000 7f82bea00640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f829c073b30 0x7f829c076040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009bc0 tx=0x7f82b00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.207+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82a8017180 con 0x7f82c0159e70 2024-09-17T03:26:31.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.423+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82c01424c0 con 0x7f82c0159e70 2024-09-17T03:26:31.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.423+0000 7f82b6a00640 1 -- 172.21.15.46:0/2311570636 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5236 (secure 0 0 0) 0x7f82a80fabc0 con 0x7f82c0159e70 2024-09-17T03:26:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:26:31.425 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"btime":"2024-09-17T03:26:24:284029+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:26:20.685438+0000","modified":"2024-09-17T03:26:23.700531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245850":{"gid":245850,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/695855747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":695855747},{"type":"v1","addr":"172.21.15.46:6835","nonce":695855747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245850,"qdb_cluster":[245850]},"id":4}]} 2024-09-17T03:26:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 35 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f829c073b30 msgr2=0x7f829c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f829c073b30 0x7f829c076040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009bc0 tx=0x7f82b00091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 msgr2=0x7f82c00abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 0x7f82c00abe70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f82a801d2a0 tx=0x7f82a80098e0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 shutdown_connections 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f829c073b30 0x7f829c076040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82c00ac3b0 0x7f82c00ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82c0159e70 0x7f82c00abe70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 --2- 172.21.15.46:0/2311570636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a4004810 0x7f82c00af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 >> 172.21.15.46:0/2311570636 conn(0x7f82c00b2040 msgr2=0x7f82c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 shutdown_connections 2024-09-17T03:26:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:31.427+0000 7f82c5800640 1 -- 172.21.15.46:0/2311570636 wait complete. 2024-09-17T03:26:31.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:26:20.685438+0000', 'modified': '2024-09-17T03:26:23.700531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 245850}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_245850': {'gid': 245850, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/695855747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 695855747}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 695855747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 245850, 'qdb_cluster': [245850]} 2024-09-17T03:26:31.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:26:31.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T03:26:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 -- 172.21.15.46:0/1580096355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 msgr2=0x7f3bf40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 --2- 172.21.15.46:0/1580096355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf40700a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40b6bf0 tx=0x7f3bf001ccf0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 -- 172.21.15.46:0/1580096355 shutdown_connections 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 --2- 172.21.15.46:0/1580096355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf40700a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 --2- 172.21.15.46:0/1580096355 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bf406cdf0 0x7f3bf406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 --2- 172.21.15.46:0/1580096355 >> v2:172.21.15.99:3300/0 conn(0x7f3bf4159130 0x7f3bf415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 -- 172.21.15.46:0/1580096355 >> 172.21.15.46:0/1580096355 conn(0x7f3bf40b35f0 msgr2=0x7f3bf40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 -- 172.21.15.46:0/1580096355 shutdown_connections 2024-09-17T03:26:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.011+0000 7f3bfc800640 1 -- 172.21.15.46:0/1580096355 wait complete. 2024-09-17T03:26:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 Processor -- start 2024-09-17T03:26:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 -- start start 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bf406cdf0 0x7f3bf414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf41fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf4159130 0x7f3bf41fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf415fa10 con 0x7f3bf406fc90 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf415f890 con 0x7f3bf4159130 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf415fb90 con 0x7f3bf406cdf0 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf41fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf41fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50078/0 (socket says 172.21.15.46:50078) 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 -- 172.21.15.46:0/230455192 learned_addr learned my addr 172.21.15.46:0/230455192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bf406cdf0 0x7f3bf414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfa800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf4159130 0x7f3bf41fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 -- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bf406cdf0 msgr2=0x7f3bf414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bf406cdf0 0x7f3bf414b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 -- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf4159130 msgr2=0x7f3bf41fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf4159130 0x7f3bf41fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 -- 172.21.15.46:0/230455192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf414b820 con 0x7f3bf406fc90 2024-09-17T03:26:32.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bf9400640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf41fcc40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3be400d680 tx=0x7f3be400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bee000640 1 -- 172.21.15.46:0/230455192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4017070 con 0x7f3bf406fc90 2024-09-17T03:26:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bee000640 1 -- 172.21.15.46:0/230455192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3be4005310 con 0x7f3bf406fc90 2024-09-17T03:26:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf41f96c0 con 0x7f3bf406fc90 2024-09-17T03:26:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bee000640 1 -- 172.21.15.46:0/230455192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4009820 con 0x7f3bf406fc90 2024-09-17T03:26:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.015+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41f9c80 con 0x7f3bf406fc90 2024-09-17T03:26:32.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3bee000640 1 -- 172.21.15.46:0/230455192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f3be40099c0 con 0x7f3bf406fc90 2024-09-17T03:26:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3bee000640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3bd8073b00 0x7f3bd8076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3bee000640 1 -- 172.21.15.46:0/230455192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3be4092af0 con 0x7f3bf406fc90 2024-09-17T03:26:32.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3bd8073b00 0x7f3bd8076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3c03d7e000 1 Processor -- start 2024-09-17T03:26:32.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3c03d7e000 1 -- start start 2024-09-17T03:26:32.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.019+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3bd8073b00 0x7f3bd8076010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3be800bad0 tx=0x7f3be800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dd1bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd1c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dba53c0 0x56014dd1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56014db0eb40 con 0x56014dbcb9c0 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56014dafdf70 con 0x56014dbba570 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56014db01d80 con 0x56014dba53c0 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dd1bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dd1bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50086/0 (socket says 172.21.15.46:50086) 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 -- 172.21.15.46:0/3329410468 learned_addr learned my addr 172.21.15.46:0/3329410468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9400640 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd1c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bfa800640 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dba53c0 0x56014dd1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 -- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dba53c0 msgr2=0x56014dd1ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dba53c0 0x56014dd1ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 -- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 msgr2=0x56014dd1c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd1c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 -- 172.21.15.46:0/3329410468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014dab13e0 con 0x56014dbcb9c0 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bfa800640 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dba53c0 0x56014dd1ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:32.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dd1bfc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3be8010f20 tx=0x7f3be8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa04f8a4d19ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3be2600640 1 -- 172.21.15.46:0/3329410468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be801e620 con 0x56014dbcb9c0 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3be2600640 1 -- 172.21.15.46:0/3329410468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3be801e7c0 con 0x56014dbcb9c0 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3be2600640 1 -- 172.21.15.46:0/3329410468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be801ebd0 con 0x56014dbcb9c0 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3329410468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 msgr2=0x56014dd1bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dd1bfc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3be8010f20 tx=0x7f3be8013000 comp rx=0 tx=0).stop 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3329410468 shutdown_connections 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dba53c0 0x56014dd1ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd1c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3329410468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dd1bfc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3329410468 >> 172.21.15.46:0/3329410468 conn(0x56014d7bb460 msgr2=0x56014dcff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3329410468 shutdown_connections 2024-09-17T03:26:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.023+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3329410468 wait complete. 2024-09-17T03:26:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 -- start start 2024-09-17T03:26:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 Processor -- start 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dcfe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dbba570 0x56014dcfeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dcff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56014daa4520 con 0x56014dbcb9c0 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56014da985f0 con 0x56014dba53c0 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56014dac9ee0 con 0x56014dbba570 2024-09-17T03:26:32.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 10 client.0 _ll_register_callbacks cb 0x56014db1f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dcff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bf9e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dcfe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bf9400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dbba570 0x56014dcfeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dcff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50102/0 (socket says 172.21.15.46:50102) 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 -- 172.21.15.46:0/2697258517 learned_addr learned my addr 172.21.15.46:0/2697258517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 -- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dbba570 msgr2=0x56014dcfeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dbba570 0x56014dcfeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 -- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 msgr2=0x56014dcfe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dcfe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 -- 172.21.15.46:0/2697258517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014daa8460 con 0x56014dbcb9c0 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dcfe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:26:32.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bf9400640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dbba570 0x56014dcfeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bfa800640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dcff030 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00b0040 tx=0x7f3bf0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8167816162a08e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bd6a00640 1 -- 172.21.15.46:0/2697258517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00bb020 con 0x56014dbcb9c0 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56014da96110 con 0x56014dbcb9c0 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bd6a00640 1 -- 172.21.15.46:0/2697258517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3bf00092c0 con 0x56014dbcb9c0 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3bd6a00640 1 -- 172.21.15.46:0/2697258517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bf00c5050 con 0x56014dbcb9c0 2024-09-17T03:26:32.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 10 client.245979 fetch_fsmap learned FSMap version 35 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.027+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x56014dac55b0 con 0x56014dbcb9c0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bd6a00640 1 -- 172.21.15.46:0/2697258517 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56014da96110 con 0x56014dbcb9c0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bd6a00640 1 -- 172.21.15.46:0/2697258517 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f3bf00c8020 con 0x56014dbcb9c0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 10 client.245979 fetch_fsmap finished waiting for FSMap version 35 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 10 client.245979 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 20 client.245979 populate_metadata read hostname 'smithi046' 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7a07d0 0x56014dd2d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 4 client.245979 mds_command: new command op to 245850 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 --> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56014db10a90 con 0x56014d7a07d0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7a07d0 0x56014dd2d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7a07d0 0x56014dd2d580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a0659a825c12ed server_cookie=e74913073fdfd466 in_seq=0 out_seq=0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bd6a00640 10 client.245979 ms_handle_connect on v2:172.21.15.46:6834/695855747 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bd6a00640 1 -- 172.21.15.46:0/2697258517 <== mds.0 v2:172.21.15.46:6834/695855747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56014db10a90 con 0x56014d7a07d0 2024-09-17T03:26:32.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bd6a00640 10 client.245979 handle_command_reply: tid=0 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 client.245979 shutdown 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 msgr2=0x56014dcff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dcff030 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00b0040 tx=0x7f3bf0005dd0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 shutdown_connections 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7a07d0 0x56014dd2d580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbcb9c0 0x56014dcff030 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014dbba570 0x56014dcfeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/2697258517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dcfe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3bd6a00640 0 client.245979 ms_handle_reset on v2:172.21.15.46:6834/695855747 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 >> 172.21.15.46:0/2697258517 conn(0x56014d7bb460 msgr2=0x56014dcd0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 shutdown_connections 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 1 -- 172.21.15.46:0/2697258517 wait complete. 2024-09-17T03:26:32.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.031+0000 7f3c03d7e000 20 client.245979 trim_cache size 0 max 16384 2024-09-17T03:26:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.051+0000 7f3c03d7e000 1 Processor -- start 2024-09-17T03:26:32.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.051+0000 7f3c03d7e000 1 -- start start 2024-09-17T03:26:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd30df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 0x56014dd32e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41653a0 con 0x56014dba53c0 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4164700 con 0x56014dbba570 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf4164d50 con 0x56014d7bb460 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 0x56014dd32e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 0x56014dd32e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50112/0 (socket says 172.21.15.46:50112) 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 -- 172.21.15.46:0/3251862647 learned_addr learned my addr 172.21.15.46:0/3251862647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd30df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bfa800640 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 -- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 msgr2=0x56014dd30df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd30df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 -- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 msgr2=0x56014dd333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 -- 172.21.15.46:0/3251862647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014da9c190 con 0x56014dba53c0 2024-09-17T03:26:32.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd30df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:32.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3bf9400640 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 0x56014dd32e60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3be405ee20 tx=0x7f3be4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fc2c9145b7772 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3be1c00640 1 -- 172.21.15.46:0/3251862647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4079070 con 0x56014dba53c0 2024-09-17T03:26:32.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3be1c00640 1 -- 172.21.15.46:0/3251862647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3be4062040 con 0x56014dba53c0 2024-09-17T03:26:32.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3be1c00640 1 -- 172.21.15.46:0/3251862647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4020040 con 0x56014dba53c0 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3251862647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 msgr2=0x56014dd32e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 0x56014dd32e60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3be405ee20 tx=0x7f3be4005850 comp rx=0 tx=0).stop 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3251862647 shutdown_connections 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dbba570 0x56014dd333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dba53c0 0x56014dd32e60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/3251862647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd30df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3251862647 >> 172.21.15.46:0/3251862647 conn(0x56014dd28e60 msgr2=0x56014dd32750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3251862647 shutdown_connections 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- 172.21.15.46:0/3251862647 wait complete. 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 -- start start 2024-09-17T03:26:32.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.055+0000 7f3c03d7e000 1 Processor -- start 2024-09-17T03:26:32.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd4daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dd4dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 0x56014dd4e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56014db01090 con 0x56014dbba570 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56014db0d910 con 0x56014dba53c0 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56014db063b0 con 0x56014d7bb460 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 10 client.0 _ll_register_callbacks cb 0x7f3bf0009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 0x56014dd4e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 0x56014dd4e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50124/0 (socket says 172.21.15.46:50124) 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 -- 172.21.15.46:0/1163837546 learned_addr learned my addr 172.21.15.46:0/1163837546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bf9400640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dd4dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd4daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 -- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 msgr2=0x56014dd4daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd4daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 -- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 msgr2=0x56014dd4dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:26:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dd4dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 -- 172.21.15.46:0/1163837546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014dac4ab0 con 0x56014dbba570 2024-09-17T03:26:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bfa800640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 0x56014dd4e520 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009bd0 tx=0x7f3bf00b8a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aee00f15373af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:26:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 1 -- 172.21.15.46:0/1163837546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00cd030 con 0x56014dbba570 2024-09-17T03:26:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 1 -- 172.21.15.46:0/1163837546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3bf0002040 con 0x56014dbba570 2024-09-17T03:26:32.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be4062040 con 0x56014dbba570 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 1 -- 172.21.15.46:0/1163837546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bf00d6050 con 0x56014dbba570 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 10 client.245985 fetch_fsmap learned FSMap version 35 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x56014daa3070 con 0x56014dbba570 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 1 -- 172.21.15.46:0/1163837546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3be4062040 con 0x56014dbba570 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 1 -- 172.21.15.46:0/1163837546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f3bf00d9020 con 0x56014dbba570 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 10 client.245985 fetch_fsmap finished waiting for FSMap version 35 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 10 client.245985 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 20 client.245985 populate_metadata read hostname 'smithi046' 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7b9e80 0x56014dd99a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 4 client.245985 mds_command: new command op to 245850 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 --> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] -- command(tid 0: {"prefix": "status"}) -- 0x56014db0c540 con 0x56014d7b9e80 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7b9e80 0x56014dd99a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bf9e00640 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7b9e80 0x56014dd99a40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11407c54fc43c2eb server_cookie=a26fbca9a1d9f94e in_seq=0 out_seq=0 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.059+0000 7f3bd6a00640 10 client.245985 ms_handle_connect on v2:172.21.15.46:6834/695855747 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bd6a00640 1 -- 172.21.15.46:0/1163837546 <== mds.0 v2:172.21.15.46:6834/695855747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56014db0c540 con 0x56014d7b9e80 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bd6a00640 10 client.245985 handle_command_reply: tid=0 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 client.245985 shutdown 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 msgr2=0x56014dd4e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 0x56014dd4e520 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009bd0 tx=0x7f3bf00b8a10 comp rx=0 tx=0).stop 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 shutdown_connections 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:6834/695855747,v1:172.21.15.46:6835/695855747] conn(0x56014d7b9e80 0x56014dd99a40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56014dbba570 0x56014dd4e520 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56014dba53c0 0x56014dd4dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 --2- 172.21.15.46:0/1163837546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56014d7bb460 0x56014dd4daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bd6a00640 0 client.245985 ms_handle_reset on v2:172.21.15.46:6834/695855747 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 >> 172.21.15.46:0/1163837546 conn(0x56014dd28e60 msgr2=0x56014dd33110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 shutdown_connections 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 1 -- 172.21.15.46:0/1163837546 wait complete. 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3c03d7e000 20 client.245985 trim_cache size 0 max 16384 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 245850, 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:26:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.812246151, 2024-09-17T03:26:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 35, 2024-09-17T03:26:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 58, 2024-09-17T03:26:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 58, 2024-09-17T03:26:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.989866768000001 2024-09-17T03:26:32.068 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3bd8073b00 msgr2=0x7f3bd8076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3bd8073b00 0x7f3bd8076010 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3be800bad0 tx=0x7f3be800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 msgr2=0x7f3bf41fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf41fcc40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3be400d680 tx=0x7f3be400db40 comp rx=0 tx=0).stop 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 shutdown_connections 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3bd8073b00 0x7f3bd8076010 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf4159130 0x7f3bf41fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bf406fc90 0x7f3bf41fcc40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 --2- 172.21.15.46:0/230455192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bf406cdf0 0x7f3bf414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 >> 172.21.15.46:0/230455192 conn(0x7f3bf40b35f0 msgr2=0x7f3bf414e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 shutdown_connections 2024-09-17T03:26:32.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.063+0000 7f3bfc800640 1 -- 172.21.15.46:0/230455192 wait complete. 2024-09-17T03:26:32.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:26:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 -- 172.21.15.46:0/3687925047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da80b4de0 msgr2=0x7f4da80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 --2- 172.21.15.46:0/3687925047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da80b4de0 0x7f4da80b51f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4da8071cb0 tx=0x7f4da001ccc0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 -- 172.21.15.46:0/3687925047 shutdown_connections 2024-09-17T03:26:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 --2- 172.21.15.46:0/3687925047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da80b4de0 0x7f4da80b51f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 --2- 172.21.15.46:0/3687925047 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4da81527b0 0x7f4da8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 --2- 172.21.15.46:0/3687925047 >> v2:172.21.15.99:3300/0 conn(0x7f4da80b5c70 0x7f4da815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 -- 172.21.15.46:0/3687925047 >> 172.21.15.46:0/3687925047 conn(0x7f4da80b2040 msgr2=0x7f4da80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 -- 172.21.15.46:0/3687925047 shutdown_connections 2024-09-17T03:26:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 -- 172.21.15.46:0/3687925047 wait complete. 2024-09-17T03:26:32.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 Processor -- start 2024-09-17T03:26:32.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.243+0000 7f4dae200640 1 -- start start 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4da80b4de0 0x7f4da8141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4da80b5c70 0x7f4da81fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 0x7f4da81fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4da815f6b0 con 0x7f4da81527b0 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4da815f530 con 0x7f4da80b5c70 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4da815f830 con 0x7f4da80b4de0 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 0x7f4da81fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 0x7f4da81fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50148/0 (socket says 172.21.15.46:50148) 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 -- 172.21.15.46:0/2796676961 learned_addr learned my addr 172.21.15.46:0/2796676961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4da80b4de0 0x7f4da8141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4d9ea00640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4da80b5c70 0x7f4da81fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 -- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4da80b4de0 msgr2=0x7f4da8141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4da80b4de0 0x7f4da8141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 -- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4da80b5c70 msgr2=0x7f4da81fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4da80b5c70 0x7f4da81fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 -- 172.21.15.46:0/2796676961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da80cf7a0 con 0x7f4da81527b0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4da7e00640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 0x7f4da81fc6c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4da001b370 tx=0x7f4da0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da00b2040 con 0x7f4da81527b0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da0004240 con 0x7f4da81527b0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0004650 con 0x7f4da81527b0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da80cfa80 con 0x7f4da81527b0 2024-09-17T03:26:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da80cc950 con 0x7f4da81527b0 2024-09-17T03:26:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.247+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da80ccb80 con 0x7f4da81527b0 2024-09-17T03:26:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.251+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f4da00ad440 con 0x7f4da81527b0 2024-09-17T03:26:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.251+0000 7f4d9fe00640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d84073b00 0x7f4d84076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.251+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4da012d150 con 0x7f4da81527b0 2024-09-17T03:26:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.251+0000 7f4da7400640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d84073b00 0x7f4d84076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.251+0000 7f4da7400640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d84073b00 0x7f4d84076010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800bad0 tx=0x7f4d9800b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.255+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4da0016970 con 0x7f4da81527b0 2024-09-17T03:26:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4da815fd50 con 0x7f4da81527b0 2024-09-17T03:26:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4d9fe00640 1 -- 172.21.15.46:0/2796676961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f4da00f9660 con 0x7f4da81527b0 2024-09-17T03:26:32.468 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:26:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d84073b00 msgr2=0x7f4d84076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d84073b00 0x7f4d84076010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800bad0 tx=0x7f4d9800b260 comp rx=0 tx=0).stop 2024-09-17T03:26:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 msgr2=0x7f4da81fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 0x7f4da81fc6c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4da001b370 tx=0x7f4da0005b10 comp rx=0 tx=0).stop 2024-09-17T03:26:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 shutdown_connections 2024-09-17T03:26:32.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d84073b00 0x7f4d84076010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4da81527b0 0x7f4da81fc6c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4da80b5c70 0x7f4da81fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 --2- 172.21.15.46:0/2796676961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4da80b4de0 0x7f4da8141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 >> 172.21.15.46:0/2796676961 conn(0x7f4da80b2040 msgr2=0x7f4da815acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 shutdown_connections 2024-09-17T03:26:32.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32.467+0000 7f4dae200640 1 -- 172.21.15.46:0/2796676961 wait complete. 2024-09-17T03:26:32.490 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:26:32.490 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:26:32.490 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:26:32.490 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:26:32.490 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:26:32.490 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:26:32.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:26:32.491 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:26:32.491 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:26:32.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:32.491 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:26:32.501 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42096sec preferred_lft 42096sec 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:26:32.502 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:26:32.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:26:32.504 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:26:32.504 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:26:32.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:32.504 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:26:32.566 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:26:32.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:32.568 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:26:32.620 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:26:32.621 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:26:32.621 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:26:32.621 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:26:32.621 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:26:32.621 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:26:32.621 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:26:32.621 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:26:32.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:26:32.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:26:32.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:26:32.622 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:26:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:26:32.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:26:32.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:32.759 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:26:32.768 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:26:32.768 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:26:32.769 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:26:32.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:32.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:26:32.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:26:32.831 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:26:32.887 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:26:32.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:26:32.887 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:26:32.938 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:26:32.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:32.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:26:33.045 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:26:33.045 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:26:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:26:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:26:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:26:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:26:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:26:33.064 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:26:33.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:33.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:26:33.085 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:26:33.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:26:33.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:26:33.192 INFO:teuthology.orchestra.run.smithi046.stdout:client245447 2024-09-17T03:26:33.192 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:33.193 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:33.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 -- 172.21.15.46:0/67597779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b4de0 msgr2=0x7f06300b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 --2- 172.21.15.46:0/67597779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b4de0 0x7f06300b51f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0628009c80 tx=0x7f062801cce0 comp rx=0 tx=0).stop 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 -- 172.21.15.46:0/67597779 shutdown_connections 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 --2- 172.21.15.46:0/67597779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b4de0 0x7f06300b51f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 --2- 172.21.15.46:0/67597779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06301527b0 0x7f063015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 -- 172.21.15.46:0/67597779 >> 172.21.15.46:0/67597779 conn(0x7f06300b2040 msgr2=0x7f06300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 -- 172.21.15.46:0/67597779 shutdown_connections 2024-09-17T03:26:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.403+0000 7f0634c00640 1 -- 172.21.15.46:0/67597779 wait complete. 2024-09-17T03:26:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 Processor -- start 2024-09-17T03:26:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 -- start start 2024-09-17T03:26:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06300b4de0 0x7f0630141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 0x7f0630158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06301527b0 0x7f06301424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0630168ab0 con 0x7f06300b5c70 2024-09-17T03:26:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0630168930 con 0x7f06301527b0 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0634c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0630168c30 con 0x7f06300b4de0 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 0x7f0630158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 0x7f0630158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39032/0 (socket says 172.21.15.46:39032) 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 -- 172.21.15.46:0/53494745 learned_addr learned my addr 172.21.15.46:0/53494745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f062de00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06300b4de0 0x7f0630141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f062e800640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06301527b0 0x7f06301424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 -- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06300b4de0 msgr2=0x7f0630141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06300b4de0 0x7f0630141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 -- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06301527b0 msgr2=0x7f06301424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06301527b0 0x7f06301424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.407+0000 7f0625400640 1 -- 172.21.15.46:0/53494745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06300cd050 con 0x7f06300b5c70 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f062e800640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06301527b0 0x7f06301424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0625400640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 0x7f0630158d10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f061800d6d0 tx=0x7f061800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618017070 con 0x7f06300b5c70 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0618005310 con 0x7f06300b5c70 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06300cd330 con 0x7f06300b5c70 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180097c0 con 0x7f06300b5c70 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06300cd8f0 con 0x7f06300b5c70 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06300b51f0 con 0x7f06300b5c70 2024-09-17T03:26:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f0618009960 con 0x7f06300b5c70 2024-09-17T03:26:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.411+0000 7f0626a00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f060c073b30 0x7f060c076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.415+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0618092a90 con 0x7f06300b5c70 2024-09-17T03:26:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.415+0000 7f062de00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f060c073b30 0x7f060c076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.415+0000 7f062de00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f060c073b30 0x7f060c076040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0620007fd0 tx=0x7f0620009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:33.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.419+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f061805eeb0 con 0x7f06300b5c70 2024-09-17T03:26:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:33.631+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f06300ccde0 con 0x7f06300b5c70 2024-09-17T03:26:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0626a00640 1 -- 172.21.15.46:0/53494745 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v36) ==== 136+0+0 (secure 0 0 0) 0x7f06180638e0 con 0x7f06300b5c70 2024-09-17T03:26:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f060c073b30 msgr2=0x7f060c076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f060c073b30 0x7f060c076040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0620007fd0 tx=0x7f0620009040 comp rx=0 tx=0).stop 2024-09-17T03:26:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 msgr2=0x7f0630158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 0x7f0630158d10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f061800d6d0 tx=0x7f061800db90 comp rx=0 tx=0).stop 2024-09-17T03:26:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 shutdown_connections 2024-09-17T03:26:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f060c073b30 0x7f060c076040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06301527b0 0x7f06301424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06300b5c70 0x7f0630158d10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 --2- 172.21.15.46:0/53494745 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06300b4de0 0x7f0630141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 >> 172.21.15.46:0/53494745 conn(0x7f06300b2040 msgr2=0x7f0630151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 shutdown_connections 2024-09-17T03:26:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.323+0000 7f0634c00640 1 -- 172.21.15.46:0/53494745 wait complete. 2024-09-17T03:26:34.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:26:34.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 -- 172.21.15.46:0/2228772090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415c5d0 msgr2=0x7ff454073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 --2- 172.21.15.46:0/2228772090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415c5d0 0x7ff454073dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff444009900 tx=0x7ff44401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 -- 172.21.15.46:0/2228772090 shutdown_connections 2024-09-17T03:26:34.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 --2- 172.21.15.46:0/2228772090 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff454074300 0x7ff45415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 --2- 172.21.15.46:0/2228772090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415c5d0 0x7ff454073dc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 -- 172.21.15.46:0/2228772090 >> 172.21.15.46:0/2228772090 conn(0x7ff4540b2c40 msgr2=0x7ff4540b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:34.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 -- 172.21.15.46:0/2228772090 shutdown_connections 2024-09-17T03:26:34.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 -- 172.21.15.46:0/2228772090 wait complete. 2024-09-17T03:26:34.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 Processor -- start 2024-09-17T03:26:34.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.515+0000 7ff45a200640 1 -- start start 2024-09-17T03:26:34.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 0x7ff45414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff45415c5d0 0x7ff4541549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415f6b0 0x7ff45414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff45406f980 con 0x7ff454074300 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540b3d50 con 0x7ff45415f6b0 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff45406fb00 con 0x7ff45415c5d0 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 0x7ff45414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 0x7ff45414a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39036/0 (socket says 172.21.15.46:39036) 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 -- 172.21.15.46:0/3977330960 learned_addr learned my addr 172.21.15.46:0/3977330960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff44aa00640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff45415c5d0 0x7ff4541549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453e00640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415f6b0 0x7ff45414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 -- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff45415c5d0 msgr2=0x7ff4541549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff45415c5d0 0x7ff4541549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 -- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415f6b0 msgr2=0x7ff45414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415f6b0 0x7ff45414f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 -- 172.21.15.46:0/3977330960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff454155320 con 0x7ff454074300 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453e00640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415f6b0 0x7ff45414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff453400640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 0x7ff45414a350 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff444009850 tx=0x7ff4440056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4440b2040 con 0x7ff454074300 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff444003600 con 0x7ff454074300 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45414e100 con 0x7ff454074300 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444003a70 con 0x7ff454074300 2024-09-17T03:26:34.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45414e6c0 con 0x7ff454074300 2024-09-17T03:26:34.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.519+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45415f170 con 0x7ff454074300 2024-09-17T03:26:34.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.523+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7ff4440bf020 con 0x7ff454074300 2024-09-17T03:26:34.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.523+0000 7ff44be00640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff430073b30 0x7ff430076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.523+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff44412d2a0 con 0x7ff454074300 2024-09-17T03:26:34.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.523+0000 7ff44aa00640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff430073b30 0x7ff430076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.523+0000 7ff44aa00640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff430073b30 0x7ff430076040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff45414f210 tx=0x7ff43c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:34.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.527+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff4440c6020 con 0x7ff454074300 2024-09-17T03:26:34.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff45406b540 con 0x7ff430073b30 2024-09-17T03:26:34.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff44be00640 1 -- 172.21.15.46:0/3977330960 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff43c011b50 con 0x7ff430073b30 2024-09-17T03:26:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:26:34.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:26:34.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:26:34.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:26:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:26:34.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff430073b30 msgr2=0x7ff430076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff430073b30 0x7ff430076040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff45414f210 tx=0x7ff43c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 msgr2=0x7ff45414a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 0x7ff45414a350 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff444009850 tx=0x7ff4440056e0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 shutdown_connections 2024-09-17T03:26:34.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff430073b30 0x7ff430076040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff45415f6b0 0x7ff45414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff45415c5d0 0x7ff4541549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 --2- 172.21.15.46:0/3977330960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff454074300 0x7ff45414a350 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 >> 172.21.15.46:0/3977330960 conn(0x7ff4540b2c40 msgr2=0x7ff454072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.723+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 shutdown_connections 2024-09-17T03:26:34.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.727+0000 7ff45a200640 1 -- 172.21.15.46:0/3977330960 wait complete. 2024-09-17T03:26:34.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:26:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 -- 172.21.15.46:0/2790942828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 msgr2=0x7fcf780b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/2790942828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf780b51f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcf78070710 tx=0x7fcf7001cae0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 -- 172.21.15.46:0/2790942828 shutdown_connections 2024-09-17T03:26:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/2790942828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf780b51f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/2790942828 >> v2:172.21.15.99:3300/0 conn(0x7fcf781527b0 0x7fcf7815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 -- 172.21.15.46:0/2790942828 >> 172.21.15.46:0/2790942828 conn(0x7fcf780b2040 msgr2=0x7fcf780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 -- 172.21.15.46:0/2790942828 shutdown_connections 2024-09-17T03:26:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.907+0000 7fcf7ee00640 1 -- 172.21.15.46:0/2790942828 wait complete. 2024-09-17T03:26:34.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 Processor -- start 2024-09-17T03:26:34.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 -- start start 2024-09-17T03:26:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf781fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf780b5c70 0x7fcf781f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf781527b0 0x7fcf7814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78168970 con 0x7fcf780b4de0 2024-09-17T03:26:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781687f0 con 0x7fcf780b5c70 2024-09-17T03:26:34.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcf78168af0 con 0x7fcf781527b0 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf781fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf781fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39046/0 (socket says 172.21.15.46:39046) 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 -- 172.21.15.46:0/1099600195 learned_addr learned my addr 172.21.15.46:0/1099600195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf6fe00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf780b5c70 0x7fcf781f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ce00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf781527b0 0x7fcf7814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 -- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf781527b0 msgr2=0x7fcf7814bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf781527b0 0x7fcf7814bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 -- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf780b5c70 msgr2=0x7fcf781f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf780b5c70 0x7fcf781f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:34.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 -- 172.21.15.46:0/1099600195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf781f9b00 con 0x7fcf780b4de0 2024-09-17T03:26:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf77e00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf781fc210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800b410 tx=0x7fcf6800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68015030 con 0x7fcf780b4de0 2024-09-17T03:26:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf68004020 con 0x7fcf780b4de0 2024-09-17T03:26:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf781fcb10 con 0x7fcf780b4de0 2024-09-17T03:26:34.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68004470 con 0x7fcf780b4de0 2024-09-17T03:26:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.911+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf781f8dc0 con 0x7fcf780b4de0 2024-09-17T03:26:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.915+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf780b51f0 con 0x7fcf780b4de0 2024-09-17T03:26:34.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.915+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fcf68018070 con 0x7fcf780b4de0 2024-09-17T03:26:34.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.915+0000 7fcf74c00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:34.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.915+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcf68092550 con 0x7fcf780b4de0 2024-09-17T03:26:34.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.915+0000 7fcf6fe00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:34.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.919+0000 7fcf6fe00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcf780cf920 tx=0x7fcf6000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:34.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:34.919+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcf6805ea00 con 0x7fcf780b4de0 2024-09-17T03:26:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.095+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcf780cc710 con 0x7fcf780b4de0 2024-09-17T03:26:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.095+0000 7fcf74c00640 1 -- 172.21.15.46:0/1099600195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7fcf68063430 con 0x7fcf780b4de0 2024-09-17T03:26:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.095+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 msgr2=0x7fcf54076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.095+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcf780cf920 tx=0x7fcf6000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.095+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 msgr2=0x7fcf781fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.095+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf781fc210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800b410 tx=0x7fcf6800b8d0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 shutdown_connections 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcf54073b00 0x7fcf54076010 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcf781527b0 0x7fcf7814bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf780b5c70 0x7fcf781f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 --2- 172.21.15.46:0/1099600195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf780b4de0 0x7fcf781fc210 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 >> 172.21.15.46:0/1099600195 conn(0x7fcf780b2040 msgr2=0x7fcf781511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 shutdown_connections 2024-09-17T03:26:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.099+0000 7fcf7ee00640 1 -- 172.21.15.46:0/1099600195 wait complete. 2024-09-17T03:26:35.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 -- 172.21.15.46:0/2990509058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 msgr2=0x7f65440b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 --2- 172.21.15.46:0/2990509058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 0x7f65440b51f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f653c009c80 tx=0x7f653c01cce0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 -- 172.21.15.46:0/2990509058 shutdown_connections 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 --2- 172.21.15.46:0/2990509058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 0x7f65440b51f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 --2- 172.21.15.46:0/2990509058 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65441527b0 0x7f654415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 -- 172.21.15.46:0/2990509058 >> 172.21.15.46:0/2990509058 conn(0x7f65440b2040 msgr2=0x7f65440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 -- 172.21.15.46:0/2990509058 shutdown_connections 2024-09-17T03:26:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.279+0000 7f654bc00640 1 -- 172.21.15.46:0/2990509058 wait complete. 2024-09-17T03:26:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 Processor -- start 2024-09-17T03:26:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 -- start start 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 0x7f654413c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65440b5c70 0x7f654413efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 0x7f654413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6544168ab0 con 0x7f65441527b0 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6544168930 con 0x7f65440b4de0 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6544168c30 con 0x7f65440b5c70 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 0x7f654413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 0x7f654413f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39056/0 (socket says 172.21.15.46:39056) 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 -- 172.21.15.46:0/493902868 learned_addr learned my addr 172.21.15.46:0/493902868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6543e00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65440b5c70 0x7f654413efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549200640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 0x7f654413c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 -- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65440b5c70 msgr2=0x7f654413efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65440b5c70 0x7f654413efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 -- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 msgr2=0x7f654413c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 0x7f654413c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 -- 172.21.15.46:0/493902868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65440af170 con 0x7f65441527b0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6549c00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 0x7f654413f4e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f653c009bd0 tx=0x7f653c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0bb070 con 0x7f65441527b0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f653c004b40 con 0x7f65441527b0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65440af450 con 0x7f65441527b0 2024-09-17T03:26:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0b2040 con 0x7f65441527b0 2024-09-17T03:26:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.283+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65440afa10 con 0x7f65441527b0 2024-09-17T03:26:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.287+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65440aeee0 con 0x7f65441527b0 2024-09-17T03:26:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.287+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f653c0be070 con 0x7f65441527b0 2024-09-17T03:26:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.287+0000 7f6540c00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524073b00 0x7f6524076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.287+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f653c12c850 con 0x7f65441527b0 2024-09-17T03:26:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.287+0000 7f6549200640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524073b00 0x7f6524076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.291+0000 7f6549200640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524073b00 0x7f6524076010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6538007fd0 tx=0x7f6538009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:35.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.291+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f653c016990 con 0x7f65441527b0 2024-09-17T03:26:35.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.467+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f65440b51f0 con 0x7f65441527b0 2024-09-17T03:26:35.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.467+0000 7f6540c00640 1 -- 172.21.15.46:0/493902868 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v38) ==== 116+0+2 (secure 0 0 0) 0x7f653c0f8d00 con 0x7f65441527b0 2024-09-17T03:26:35.470 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524073b00 msgr2=0x7f6524076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524073b00 0x7f6524076010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6538007fd0 tx=0x7f6538009040 comp rx=0 tx=0).stop 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 msgr2=0x7f654413f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 0x7f654413f4e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f653c009bd0 tx=0x7f653c005b00 comp rx=0 tx=0).stop 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 shutdown_connections 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524073b00 0x7f6524076010 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65441527b0 0x7f654413f4e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65440b5c70 0x7f654413efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 --2- 172.21.15.46:0/493902868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65440b4de0 0x7f654413c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 >> 172.21.15.46:0/493902868 conn(0x7f65440b2040 msgr2=0x7f65440b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 shutdown_connections 2024-09-17T03:26:35.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.471+0000 7f654bc00640 1 -- 172.21.15.46:0/493902868 wait complete. 2024-09-17T03:26:35.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:26:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 -- 172.21.15.46:0/3110979135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 msgr2=0x7f51a80cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 --2- 172.21.15.46:0/3110979135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80cf5d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f51a815fa90 tx=0x7f51a001cd10 comp rx=0 tx=0).stop 2024-09-17T03:26:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 -- 172.21.15.46:0/3110979135 shutdown_connections 2024-09-17T03:26:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 --2- 172.21.15.46:0/3110979135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80cf5d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 --2- 172.21.15.46:0/3110979135 >> v2:172.21.15.99:3300/0 conn(0x7f51a80d0ce0 0x7f51a80d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 -- 172.21.15.46:0/3110979135 >> 172.21.15.46:0/3110979135 conn(0x7f51a8073a10 msgr2=0x7f51a806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 -- 172.21.15.46:0/3110979135 shutdown_connections 2024-09-17T03:26:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 -- 172.21.15.46:0/3110979135 wait complete. 2024-09-17T03:26:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 Processor -- start 2024-09-17T03:26:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.655+0000 7f51ad000640 1 -- start start 2024-09-17T03:26:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 0x7f51a8154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51a81625f0 0x7f51a8154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f51a80d3140 con 0x7f51a80d0ce0 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f51a80d2fc0 con 0x7f51a81625f0 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f51a80d32c0 con 0x7f51a80cf1c0 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 0x7f51a8154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51a7400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51a7400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:53006/0 (socket says 172.21.15.46:53006) 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51a7400640 1 -- 172.21.15.46:0/188221683 learned_addr learned my addr 172.21.15.46:0/188221683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 0x7f51a8154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39058/0 (socket says 172.21.15.46:39058) 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51a7e00640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51a81625f0 0x7f51a8154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 -- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 msgr2=0x7f51a80b9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80b9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 -- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51a81625f0 msgr2=0x7f51a8154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51a81625f0 0x7f51a8154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 -- 172.21.15.46:0/188221683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a8140e30 con 0x7f51a80d0ce0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51a7400640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80b9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519fe00640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 0x7f51a8154590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f519000c390 tx=0x7f519000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190017070 con 0x7f51a80d0ce0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5190003600 con 0x7f51a80d0ce0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190003a50 con 0x7f51a80d0ce0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a80c8190 con 0x7f51a80d0ce0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a80c9ff0 con 0x7f51a80d0ce0 2024-09-17T03:26:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.659+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a80cf5d0 con 0x7f51a80d0ce0 2024-09-17T03:26:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.663+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f519001a070 con 0x7f51a80d0ce0 2024-09-17T03:26:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.663+0000 7f519f400640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5184073b00 0x7f5184076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.663+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5190091c50 con 0x7f51a80d0ce0 2024-09-17T03:26:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.663+0000 7f51a7400640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5184073b00 0x7f5184076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:35.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.663+0000 7f51a7400640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5184073b00 0x7f5184076010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5198009680 tx=0x7f51980091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:35.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.667+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f519005e0e0 con 0x7f51a80d0ce0 2024-09-17T03:26:35.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.839+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f51a80ce7a0 con 0x7f51a80d0ce0 2024-09-17T03:26:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.843+0000 7f519f400640 1 -- 172.21.15.46:0/188221683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7f5190062b10 con 0x7f51a80d0ce0 2024-09-17T03:26:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5184073b00 msgr2=0x7f5184076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5184073b00 0x7f5184076010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5198009680 tx=0x7f51980091e0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 msgr2=0x7f51a8154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 0x7f51a8154590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f519000c390 tx=0x7f519000c850 comp rx=0 tx=0).stop 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 shutdown_connections 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5184073b00 0x7f5184076010 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51a81625f0 0x7f51a8154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80d0ce0 0x7f51a8154590 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 --2- 172.21.15.46:0/188221683 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f51a80cf1c0 0x7f51a80b9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 >> 172.21.15.46:0/188221683 conn(0x7f51a8073a10 msgr2=0x7f51a8162a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 shutdown_connections 2024-09-17T03:26:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:35.847+0000 7f51ad000640 1 -- 172.21.15.46:0/188221683 wait complete. 2024-09-17T03:26:35.864 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:35.863+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:35.864 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:35.863+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:35.864 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:35.863+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:35.865 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:35.863+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:35.865 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:35.863+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:35.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.031+0000 7f2d98400640 1 -- 172.21.15.46:0/746583611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d901527b0 msgr2=0x7f2d90074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.031+0000 7f2d98400640 1 --2- 172.21.15.46:0/746583611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d901527b0 0x7f2d90074e30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2d90071cb0 tx=0x7f2d7c01c890 comp rx=0 tx=0).stop 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- 172.21.15.46:0/746583611 shutdown_connections 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 --2- 172.21.15.46:0/746583611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d901527b0 0x7f2d90074e30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 --2- 172.21.15.46:0/746583611 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2d900b4de0 0x7f2d900b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 --2- 172.21.15.46:0/746583611 >> v2:172.21.15.99:3300/0 conn(0x7f2d900b5c70 0x7f2d9015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- 172.21.15.46:0/746583611 >> 172.21.15.46:0/746583611 conn(0x7f2d900b2040 msgr2=0x7f2d900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- 172.21.15.46:0/746583611 shutdown_connections 2024-09-17T03:26:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- 172.21.15.46:0/746583611 wait complete. 2024-09-17T03:26:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 Processor -- start 2024-09-17T03:26:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- start start 2024-09-17T03:26:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 0x7f2d90150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2d900b5c70 0x7f2d900aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d901527b0 0x7f2d901577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9015f5f0 con 0x7f2d900b4de0 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9015f470 con 0x7f2d901527b0 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.035+0000 7f2d98400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9015f770 con 0x7f2d900b5c70 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 0x7f2d90150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 0x7f2d90150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39086/0 (socket says 172.21.15.46:39086) 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 -- 172.21.15.46:0/2484981625 learned_addr learned my addr 172.21.15.46:0/2484981625 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d96400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d901527b0 0x7f2d901577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95000640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2d900b5c70 0x7f2d900aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 -- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2d900b5c70 msgr2=0x7f2d900aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2d900b5c70 0x7f2d900aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 -- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d901527b0 msgr2=0x7f2d901577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d901527b0 0x7f2d901577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 -- 172.21.15.46:0/2484981625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901518d0 con 0x7f2d900b4de0 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d95a00640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 0x7f2d90150e30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d750 tx=0x7f2d8400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84017070 con 0x7f2d900b4de0 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d84004310 con 0x7f2d900b4de0 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84005280 con 0x7f2d900b4de0 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d90151bb0 con 0x7f2d900b4de0 2024-09-17T03:26:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9013ce00 con 0x7f2d900b4de0 2024-09-17T03:26:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9015fcd0 con 0x7f2d900b4de0 2024-09-17T03:26:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.039+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2d84005420 con 0x7f2d900b4de0 2024-09-17T03:26:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.043+0000 7f2d8d600640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2d70073b00 0x7f2d70076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.043+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2d84092e50 con 0x7f2d900b4de0 2024-09-17T03:26:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.043+0000 7f2d95000640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2d70073b00 0x7f2d70076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.043+0000 7f2d95000640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2d70073b00 0x7f2d70076010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0af000 tx=0x7f2d7c01b0d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.047+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2d84020020 con 0x7f2d900b4de0 2024-09-17T03:26:36.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.247+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f2d9006b540 con 0x7f2d70073b00 2024-09-17T03:26:36.256 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:36.255+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:36.256 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:36.255+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:36.256 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:36.255+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:36.256 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:36.255+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:36.256 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:26:36.255+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:26:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.291+0000 7f2d8d600640 1 -- 172.21.15.46:0/2484981625 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d7c0b3040 con 0x7f2d70073b00 2024-09-17T03:26:36.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2d70073b00 msgr2=0x7f2d70076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2d70073b00 0x7f2d70076010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0af000 tx=0x7f2d7c01b0d0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 msgr2=0x7f2d90150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 0x7f2d90150e30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d750 tx=0x7f2d8400db50 comp rx=0 tx=0).stop 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 shutdown_connections 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2d70073b00 0x7f2d70076010 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d901527b0 0x7f2d901577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2d900b5c70 0x7f2d900aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 --2- 172.21.15.46:0/2484981625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d900b4de0 0x7f2d90150e30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 >> 172.21.15.46:0/2484981625 conn(0x7f2d900b2040 msgr2=0x7f2d900b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 shutdown_connections 2024-09-17T03:26:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.295+0000 7f2d98400640 1 -- 172.21.15.46:0/2484981625 wait complete. 2024-09-17T03:26:36.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:26:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- 172.21.15.46:0/2267568605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 msgr2=0x7fa64815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 --2- 172.21.15.46:0/2267568605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 0x7fa64815cf90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa640009c80 tx=0x7fa64001cda0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- 172.21.15.46:0/2267568605 shutdown_connections 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 --2- 172.21.15.46:0/2267568605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 0x7fa64815cf90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 --2- 172.21.15.46:0/2267568605 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6480b5a80 0x7fa648074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- 172.21.15.46:0/2267568605 >> 172.21.15.46:0/2267568605 conn(0x7fa6480708c0 msgr2=0x7fa648070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- 172.21.15.46:0/2267568605 shutdown_connections 2024-09-17T03:26:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- 172.21.15.46:0/2267568605 wait complete. 2024-09-17T03:26:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 Processor -- start 2024-09-17T03:26:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- start start 2024-09-17T03:26:36.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 0x7fa6480b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 0x7fa648148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa648157030 0x7fa648151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa64816a3c0 con 0x7fa6480b5a80 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa64816a240 con 0x7fa648074950 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.479+0000 7fa650200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa64816a540 con 0x7fa648157030 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 0x7fa648148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 0x7fa648148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39114/0 (socket says 172.21.15.46:39114) 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 -- 172.21.15.46:0/3869534737 learned_addr learned my addr 172.21.15.46:0/3869534737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64d800640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 0x7fa6480b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64e200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa648157030 0x7fa648151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 -- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa648157030 msgr2=0x7fa648151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa648157030 0x7fa648151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 -- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 msgr2=0x7fa6480b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 0x7fa6480b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 -- 172.21.15.46:0/3869534737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64814a000 con 0x7fa6480b5a80 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa64ce00640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 0x7fa648148e50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e770 tx=0x7fa63400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63400ccc0 con 0x7fa6480b5a80 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa63400ee80 con 0x7fa6480b5a80 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64814a2e0 con 0x7fa6480b5a80 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634010670 con 0x7fa6480b5a80 2024-09-17T03:26:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6481f9ef0 con 0x7fa6480b5a80 2024-09-17T03:26:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64815cf90 con 0x7fa6480b5a80 2024-09-17T03:26:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.483+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa634010810 con 0x7fa6480b5a80 2024-09-17T03:26:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.487+0000 7fa645600640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa628073b00 0x7fa628076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:26:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.487+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa634093ce0 con 0x7fa6480b5a80 2024-09-17T03:26:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.487+0000 7fa64d800640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa628073b00 0x7fa628076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:26:36.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.487+0000 7fa64d800640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa628073b00 0x7fa628076010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001650 tx=0x7fa63c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:26:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.487+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6340607e0 con 0x7fa6480b5a80 2024-09-17T03:26:36.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.691+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa64806b540 con 0x7fa628073b00 2024-09-17T03:26:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.695+0000 7fa645600640 1 -- 172.21.15.46:0/3869534737 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa63c0050d0 con 0x7fa628073b00 2024-09-17T03:26:36.698 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa628073b00 msgr2=0x7fa628076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa628073b00 0x7fa628076010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001650 tx=0x7fa63c009040 comp rx=0 tx=0).stop 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 msgr2=0x7fa648148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 0x7fa648148e50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e770 tx=0x7fa63400ec30 comp rx=0 tx=0).stop 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 shutdown_connections 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa628073b00 0x7fa628076010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa648157030 0x7fa648151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6480b5a80 0x7fa648148e50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 --2- 172.21.15.46:0/3869534737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa648074950 0x7fa6480b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 >> 172.21.15.46:0/3869534737 conn(0x7fa6480708c0 msgr2=0x7fa64815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 shutdown_connections 2024-09-17T03:26:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:26:36.699+0000 7fa650200640 1 -- 172.21.15.46:0/3869534737 wait complete. 2024-09-17T03:26:36.721 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:26:36.721 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:36.721 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:36.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:36.731 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:26:45.351 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:26:45.352 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:26:45.352 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.56014 s, 125 MB/s 2024-09-17T03:26:45.353 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:45.353 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:45.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:45.363 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:26:53.764 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:26:53.764 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:26:53.765 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.34779 s, 129 MB/s 2024-09-17T03:26:53.766 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:26:53.766 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:26:53.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:26:53.775 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:26:59.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:26:59.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:27:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.36571 s, 128 MB/s 2024-09-17T03:27:02.197 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:02.197 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:02.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:02.207 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.48874 s, 126 MB/s 2024-09-17T03:27:10.747 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:10.748 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:10.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:10.757 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.24357 s, 130 MB/s 2024-09-17T03:27:19.058 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:19.058 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:19.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:19.067 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.36055 s, 128 MB/s 2024-09-17T03:27:27.482 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:27.482 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:27.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:27.493 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:29.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:27:29.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:27:35.782 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:35.782 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:35.782 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.23378 s, 130 MB/s 2024-09-17T03:27:35.784 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:35.784 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:35.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:35.793 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.67274 s, 124 MB/s 2024-09-17T03:27:44.523 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:44.523 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:44.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:44.533 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:27:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:27:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.12326 s, 132 MB/s 2024-09-17T03:27:52.708 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:27:52.709 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:27:52.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:27:52.718 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:27:59.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:27:59.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:28:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:28:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:28:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.33965 s, 129 MB/s 2024-09-17T03:28:01.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:28:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 -- 172.21.15.46:0/635516843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 msgr2=0x7f0af80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 --2- 172.21.15.46:0/635516843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af80b51f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0af8071cb0 tx=0x7f0af001ccc0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 -- 172.21.15.46:0/635516843 shutdown_connections 2024-09-17T03:28:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 --2- 172.21.15.46:0/635516843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af80b51f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 --2- 172.21.15.46:0/635516843 >> v2:172.21.15.99:3300/0 conn(0x7f0af80b5c70 0x7f0af815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 -- 172.21.15.46:0/635516843 >> 172.21.15.46:0/635516843 conn(0x7f0af80b2040 msgr2=0x7f0af80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:01.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 -- 172.21.15.46:0/635516843 shutdown_connections 2024-09-17T03:28:01.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 -- 172.21.15.46:0/635516843 wait complete. 2024-09-17T03:28:01.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 Processor -- start 2024-09-17T03:28:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.274+0000 7f0afe000640 1 -- start start 2024-09-17T03:28:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af81f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0af80b5c70 0x7f0af814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af81527b0 0x7f0af815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0af8168920 con 0x7f0af80b4de0 2024-09-17T03:28:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0af81687a0 con 0x7f0af81527b0 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0af8168aa0 con 0x7f0af80b5c70 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af81f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af81f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46160/0 (socket says 172.21.15.46:46160) 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 -- 172.21.15.46:0/3015442215 learned_addr learned my addr 172.21.15.46:0/3015442215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0aefe00640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0af80b5c70 0x7f0af814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7e00640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af81527b0 0x7f0af815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 -- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0af80b5c70 msgr2=0x7f0af814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0af80b5c70 0x7f0af814e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 -- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af81527b0 msgr2=0x7f0af815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af81527b0 0x7f0af815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 -- 172.21.15.46:0/3015442215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af80cdf00 con 0x7f0af80b4de0 2024-09-17T03:28:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0af7400640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af81f6760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800e7c0 tx=0x7f0ae800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800ccc0 con 0x7f0af80b4de0 2024-09-17T03:28:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ae800ce80 con 0x7f0af80b4de0 2024-09-17T03:28:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af80ce130 con 0x7f0af80b4de0 2024-09-17T03:28:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8005280 con 0x7f0af80b4de0 2024-09-17T03:28:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af80cd2c0 con 0x7f0af80b4de0 2024-09-17T03:28:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.278+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af80b51f0 con 0x7f0af80b4de0 2024-09-17T03:28:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.282+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 98050+0+0 (secure 0 0 0) 0x7f0ae8005420 con 0x7f0af80b4de0 2024-09-17T03:28:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.282+0000 7f0aef400640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ad4073b00 0x7f0ad4076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.282+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0ae80933a0 con 0x7f0af80b4de0 2024-09-17T03:28:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.282+0000 7f0aefe00640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ad4073b00 0x7f0ad4076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.282+0000 7f0aefe00640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ad4073b00 0x7f0ad4076010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0af80afa20 tx=0x7f0ae0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:01.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.286+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ae805f850 con 0x7f0af80b4de0 2024-09-17T03:28:01.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.490+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f0af806b540 con 0x7f0ad4073b00 2024-09-17T03:28:01.500 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:01.494+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:01.501 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:01.494+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:01.501 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:01.494+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:01.501 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:01.494+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:01.501 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:01.494+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.518+0000 7f0aef400640 1 -- 172.21.15.46:0/3015442215 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ae0001d20 con 0x7f0ad4073b00 2024-09-17T03:28:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ad4073b00 msgr2=0x7f0ad4076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ad4073b00 0x7f0ad4076010 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0af80afa20 tx=0x7f0ae0009040 comp rx=0 tx=0).stop 2024-09-17T03:28:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 msgr2=0x7f0af81f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af81f6760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800e7c0 tx=0x7f0ae800ec80 comp rx=0 tx=0).stop 2024-09-17T03:28:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 shutdown_connections 2024-09-17T03:28:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ad4073b00 0x7f0ad4076010 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af81527b0 0x7f0af815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0af80b5c70 0x7f0af814e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 --2- 172.21.15.46:0/3015442215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0af80b4de0 0x7f0af81f6760 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 >> 172.21.15.46:0/3015442215 conn(0x7f0af80b2040 msgr2=0x7f0af8151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 shutdown_connections 2024-09-17T03:28:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.522+0000 7f0afe000640 1 -- 172.21.15.46:0/3015442215 wait complete. 2024-09-17T03:28:01.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:28:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 -- 172.21.15.46:0/1073207795 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de81527b0 msgr2=0x7f8de815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 --2- 172.21.15.46:0/1073207795 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de81527b0 0x7f8de815bb60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009900 tx=0x7f8dd001c820 comp rx=0 tx=0).stop 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 -- 172.21.15.46:0/1073207795 shutdown_connections 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 --2- 172.21.15.46:0/1073207795 >> v2:172.21.15.99:3300/0 conn(0x7f8de80b4de0 0x7f8de80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 --2- 172.21.15.46:0/1073207795 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de81527b0 0x7f8de815bb60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 -- 172.21.15.46:0/1073207795 >> 172.21.15.46:0/1073207795 conn(0x7f8de80b2040 msgr2=0x7f8de80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 -- 172.21.15.46:0/1073207795 shutdown_connections 2024-09-17T03:28:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 -- 172.21.15.46:0/1073207795 wait complete. 2024-09-17T03:28:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 Processor -- start 2024-09-17T03:28:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 -- start start 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.706+0000 7f8decc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80b4de0 0x7f8de81fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de80b5c70 0x7f8de81fa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 0x7f8de80ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8de81689b0 con 0x7f8de81527b0 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8de8168830 con 0x7f8de80b4de0 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8de8168b30 con 0x7f8de80b5c70 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 0x7f8de80ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 0x7f8de80ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46186/0 (socket says 172.21.15.46:46186) 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 -- 172.21.15.46:0/1981586822 learned_addr learned my addr 172.21.15.46:0/1981586822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8ddfe00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de80b5c70 0x7f8de81fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de5e00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80b4de0 0x7f8de81fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 -- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de80b5c70 msgr2=0x7f8de81fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de80b5c70 0x7f8de81fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 -- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80b4de0 msgr2=0x7f8de81fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80b4de0 0x7f8de81fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 -- 172.21.15.46:0/1981586822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de80cf3c0 con 0x7f8de81527b0 2024-09-17T03:28:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8de6800640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 0x7f8de80ab4e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8de000ec70 tx=0x7f8de000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003b80 con 0x7f8de81527b0 2024-09-17T03:28:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8de0003d20 con 0x7f8de81527b0 2024-09-17T03:28:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de80cf6a0 con 0x7f8de81527b0 2024-09-17T03:28:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0004280 con 0x7f8de81527b0 2024-09-17T03:28:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de80cfc60 con 0x7f8de81527b0 2024-09-17T03:28:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de80b51f0 con 0x7f8de81527b0 2024-09-17T03:28:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.710+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 98050+0+0 (secure 0 0 0) 0x7f8de0004420 con 0x7f8de81527b0 2024-09-17T03:28:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.714+0000 7f8dde000640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc4073b00 0x7f8dc4076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.714+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8de0093770 con 0x7f8de81527b0 2024-09-17T03:28:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.714+0000 7f8de5e00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc4073b00 0x7f8dc4076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.714+0000 7f8de5e00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc4073b00 0x7f8dc4076010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8007fd0 tx=0x7f8dd8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.718+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8de005fb10 con 0x7f8de81527b0 2024-09-17T03:28:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.922+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8de806b540 con 0x7f8dc4073b00 2024-09-17T03:28:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.926+0000 7f8dde000640 1 -- 172.21.15.46:0/1981586822 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8dd8001d20 con 0x7f8dc4073b00 2024-09-17T03:28:01.930 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:01.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc4073b00 msgr2=0x7f8dc4076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc4073b00 0x7f8dc4076010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8007fd0 tx=0x7f8dd8009040 comp rx=0 tx=0).stop 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 msgr2=0x7f8de80ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 0x7f8de80ab4e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8de000ec70 tx=0x7f8de000c6a0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 shutdown_connections 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc4073b00 0x7f8dc4076010 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de81527b0 0x7f8de80ab4e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de80b5c70 0x7f8de81fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 --2- 172.21.15.46:0/1981586822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80b4de0 0x7f8de81fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 >> 172.21.15.46:0/1981586822 conn(0x7f8de80b2040 msgr2=0x7f8de80b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 shutdown_connections 2024-09-17T03:28:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:01.930+0000 7f8decc00640 1 -- 172.21.15.46:0/1981586822 wait complete. 2024-09-17T03:28:01.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:01.953 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:01.953 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:28:01.962 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:01.962 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10405347328" 2024-09-17T03:28:01.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:01.963 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-17T03:28:03.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:03.964 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:03.964 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:28:03.975 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:03.975 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10405347328" 2024-09-17T03:28:03.975 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:03.976 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-17T03:28:05.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:05.977 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:05.977 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:28:05.988 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:05.988 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10405347328" 2024-09-17T03:28:05.988 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:05.989 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-17T03:28:07.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:07.990 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:07.990 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:28:08.001 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/a27d7291-30f8-4dc2-a0fa-bed5ce8adec4 2024-09-17T03:28:08.001 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="10737418240" 2024-09-17T03:28:08.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:08.002 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-17T03:28:08.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f998e800640 1 -- 172.21.15.46:0/1756638108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f996c003c20 msgr2=0x7f996c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f998e800640 1 --2- 172.21.15.46:0/1756638108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f996c003c20 0x7f996c004070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9978009b60 tx=0x7f997801d290 comp rx=0 tx=0).stop 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f9987e00640 1 -- 172.21.15.46:0/1756638108 reap_dead start 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f998e800640 1 -- 172.21.15.46:0/1756638108 shutdown_connections 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f998e800640 1 -- 172.21.15.46:0/1756638108 >> 172.21.15.46:0/1756638108 conn(0x7f99880708c0 msgr2=0x7f9988070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f998e800640 1 -- 172.21.15.46:0/1756638108 shutdown_connections 2024-09-17T03:28:08.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.218+0000 7f998e800640 1 -- 172.21.15.46:0/1756638108 wait complete. 2024-09-17T03:28:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 Processor -- start 2024-09-17T03:28:08.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 -- start start 2024-09-17T03:28:08.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f996c003c20 0x7f99881f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 0x7f99881fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9988150be0 0x7f99880b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99881693c0 con 0x7f99881507d0 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9988169240 con 0x7f996c003c20 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9988169540 con 0x7f9988150be0 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 0x7f99881fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 0x7f99881fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53700/0 (socket says 172.21.15.46:53700) 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 -- 172.21.15.46:0/2810191967 learned_addr learned my addr 172.21.15.46:0/2810191967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f9987400640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f996c003c20 0x7f99881f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f9987e00640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9988150be0 0x7f99880b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 -- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9988150be0 msgr2=0x7f99880b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9988150be0 0x7f99880b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 -- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f996c003c20 msgr2=0x7f99881f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f996c003c20 0x7f99881f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 -- 172.21.15.46:0/2810191967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880b4880 con 0x7f99881507d0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997fe00640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 0x7f99881fd2b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9970006630 tx=0x7f99700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970015070 con 0x7f99881507d0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9970011840 con 0x7f99881507d0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99880b4b60 con 0x7f99881507d0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970011c50 con 0x7f99881507d0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.222+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881fcd90 con 0x7f99881507d0 2024-09-17T03:28:08.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.226+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998815cf90 con 0x7f99881507d0 2024-09-17T03:28:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.226+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 98101+0+0 (secure 0 0 0) 0x7f9970018070 con 0x7f99881507d0 2024-09-17T03:28:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.226+0000 7f997f400640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9964073bc0 0x7f99640760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.226+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f99700934f0 con 0x7f99881507d0 2024-09-17T03:28:08.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.226+0000 7f9987400640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9964073bc0 0x7f99640760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.226+0000 7f9987400640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9964073bc0 0x7f99640760d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99780ae000 tx=0x7f997801a030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:08.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.230+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f997005fb50 con 0x7f99881507d0 2024-09-17T03:28:08.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.438+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f99881f9b60 con 0x7f9964073bc0 2024-09-17T03:28:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.514+0000 7f997f400640 1 -- 172.21.15.46:0/2810191967 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99780052f0 con 0x7f9964073bc0 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9964073bc0 msgr2=0x7f99640760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9964073bc0 0x7f99640760d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99780ae000 tx=0x7f997801a030 comp rx=0 tx=0).stop 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 msgr2=0x7f99881fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 0x7f99881fd2b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9970006630 tx=0x7f99700039d0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 shutdown_connections 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9964073bc0 0x7f99640760d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9988150be0 0x7f99880b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99881507d0 0x7f99881fd2b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 --2- 172.21.15.46:0/2810191967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f996c003c20 0x7f99881f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 >> 172.21.15.46:0/2810191967 conn(0x7f99880708c0 msgr2=0x7f998815ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 shutdown_connections 2024-09-17T03:28:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.518+0000 7f998e800640 1 -- 172.21.15.46:0/2810191967 wait complete. 2024-09-17T03:28:08.521 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.518+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.521 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.518+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.521 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.518+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.521 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.518+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.521 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.518+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.540 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.538+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.541 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.538+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.541 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.538+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.541 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.538+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.541 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:08.538+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:08.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:28:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 -- 172.21.15.46:0/3613984627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 msgr2=0x7fee64074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 --2- 172.21.15.46:0/3613984627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee64074950 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fee640713e0 tx=0x7fee5c01cd80 comp rx=0 tx=0).stop 2024-09-17T03:28:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 -- 172.21.15.46:0/3613984627 shutdown_connections 2024-09-17T03:28:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 --2- 172.21.15.46:0/3613984627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee64074950 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 --2- 172.21.15.46:0/3613984627 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee64160d50 0x7fee64074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 --2- 172.21.15.46:0/3613984627 >> v2:172.21.15.99:3300/0 conn(0x7fee64157030 0x7fee640b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 -- 172.21.15.46:0/3613984627 >> 172.21.15.46:0/3613984627 conn(0x7fee640708c0 msgr2=0x7fee64070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 -- 172.21.15.46:0/3613984627 shutdown_connections 2024-09-17T03:28:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.706+0000 7fee6a200640 1 -- 172.21.15.46:0/3613984627 wait complete. 2024-09-17T03:28:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 Processor -- start 2024-09-17T03:28:08.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 -- start start 2024-09-17T03:28:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64157030 0x7fee640b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee64160d50 0x7fee640b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee6414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee640722c0 con 0x7fee64161160 2024-09-17T03:28:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee64072140 con 0x7fee64157030 2024-09-17T03:28:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee6a200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fee64072440 con 0x7fee64160d50 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee6414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee6414cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53766/0 (socket says 172.21.15.46:53766) 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee63e00640 1 -- 172.21.15.46:0/2487112796 learned_addr learned my addr 172.21.15.46:0/2487112796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee63400640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64157030 0x7fee640b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.710+0000 7fee5be00640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee64160d50 0x7fee640b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63e00640 1 -- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee64160d50 msgr2=0x7fee640b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63e00640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee64160d50 0x7fee640b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63e00640 1 -- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64157030 msgr2=0x7fee640b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63e00640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64157030 0x7fee640b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63e00640 1 -- 172.21.15.46:0/2487112796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee640b2020 con 0x7fee64161160 2024-09-17T03:28:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63400640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64157030 0x7fee640b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee63e00640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee6414cc90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009bd0 tx=0x7fee5c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0bb070 con 0x7fee64161160 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee5c004b40 con 0x7fee64161160 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0b2040 con 0x7fee64161160 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee64150200 con 0x7fee64161160 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641507c0 con 0x7fee64161160 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee640728f0 con 0x7fee64161160 2024-09-17T03:28:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 98101+0+0 (secure 0 0 0) 0x7fee5c0be070 con 0x7fee64161160 2024-09-17T03:28:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.714+0000 7fee5b400640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee40073bc0 0x7fee400760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.718+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fee5c12caa0 con 0x7fee64161160 2024-09-17T03:28:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.718+0000 7fee63400640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee40073bc0 0x7fee400760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.718+0000 7fee63400640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee40073bc0 0x7fee400760d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fee54009b10 tx=0x7fee540091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.718+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fee5c016970 con 0x7fee64161160 2024-09-17T03:28:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.958+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fee640d2260 con 0x7fee64161160 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.958+0000 7fee5b400640 1 -- 172.21.15.46:0/2487112796 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1915 (secure 0 0 0) 0x7fee5c100e20 con 0x7fee64161160 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:28:08.961 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 446, 2024-09-17T03:28:08.962 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 58, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.963 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2585, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10738564099, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 21886017536, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 751208095744, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:28:08.964 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 85, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 121610170, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 0, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 31 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 36, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:26:34:319773+0000", 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 4, 2024-09-17T03:28:08.965 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 245850 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:28:08.966 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 10, 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:27:27.320954+0000", 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:28:08.967 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee40073bc0 msgr2=0x7fee400760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee40073bc0 0x7fee400760d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fee54009b10 tx=0x7fee540091e0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 msgr2=0x7fee6414cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee6414cc90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009bd0 tx=0x7fee5c003f60 comp rx=0 tx=0).stop 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 shutdown_connections 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fee40073bc0 0x7fee400760d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee64161160 0x7fee6414cc90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fee64160d50 0x7fee640b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 --2- 172.21.15.46:0/2487112796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64157030 0x7fee640b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 >> 172.21.15.46:0/2487112796 conn(0x7fee640708c0 msgr2=0x7fee640d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 shutdown_connections 2024-09-17T03:28:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:08.962+0000 7fee6a200640 1 -- 172.21.15.46:0/2487112796 wait complete. 2024-09-17T03:28:09.525 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:09.522+0000 7fbe40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:09.526 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:09.522+0000 7fbe40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:09.526 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:09.522+0000 7fbe40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:09.526 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:09.522+0000 7fbe40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:09.526 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:09.522+0000 7fbe40000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:09.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:28:10.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.314+0000 7f44b3e00640 1 -- 172.21.15.46:0/573459433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0ce4c0 msgr2=0x7f44ac15a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:10.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.314+0000 7f44b3e00640 1 --2- 172.21.15.46:0/573459433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0ce4c0 0x7f44ac15a880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44ac06f210 tx=0x7f449801c600 comp rx=0 tx=0).stop 2024-09-17T03:28:10.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 -- 172.21.15.46:0/573459433 shutdown_connections 2024-09-17T03:28:10.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 --2- 172.21.15.46:0/573459433 >> v2:172.21.15.46:3300/0 conn(0x7f44ac0cdbb0 0x7f44ac15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 --2- 172.21.15.46:0/573459433 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44ac0cd260 0x7f44ac0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 --2- 172.21.15.46:0/573459433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0ce4c0 0x7f44ac15a880 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 -- 172.21.15.46:0/573459433 >> 172.21.15.46:0/573459433 conn(0x7f44ac0b2040 msgr2=0x7f44ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:10.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 -- 172.21.15.46:0/573459433 shutdown_connections 2024-09-17T03:28:10.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.318+0000 7f44b3e00640 1 -- 172.21.15.46:0/573459433 wait complete. 2024-09-17T03:28:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 Processor -- start 2024-09-17T03:28:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 -- start start 2024-09-17T03:28:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44ac0cd260 0x7f44ac159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 0x7f44ac1458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44ac0ce4c0 0x7f44ac1fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0b3a80 con 0x7f44ac0ce4c0 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac0b3900 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b3e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0b3c00 con 0x7f44ac0cd260 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 0x7f44ac1458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 0x7f44ac1458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:55360/0 (socket says 172.21.15.46:55360) 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b0a00640 1 -- 172.21.15.46:0/920449342 learned_addr learned my addr 172.21.15.46:0/920449342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.322+0000 7f44b1400640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44ac0cd260 0x7f44ac159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.326+0000 7f44b0a00640 1 -- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44ac0cd260 msgr2=0x7f44ac159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.326+0000 7f44b0a00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44ac0cd260 0x7f44ac159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.326+0000 7f44b0a00640 1 -- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44ac0ce4c0 msgr2=0x7f44ac1fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.326+0000 7f44b0a00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44ac0ce4c0 0x7f44ac1fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.326+0000 7f44b0a00640 1 -- 172.21.15.46:0/920449342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac165810 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.326+0000 7f44b0a00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 0x7f44ac1458f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c790 tx=0x7f44a000cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.330+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0007bb0 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.330+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44a0007e40 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.330+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0005280 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.330+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac146440 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.330+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac165db0 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.330+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac15e4e0 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.334+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 98203+0+0 (secure 0 0 0) 0x7f44a0005420 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.338+0000 7f44a9600640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f448c073cb0 0x7f448c0761c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:10.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.338+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f44a0093300 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.342+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f44a0014030 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.342+0000 7f44b1400640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f448c073cb0 0x7f448c0761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.354+0000 7f44b1400640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f448c073cb0 0x7f448c0761c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f44980af000 tx=0x7f449801ae40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:10.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.786+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f44ac0b0b10 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.794+0000 7f44a9600640 1 -- 172.21.15.46:0/920449342 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1914 (secure 0 0 0) 0x7f44a005f7f0 con 0x7f44ac0cdbb0 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:28:10.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 448, 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:28:10.798 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 58, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:28:10.799 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2585, 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10738564099, 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 21886017536, 2024-09-17T03:28:10.800 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 751208095744, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 85, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 89454184, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 0, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 22 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 36, 2024-09-17T03:28:10.801 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:26:34:319773+0000", 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4, 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 4, 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:28:10.802 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 245850 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:28:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 10, 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:27:27.320954+0000", 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:28:10.804 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f448c073cb0 msgr2=0x7f448c0761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f448c073cb0 0x7f448c0761c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f44980af000 tx=0x7f449801ae40 comp rx=0 tx=0).stop 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 msgr2=0x7f44ac1458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 0x7f44ac1458f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c790 tx=0x7f44a000cc50 comp rx=0 tx=0).stop 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 shutdown_connections 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f448c073cb0 0x7f448c0761c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44ac0ce4c0 0x7f44ac1fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0cdbb0 0x7f44ac1458f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 --2- 172.21.15.46:0/920449342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44ac0cd260 0x7f44ac159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 >> 172.21.15.46:0/920449342 conn(0x7f44ac0b2040 msgr2=0x7f44ac0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 shutdown_connections 2024-09-17T03:28:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:10.798+0000 7f44b3e00640 1 -- 172.21.15.46:0/920449342 wait complete. 2024-09-17T03:28:11.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:28:12.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 -- 172.21.15.46:0/1643286321 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a806e330 msgr2=0x7fb4a80b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:12.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 --2- 172.21.15.46:0/1643286321 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a806e330 0x7fb4a80b51e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009c80 tx=0x7fb49c01cdb0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 -- 172.21.15.46:0/1643286321 shutdown_connections 2024-09-17T03:28:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 --2- 172.21.15.46:0/1643286321 >> v2:172.21.15.99:3300/0 conn(0x7fb4a8070530 0x7fb4a8070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 --2- 172.21.15.46:0/1643286321 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a806e330 0x7fb4a80b51e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 -- 172.21.15.46:0/1643286321 >> 172.21.15.46:0/1643286321 conn(0x7fb4a80b3a20 msgr2=0x7fb4a80d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 -- 172.21.15.46:0/1643286321 shutdown_connections 2024-09-17T03:28:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.178+0000 7fb4b0000640 1 -- 172.21.15.46:0/1643286321 wait complete. 2024-09-17T03:28:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.182+0000 7fb4b0000640 1 Processor -- start 2024-09-17T03:28:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.182+0000 7fb4b0000640 1 -- start start 2024-09-17T03:28:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4b0000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a806e330 0x7fb4a8146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4b0000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a8070530 0x7fb4a80ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4b0000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 0x7fb4a80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4b0000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8159140 con 0x7fb4a80b44a0 2024-09-17T03:28:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4b0000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8158fc0 con 0x7fb4a806e330 2024-09-17T03:28:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4b0000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a81592c0 con 0x7fb4a8070530 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 0x7fb4a80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 0x7fb4a80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53814/0 (socket says 172.21.15.46:53814) 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4acc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a8070530 0x7fb4a80ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 -- 172.21.15.46:0/558064836 learned_addr learned my addr 172.21.15.46:0/558064836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ad600640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a806e330 0x7fb4a8146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 -- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a8070530 msgr2=0x7fb4a80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a8070530 0x7fb4a80ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 -- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a806e330 msgr2=0x7fb4a8146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a806e330 0x7fb4a8146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ae000640 1 -- 172.21.15.46:0/558064836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a813fd60 con 0x7fb4a80b44a0 2024-09-17T03:28:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4ad600640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a806e330 0x7fb4a8146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:12.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.186+0000 7fb4acc00640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a8070530 0x7fb4a80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.190+0000 7fb4ae000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 0x7fb4a80ce930 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000c390 tx=0x7fb4a000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.190+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0005220 con 0x7fb4a80b44a0 2024-09-17T03:28:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.190+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a00054b0 con 0x7fb4a80b44a0 2024-09-17T03:28:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.190+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a8142860 con 0x7fb4a80b44a0 2024-09-17T03:28:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.190+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0006a40 con 0x7fb4a80b44a0 2024-09-17T03:28:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.190+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a8142e20 con 0x7fb4a80b44a0 2024-09-17T03:28:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.194+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a81425d0 con 0x7fb4a80b44a0 2024-09-17T03:28:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.194+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7fb4a0019020 con 0x7fb4a80b44a0 2024-09-17T03:28:12.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.194+0000 7fb4a5600640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb488073d70 0x7fb488076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:12.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.194+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb4a0091cb0 con 0x7fb4a80b44a0 2024-09-17T03:28:12.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.194+0000 7fb4ad600640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb488073d70 0x7fb488076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.198+0000 7fb4ad600640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb488073d70 0x7fb488076280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0af000 tx=0x7fb49c01b5f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:12.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.206+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb4a005e7b0 con 0x7fb4a80b44a0 2024-09-17T03:28:12.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.646+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4a806af80 con 0x7fb4a80b44a0 2024-09-17T03:28:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.646+0000 7fb4a5600640 1 -- 172.21.15.46:0/558064836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2184 (secure 0 0 0) 0x7fb4a005e0b0 con 0x7fb4a80b44a0 2024-09-17T03:28:12.651 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:12.651 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:28:12.652 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 450, 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 58, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:28:12.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2624, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10902141955, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 22116708352, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 750977404928, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 13627579, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 103082834, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 3, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 25 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 36, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:26:34:319773+0000", 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 4, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:28:12.656 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 245850 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:28:12.657 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 10, 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:27:27.320954+0000", 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "1 ongoing clones - average progress is 4.766% (1s)\n [=...........................] (remaining: 21s)", 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.047660000622272491, 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:28:12.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:12.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:12.659 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:28:12.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb488073d70 msgr2=0x7fb488076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb488073d70 0x7fb488076280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0af000 tx=0x7fb49c01b5f0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 msgr2=0x7fb4a80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 0x7fb4a80ce930 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000c390 tx=0x7fb4a000c790 comp rx=0 tx=0).stop 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 shutdown_connections 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb488073d70 0x7fb488076280 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a80b44a0 0x7fb4a80ce930 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a8070530 0x7fb4a80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 --2- 172.21.15.46:0/558064836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a806e330 0x7fb4a8146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 >> 172.21.15.46:0/558064836 conn(0x7fb4a80b3a20 msgr2=0x7fb4a8157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 shutdown_connections 2024-09-17T03:28:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:12.650+0000 7fb4b0000640 1 -- 172.21.15.46:0/558064836 wait complete. 2024-09-17T03:28:12.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-17T03:28:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 -- 172.21.15.46:0/3490870876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf210 msgr2=0x7fcdec0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 --2- 172.21.15.46:0/3490870876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf210 0x7fcdec0ae8e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcdec06edb0 tx=0x7fcdd401c7a0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 -- 172.21.15.46:0/3490870876 shutdown_connections 2024-09-17T03:28:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 --2- 172.21.15.46:0/3490870876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf210 0x7fcdec0ae8e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 --2- 172.21.15.46:0/3490870876 >> v2:172.21.15.99:3300/0 conn(0x7fcdec15f8f0 0x7fcdec1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 -- 172.21.15.46:0/3490870876 >> 172.21.15.46:0/3490870876 conn(0x7fcdec06e1d0 msgr2=0x7fcdec06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 -- 172.21.15.46:0/3490870876 shutdown_connections 2024-09-17T03:28:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 -- 172.21.15.46:0/3490870876 wait complete. 2024-09-17T03:28:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 Processor -- start 2024-09-17T03:28:13.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.014+0000 7fcdf3400640 1 -- start start 2024-09-17T03:28:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 0x7fcdec147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 0x7fcdec147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:13.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf620 0x7fcdec14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 0x7fcdec147800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:58734/0 (socket says 172.21.15.46:58734) 2024-09-17T03:28:13.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdec15f8f0 0x7fcdec14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf0a00640 1 -- 172.21.15.46:0/907877442 learned_addr learned my addr 172.21.15.46:0/907877442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec16c740 con 0x7fcdec0cf620 2024-09-17T03:28:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec16c5c0 con 0x7fcdec0cf210 2024-09-17T03:28:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec16c8c0 con 0x7fcdec15f8f0 2024-09-17T03:28:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf0a00640 1 -- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdec15f8f0 msgr2=0x7fcdec14e050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:28:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.018+0000 7fcdf0a00640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdec15f8f0 0x7fcdec14e050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf0a00640 1 -- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf620 msgr2=0x7fcdec14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf0a00640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf620 0x7fcdec14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf0a00640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec0b1d80 con 0x7fcdec0cf210 2024-09-17T03:28:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf0a00640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 0x7fcdec147800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00b410 tx=0x7fcddc00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc015020 con 0x7fcdec0cf210 2024-09-17T03:28:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcddc011920 con 0x7fcdec0cf210 2024-09-17T03:28:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc011d70 con 0x7fcdec0cf210 2024-09-17T03:28:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec0b1aa0 con 0x7fcdec0cf210 2024-09-17T03:28:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec0aff30 con 0x7fcdec0cf210 2024-09-17T03:28:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.022+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec0ae8e0 con 0x7fcdec0cf210 2024-09-17T03:28:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.026+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7fcddc018020 con 0x7fcdec0cf210 2024-09-17T03:28:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.026+0000 7fcde9600640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcdc8073d40 0x7fcdc8076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.026+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcddc092b00 con 0x7fcdec0cf210 2024-09-17T03:28:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.034+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcddc01f020 con 0x7fcdec0cf210 2024-09-17T03:28:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.038+0000 7fcde3e00640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcdc8073d40 0x7fcdc8076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:13.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.046+0000 7fcde3e00640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcdc8073d40 0x7fcdc8076250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40ae000 tx=0x7fcdd4019540 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:13.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.462+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fcdec06b540 con 0x7fcdc8073d40 2024-09-17T03:28:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.610+0000 7fcde9600640 1 -- 172.21.15.46:0/907877442 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcdd4004050 con 0x7fcdc8073d40 2024-09-17T03:28:13.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcdc8073d40 msgr2=0x7fcdc8076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcdc8073d40 0x7fcdc8076250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40ae000 tx=0x7fcdd4019540 comp rx=0 tx=0).stop 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 msgr2=0x7fcdec147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 0x7fcdec147800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00b410 tx=0x7fcddc00b8d0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 shutdown_connections 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcdc8073d40 0x7fcdc8076250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdec15f8f0 0x7fcdec14e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec0cf620 0x7fcdec14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 --2- 172.21.15.46:0/907877442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdec0cf210 0x7fcdec147800 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 >> 172.21.15.46:0/907877442 conn(0x7fcdec06e1d0 msgr2=0x7fcdec073040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 shutdown_connections 2024-09-17T03:28:13.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.614+0000 7fcdf3400640 1 -- 172.21.15.46:0/907877442 wait complete. 2024-09-17T03:28:13.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 -- 172.21.15.46:0/88036869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 msgr2=0x7f49ec004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 --2- 172.21.15.46:0/88036869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f49ec004070 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f49f8006fa0 tx=0x7f49f801cee0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a09c00640 1 -- 172.21.15.46:0/88036869 reap_dead start 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 -- 172.21.15.46:0/88036869 shutdown_connections 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 -- 172.21.15.46:0/88036869 >> 172.21.15.46:0/88036869 conn(0x7f4a040b4560 msgr2=0x7f4a040d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 -- 172.21.15.46:0/88036869 shutdown_connections 2024-09-17T03:28:13.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 -- 172.21.15.46:0/88036869 wait complete. 2024-09-17T03:28:13.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 Processor -- start 2024-09-17T03:28:13.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.794+0000 7f4a0a600640 1 -- start start 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f4a040d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4a0415f760 0x7f4a040d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0414f990 0x7f4a04155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040b5e80 con 0x7f49ec003c20 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a040b5d00 con 0x7f4a0414f990 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4a041681f0 con 0x7f4a0415f760 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f4a040d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:13.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f4a040d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38720/0 (socket says 172.21.15.46:38720) 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 -- 172.21.15.46:0/4268682789 learned_addr learned my addr 172.21.15.46:0/4268682789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09c00640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0414f990 0x7f4a04155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a03e00640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4a0415f760 0x7f4a040d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 -- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4a0415f760 msgr2=0x7f4a040d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4a0415f760 0x7f4a040d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 -- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0414f990 msgr2=0x7f4a04155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0414f990 0x7f4a04155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 -- 172.21.15.46:0/4268682789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0414a540 con 0x7f49ec003c20 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09c00640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0414f990 0x7f4a04155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:13.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a09200640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f4a040d16c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f49f801b5e0 tx=0x7f49f8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f80a5b70 con 0x7f49ec003c20 2024-09-17T03:28:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f8001500 con 0x7f49ec003c20 2024-09-17T03:28:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8006330 con 0x7f49ec003c20 2024-09-17T03:28:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0414e9c0 con 0x7f49ec003c20 2024-09-17T03:28:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0415ef30 con 0x7f49ec003c20 2024-09-17T03:28:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.798+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a04074240 con 0x7f49ec003c20 2024-09-17T03:28:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.802+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7f49f8003600 con 0x7f49ec003c20 2024-09-17T03:28:13.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.802+0000 7f4a00c00640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49e4073d40 0x7f49e4076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:13.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.802+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f49f812c6b0 con 0x7f49ec003c20 2024-09-17T03:28:13.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.802+0000 7f4a03e00640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49e4073d40 0x7f49e4076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:13.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.802+0000 7f4a03e00640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49e4073d40 0x7f49e4076250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b1d00 tx=0x7f49f0006b20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:13.806+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f49f8016c90 con 0x7f49ec003c20 2024-09-17T03:28:14.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f4a0414e770 con 0x7f49e4073d40 2024-09-17T03:28:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a00c00640 1 -- 172.21.15.46:0/4268682789 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f4a0414e770 con 0x7f49e4073d40 2024-09-17T03:28:14.005 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:28:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:28:14.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:14.005 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:28:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49e4073d40 msgr2=0x7f49e4076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49e4073d40 0x7f49e4076250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b1d00 tx=0x7f49f0006b20 comp rx=0 tx=0).stop 2024-09-17T03:28:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 msgr2=0x7f4a040d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f4a040d16c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f49f801b5e0 tx=0x7f49f8001cb0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 shutdown_connections 2024-09-17T03:28:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f49e4073d40 0x7f49e4076250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0414f990 0x7f4a04155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4a0415f760 0x7f4a040d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 --2- 172.21.15.46:0/4268682789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec003c20 0x7f4a040d16c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 >> 172.21.15.46:0/4268682789 conn(0x7f4a040b4560 msgr2=0x7f4a0406e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 shutdown_connections 2024-09-17T03:28:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.002+0000 7f4a0a600640 1 -- 172.21.15.46:0/4268682789 wait complete. 2024-09-17T03:28:14.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:28:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.182+0000 7f963f600640 1 -- 172.21.15.46:0/2924530247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96380cf900 msgr2=0x7f963815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.182+0000 7f963f600640 1 --2- 172.21.15.46:0/2924530247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96380cf900 0x7f963815e8c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f962800a980 tx=0x7f962801db60 comp rx=0 tx=0).stop 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 -- 172.21.15.46:0/2924530247 shutdown_connections 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 --2- 172.21.15.46:0/2924530247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96380cf900 0x7f963815e8c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 -- 172.21.15.46:0/2924530247 >> 172.21.15.46:0/2924530247 conn(0x7f96380b2040 msgr2=0x7f96380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 -- 172.21.15.46:0/2924530247 shutdown_connections 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 -- 172.21.15.46:0/2924530247 wait complete. 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 Processor -- start 2024-09-17T03:28:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 -- start start 2024-09-17T03:28:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 0x7f96381f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.186+0000 7f963f600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f963815e290 0x7f9638164c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f963815ee00 0x7f96381f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9638168be0 con 0x7f96380cf900 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9638168a60 con 0x7f963815ee00 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9638168d60 con 0x7f963815e290 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 0x7f96381f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 0x7f96381f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38728/0 (socket says 172.21.15.46:38728) 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 -- 172.21.15.46:0/4241453296 learned_addr learned my addr 172.21.15.46:0/4241453296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f962fe00640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f963815e290 0x7f9638164c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963d600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f963815ee00 0x7f96381f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 -- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f963815e290 msgr2=0x7f9638164c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f963815e290 0x7f9638164c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 -- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f963815ee00 msgr2=0x7f96381f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f963815ee00 0x7f96381f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 -- 172.21.15.46:0/4241453296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96381f7bd0 con 0x7f96380cf900 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963cc00640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 0x7f96381f7690 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f962801a770 tx=0x7f9628004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96280be070 con 0x7f96380cf900 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96280038f0 con 0x7f96380cf900 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96380acfd0 con 0x7f96380cf900 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96280b3040 con 0x7f96380cf900 2024-09-17T03:28:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96380ad590 con 0x7f96380cf900 2024-09-17T03:28:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963815e8c0 con 0x7f96380cf900 2024-09-17T03:28:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.190+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7f96280c1070 con 0x7f96380cf900 2024-09-17T03:28:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.194+0000 7f9635600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9614073d40 0x7f9614076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.194+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f962812d3a0 con 0x7f96380cf900 2024-09-17T03:28:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.194+0000 7f962fe00640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9614073d40 0x7f9614076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.194+0000 7f962fe00640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9614073d40 0x7f9614076250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9620007fd0 tx=0x7f9620009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.198+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9628017780 con 0x7f96380cf900 2024-09-17T03:28:14.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.394+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f963806eb80 con 0x7f9614073d40 2024-09-17T03:28:14.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.398+0000 7f9635600640 1 -- 172.21.15.46:0/4241453296 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f963806eb80 con 0x7f9614073d40 2024-09-17T03:28:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:14.402 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:28:14.402 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:28:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1clone1" 2024-09-17T03:28:14.403 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:14.403 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9614073d40 msgr2=0x7f9614076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9614073d40 0x7f9614076250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9620007fd0 tx=0x7f9620009040 comp rx=0 tx=0).stop 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 msgr2=0x7f96381f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 0x7f96381f7690 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f962801a770 tx=0x7f9628004620 comp rx=0 tx=0).stop 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 shutdown_connections 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9614073d40 0x7f9614076250 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f963815ee00 0x7f96381f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f963815e290 0x7f9638164c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 --2- 172.21.15.46:0/4241453296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96380cf900 0x7f96381f7690 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 >> 172.21.15.46:0/4241453296 conn(0x7f96380b2040 msgr2=0x7f96380cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:14.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 shutdown_connections 2024-09-17T03:28:14.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.402+0000 7f963f600640 1 -- 172.21.15.46:0/4241453296 wait complete. 2024-09-17T03:28:14.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:28:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 -- 172.21.15.46:0/1150045124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 msgr2=0x7fe0b015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/1150045124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b015bb60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe098009900 tx=0x7fe09801c6d0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 -- 172.21.15.46:0/1150045124 shutdown_connections 2024-09-17T03:28:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/1150045124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0b00b4de0 0x7fe0b00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/1150045124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b015bb60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 -- 172.21.15.46:0/1150045124 >> 172.21.15.46:0/1150045124 conn(0x7fe0b00b2040 msgr2=0x7fe0b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 -- 172.21.15.46:0/1150045124 shutdown_connections 2024-09-17T03:28:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.586+0000 7fe0b7a00640 1 -- 172.21.15.46:0/1150045124 wait complete. 2024-09-17T03:28:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 Processor -- start 2024-09-17T03:28:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 -- start start 2024-09-17T03:28:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0b00b4de0 0x7fe0b0154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b00b5c70 0x7fe0b0140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b0140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0168980 con 0x7fe0b01527b0 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0168800 con 0x7fe0b00b5c70 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b0168b00 con 0x7fe0b00b4de0 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b0140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b0140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38760/0 (socket says 172.21.15.46:38760) 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 -- 172.21.15.46:0/4182517107 learned_addr learned my addr 172.21.15.46:0/4182517107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0a7e00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b00b5c70 0x7fe0b0140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5000640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0b00b4de0 0x7fe0b0154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 -- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0b00b4de0 msgr2=0x7fe0b0154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0b00b4de0 0x7fe0b0154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 -- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b00b5c70 msgr2=0x7fe0b0140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b00b5c70 0x7fe0b0140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 -- 172.21.15.46:0/4182517107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b0150160 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b5a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b0140d60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800ca30 tx=0x7fe0a800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8007d90 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0a8004c20 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8005280 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b0155710 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.590+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b013ea30 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.594+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b00b51f0 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.594+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7fe0a8005420 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.594+0000 7fe0ad600640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe08c073d40 0x7fe08c076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.594+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a8093800 con 0x7fe0b01527b0 2024-09-17T03:28:14.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.594+0000 7fe0b5000640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe08c073d40 0x7fe08c076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.594+0000 7fe0b5000640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe08c073d40 0x7fe08c076250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00098e0 tx=0x7fe0a0009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.598+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe0b0150160 con 0x7fe0b01527b0 2024-09-17T03:28:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.802+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0b006b540 con 0x7fe08c073d40 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.818+0000 7fe0ad600640 1 -- 172.21.15.46:0/4182517107 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe0a0009530 con 0x7fe08c073d40 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:14.822 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe08c073d40 msgr2=0x7fe08c076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe08c073d40 0x7fe08c076250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00098e0 tx=0x7fe0a0009060 comp rx=0 tx=0).stop 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 msgr2=0x7fe0b0140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b0140d60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800ca30 tx=0x7fe0a800cef0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 shutdown_connections 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe08c073d40 0x7fe08c076250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0b01527b0 0x7fe0b0140d60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b00b5c70 0x7fe0b0140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 --2- 172.21.15.46:0/4182517107 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0b00b4de0 0x7fe0b0154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 >> 172.21.15.46:0/4182517107 conn(0x7fe0b00b2040 msgr2=0x7fe0b015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 shutdown_connections 2024-09-17T03:28:14.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:14.822+0000 7fe0b7a00640 1 -- 172.21.15.46:0/4182517107 wait complete. 2024-09-17T03:28:14.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:28:15.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 -- 172.21.15.46:0/1881186771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 msgr2=0x7f432c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 --2- 172.21.15.46:0/1881186771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15bb60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f432c074b20 tx=0x7f431401c820 comp rx=0 tx=0).stop 2024-09-17T03:28:15.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 -- 172.21.15.46:0/1881186771 shutdown_connections 2024-09-17T03:28:15.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 --2- 172.21.15.46:0/1881186771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15bb60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 -- 172.21.15.46:0/1881186771 >> 172.21.15.46:0/1881186771 conn(0x7f432c0b2040 msgr2=0x7f432c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 -- 172.21.15.46:0/1881186771 shutdown_connections 2024-09-17T03:28:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.006+0000 7f4332200640 1 -- 172.21.15.46:0/1881186771 wait complete. 2024-09-17T03:28:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 Processor -- start 2024-09-17T03:28:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 -- start start 2024-09-17T03:28:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f432c0b4de0 0x7f432c14aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f432c0b5c70 0x7f432c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f432c169b30 con 0x7f432c1527b0 2024-09-17T03:28:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1699b0 con 0x7f432c0b5c70 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f432c169cb0 con 0x7f432c0b4de0 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38774/0 (socket says 172.21.15.46:38774) 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 -- 172.21.15.46:0/1978373464 learned_addr learned my addr 172.21.15.46:0/1978373464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432b400640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f432c0b4de0 0x7f432c14aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4323e00640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f432c0b5c70 0x7f432c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 -- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f432c0b4de0 msgr2=0x7f432c14aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f432c0b4de0 0x7f432c14aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 -- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f432c0b5c70 msgr2=0x7f432c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f432c0b5c70 0x7f432c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 -- 172.21.15.46:0/1978373464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c157e30 con 0x7f432c1527b0 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432b400640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f432c0b4de0 0x7f432c14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f432be00640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15a760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f432400c6b0 tx=0x7f432400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324003b80 con 0x7f432c1527b0 2024-09-17T03:28:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4324003d20 con 0x7f432c1527b0 2024-09-17T03:28:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324005330 con 0x7f432c1527b0 2024-09-17T03:28:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c158110 con 0x7f432c1527b0 2024-09-17T03:28:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.010+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c1586d0 con 0x7f432c1527b0 2024-09-17T03:28:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.014+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c15bb60 con 0x7f432c1527b0 2024-09-17T03:28:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.014+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7f43240054d0 con 0x7f432c1527b0 2024-09-17T03:28:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.014+0000 7f4323400640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4308073d40 0x7f4308076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.014+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4324092b70 con 0x7f432c1527b0 2024-09-17T03:28:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.014+0000 7f432b400640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4308073d40 0x7f4308076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.014+0000 7f432b400640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4308073d40 0x7f4308076250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f431c0016b0 tx=0x7f431c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.018+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f432405f420 con 0x7f432c1527b0 2024-09-17T03:28:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.226+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f432c14bb80 con 0x7f4308073d40 2024-09-17T03:28:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.250+0000 7f4323400640 1 -- 172.21.15.46:0/1978373464 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f432c14bb80 con 0x7f4308073d40 2024-09-17T03:28:15.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4308073d40 msgr2=0x7f4308076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4308073d40 0x7f4308076250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f431c0016b0 tx=0x7f431c009040 comp rx=0 tx=0).stop 2024-09-17T03:28:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 msgr2=0x7f432c15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15a760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f432400c6b0 tx=0x7f432400cb70 comp rx=0 tx=0).stop 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 shutdown_connections 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4308073d40 0x7f4308076250 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432c1527b0 0x7f432c15a760 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f432c0b5c70 0x7f432c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 --2- 172.21.15.46:0/1978373464 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f432c0b4de0 0x7f432c14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 >> 172.21.15.46:0/1978373464 conn(0x7f432c0b2040 msgr2=0x7f432c14c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 shutdown_connections 2024-09-17T03:28:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.254+0000 7f4332200640 1 -- 172.21.15.46:0/1978373464 wait complete. 2024-09-17T03:28:15.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:28:15.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 -- 172.21.15.46:0/1710311517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2006fc90 msgr2=0x7fbc200700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 --2- 172.21.15.46:0/1710311517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2006fc90 0x7fbc200700a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc200d14d0 tx=0x7fbc1801caf0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 -- 172.21.15.46:0/1710311517 shutdown_connections 2024-09-17T03:28:15.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 --2- 172.21.15.46:0/1710311517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2006fc90 0x7fbc200700a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 --2- 172.21.15.46:0/1710311517 >> v2:172.21.15.99:3300/0 conn(0x7fbc2006cdf0 0x7fbc2006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 -- 172.21.15.46:0/1710311517 >> 172.21.15.46:0/1710311517 conn(0x7fbc200b35f0 msgr2=0x7fbc200b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:15.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 -- 172.21.15.46:0/1710311517 shutdown_connections 2024-09-17T03:28:15.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 -- 172.21.15.46:0/1710311517 wait complete. 2024-09-17T03:28:15.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 Processor -- start 2024-09-17T03:28:15.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.438+0000 7fbc28200640 1 -- start start 2024-09-17T03:28:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc2006cdf0 0x7fbc200aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc2006fc90 0x7fbc2015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 0x7fbc2015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20168da0 con 0x7fbc20159130 2024-09-17T03:28:15.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20168c20 con 0x7fbc2006fc90 2024-09-17T03:28:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbc20168f20 con 0x7fbc2006cdf0 2024-09-17T03:28:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 0x7fbc2015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 0x7fbc2015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38800/0 (socket says 172.21.15.46:38800) 2024-09-17T03:28:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 -- 172.21.15.46:0/3366016974 learned_addr learned my addr 172.21.15.46:0/3366016974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc24e00640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc2006fc90 0x7fbc2015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc25800640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc2006cdf0 0x7fbc200aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 -- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc2006cdf0 msgr2=0x7fbc200aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc2006cdf0 0x7fbc200aa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 -- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc2006fc90 msgr2=0x7fbc2015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc2006fc90 0x7fbc2015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 -- 172.21.15.46:0/3366016974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2015d4d0 con 0x7fbc20159130 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc24e00640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc2006fc90 0x7fbc2015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc26200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 0x7fbc2015a9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009b10 tx=0x7fbc18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc180bc070 con 0x7fbc20159130 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc180042f0 con 0x7fbc20159130 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc180b2040 con 0x7fbc20159130 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc200aaf80 con 0x7fbc20159130 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc200ab540 con 0x7fbc20159130 2024-09-17T03:28:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.442+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2006d200 con 0x7fbc20159130 2024-09-17T03:28:15.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.446+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7fbc180bf070 con 0x7fbc20159130 2024-09-17T03:28:15.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.446+0000 7fbc1d600640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc00073d40 0x7fbc00076250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.446+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fbc1812cb80 con 0x7fbc20159130 2024-09-17T03:28:15.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.446+0000 7fbc25800640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc00073d40 0x7fbc00076250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.446+0000 7fbc25800640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc00073d40 0x7fbc00076250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbc140015f0 tx=0x7fbc14009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.450+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc18016770 con 0x7fbc20159130 2024-09-17T03:28:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.650+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fbc2006b540 con 0x7fbc00073d40 2024-09-17T03:28:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.666+0000 7fbc1d600640 1 -- 172.21.15.46:0/3366016974 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc1400e4b0 con 0x7fbc00073d40 2024-09-17T03:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc00073d40 msgr2=0x7fbc00076250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc00073d40 0x7fbc00076250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbc140015f0 tx=0x7fbc14009040 comp rx=0 tx=0).stop 2024-09-17T03:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 msgr2=0x7fbc2015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 0x7fbc2015a9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009b10 tx=0x7fbc18009260 comp rx=0 tx=0).stop 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 shutdown_connections 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbc00073d40 0x7fbc00076250 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc20159130 0x7fbc2015a9c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc2006fc90 0x7fbc2015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 --2- 172.21.15.46:0/3366016974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbc2006cdf0 0x7fbc200aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 >> 172.21.15.46:0/3366016974 conn(0x7fbc200b35f0 msgr2=0x7fbc20158370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 shutdown_connections 2024-09-17T03:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbc28200640 1 -- 172.21.15.46:0/3366016974 wait complete. 2024-09-17T03:28:15.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.670+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.694 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.690+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.694 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.690+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.694 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.690+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.694 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.690+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.694 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:15.690+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:15.697 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T03:28:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 -- 172.21.15.46:0/3366311844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740cefa0 msgr2=0x7f7c740cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 --2- 172.21.15.46:0/3366311844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740cefa0 0x7f7c740cf3b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009900 tx=0x7f7c6001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 -- 172.21.15.46:0/3366311844 shutdown_connections 2024-09-17T03:28:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 --2- 172.21.15.46:0/3366311844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740cefa0 0x7f7c740cf3b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 --2- 172.21.15.46:0/3366311844 >> v2:172.21.15.99:3300/0 conn(0x7f7c740749a0 0x7f7c74074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 -- 172.21.15.46:0/3366311844 >> 172.21.15.46:0/3366311844 conn(0x7f7c740b2040 msgr2=0x7f7c740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 -- 172.21.15.46:0/3366311844 shutdown_connections 2024-09-17T03:28:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.866+0000 7f7c7c600640 1 -- 172.21.15.46:0/3366311844 wait complete. 2024-09-17T03:28:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 Processor -- start 2024-09-17T03:28:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 -- start start 2024-09-17T03:28:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 0x7f7c74159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c740cefa0 0x7f7c741f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 0x7f7c741fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7416a060 con 0x7f7c74160330 2024-09-17T03:28:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74169ee0 con 0x7f7c740cefa0 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7416a1e0 con 0x7f7c740749a0 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 0x7f7c741fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 0x7f7c74159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c740cefa0 0x7f7c741f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 0x7f7c74159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41942/0 (socket says 172.21.15.46:41942) 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 0x7f7c741fa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38830/0 (socket says 172.21.15.46:38830) 2024-09-17T03:28:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 -- 172.21.15.46:0/4259245607 learned_addr learned my addr 172.21.15.46:0/4259245607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 -- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c740cefa0 msgr2=0x7f7c741f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c740cefa0 0x7f7c741f9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 -- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 msgr2=0x7f7c741fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 0x7f7c741fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:15.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 -- 172.21.15.46:0/4259245607 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7413d130 con 0x7f7c740749a0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7a600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 0x7f7c741fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c79c00640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 0x7f7c74159810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e7d0 tx=0x7f7c6800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800ccc0 con 0x7f7c740749a0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c68010430 con 0x7f7c740749a0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68010860 con 0x7f7c740749a0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7413bf10 con 0x7f7c740749a0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.870+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7413c4d0 con 0x7f7c740749a0 2024-09-17T03:28:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.874+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c74074db0 con 0x7f7c740749a0 2024-09-17T03:28:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.874+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 26) ==== 98254+0+0 (secure 0 0 0) 0x7f7c68010a00 con 0x7f7c740749a0 2024-09-17T03:28:15.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.874+0000 7f7c71600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c54073d70 0x7f7c54076280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:15.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.874+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7c68093d60 con 0x7f7c740749a0 2024-09-17T03:28:15.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.874+0000 7f7c79200640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c54073d70 0x7f7c54076280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:15.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.874+0000 7f7c79200640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c54073d70 0x7f7c54076280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c600af000 tx=0x7f7c6001b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:15.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:15.878+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c7413d130 con 0x7f7c740749a0 2024-09-17T03:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.078+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f7c74159d50 con 0x7f7c54073d70 2024-09-17T03:28:16.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c71600640 1 -- 172.21.15.46:0/4259245607 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c600a5a90 con 0x7f7c54073d70 2024-09-17T03:28:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c54073d70 msgr2=0x7f7c54076280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c54073d70 0x7f7c54076280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c600af000 tx=0x7f7c6001b120 comp rx=0 tx=0).stop 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 msgr2=0x7f7c74159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 0x7f7c74159810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e7d0 tx=0x7f7c6800ec90 comp rx=0 tx=0).stop 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 shutdown_connections 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7c54073d70 0x7f7c54076280 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c74160330 0x7f7c741fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c740cefa0 0x7f7c741f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 --2- 172.21.15.46:0/4259245607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c740749a0 0x7f7c74159810 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.102+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 >> 172.21.15.46:0/4259245607 conn(0x7f7c740b2040 msgr2=0x7f7c740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.106+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 shutdown_connections 2024-09-17T03:28:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:16.106+0000 7f7c7c600640 1 -- 172.21.15.46:0/4259245607 wait complete. 2024-09-17T03:28:16.119 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:16.114+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:16.119 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:16.114+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:16.119 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:16.114+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:16.120 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:16.114+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:16.120 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:16.114+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:16.135 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:28:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:28:17.617 INFO:teuthology.orchestra.run.smithi046.stdout:3 2024-09-17T03:28:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:28:22.625 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:28:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:28:22.685 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:28:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:28:22.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 -- 172.21.15.46:0/1641809541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c005360 msgr2=0x7fbd0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1641809541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c005360 0x7fbd0c00b9d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009e90 tx=0x7fbd2001d5f0 comp rx=0 tx=0).stop 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 -- 172.21.15.46:0/1641809541 shutdown_connections 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1641809541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c005360 0x7fbd0c00b9d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 -- 172.21.15.46:0/1641809541 >> 172.21.15.46:0/1641809541 conn(0x7fbd280b2040 msgr2=0x7fbd280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 -- 172.21.15.46:0/1641809541 shutdown_connections 2024-09-17T03:28:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.858+0000 7fbd2d000640 1 -- 172.21.15.46:0/1641809541 wait complete. 2024-09-17T03:28:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 Processor -- start 2024-09-17T03:28:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 -- start start 2024-09-17T03:28:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c005360 0x7fbd281f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd281548c0 0x7fbd2814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 0x7fbd281f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd280751c0 con 0x7fbd2814e090 2024-09-17T03:28:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd28075040 con 0x7fbd0c005360 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbd28075340 con 0x7fbd281548c0 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 0x7fbd281f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 0x7fbd281f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34554/0 (socket says 172.21.15.46:34554) 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 -- 172.21.15.46:0/1969693193 learned_addr learned my addr 172.21.15.46:0/1969693193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd1fe00640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd281548c0 0x7fbd2814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd25e00640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c005360 0x7fbd281f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 -- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd281548c0 msgr2=0x7fbd2814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd281548c0 0x7fbd2814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 -- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c005360 msgr2=0x7fbd281f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c005360 0x7fbd281f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 -- 172.21.15.46:0/1969693193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd281f9270 con 0x7fbd2814e090 2024-09-17T03:28:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd26800640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 0x7fbd281f5c60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbd2001d420 tx=0x7fbd200092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:22.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd200ba070 con 0x7fbd2814e090 2024-09-17T03:28:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd200a77c0 con 0x7fbd2814e090 2024-09-17T03:28:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd200a7bf0 con 0x7fbd2814e090 2024-09-17T03:28:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.862+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd281f9550 con 0x7fbd2814e090 2024-09-17T03:28:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2814d780 con 0x7fbd2814e090 2024-09-17T03:28:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd280b4fa0 con 0x7fbd2814e090 2024-09-17T03:28:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fbd20005040 con 0x7fbd2814e090 2024-09-17T03:28:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd1e000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd04073f50 0x7fbd04076460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7fbd2012c8a0 con 0x7fbd2814e090 2024-09-17T03:28:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd25e00640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd04073f50 0x7fbd04076460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.866+0000 7fbd25e00640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd04073f50 0x7fbd04076460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbd18006fd0 tx=0x7fbd18006b80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:22.870+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd20017300 con 0x7fbd2814e090 2024-09-17T03:28:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fbd280cc870 con 0x7fbd2814e090 2024-09-17T03:28:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd1e000640 1 -- 172.21.15.46:0/1969693193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7fbd200f9310 con 0x7fbd2814e090 2024-09-17T03:28:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd04073f50 msgr2=0x7fbd04076460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd04073f50 0x7fbd04076460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbd18006fd0 tx=0x7fbd18006b80 comp rx=0 tx=0).stop 2024-09-17T03:28:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 msgr2=0x7fbd281f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 0x7fbd281f5c60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbd2001d420 tx=0x7fbd200092f0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 shutdown_connections 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd04073f50 0x7fbd04076460 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd2814e090 0x7fbd281f5c60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd281548c0 0x7fbd2814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 --2- 172.21.15.46:0/1969693193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c005360 0x7fbd281f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 >> 172.21.15.46:0/1969693193 conn(0x7fbd280b2040 msgr2=0x7fbd280b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 shutdown_connections 2024-09-17T03:28:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.050+0000 7fbd2d000640 1 -- 172.21.15.46:0/1969693193 wait complete. 2024-09-17T03:28:23.075 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:28:23.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:28:23.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:28:23.241 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:28:23.241 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:28:23.242 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:23.242 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:23.251 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:23.252 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:23.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:28:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 -- 172.21.15.46:0/3395294902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc0cf7f0 msgr2=0x7f12dc15d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 --2- 172.21.15.46:0/3395294902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc0cf7f0 0x7f12dc15d180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009900 tx=0x7f12c401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 -- 172.21.15.46:0/3395294902 shutdown_connections 2024-09-17T03:28:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 --2- 172.21.15.46:0/3395294902 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12dc15d6c0 0x7f12dc15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 --2- 172.21.15.46:0/3395294902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc0cf7f0 0x7f12dc15d180 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 -- 172.21.15.46:0/3395294902 >> 172.21.15.46:0/3395294902 conn(0x7f12dc0b2040 msgr2=0x7f12dc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 -- 172.21.15.46:0/3395294902 shutdown_connections 2024-09-17T03:28:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.506+0000 7f12e0a00640 1 -- 172.21.15.46:0/3395294902 wait complete. 2024-09-17T03:28:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 Processor -- start 2024-09-17T03:28:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 -- start start 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12dc0cf7f0 0x7f12dc1f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12dc15d6c0 0x7f12dc15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 0x7f12dc153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc169490 con 0x7f12dc15eeb0 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc169310 con 0x7f12dc0cf7f0 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.510+0000 7f12e0a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f12dc169610 con 0x7f12dc15d6c0 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 0x7f12dc153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 0x7f12dc153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34566/0 (socket says 172.21.15.46:34566) 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 -- 172.21.15.46:0/2375219577 learned_addr learned my addr 172.21.15.46:0/2375219577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12d3e00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12dc15d6c0 0x7f12dc15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12d9e00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12dc0cf7f0 0x7f12dc1f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 -- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12dc15d6c0 msgr2=0x7f12dc15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12dc15d6c0 0x7f12dc15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 -- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12dc0cf7f0 msgr2=0x7f12dc1f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12dc0cf7f0 0x7f12dc1f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 -- 172.21.15.46:0/2375219577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc139ff0 con 0x7f12dc15eeb0 2024-09-17T03:28:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12da800640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 0x7f12dc153db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12d400ea80 tx=0x7f12d400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400ce40 con 0x7f12dc15eeb0 2024-09-17T03:28:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d4004b30 con 0x7f12dc15eeb0 2024-09-17T03:28:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc150c10 con 0x7f12dc15eeb0 2024-09-17T03:28:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4005020 con 0x7f12dc15eeb0 2024-09-17T03:28:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1511d0 con 0x7f12dc15eeb0 2024-09-17T03:28:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc150980 con 0x7f12dc15eeb0 2024-09-17T03:28:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.514+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f12d40051c0 con 0x7f12dc15eeb0 2024-09-17T03:28:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.518+0000 7f12d2000640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f12b8073f20 0x7f12b8076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.518+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7f12d4093600 con 0x7f12dc15eeb0 2024-09-17T03:28:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.518+0000 7f12d9e00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f12b8073f20 0x7f12b8076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.518+0000 7f12d9e00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f12b8073f20 0x7f12b8076430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f12cc001780 tx=0x7f12cc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.522+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12d405fab0 con 0x7f12dc15eeb0 2024-09-17T03:28:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12dc15d180 con 0x7f12dc15eeb0 2024-09-17T03:28:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12d2000640 1 -- 172.21.15.46:0/2375219577 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) ==== 80+0+0 (secure 0 0 0) 0x7f12d40644e0 con 0x7f12dc15eeb0 2024-09-17T03:28:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f12b8073f20 msgr2=0x7f12b8076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f12b8073f20 0x7f12b8076430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f12cc001780 tx=0x7f12cc009040 comp rx=0 tx=0).stop 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 msgr2=0x7f12dc153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 0x7f12dc153db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12d400ea80 tx=0x7f12d400ef40 comp rx=0 tx=0).stop 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 shutdown_connections 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f12b8073f20 0x7f12b8076430 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12dc15eeb0 0x7f12dc153db0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12dc15d6c0 0x7f12dc15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 --2- 172.21.15.46:0/2375219577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12dc0cf7f0 0x7f12dc1f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:23.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 >> 172.21.15.46:0/2375219577 conn(0x7f12dc0b2040 msgr2=0x7f12dc15b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:23.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 shutdown_connections 2024-09-17T03:28:23.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:23.710+0000 7f12e0a00640 1 -- 172.21.15.46:0/2375219577 wait complete. 2024-09-17T03:28:23.735 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:28:23.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:23.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:28:23.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:28:23.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:23.792 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:28:23.816 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:28:23.816 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:28:23.817 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:23.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:28:23.826 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:28:23.826 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:23.826 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:23.873 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:28:23.873 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:28:23.873 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:28:23.873 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:23.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:28:23.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 -- 172.21.15.46:0/687047484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b5c70 msgr2=0x7fe12015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 --2- 172.21.15.46:0/687047484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b5c70 0x7fe12015bb60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe120074b20 tx=0x7fe11401c760 comp rx=0 tx=0).stop 2024-09-17T03:28:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 -- 172.21.15.46:0/687047484 shutdown_connections 2024-09-17T03:28:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 --2- 172.21.15.46:0/687047484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b5c70 0x7fe12015bb60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 -- 172.21.15.46:0/687047484 >> 172.21.15.46:0/687047484 conn(0x7fe1200b2040 msgr2=0x7fe1200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 -- 172.21.15.46:0/687047484 shutdown_connections 2024-09-17T03:28:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.070+0000 7fe128000640 1 -- 172.21.15.46:0/687047484 wait complete. 2024-09-17T03:28:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 Processor -- start 2024-09-17T03:28:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 -- start start 2024-09-17T03:28:24.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 0x7fe120150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1200b5c70 0x7fe12014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1201527b0 0x7fe120158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe120169bd0 con 0x7fe1200b4de0 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe120169a50 con 0x7fe1200b5c70 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe120169d50 con 0x7fe1201527b0 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 0x7fe120150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 0x7fe120150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34580/0 (socket says 172.21.15.46:34580) 2024-09-17T03:28:24.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 -- 172.21.15.46:0/2099111590 learned_addr learned my addr 172.21.15.46:0/2099111590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe124c00640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1200b5c70 0x7fe12014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe126000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1201527b0 0x7fe120158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 -- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1201527b0 msgr2=0x7fe120158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1201527b0 0x7fe120158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 -- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1200b5c70 msgr2=0x7fe12014ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1200b5c70 0x7fe12014ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 -- 172.21.15.46:0/2099111590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12014f1a0 con 0x7fe1200b4de0 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe124c00640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1200b5c70 0x7fe12014ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe125600640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 0x7fe120150270 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe114000c10 tx=0x7fe1140a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:24.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140bb070 con 0x7fe1200b4de0 2024-09-17T03:28:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe114004470 con 0x7fe1200b4de0 2024-09-17T03:28:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12014f480 con 0x7fe1200b4de0 2024-09-17T03:28:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140b2040 con 0x7fe1200b4de0 2024-09-17T03:28:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.074+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12014b4c0 con 0x7fe1200b4de0 2024-09-17T03:28:24.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.078+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12015bb60 con 0x7fe1200b4de0 2024-09-17T03:28:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.078+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fe1140be070 con 0x7fe1200b4de0 2024-09-17T03:28:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.078+0000 7fe11d600640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe100073f20 0x7fe100076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.078+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7fe11412c5e0 con 0x7fe1200b4de0 2024-09-17T03:28:24.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.078+0000 7fe124c00640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe100073f20 0x7fe100076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.078+0000 7fe124c00640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe100073f20 0x7fe100076430 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe10c007f30 tx=0x7fe10c007ec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:24.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.082+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe114016410 con 0x7fe1200b4de0 2024-09-17T03:28:24.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.294+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1200b49c0 con 0x7fe1200b4de0 2024-09-17T03:28:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.294+0000 7fe11d600640 1 -- 172.21.15.46:0/2099111590 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5236 (secure 0 0 0) 0x7fe1140f8a70 con 0x7fe1200b4de0 2024-09-17T03:28:24.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:24.299 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"btime":"2024-09-17T03:26:34:319773+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:26:20.685438+0000","modified":"2024-09-17T03:26:23.700531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245850":{"gid":245850,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/695855747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":695855747},{"type":"v1","addr":"172.21.15.46:6835","nonce":695855747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245850,"qdb_cluster":[245850]},"id":4}]} 2024-09-17T03:28:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe100073f20 msgr2=0x7fe100076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe100073f20 0x7fe100076430 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe10c007f30 tx=0x7fe10c007ec0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 msgr2=0x7fe120150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 0x7fe120150270 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe114000c10 tx=0x7fe1140a56a0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 shutdown_connections 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe100073f20 0x7fe100076430 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1201527b0 0x7fe120158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1200b5c70 0x7fe12014ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 --2- 172.21.15.46:0/2099111590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1200b4de0 0x7fe120150270 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 >> 172.21.15.46:0/2099111590 conn(0x7fe1200b2040 msgr2=0x7fe12015b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 shutdown_connections 2024-09-17T03:28:24.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.298+0000 7fe128000640 1 -- 172.21.15.46:0/2099111590 wait complete. 2024-09-17T03:28:24.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 -- 172.21.15.46:0/4137231998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 msgr2=0x7fada40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 --2- 172.21.15.46:0/4137231998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40b51f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009c80 tx=0x7fad9c01cd70 comp rx=0 tx=0).stop 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 -- 172.21.15.46:0/4137231998 shutdown_connections 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 --2- 172.21.15.46:0/4137231998 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40b51f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 --2- 172.21.15.46:0/4137231998 >> v2:172.21.15.99:3300/0 conn(0x7fada40b5c70 0x7fada415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 -- 172.21.15.46:0/4137231998 >> 172.21.15.46:0/4137231998 conn(0x7fada40b2040 msgr2=0x7fada40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 -- 172.21.15.46:0/4137231998 shutdown_connections 2024-09-17T03:28:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.478+0000 7fada9c00640 1 -- 172.21.15.46:0/4137231998 wait complete. 2024-09-17T03:28:24.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 Processor -- start 2024-09-17T03:28:24.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 -- start start 2024-09-17T03:28:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada40b5c70 0x7fada4141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fada41527b0 0x7fada4141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fada41689d0 con 0x7fada40b5c70 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fada4168850 con 0x7fada41527b0 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada9c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fada4168b50 con 0x7fada40b4de0 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fad9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada40b5c70 0x7fada4141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:44348/0 (socket says 172.21.15.46:44348) 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 -- 172.21.15.46:0/1144034378 learned_addr learned my addr 172.21.15.46:0/1144034378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada3400640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fada41527b0 0x7fada4141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 -- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fada41527b0 msgr2=0x7fada4141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fada41527b0 0x7fada4141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 -- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada40b5c70 msgr2=0x7fada4141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada40b5c70 0x7fada4141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 -- 172.21.15.46:0/1144034378 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada4159030 con 0x7fada40b4de0 2024-09-17T03:28:24.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fad9be00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada40b5c70 0x7fada4141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.482+0000 7fada2a00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40cf0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e9a0 tx=0x7fad9400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9400cd80 con 0x7fada40b4de0 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad94004020 con 0x7fada40b4de0 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94004450 con 0x7fada40b4de0 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada413f770 con 0x7fada40b4de0 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fada4140ff0 con 0x7fada40b4de0 2024-09-17T03:28:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada4141250 con 0x7fada40b4de0 2024-09-17T03:28:24.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fad940047d0 con 0x7fada40b4de0 2024-09-17T03:28:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fad9aa00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad80073ef0 0x7fad80076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.486+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7fad94093770 con 0x7fada40b4de0 2024-09-17T03:28:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.490+0000 7fad9be00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad80073ef0 0x7fad80076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.490+0000 7fad9be00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad80073ef0 0x7fad80076400 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fada41599e0 tx=0x7fad8c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.490+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad9405fc20 con 0x7fada40b4de0 2024-09-17T03:28:24.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.702+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fada40b51f0 con 0x7fada40b4de0 2024-09-17T03:28:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.702+0000 7fad9aa00640 1 -- 172.21.15.46:0/1144034378 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5236 (secure 0 0 0) 0x7fad94064650 con 0x7fada40b4de0 2024-09-17T03:28:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:24.707 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"btime":"2024-09-17T03:26:34:319773+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:26:20.685438+0000","modified":"2024-09-17T03:26:23.700531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245850":{"gid":245850,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/695855747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":695855747},{"type":"v1","addr":"172.21.15.46:6835","nonce":695855747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245850,"qdb_cluster":[245850]},"id":4}]} 2024-09-17T03:28:24.707 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-09-17T03:28:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad80073ef0 msgr2=0x7fad80076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad80073ef0 0x7fad80076400 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fada41599e0 tx=0x7fad8c009040 comp rx=0 tx=0).stop 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 msgr2=0x7fada40cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40cf0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e9a0 tx=0x7fad9400ee60 comp rx=0 tx=0).stop 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 shutdown_connections 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad80073ef0 0x7fad80076400 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fada41527b0 0x7fada4141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fada40b5c70 0x7fada4141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 --2- 172.21.15.46:0/1144034378 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fada40b4de0 0x7fada40cf0e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 >> 172.21.15.46:0/1144034378 conn(0x7fada40b2040 msgr2=0x7fada4151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 shutdown_connections 2024-09-17T03:28:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.706+0000 7fada9c00640 1 -- 172.21.15.46:0/1144034378 wait complete. 2024-09-17T03:28:24.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:28:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 -- 172.21.15.46:0/2340650795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c074b20 msgr2=0x7f796c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 --2- 172.21.15.46:0/2340650795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c074b20 0x7f796c15cf90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7968009bc0 tx=0x7f796801cb10 comp rx=0 tx=0).stop 2024-09-17T03:28:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 -- 172.21.15.46:0/2340650795 shutdown_connections 2024-09-17T03:28:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 --2- 172.21.15.46:0/2340650795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c074b20 0x7f796c15cf90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 -- 172.21.15.46:0/2340650795 >> 172.21.15.46:0/2340650795 conn(0x7f796c0708c0 msgr2=0x7f796c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 -- 172.21.15.46:0/2340650795 shutdown_connections 2024-09-17T03:28:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.886+0000 7f7974800640 1 -- 172.21.15.46:0/2340650795 wait complete. 2024-09-17T03:28:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 Processor -- start 2024-09-17T03:28:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 -- start start 2024-09-17T03:28:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f796c0b5a80 0x7f796c0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 0x7f796c146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f796c1f6be0 0x7f796c14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f796c16b590 con 0x7f796c157030 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f796c16b410 con 0x7f796c0b5a80 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f796c16b710 con 0x7f796c1f6be0 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 0x7f796c146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 0x7f796c146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34614/0 (socket says 172.21.15.46:34614) 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 -- 172.21.15.46:0/1745471306 learned_addr learned my addr 172.21.15.46:0/1745471306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971e00640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f796c0b5a80 0x7f796c0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7972800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f796c1f6be0 0x7f796c14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 -- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f796c1f6be0 msgr2=0x7f796c14af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f796c1f6be0 0x7f796c14af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 -- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f796c0b5a80 msgr2=0x7f796c0b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f796c0b5a80 0x7f796c0b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 -- 172.21.15.46:0/1745471306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c146ad0 con 0x7f796c157030 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7971400640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 0x7f796c146590 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c420 tx=0x7f795c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c011790 con 0x7f796c157030 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f795c017070 con 0x7f796c157030 2024-09-17T03:28:24.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c011ca0 con 0x7f796c157030 2024-09-17T03:28:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c1f7340 con 0x7f796c157030 2024-09-17T03:28:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.890+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c1f7900 con 0x7f796c157030 2024-09-17T03:28:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.894+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c0d2260 con 0x7f796c157030 2024-09-17T03:28:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.894+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f795c01c460 con 0x7f796c157030 2024-09-17T03:28:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.894+0000 7f7966000640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7950073ef0 0x7f7950076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.894+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7f795c092850 con 0x7f796c157030 2024-09-17T03:28:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.894+0000 7f7971e00640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7950073ef0 0x7f7950076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.898+0000 7f7971e00640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7950073ef0 0x7f7950076400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7960007fd0 tx=0x7f7960009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:24.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:24.898+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f795c05ed00 con 0x7f796c157030 2024-09-17T03:28:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.074+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f796c1f70d0 con 0x7f796c157030 2024-09-17T03:28:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7966000640 1 -- 172.21.15.46:0/1745471306 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16797 (secure 0 0 0) 0x7f795c063730 con 0x7f796c157030 2024-09-17T03:28:25.080 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:25.080 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:28:16.436841+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:26:18.439366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T03:26:19.646068+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":59,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:28:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7950073ef0 msgr2=0x7f7950076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7950073ef0 0x7f7950076400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7960007fd0 tx=0x7f7960009040 comp rx=0 tx=0).stop 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 msgr2=0x7f796c146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 0x7f796c146590 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c420 tx=0x7f795c00c8e0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 shutdown_connections 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7950073ef0 0x7f7950076400 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f796c1f6be0 0x7f796c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f796c157030 0x7f796c146590 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 --2- 172.21.15.46:0/1745471306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f796c0b5a80 0x7f796c0b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 >> 172.21.15.46:0/1745471306 conn(0x7f796c0708c0 msgr2=0x7f796c15f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 shutdown_connections 2024-09-17T03:28:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.078+0000 7f7974800640 1 -- 172.21.15.46:0/1745471306 wait complete. 2024-09-17T03:28:25.104 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:28:25.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.258+0000 7fe82b400640 1 -- 172.21.15.46:0/794744977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d25d0 msgr2=0x7fe8240d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.258+0000 7fe82b400640 1 --2- 172.21.15.46:0/794744977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d25d0 0x7fe8240d0fa0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe818009840 tx=0x7fe81801c790 comp rx=0 tx=0).stop 2024-09-17T03:28:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 -- 172.21.15.46:0/794744977 shutdown_connections 2024-09-17T03:28:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 --2- 172.21.15.46:0/794744977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d25d0 0x7fe8240d0fa0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 -- 172.21.15.46:0/794744977 >> 172.21.15.46:0/794744977 conn(0x7fe824073e00 msgr2=0x7fe82406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 -- 172.21.15.46:0/794744977 shutdown_connections 2024-09-17T03:28:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 -- 172.21.15.46:0/794744977 wait complete. 2024-09-17T03:28:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 Processor -- start 2024-09-17T03:28:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 -- start start 2024-09-17T03:28:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 0x7fe824153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.262+0000 7fe82b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8240d25d0 0x7fe824155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240d29e0 0x7fe824155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe82416d200 con 0x7fe8240d14e0 2024-09-17T03:28:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe82416d080 con 0x7fe8240d29e0 2024-09-17T03:28:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe82416d380 con 0x7fe8240d25d0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 0x7fe824153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 0x7fe824153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34632/0 (socket says 172.21.15.46:34632) 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 -- 172.21.15.46:0/3979106927 learned_addr learned my addr 172.21.15.46:0/3979106927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 -- 172.21.15.46:0/3979106927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8240d25d0 msgr2=0x7fe824155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8240d25d0 0x7fe824155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 -- 172.21.15.46:0/3979106927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240d29e0 msgr2=0x7fe824155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240d29e0 0x7fe824155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 -- 172.21.15.46:0/3979106927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8240c93e0 con 0x7fe8240d14e0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe828a00640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 0x7fe824153ad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe818009790 tx=0x7fe818005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8180bb070 con 0x7fe8240d14e0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe818004330 con 0x7fe8240d14e0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8180b2040 con 0x7fe8240d14e0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8240c8810 con 0x7fe8240d14e0 2024-09-17T03:28:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8241f9890 con 0x7fe8240d14e0 2024-09-17T03:28:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8240d0fa0 con 0x7fe8240d14e0 2024-09-17T03:28:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.266+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fe8180be070 con 0x7fe8240d14e0 2024-09-17T03:28:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.270+0000 7fe820c00640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe804073ef0 0x7fe804076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.270+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7fe81812cdd0 con 0x7fe8240d14e0 2024-09-17T03:28:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.270+0000 7fe823e00640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe804073ef0 0x7fe804076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.270+0000 7fe823e00640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe804073ef0 0x7fe804076400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe810000f70 tx=0x7fe8100079c0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:25.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.274+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8180c5070 con 0x7fe8240d14e0 2024-09-17T03:28:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.486+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8240cf490 con 0x7fe8240d14e0 2024-09-17T03:28:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.486+0000 7fe820c00640 1 -- 172.21.15.46:0/3979106927 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5236 (secure 0 0 0) 0x7fe818016410 con 0x7fe8240d14e0 2024-09-17T03:28:25.490 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:25.490 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"btime":"2024-09-17T03:26:34:319773+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:26:20.685438+0000","modified":"2024-09-17T03:26:23.700531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245850":{"gid":245850,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/695855747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":695855747},{"type":"v1","addr":"172.21.15.46:6835","nonce":695855747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245850,"qdb_cluster":[245850]},"id":4}]} 2024-09-17T03:28:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-09-17T03:28:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe804073ef0 msgr2=0x7fe804076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe804073ef0 0x7fe804076400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe810000f70 tx=0x7fe8100079c0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 msgr2=0x7fe824153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 0x7fe824153ad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe818009790 tx=0x7fe818005ca0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 shutdown_connections 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe804073ef0 0x7fe804076400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8240d29e0 0x7fe824155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8240d25d0 0x7fe824155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 --2- 172.21.15.46:0/3979106927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8240d14e0 0x7fe824153ad0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 >> 172.21.15.46:0/3979106927 conn(0x7fe824073e00 msgr2=0x7fe8240ca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 shutdown_connections 2024-09-17T03:28:25.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.490+0000 7fe82b400640 1 -- 172.21.15.46:0/3979106927 wait complete. 2024-09-17T03:28:25.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:25.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 -- 172.21.15.46:0/642053484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40cefa0 msgr2=0x7f73d40cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 --2- 172.21.15.46:0/642053484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40cefa0 0x7f73d40cf3b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009840 tx=0x7f73bc01c820 comp rx=0 tx=0).stop 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 -- 172.21.15.46:0/642053484 shutdown_connections 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 --2- 172.21.15.46:0/642053484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40cefa0 0x7f73d40cf3b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 -- 172.21.15.46:0/642053484 >> 172.21.15.46:0/642053484 conn(0x7f73d40b2040 msgr2=0x7f73d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 -- 172.21.15.46:0/642053484 shutdown_connections 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.670+0000 7f73d9c00640 1 -- 172.21.15.46:0/642053484 wait complete. 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 Processor -- start 2024-09-17T03:28:25.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 -- start start 2024-09-17T03:28:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 0x7f73d4159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d40b3340 0x7f73d40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d40cefa0 0x7f73d40aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73d416b350 con 0x7f73d40749a0 2024-09-17T03:28:25.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73d416b1d0 con 0x7f73d40cefa0 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f73d416b4d0 con 0x7f73d40b3340 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 0x7f73d4159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 0x7f73d4159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34646/0 (socket says 172.21.15.46:34646) 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 -- 172.21.15.46:0/2909896943 learned_addr learned my addr 172.21.15.46:0/2909896943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d3400640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d40cefa0 0x7f73d40aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73cbe00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d40b3340 0x7f73d40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 -- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d40b3340 msgr2=0x7f73d40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d40b3340 0x7f73d40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 -- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d40cefa0 msgr2=0x7f73d40aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d40cefa0 0x7f73d40aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 -- 172.21.15.46:0/2909896943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d415a4d0 con 0x7f73d40749a0 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73cbe00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d40b3340 0x7f73d40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d3400640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d40cefa0 0x7f73d40aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d2a00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 0x7f73d4159f90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c3f0 tx=0x7f73c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4017070 con 0x7f73d40749a0 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73c40053c0 con 0x7f73d40749a0 2024-09-17T03:28:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d40cd9a0 con 0x7f73d40749a0 2024-09-17T03:28:25.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4003860 con 0x7f73d40749a0 2024-09-17T03:28:25.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.674+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d413a330 con 0x7f73d40749a0 2024-09-17T03:28:25.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.678+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d40cf3b0 con 0x7f73d40749a0 2024-09-17T03:28:25.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.678+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f73c401a070 con 0x7f73d40749a0 2024-09-17T03:28:25.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.678+0000 7f73caa00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73b0073ef0 0x7f73b0076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:25.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.678+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5765+0+0 (secure 0 0 0) 0x7f73c4092010 con 0x7f73d40749a0 2024-09-17T03:28:25.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.678+0000 7f73cbe00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73b0073ef0 0x7f73b0076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:25.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.678+0000 7f73cbe00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73b0073ef0 0x7f73b0076400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0af000 tx=0x7f73bc01b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.682+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f73c405eae0 con 0x7f73d40749a0 2024-09-17T03:28:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.890+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73d4139de0 con 0x7f73d40749a0 2024-09-17T03:28:25.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73caa00640 1 -- 172.21.15.46:0/2909896943 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5236 (secure 0 0 0) 0x7f73c405e3e0 con 0x7f73d40749a0 2024-09-17T03:28:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:25.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"btime":"2024-09-17T03:26:34:319773+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:26:20.685438+0000","modified":"2024-09-17T03:26:23.700531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":245850},"failed":[],"damaged":[],"stopped":[],"info":{"gid_245850":{"gid":245850,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/695855747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":695855747},{"type":"v1","addr":"172.21.15.46:6835","nonce":695855747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":245850,"qdb_cluster":[245850]},"id":4}]} 2024-09-17T03:28:25.896 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-09-17T03:28:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73b0073ef0 msgr2=0x7f73b0076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73b0073ef0 0x7f73b0076400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0af000 tx=0x7f73bc01b060 comp rx=0 tx=0).stop 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 msgr2=0x7f73d4159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 0x7f73d4159f90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f73c400c3f0 tx=0x7f73c400c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 shutdown_connections 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73b0073ef0 0x7f73b0076400 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73d40cefa0 0x7f73d40aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73d40b3340 0x7f73d40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 --2- 172.21.15.46:0/2909896943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73d40749a0 0x7f73d4159f90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 >> 172.21.15.46:0/2909896943 conn(0x7f73d40b2040 msgr2=0x7f73d4072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 shutdown_connections 2024-09-17T03:28:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:25.894+0000 7f73d9c00640 1 -- 172.21.15.46:0/2909896943 wait complete. 2024-09-17T03:28:25.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 -- 172.21.15.46:0/2649221823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b5c70 msgr2=0x7fc3ac15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 --2- 172.21.15.46:0/2649221823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b5c70 0x7fc3ac15bb60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac074b20 tx=0x7fc39c01c760 comp rx=0 tx=0).stop 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 -- 172.21.15.46:0/2649221823 shutdown_connections 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 --2- 172.21.15.46:0/2649221823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b5c70 0x7fc3ac15bb60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 -- 172.21.15.46:0/2649221823 >> 172.21.15.46:0/2649221823 conn(0x7fc3ac0b2040 msgr2=0x7fc3ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 -- 172.21.15.46:0/2649221823 shutdown_connections 2024-09-17T03:28:26.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 -- 172.21.15.46:0/2649221823 wait complete. 2024-09-17T03:28:26.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.078+0000 7fc3b3200640 1 Processor -- start 2024-09-17T03:28:26.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 -- start start 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b4de0 0x7fc3ac1f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3ac0b5c70 0x7fc3ac150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ac1527b0 0x7fc3ac1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac169b60 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac1699e0 con 0x7fc3ac1527b0 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b3200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ac169ce0 con 0x7fc3ac0b5c70 2024-09-17T03:28:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3a3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3ac0b5c70 0x7fc3ac150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b4de0 0x7fc3ac1f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3a3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3ac0b5c70 0x7fc3ac150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:44382/0 (socket says 172.21.15.46:44382) 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3a3e00640 1 -- 172.21.15.46:0/1862324292 learned_addr learned my addr 172.21.15.46:0/1862324292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3b1200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ac1527b0 0x7fc3ac1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 -- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3ac0b5c70 msgr2=0x7fc3ac150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3ac0b5c70 0x7fc3ac150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 -- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ac1527b0 msgr2=0x7fc3ac1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ac1527b0 0x7fc3ac1587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 -- 172.21.15.46:0/1862324292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac1506c0 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.082+0000 7fc3abe00640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b4de0 0x7fc3ac1f7640 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0ac000 tx=0x7fc39c0a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0bb070 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc39c004470 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac156c90 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0b2040 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac150c30 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac15bb60 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fc39c0be070 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.086+0000 7fc3a8c00640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc388073ef0 0x7fc388076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.090+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fc39c12c820 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.090+0000 7fc3a3e00640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc388073ef0 0x7fc388076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.090+0000 7fc3a3e00640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc388073ef0 0x7fc388076400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac1591c0 tx=0x7fc394009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.094+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc39c016410 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3ac0b48d0 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3a8c00640 1 -- 172.21.15.46:0/1862324292 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16754 (secure 0 0 0) 0x7fc39c0f8c50 con 0x7fc3ac0b4de0 2024-09-17T03:28:26.273 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:26.274 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:28:25.987818+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:26:18.439366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T03:26:19.646068+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":59,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:28:26.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc388073ef0 msgr2=0x7fc388076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc388073ef0 0x7fc388076400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac1591c0 tx=0x7fc394009040 comp rx=0 tx=0).stop 2024-09-17T03:28:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b4de0 msgr2=0x7fc3ac1f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b4de0 0x7fc3ac1f7640 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0ac000 tx=0x7fc39c0a56a0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 shutdown_connections 2024-09-17T03:28:26.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc388073ef0 0x7fc388076400 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ac1527b0 0x7fc3ac1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc3ac0b5c70 0x7fc3ac150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 --2- 172.21.15.46:0/1862324292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3ac0b4de0 0x7fc3ac1f7640 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 >> 172.21.15.46:0/1862324292 conn(0x7fc3ac0b2040 msgr2=0x7fc3ac14bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 shutdown_connections 2024-09-17T03:28:26.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.270+0000 7fc3b3200640 1 -- 172.21.15.46:0/1862324292 wait complete. 2024-09-17T03:28:26.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ef600640 1 -- 172.21.15.46:0/2406307212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27cc004d50 msgr2=0x7f27cc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ef600640 1 --2- 172.21.15.46:0/2406307212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27cc004d50 0x7f27cc005160 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009ee0 tx=0x7f27e001d630 comp rx=0 tx=0).stop 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ed600640 1 -- 172.21.15.46:0/2406307212 reap_dead start 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ef600640 1 -- 172.21.15.46:0/2406307212 shutdown_connections 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ef600640 1 -- 172.21.15.46:0/2406307212 >> 172.21.15.46:0/2406307212 conn(0x7f27e80b2040 msgr2=0x7f27e80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ef600640 1 -- 172.21.15.46:0/2406307212 shutdown_connections 2024-09-17T03:28:26.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.454+0000 7f27ef600640 1 -- 172.21.15.46:0/2406307212 wait complete. 2024-09-17T03:28:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 Processor -- start 2024-09-17T03:28:26.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 -- start start 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27cc004d50 0x7f27e8152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27e81e2980 0x7f27e81e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 0x7f27e80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27e815cf30 con 0x7f27e815a8e0 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e815cdb0 con 0x7f27cc004d50 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f27e815d0b0 con 0x7f27e81e2980 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 0x7f27e80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 0x7f27e80ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34682/0 (socket says 172.21.15.46:34682) 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 -- 172.21.15.46:0/3532648437 learned_addr learned my addr 172.21.15.46:0/3532648437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27dfe00640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27e81e2980 0x7f27e81e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ecc00640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27cc004d50 0x7f27e8152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 -- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27e81e2980 msgr2=0x7f27e81e2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27e81e2980 0x7f27e81e2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 -- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27cc004d50 msgr2=0x7f27e8152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27cc004d50 0x7f27e8152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 -- 172.21.15.46:0/3532648437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e80b1200 con 0x7f27e815a8e0 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ed600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 0x7f27e80ae8e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f27e001d0a0 tx=0x7f27e0005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0009ab0 con 0x7f27e815a8e0 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27e00aa500 con 0x7f27e815a8e0 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0009200 con 0x7f27e815a8e0 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e80b14e0 con 0x7f27e815a8e0 2024-09-17T03:28:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.458+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e80b1aa0 con 0x7f27e815a8e0 2024-09-17T03:28:26.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.462+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e80768c0 con 0x7f27e815a8e0 2024-09-17T03:28:26.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.462+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f27e00aacc0 con 0x7f27e815a8e0 2024-09-17T03:28:26.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.462+0000 7f27e5600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27c4073f20 0x7f27c4076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.462+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f27e012d7c0 con 0x7f27e815a8e0 2024-09-17T03:28:26.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.462+0000 7f27ecc00640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27c4073f20 0x7f27c4076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.462+0000 7f27ecc00640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27c4073f20 0x7f27c4076430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009c60 tx=0x7f27d80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.466+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27e00c3070 con 0x7f27e815a8e0 2024-09-17T03:28:26.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.678+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27e815d640 con 0x7f27e815a8e0 2024-09-17T03:28:26.682 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:28:26.678+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:28:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.682+0000 7f27e5600640 1 -- 172.21.15.46:0/3532648437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f27e00ae020 con 0x7f27e815a8e0 2024-09-17T03:28:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:28:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27c4073f20 msgr2=0x7f27c4076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27c4073f20 0x7f27c4076430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009c60 tx=0x7f27d80091e0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 msgr2=0x7f27e80ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 0x7f27e80ae8e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f27e001d0a0 tx=0x7f27e0005c60 comp rx=0 tx=0).stop 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 shutdown_connections 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f27c4073f20 0x7f27c4076430 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27e815a8e0 0x7f27e80ae8e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27e81e2980 0x7f27e81e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 --2- 172.21.15.46:0/3532648437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27cc004d50 0x7f27e8152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 >> 172.21.15.46:0/3532648437 conn(0x7f27e80b2040 msgr2=0x7f27e80af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.686+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 shutdown_connections 2024-09-17T03:28:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.690+0000 7f27ef600640 1 -- 172.21.15.46:0/3532648437 wait complete. 2024-09-17T03:28:26.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:28:26.830 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 -- 172.21.15.46:0/2614976471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c0b5c70 msgr2=0x7f315c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 --2- 172.21.15.46:0/2614976471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c0b5c70 0x7f315c15bb60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f315c074b20 tx=0x7f314c01c760 comp rx=0 tx=0).stop 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 -- 172.21.15.46:0/2614976471 shutdown_connections 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 --2- 172.21.15.46:0/2614976471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c0b5c70 0x7f315c15bb60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 -- 172.21.15.46:0/2614976471 >> 172.21.15.46:0/2614976471 conn(0x7f315c0b2040 msgr2=0x7f315c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 -- 172.21.15.46:0/2614976471 shutdown_connections 2024-09-17T03:28:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.878+0000 7f3162400640 1 -- 172.21.15.46:0/2614976471 wait complete. 2024-09-17T03:28:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 Processor -- start 2024-09-17T03:28:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 -- start start 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f315c0b4de0 0x7f315c149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f315c0b5c70 0x7f315c1f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 0x7f315c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f315c169bd0 con 0x7f315c1527b0 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f315c169a50 con 0x7f315c0b4de0 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3162400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f315c169d50 con 0x7f315c0b5c70 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 0x7f315c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 0x7f315c14bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34710/0 (socket says 172.21.15.46:34710) 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 -- 172.21.15.46:0/1220428130 learned_addr learned my addr 172.21.15.46:0/1220428130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315b400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f315c0b4de0 0x7f315c149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f3152a00640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f315c0b5c70 0x7f315c1f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 -- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f315c0b5c70 msgr2=0x7f315c1f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f315c0b5c70 0x7f315c1f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 -- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f315c0b4de0 msgr2=0x7f315c149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f315c0b4de0 0x7f315c149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.882+0000 7f315be00640 1 -- 172.21.15.46:0/1220428130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c1394c0 con 0x7f315c1527b0 2024-09-17T03:28:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f315be00640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 0x7f315c14bcc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f315400c3c0 tx=0x7f315400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154005220 con 0x7f315c1527b0 2024-09-17T03:28:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31540054b0 con 0x7f315c1527b0 2024-09-17T03:28:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c14a480 con 0x7f315c1527b0 2024-09-17T03:28:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154006a60 con 0x7f315c1527b0 2024-09-17T03:28:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c14ac70 con 0x7f315c1527b0 2024-09-17T03:28:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c14a6e0 con 0x7f315c1527b0 2024-09-17T03:28:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f3154019020 con 0x7f315c1527b0 2024-09-17T03:28:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3153e00640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3138073ef0 0x7f3138076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5795+0+0 (secure 0 0 0) 0x7f3154091f30 con 0x7f315c1527b0 2024-09-17T03:28:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.886+0000 7f315b400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3138073ef0 0x7f3138076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.890+0000 7f315b400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3138073ef0 0x7f3138076400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f314c0af000 tx=0x7f314c01afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:26.894+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f315405e350 con 0x7f315c1527b0 2024-09-17T03:28:27.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.102+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f315c15bb60 con 0x7f315c1527b0 2024-09-17T03:28:27.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3153e00640 1 -- 172.21.15.46:0/1220428130 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f3154062d80 con 0x7f315c1527b0 2024-09-17T03:28:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3138073ef0 msgr2=0x7f3138076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3138073ef0 0x7f3138076400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f314c0af000 tx=0x7f314c01afa0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 msgr2=0x7f315c14bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 0x7f315c14bcc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f315400c3c0 tx=0x7f315400c7c0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 shutdown_connections 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3138073ef0 0x7f3138076400 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f315c1527b0 0x7f315c14bcc0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f315c0b5c70 0x7f315c1f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 --2- 172.21.15.46:0/1220428130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f315c0b4de0 0x7f315c149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 >> 172.21.15.46:0/1220428130 conn(0x7f315c0b2040 msgr2=0x7f315c15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 shutdown_connections 2024-09-17T03:28:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.690+0000 7f3162400640 1 -- 172.21.15.46:0/1220428130 wait complete. 2024-09-17T03:28:27.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:28:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 -- 172.21.15.46:0/310503631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 msgr2=0x7effe00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 --2- 172.21.15.46:0/310503631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe00b5a80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7effd0009840 tx=0x7effd001c790 comp rx=0 tx=0).stop 2024-09-17T03:28:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 -- 172.21.15.46:0/310503631 shutdown_connections 2024-09-17T03:28:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 --2- 172.21.15.46:0/310503631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe00b5a80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 -- 172.21.15.46:0/310503631 >> 172.21.15.46:0/310503631 conn(0x7effe00708c0 msgr2=0x7effe0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:27.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 -- 172.21.15.46:0/310503631 shutdown_connections 2024-09-17T03:28:27.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.878+0000 7effe5800640 1 -- 172.21.15.46:0/310503631 wait complete. 2024-09-17T03:28:27.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 Processor -- start 2024-09-17T03:28:27.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 -- start start 2024-09-17T03:28:27.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:27.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0160d50 0x7effe0155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:27.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effe014de20 0x7effe0074410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:27.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7effe016b820 con 0x7effe0157030 2024-09-17T03:28:27.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effe016b6a0 con 0x7effe0160d50 2024-09-17T03:28:27.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7effe016b9a0 con 0x7effe014de20 2024-09-17T03:28:27.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:27.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe013ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34714/0 (socket says 172.21.15.46:34714) 2024-09-17T03:28:27.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 -- 172.21.15.46:0/2653379826 learned_addr learned my addr 172.21.15.46:0/2653379826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:27.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effd7e00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0160d50 0x7effe0155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:27.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdf400640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effe014de20 0x7effe0074410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:27.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 -- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effe014de20 msgr2=0x7effe0074410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:27.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effe014de20 0x7effe0074410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 -- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0160d50 msgr2=0x7effe0155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:27.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0160d50 0x7effe0155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:27.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 -- 172.21.15.46:0/2653379826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe0155980 con 0x7effe0157030 2024-09-17T03:28:27.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdf400640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effe014de20 0x7effe0074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:27.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effdea00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe013ff50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c10 tx=0x7effd00a65c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:27.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0005250 con 0x7effe0157030 2024-09-17T03:28:27.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effd00053f0 con 0x7effe0157030 2024-09-17T03:28:27.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe0155c30 con 0x7effe0157030 2024-09-17T03:28:27.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd00046d0 con 0x7effe0157030 2024-09-17T03:28:27.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.882+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe01fcc10 con 0x7effe0157030 2024-09-17T03:28:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.886+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe00d2260 con 0x7effe0157030 2024-09-17T03:28:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.886+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7effd0004890 con 0x7effe0157030 2024-09-17T03:28:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.886+0000 7effd6a00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effbc073ef0 0x7effbc076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.886+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5795+0+0 (secure 0 0 0) 0x7effd012d1e0 con 0x7effe0157030 2024-09-17T03:28:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.886+0000 7effd7e00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effbc073ef0 0x7effbc076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:27.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.886+0000 7effd7e00640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effbc073ef0 0x7effbc076400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effe006ee70 tx=0x7effc8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:27.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:27.890+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7effd00ac030 con 0x7effe0157030 2024-09-17T03:28:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.066+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7effe014ebe0 con 0x7effe0157030 2024-09-17T03:28:28.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.690+0000 7effd6a00640 1 -- 172.21.15.46:0/2653379826 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) ==== 181+0+0 (secure 0 0 0) 0x7effd0016410 con 0x7effe0157030 2024-09-17T03:28:28.694 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:28:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effbc073ef0 msgr2=0x7effbc076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effbc073ef0 0x7effbc076400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effe006ee70 tx=0x7effc8009040 comp rx=0 tx=0).stop 2024-09-17T03:28:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 msgr2=0x7effe013ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe013ff50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c10 tx=0x7effd00a65c0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 shutdown_connections 2024-09-17T03:28:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effbc073ef0 0x7effbc076400 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effe014de20 0x7effe0074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0160d50 0x7effe0155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 --2- 172.21.15.46:0/2653379826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe0157030 0x7effe013ff50 secure :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c10 tx=0x7effd00a65c0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 >> 172.21.15.46:0/2653379826 conn(0x7effe00708c0 msgr2=0x7effe0151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:28.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.694+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 shutdown_connections 2024-09-17T03:28:28.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.698+0000 7effe5800640 1 -- 172.21.15.46:0/2653379826 wait complete. 2024-09-17T03:28:28.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:28:28.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.882+0000 7f4c0e400640 1 -- 172.21.15.46:0/1995406600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c080cefa0 msgr2=0x7f4c080cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.882+0000 7f4c0e400640 1 --2- 172.21.15.46:0/1995406600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c080cefa0 0x7f4c080cf3b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009870 tx=0x7f4bf801c6d0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.882+0000 7f4c0e400640 1 -- 172.21.15.46:0/1995406600 shutdown_connections 2024-09-17T03:28:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.882+0000 7f4c0e400640 1 --2- 172.21.15.46:0/1995406600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c080cefa0 0x7f4c080cf3b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.882+0000 7f4c0e400640 1 -- 172.21.15.46:0/1995406600 >> 172.21.15.46:0/1995406600 conn(0x7f4c080b2040 msgr2=0x7f4c080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 -- 172.21.15.46:0/1995406600 shutdown_connections 2024-09-17T03:28:28.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 -- 172.21.15.46:0/1995406600 wait complete. 2024-09-17T03:28:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 Processor -- start 2024-09-17T03:28:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 -- start start 2024-09-17T03:28:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c08072100 0x7f4c0813d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c080cefa0 0x7f4c080b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 0x7f4c080b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0816b0d0 con 0x7f4c0815cbb0 2024-09-17T03:28:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.886+0000 7f4c0e400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0816af50 con 0x7f4c08072100 2024-09-17T03:28:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c0e400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0816b250 con 0x7f4c080cefa0 2024-09-17T03:28:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 0x7f4c080b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c08072100 0x7f4c0813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 0x7f4c080b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34734/0 (socket says 172.21.15.46:34734) 2024-09-17T03:28:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c08072100 0x7f4c0813d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:50568/0 (socket says 172.21.15.46:50568) 2024-09-17T03:28:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 -- 172.21.15.46:0/2989050597 learned_addr learned my addr 172.21.15.46:0/2989050597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c06a00640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c080cefa0 0x7f4c080b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 -- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c080cefa0 msgr2=0x7f4c080b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c080cefa0 0x7f4c080b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 -- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c08072100 msgr2=0x7f4c0813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c08072100 0x7f4c0813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 -- 172.21.15.46:0/2989050597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c08155e30 con 0x7f4c0815cbb0 2024-09-17T03:28:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c07e00640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 0x7f4c080b1540 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c730 tx=0x7f4c0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00003b80 con 0x7f4c0815cbb0 2024-09-17T03:28:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c00003d20 con 0x7f4c0815cbb0 2024-09-17T03:28:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000aab0 con 0x7f4c0815cbb0 2024-09-17T03:28:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c08156110 con 0x7f4c0815cbb0 2024-09-17T03:28:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081566d0 con 0x7f4c0815cbb0 2024-09-17T03:28:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c080cf3b0 con 0x7f4c0815cbb0 2024-09-17T03:28:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.890+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f4c0000ac50 con 0x7f4c0815cbb0 2024-09-17T03:28:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.894+0000 7f4bff400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4be8073ef0 0x7f4be8076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.894+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5357+0+0 (secure 0 0 0) 0x7f4c00092970 con 0x7f4c0815cbb0 2024-09-17T03:28:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.894+0000 7f4c07400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4be8073ef0 0x7f4be8076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.894+0000 7f4c07400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4be8073ef0 0x7f4be8076400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80af000 tx=0x7f4bf801af10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:28.898+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4c0815d6d0 con 0x7f4c0815cbb0 2024-09-17T03:28:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.070+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4c08144cf0 con 0x7f4c0815cbb0 2024-09-17T03:28:29.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:28:29.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:28:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.698+0000 7f4bff400640 1 -- 172.21.15.46:0/2989050597 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) ==== 169+0+0 (secure 0 0 0) 0x7f4c0005edd0 con 0x7f4c0815cbb0 2024-09-17T03:28:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:28:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4be8073ef0 msgr2=0x7f4be8076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4be8073ef0 0x7f4be8076400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80af000 tx=0x7f4bf801af10 comp rx=0 tx=0).stop 2024-09-17T03:28:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 msgr2=0x7f4c080b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 0x7f4c080b1540 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c730 tx=0x7f4c0000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 shutdown_connections 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4be8073ef0 0x7f4be8076400 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0815cbb0 0x7f4c080b1540 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c080cefa0 0x7f4c080b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 --2- 172.21.15.46:0/2989050597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c08072100 0x7f4c0813d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 >> 172.21.15.46:0/2989050597 conn(0x7f4c080b2040 msgr2=0x7f4c080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 shutdown_connections 2024-09-17T03:28:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.702+0000 7f4c0e400640 1 -- 172.21.15.46:0/2989050597 wait complete. 2024-09-17T03:28:29.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 -- 172.21.15.46:0/1428809688 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6580cefa0 msgr2=0x7fa6580cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 --2- 172.21.15.46:0/1428809688 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6580cefa0 0x7fa6580cf3b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa644009840 tx=0x7fa64401c820 comp rx=0 tx=0).stop 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 -- 172.21.15.46:0/1428809688 shutdown_connections 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 --2- 172.21.15.46:0/1428809688 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6580cefa0 0x7fa6580cf3b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 --2- 172.21.15.46:0/1428809688 >> v2:172.21.15.99:3300/0 conn(0x7fa6580749a0 0x7fa658074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 -- 172.21.15.46:0/1428809688 >> 172.21.15.46:0/1428809688 conn(0x7fa6580b2040 msgr2=0x7fa6580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 -- 172.21.15.46:0/1428809688 shutdown_connections 2024-09-17T03:28:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.950+0000 7fa65f600640 1 -- 172.21.15.46:0/1428809688 wait complete. 2024-09-17T03:28:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 Processor -- start 2024-09-17T03:28:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 -- start start 2024-09-17T03:28:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6580749a0 0x7fa658145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 0x7fa65813dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:29.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa658160330 0x7fa6581fc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa65816a060 con 0x7fa6580cefa0 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa658169ee0 con 0x7fa6580749a0 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa65816a1e0 con 0x7fa658160330 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 0x7fa65813dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 0x7fa65813dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34746/0 (socket says 172.21.15.46:34746) 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 -- 172.21.15.46:0/740809619 learned_addr learned my addr 172.21.15.46:0/740809619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65cc00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6580749a0 0x7fa658145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65d600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa658160330 0x7fa6581fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 -- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa658160330 msgr2=0x7fa6581fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa658160330 0x7fa6581fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 -- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6580749a0 msgr2=0x7fa658145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:29.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6580749a0 0x7fa658145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 -- 172.21.15.46:0/740809619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6580b1720 con 0x7fa6580cefa0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65cc00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6580749a0 0x7fa658145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa657e00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 0x7fa65813dbe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa644009790 tx=0x7fa6440a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b2040 con 0x7fa6580cefa0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa644004400 con 0x7fa6580cefa0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6580ccd90 con 0x7fa6580cefa0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.954+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440052b0 con 0x7fa6580cefa0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.958+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6580b0370 con 0x7fa6580cefa0 2024-09-17T03:28:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.958+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658074db0 con 0x7fa6580cefa0 2024-09-17T03:28:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.958+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fa644005450 con 0x7fa6580cefa0 2024-09-17T03:28:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.958+0000 7fa654c00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa638073f20 0x7fa638076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.958+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7fa64412ce40 con 0x7fa6580cefa0 2024-09-17T03:28:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.958+0000 7fa65cc00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa638073f20 0x7fa638076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:29.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.962+0000 7fa65cc00640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa638073f20 0x7fa638076430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009b90 tx=0x7fa64c009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:29.962+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa644016410 con 0x7fa6580cefa0 2024-09-17T03:28:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.138+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fa6580ccf60 con 0x7fa6580cefa0 2024-09-17T03:28:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa654c00640 1 -- 172.21.15.46:0/740809619 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7fa6440f9940 con 0x7fa6580cefa0 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa638073f20 msgr2=0x7fa638076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa638073f20 0x7fa638076430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009b90 tx=0x7fa64c009260 comp rx=0 tx=0).stop 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 msgr2=0x7fa65813dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 0x7fa65813dbe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa644009790 tx=0x7fa6440a57f0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 shutdown_connections 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa638073f20 0x7fa638076430 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa658160330 0x7fa6581fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6580cefa0 0x7fa65813dbe0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 --2- 172.21.15.46:0/740809619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6580749a0 0x7fa658145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 >> 172.21.15.46:0/740809619 conn(0x7fa6580b2040 msgr2=0x7fa658160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 shutdown_connections 2024-09-17T03:28:30.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.146+0000 7fa65f600640 1 -- 172.21.15.46:0/740809619 wait complete. 2024-09-17T03:28:30.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-17T03:28:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 -- 172.21.15.46:0/77258895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce381527b0 msgr2=0x7fce3815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 --2- 172.21.15.46:0/77258895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce381527b0 0x7fce3815bb60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fce20009900 tx=0x7fce2001c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 -- 172.21.15.46:0/77258895 shutdown_connections 2024-09-17T03:28:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 --2- 172.21.15.46:0/77258895 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce380b4de0 0x7fce380b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 --2- 172.21.15.46:0/77258895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce381527b0 0x7fce3815bb60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 -- 172.21.15.46:0/77258895 >> 172.21.15.46:0/77258895 conn(0x7fce380b2040 msgr2=0x7fce380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 -- 172.21.15.46:0/77258895 shutdown_connections 2024-09-17T03:28:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 -- 172.21.15.46:0/77258895 wait complete. 2024-09-17T03:28:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 Processor -- start 2024-09-17T03:28:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.330+0000 7fce3d400640 1 -- start start 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce380b4de0 0x7fce38158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 0x7fce380ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce381527b0 0x7fce380aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce38168a00 con 0x7fce380b5c70 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce38168880 con 0x7fce380b4de0 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fce38168b80 con 0x7fce381527b0 2024-09-17T03:28:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 0x7fce380ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 0x7fce380ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34758/0 (socket says 172.21.15.46:34758) 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 -- 172.21.15.46:0/914909757 learned_addr learned my addr 172.21.15.46:0/914909757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce37200640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce381527b0 0x7fce380aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce36800640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce380b4de0 0x7fce38158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 -- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce381527b0 msgr2=0x7fce380aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce381527b0 0x7fce380aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 -- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce380b4de0 msgr2=0x7fce38158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce380b4de0 0x7fce38158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 -- 172.21.15.46:0/914909757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3814feb0 con 0x7fce380b5c70 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2fe00640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 0x7fce380ae8e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fce2001a4e0 tx=0x7fce20005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce200b2040 con 0x7fce380b5c70 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce20005320 con 0x7fce380b5c70 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce38150150 con 0x7fce380b5c70 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce200046d0 con 0x7fce380b5c70 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce38150710 con 0x7fce380b5c70 2024-09-17T03:28:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.334+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce380b51f0 con 0x7fce380b5c70 2024-09-17T03:28:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.338+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fce200bd060 con 0x7fce380b5c70 2024-09-17T03:28:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.338+0000 7fce2ea00640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce14073f20 0x7fce14076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.338+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7fce2012c8d0 con 0x7fce380b5c70 2024-09-17T03:28:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.338+0000 7fce36800640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce14073f20 0x7fce14076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.338+0000 7fce36800640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce14073f20 0x7fce14076430 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fce28007fd0 tx=0x7fce28009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.342+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce200164d0 con 0x7fce380b5c70 2024-09-17T03:28:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.578+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7fce381546f0 con 0x7fce380b5c70 2024-09-17T03:28:30.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.698+0000 7fce2ea00640 1 -- 172.21.15.46:0/914909757 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v338) ==== 167+0+0 (secure 0 0 0) 0x7fce200f8d80 con 0x7fce380b5c70 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce14073f20 msgr2=0x7fce14076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce14073f20 0x7fce14076430 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fce28007fd0 tx=0x7fce28009040 comp rx=0 tx=0).stop 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 msgr2=0x7fce380ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 0x7fce380ae8e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fce2001a4e0 tx=0x7fce20005760 comp rx=0 tx=0).stop 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 shutdown_connections 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce14073f20 0x7fce14076430 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce381527b0 0x7fce380aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce380b5c70 0x7fce380ae8e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 --2- 172.21.15.46:0/914909757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce380b4de0 0x7fce38158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 >> 172.21.15.46:0/914909757 conn(0x7fce380b2040 msgr2=0x7fce381538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 shutdown_connections 2024-09-17T03:28:30.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.702+0000 7fce3d400640 1 -- 172.21.15.46:0/914909757 wait complete. 2024-09-17T03:28:30.725 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-17T03:28:30.726 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-17T03:28:30.726 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:28:30.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 -- 172.21.15.46:0/4210199104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d041527b0 msgr2=0x7f9d0415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 --2- 172.21.15.46:0/4210199104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d041527b0 0x7f9d0415bb60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009900 tx=0x7f9cf001c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 -- 172.21.15.46:0/4210199104 shutdown_connections 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 --2- 172.21.15.46:0/4210199104 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d040b4de0 0x7f9d040b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 --2- 172.21.15.46:0/4210199104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d041527b0 0x7f9d0415bb60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 -- 172.21.15.46:0/4210199104 >> 172.21.15.46:0/4210199104 conn(0x7f9d040b2040 msgr2=0x7f9d040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 -- 172.21.15.46:0/4210199104 shutdown_connections 2024-09-17T03:28:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 -- 172.21.15.46:0/4210199104 wait complete. 2024-09-17T03:28:30.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 Processor -- start 2024-09-17T03:28:30.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.882+0000 7f9d0a000640 1 -- start start 2024-09-17T03:28:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d0a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 0x7f9d040af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d0a000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d040b5c70 0x7f9d04143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d0a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d041527b0 0x7f9d041f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d0a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04168a00 con 0x7f9d040b4de0 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d0a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04168880 con 0x7f9d041527b0 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d0a000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9d04168b80 con 0x7f9d040b5c70 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 0x7f9d040af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 0x7f9d040af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34780/0 (socket says 172.21.15.46:34780) 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 -- 172.21.15.46:0/941279945 learned_addr learned my addr 172.21.15.46:0/941279945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d02a00640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d040b5c70 0x7f9d04143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03e00640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d041527b0 0x7f9d041f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 -- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d040b5c70 msgr2=0x7f9d04143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d040b5c70 0x7f9d04143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 -- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d041527b0 msgr2=0x7f9d041f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d041527b0 0x7f9d041f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03400640 1 -- 172.21.15.46:0/941279945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d040b19a0 con 0x7f9d040b4de0 2024-09-17T03:28:30.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.886+0000 7f9d03e00640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d041527b0 0x7f9d041f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:30.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.890+0000 7f9d03400640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 0x7f9d040af500 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c3c0 tx=0x7f9cf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.890+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4017070 con 0x7f9d040b4de0 2024-09-17T03:28:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.890+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d040b13e0 con 0x7f9d040b4de0 2024-09-17T03:28:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.890+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cf40053c0 con 0x7f9d040b4de0 2024-09-17T03:28:30.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.890+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041f6410 con 0x7f9d040b4de0 2024-09-17T03:28:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.890+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4021060 con 0x7f9d040b4de0 2024-09-17T03:28:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.894+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cdc0052a0 con 0x7f9d040b4de0 2024-09-17T03:28:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.894+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f9cf401a070 con 0x7f9d040b4de0 2024-09-17T03:28:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.898+0000 7f9cfb400640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ce4073ef0 0x7f9ce4076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.898+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f9cf40923a0 con 0x7f9d040b4de0 2024-09-17T03:28:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.898+0000 7f9d02a00640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ce4073ef0 0x7f9ce4076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.902+0000 7f9d02a00640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ce4073ef0 0x7f9ce4076400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00af000 tx=0x7f9cf001b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:30.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:30.902+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9d040b5730 con 0x7f9d040b4de0 2024-09-17T03:28:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.078+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cdc005860 con 0x7f9d040b4de0 2024-09-17T03:28:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9cfb400640 1 -- 172.21.15.46:0/941279945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+12994 (secure 0 0 0) 0x7f9cf405e820 con 0x7f9d040b4de0 2024-09-17T03:28:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:31.084 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":63,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:28:29.696744+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/695855747":"2024-09-18T03:28:26.685497+0000","172.21.15.46:6834/695855747":"2024-09-18T03:28:26.685497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ce4073ef0 msgr2=0x7f9ce4076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ce4073ef0 0x7f9ce4076400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00af000 tx=0x7f9cf001b0f0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 msgr2=0x7f9d040af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 0x7f9d040af500 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c3c0 tx=0x7f9cf400c880 comp rx=0 tx=0).stop 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 shutdown_connections 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ce4073ef0 0x7f9ce4076400 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d041527b0 0x7f9d041f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d040b5c70 0x7f9d04143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 --2- 172.21.15.46:0/941279945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d040b4de0 0x7f9d040af500 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 >> 172.21.15.46:0/941279945 conn(0x7f9d040b2040 msgr2=0x7f9d04153910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 shutdown_connections 2024-09-17T03:28:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.082+0000 7f9d0a000640 1 -- 172.21.15.46:0/941279945 wait complete. 2024-09-17T03:28:31.106 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:28:31.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:28:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 -- 172.21.15.46:0/973710044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b81527b0 msgr2=0x7f21b8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 --2- 172.21.15.46:0/973710044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b81527b0 0x7f21b8074e30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009840 tx=0x7f21a001c760 comp rx=0 tx=0).stop 2024-09-17T03:28:31.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 -- 172.21.15.46:0/973710044 shutdown_connections 2024-09-17T03:28:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 --2- 172.21.15.46:0/973710044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b81527b0 0x7f21b8074e30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 --2- 172.21.15.46:0/973710044 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b80b5c70 0x7f21b815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 -- 172.21.15.46:0/973710044 >> 172.21.15.46:0/973710044 conn(0x7f21b80b2040 msgr2=0x7f21b80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 -- 172.21.15.46:0/973710044 shutdown_connections 2024-09-17T03:28:31.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.266+0000 7f21bda00640 1 -- 172.21.15.46:0/973710044 wait complete. 2024-09-17T03:28:31.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 Processor -- start 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 -- start start 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21b80b4de0 0x7f21b80b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 0x7f21b80cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b81527b0 0x7f21b80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8168ca0 con 0x7f21b80b5c70 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8168b20 con 0x7f21b80b4de0 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8168e20 con 0x7f21b81527b0 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 0x7f21b80cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 0x7f21b80cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34784/0 (socket says 172.21.15.46:34784) 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 -- 172.21.15.46:0/4264383508 learned_addr learned my addr 172.21.15.46:0/4264383508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:31.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21b6a00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21b80b4de0 0x7f21b80b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21b7400640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b81527b0 0x7f21b80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 -- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b81527b0 msgr2=0x7f21b80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b81527b0 0x7f21b80ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 -- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21b80b4de0 msgr2=0x7f21b80b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21b80b4de0 0x7f21b80b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 -- 172.21.15.46:0/4264383508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b80cfa30 con 0x7f21b80b5c70 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21b7400640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b81527b0 0x7f21b80ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21afe00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 0x7f21b80cdf20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009790 tx=0x7f21a00a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00bb070 con 0x7f21b80b5c70 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21a0005030 con 0x7f21b80b5c70 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00b2040 con 0x7f21b80b5c70 2024-09-17T03:28:31.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b8153da0 con 0x7f21b80b5c70 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.270+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b8154360 con 0x7f21b80b5c70 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.274+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b815bb60 con 0x7f21b80b5c70 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.274+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f21a00be070 con 0x7f21b80b5c70 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.274+0000 7f21aea00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2194073f20 0x7f2194076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.274+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f21a00de080 con 0x7f21b80b5c70 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.274+0000 7f21b6a00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2194073f20 0x7f2194076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.274+0000 7f21b6a00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2194073f20 0x7f2194076430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21a80058d0 tx=0x7f21a8005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:31.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.278+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21b8075580 con 0x7f21b80b5c70 2024-09-17T03:28:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.454+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21b80d0070 con 0x7f21b80b5c70 2024-09-17T03:28:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.454+0000 7f21aea00640 1 -- 172.21.15.46:0/4264383508 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f21a0016410 con 0x7f21b80b5c70 2024-09-17T03:28:31.459 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:28:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2194073f20 msgr2=0x7f2194076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2194073f20 0x7f2194076430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21a80058d0 tx=0x7f21a8005860 comp rx=0 tx=0).stop 2024-09-17T03:28:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 msgr2=0x7f21b80cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 0x7f21b80cdf20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009790 tx=0x7f21a00a57a0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 shutdown_connections 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2194073f20 0x7f2194076430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f21b81527b0 0x7f21b80ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21b80b5c70 0x7f21b80cdf20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 --2- 172.21.15.46:0/4264383508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21b80b4de0 0x7f21b80b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 >> 172.21.15.46:0/4264383508 conn(0x7f21b80b2040 msgr2=0x7f21b8151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 shutdown_connections 2024-09-17T03:28:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.458+0000 7f21bda00640 1 -- 172.21.15.46:0/4264383508 wait complete. 2024-09-17T03:28:31.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-17T03:28:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 -- 172.21.15.46:0/3995809504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 msgr2=0x7f93040b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 --2- 172.21.15.46:0/3995809504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f93040b51f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f93040703e0 tx=0x7f92fc01ccc0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 -- 172.21.15.46:0/3995809504 shutdown_connections 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 --2- 172.21.15.46:0/3995809504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f93040b51f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 --2- 172.21.15.46:0/3995809504 >> v2:172.21.15.99:3300/0 conn(0x7f93041527b0 0x7f9304074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 --2- 172.21.15.46:0/3995809504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93040b5c70 0x7f930415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 -- 172.21.15.46:0/3995809504 >> 172.21.15.46:0/3995809504 conn(0x7f93040b2040 msgr2=0x7f93040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 -- 172.21.15.46:0/3995809504 shutdown_connections 2024-09-17T03:28:31.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.638+0000 7f930b600640 1 -- 172.21.15.46:0/3995809504 wait complete. 2024-09-17T03:28:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 Processor -- start 2024-09-17T03:28:31.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 -- start start 2024-09-17T03:28:31.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f9304142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93040b5c70 0x7f930415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041527b0 0x7f93040af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f930415f730 con 0x7f93040b4de0 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f930415f5b0 con 0x7f93041527b0 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f930415f8b0 con 0x7f93040b5c70 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f9304142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f9304142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34800/0 (socket says 172.21.15.46:34800) 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 -- 172.21.15.46:0/3928096149 learned_addr learned my addr 172.21.15.46:0/3928096149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9309600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041527b0 0x7f93040af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9303e00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93040b5c70 0x7f930415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 -- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93040b5c70 msgr2=0x7f930415a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93040b5c70 0x7f930415a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 -- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041527b0 msgr2=0x7f93040af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:31.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041527b0 0x7f93040af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 -- 172.21.15.46:0/3928096149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93041523f0 con 0x7f93040b4de0 2024-09-17T03:28:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9308c00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f9304142b70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f92f800d660 tx=0x7f92f800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8017070 con 0x7f93040b4de0 2024-09-17T03:28:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92f8005310 con 0x7f93040b4de0 2024-09-17T03:28:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93040aa4d0 con 0x7f93040b4de0 2024-09-17T03:28:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8009820 con 0x7f93040b4de0 2024-09-17T03:28:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.642+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9304158700 con 0x7f93040b4de0 2024-09-17T03:28:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.646+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930415fdd0 con 0x7f93040b4de0 2024-09-17T03:28:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.646+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f92f80099c0 con 0x7f93040b4de0 2024-09-17T03:28:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.646+0000 7f9300c00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f92e4073f20 0x7f92e4076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.646+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f92f8092cf0 con 0x7f93040b4de0 2024-09-17T03:28:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.646+0000 7f9303e00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f92e4073f20 0x7f92e4076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.646+0000 7f9303e00640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f92e4073f20 0x7f92e4076430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f92f0005fd0 tx=0x7f92f0005d10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:31.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.650+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92f805f110 con 0x7f93040b4de0 2024-09-17T03:28:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:31.886+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f9304155c60 con 0x7f93040b4de0 2024-09-17T03:28:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f9300c00640 1 -- 172.21.15.46:0/3928096149 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v340) ==== 170+0+0 (secure 0 0 0) 0x7f92f8063b40 con 0x7f93040b4de0 2024-09-17T03:28:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f92e4073f20 msgr2=0x7f92e4076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f92e4073f20 0x7f92e4076430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f92f0005fd0 tx=0x7f92f0005d10 comp rx=0 tx=0).stop 2024-09-17T03:28:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 msgr2=0x7f9304142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f9304142b70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f92f800d660 tx=0x7f92f800db20 comp rx=0 tx=0).stop 2024-09-17T03:28:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 shutdown_connections 2024-09-17T03:28:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f92e4073f20 0x7f92e4076430 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041527b0 0x7f93040af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93040b5c70 0x7f930415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 --2- 172.21.15.46:0/3928096149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93040b4de0 0x7f9304142b70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 >> 172.21.15.46:0/3928096149 conn(0x7f93040b2040 msgr2=0x7f930406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:32.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 shutdown_connections 2024-09-17T03:28:32.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.710+0000 7f930b600640 1 -- 172.21.15.46:0/3928096149 wait complete. 2024-09-17T03:28:32.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 -- 172.21.15.46:0/3405065461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf700b4de0 msgr2=0x7faf700b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 --2- 172.21.15.46:0/3405065461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf700b4de0 0x7faf700b51f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf68009c80 tx=0x7faf6801cce0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 -- 172.21.15.46:0/3405065461 shutdown_connections 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 --2- 172.21.15.46:0/3405065461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf700b4de0 0x7faf700b51f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 --2- 172.21.15.46:0/3405065461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf701527b0 0x7faf7015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 -- 172.21.15.46:0/3405065461 >> 172.21.15.46:0/3405065461 conn(0x7faf700b2040 msgr2=0x7faf700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 -- 172.21.15.46:0/3405065461 shutdown_connections 2024-09-17T03:28:32.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 -- 172.21.15.46:0/3405065461 wait complete. 2024-09-17T03:28:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 Processor -- start 2024-09-17T03:28:32.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.902+0000 7faf77600640 1 -- start start 2024-09-17T03:28:32.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf700b4de0 0x7faf7013d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf700b5c70 0x7faf700af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 0x7faf700b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf70168ab0 con 0x7faf701527b0 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf70168930 con 0x7faf700b5c70 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faf70168c30 con 0x7faf700b4de0 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 0x7faf700b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 0x7faf700b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35730/0 (socket says 172.21.15.46:35730) 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 -- 172.21.15.46:0/1571234460 learned_addr learned my addr 172.21.15.46:0/1571234460 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf74c00640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf700b4de0 0x7faf7013d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf6fe00640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf700b5c70 0x7faf700af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 -- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf700b4de0 msgr2=0x7faf7013d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf700b4de0 0x7faf7013d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 -- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf700b5c70 msgr2=0x7faf700af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf700b5c70 0x7faf700af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 -- 172.21.15.46:0/1571234460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7013f6d0 con 0x7faf701527b0 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf6fe00640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf700b5c70 0x7faf700af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf75600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 0x7faf700b12c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faf68009bd0 tx=0x7faf68009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680bb070 con 0x7faf701527b0 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf68004d40 con 0x7faf701527b0 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf680b2040 con 0x7faf701527b0 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7013f9b0 con 0x7faf701527b0 2024-09-17T03:28:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7013ff70 con 0x7faf701527b0 2024-09-17T03:28:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7013f440 con 0x7faf701527b0 2024-09-17T03:28:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.906+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7faf680ad850 con 0x7faf701527b0 2024-09-17T03:28:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.910+0000 7faf6cc00640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf50073f20 0x7faf50076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.910+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7faf6812cea0 con 0x7faf701527b0 2024-09-17T03:28:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.910+0000 7faf74c00640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf50073f20 0x7faf50076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:32.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.910+0000 7faf74c00640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf50073f20 0x7faf50076430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf64007fd0 tx=0x7faf64009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:32.914+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf68016990 con 0x7faf701527b0 2024-09-17T03:28:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.086+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7faf700b51f0 con 0x7faf701527b0 2024-09-17T03:28:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf6cc00640 1 -- 172.21.15.46:0/1571234460 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+2294 (secure 0 0 0) 0x7faf680f8fb0 con 0x7faf701527b0 2024-09-17T03:28:33.091 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:28:33.092 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:28:33.093 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:28:33.094 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:28:33.094 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:28:33.094 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:28:33.094 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:28:33.094 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:28:33.094 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf50073f20 msgr2=0x7faf50076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf50073f20 0x7faf50076430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf64007fd0 tx=0x7faf64009040 comp rx=0 tx=0).stop 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 msgr2=0x7faf700b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 0x7faf700b12c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faf68009bd0 tx=0x7faf68009350 comp rx=0 tx=0).stop 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 shutdown_connections 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf50073f20 0x7faf50076430 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf701527b0 0x7faf700b12c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf700b5c70 0x7faf700af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 --2- 172.21.15.46:0/1571234460 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf700b4de0 0x7faf7013d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 >> 172.21.15.46:0/1571234460 conn(0x7faf700b2040 msgr2=0x7faf701581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 shutdown_connections 2024-09-17T03:28:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.090+0000 7faf77600640 1 -- 172.21.15.46:0/1571234460 wait complete. 2024-09-17T03:28:33.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:28:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 -- 172.21.15.46:0/564568216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35040cefa0 msgr2=0x7f35040cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 --2- 172.21.15.46:0/564568216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35040cefa0 0x7f35040cf3b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009870 tx=0x7f34f401c6d0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 -- 172.21.15.46:0/564568216 shutdown_connections 2024-09-17T03:28:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 --2- 172.21.15.46:0/564568216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35040cefa0 0x7f35040cf3b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 -- 172.21.15.46:0/564568216 >> 172.21.15.46:0/564568216 conn(0x7f35040b2040 msgr2=0x7f35040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:33.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 -- 172.21.15.46:0/564568216 shutdown_connections 2024-09-17T03:28:33.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 -- 172.21.15.46:0/564568216 wait complete. 2024-09-17T03:28:33.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 Processor -- start 2024-09-17T03:28:33.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.274+0000 7f3509c00640 1 -- start start 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 0x7f350413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35040cefa0 0x7f35041f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f350415cbb0 0x7f35040b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f350416b0d0 con 0x7f3504072100 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f350416af50 con 0x7f35040cefa0 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f350416b250 con 0x7f350415cbb0 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 0x7f350413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 0x7f350413d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35752/0 (socket says 172.21.15.46:35752) 2024-09-17T03:28:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 -- 172.21.15.46:0/4238549686 learned_addr learned my addr 172.21.15.46:0/4238549686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f34fa000640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35040cefa0 0x7f35041f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3503400640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f350415cbb0 0x7f35040b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 -- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f350415cbb0 msgr2=0x7f35040b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f350415cbb0 0x7f35040b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 -- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35040cefa0 msgr2=0x7f35041f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35040cefa0 0x7f35041f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 -- 172.21.15.46:0/4238549686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350413ff10 con 0x7f3504072100 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3502a00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 0x7f350413d660 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f34f40b0040 tx=0x7f34f40a65e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40bb020 con 0x7f3504072100 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f40a6ca0 con 0x7f3504072100 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3504140380 con 0x7f3504072100 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40042a0 con 0x7f3504072100 2024-09-17T03:28:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35040cce80 con 0x7f3504072100 2024-09-17T03:28:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.278+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35040cf3b0 con 0x7f3504072100 2024-09-17T03:28:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.282+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f34f40be020 con 0x7f3504072100 2024-09-17T03:28:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.282+0000 7f34fb400640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34e0073f20 0x7f34e0076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.282+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7f34f412ca90 con 0x7f3504072100 2024-09-17T03:28:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.282+0000 7f34fa000640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34e0073f20 0x7f34e0076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.282+0000 7f34fa000640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34e0073f20 0x7f34e0076430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35040cef70 tx=0x7f34ec009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:33.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.286+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34f4016410 con 0x7f3504072100 2024-09-17T03:28:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.458+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3504144cf0 con 0x7f3504072100 2024-09-17T03:28:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.462+0000 7f34fb400640 1 -- 172.21.15.46:0/4238549686 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f34f40f8f40 con 0x7f3504072100 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34e0073f20 msgr2=0x7f34e0076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34e0073f20 0x7f34e0076430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f35040cef70 tx=0x7f34ec009040 comp rx=0 tx=0).stop 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 msgr2=0x7f350413d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 0x7f350413d660 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f34f40b0040 tx=0x7f34f40a65e0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 shutdown_connections 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34e0073f20 0x7f34e0076430 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f350415cbb0 0x7f35040b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35040cefa0 0x7f35041f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 --2- 172.21.15.46:0/4238549686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3504072100 0x7f350413d660 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 >> 172.21.15.46:0/4238549686 conn(0x7f35040b2040 msgr2=0x7f35040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 shutdown_connections 2024-09-17T03:28:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.466+0000 7f3509c00640 1 -- 172.21.15.46:0/4238549686 wait complete. 2024-09-17T03:28:33.489 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:28:33.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:28:33.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:28:33.680 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:28:33.681 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:33.681 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:33.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:28:33.690 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:28:33.690 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:33.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:28:33.700 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:33.700 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:33.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:28:33.744 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:28:33.744 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:33.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:28:33.753 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:28:33.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:33.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 -- 172.21.15.46:0/2844680737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 msgr2=0x7fb7080b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 --2- 172.21.15.46:0/2844680737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb7080b51f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb700009bc0 tx=0x7fb70001cae0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 -- 172.21.15.46:0/2844680737 shutdown_connections 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 --2- 172.21.15.46:0/2844680737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb7080b51f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 --2- 172.21.15.46:0/2844680737 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7081527b0 0x7fb70815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 -- 172.21.15.46:0/2844680737 >> 172.21.15.46:0/2844680737 conn(0x7fb7080b2040 msgr2=0x7fb7080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 -- 172.21.15.46:0/2844680737 shutdown_connections 2024-09-17T03:28:33.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.946+0000 7fb70ee00640 1 -- 172.21.15.46:0/2844680737 wait complete. 2024-09-17T03:28:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 Processor -- start 2024-09-17T03:28:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 -- start start 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb708143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7080b5c70 0x7fb7080aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7081527b0 0x7fb708158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb708168a00 con 0x7fb7080b4de0 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb708168880 con 0x7fb7081527b0 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb708168b80 con 0x7fb7080b5c70 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb708143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb708143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35770/0 (socket says 172.21.15.46:35770) 2024-09-17T03:28:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 -- 172.21.15.46:0/3245343947 learned_addr learned my addr 172.21.15.46:0/3245343947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ce00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7081527b0 0x7fb708158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb6ffe00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7080b5c70 0x7fb7080aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 -- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7080b5c70 msgr2=0x7fb7080aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7080b5c70 0x7fb7080aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 -- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7081527b0 msgr2=0x7fb708158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7081527b0 0x7fb708158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 -- 172.21.15.46:0/3245343947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb708144130 con 0x7fb7080b4de0 2024-09-17T03:28:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb707e00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb708143bf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800c760 tx=0x7fb6f800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8016020 con 0x7fb7080b4de0 2024-09-17T03:28:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6f800a950 con 0x7fb7080b4de0 2024-09-17T03:28:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f800ada0 con 0x7fb7080b4de0 2024-09-17T03:28:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70815ac00 con 0x7fb7080b4de0 2024-09-17T03:28:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.950+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70815b1c0 con 0x7fb7080b4de0 2024-09-17T03:28:33.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.954+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7080b51f0 con 0x7fb7080b4de0 2024-09-17T03:28:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.954+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fb6f8019020 con 0x7fb7080b4de0 2024-09-17T03:28:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.954+0000 7fb704c00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6e4073ef0 0x7fb6e4076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.954+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4943+0+0 (secure 0 0 0) 0x7fb6f80929d0 con 0x7fb7080b4de0 2024-09-17T03:28:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.954+0000 7fb6ffe00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6e4073ef0 0x7fb6e4076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.954+0000 7fb6ffe00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6e4073ef0 0x7fb6e4076400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb7080b4c30 tx=0x7fb6f000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:33.958+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb6f8020020 con 0x7fb7080b4de0 2024-09-17T03:28:34.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.166+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7080b4980 con 0x7fb7080b4de0 2024-09-17T03:28:34.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.166+0000 7fb704c00640 1 -- 172.21.15.46:0/3245343947 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3919 (secure 0 0 0) 0x7fb7080b5730 con 0x7fb7080b4de0 2024-09-17T03:28:34.170 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:34.170 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"btime":"2024-09-17T03:28:27:688015+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":245327,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/4271785499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4271785499},{"type":"v1","addr":"172.21.15.99:6834","nonce":4271785499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/4050631015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":4050631015},{"type":"v1","addr":"172.21.15.99:6835","nonce":4050631015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":245680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3110002312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3110002312},{"type":"v1","addr":"172.21.15.46:6837","nonce":3110002312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":246210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3924491368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3924491368},{"type":"v1","addr":"172.21.15.46:6835","nonce":3924491368}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-09-17T03:28:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 38 2024-09-17T03:28:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6e4073ef0 msgr2=0x7fb6e4076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6e4073ef0 0x7fb6e4076400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb7080b4c30 tx=0x7fb6f000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 msgr2=0x7fb708143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb708143bf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800c760 tx=0x7fb6f800cc20 comp rx=0 tx=0).stop 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 shutdown_connections 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6e4073ef0 0x7fb6e4076400 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7081527b0 0x7fb708158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7080b5c70 0x7fb7080aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 --2- 172.21.15.46:0/3245343947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7080b4de0 0x7fb708143bf0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 >> 172.21.15.46:0/3245343947 conn(0x7fb7080b2040 msgr2=0x7fb7080b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 shutdown_connections 2024-09-17T03:28:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:34.170+0000 7fb70ee00640 1 -- 172.21.15.46:0/3245343947 wait complete. 2024-09-17T03:28:34.193 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:28:34.193 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:28:34.194 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:28:34.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:28:34.194 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:28:34.194 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:28:34.194 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:28:34.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:28:34.194 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:28:34.195 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:28:34.195 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:28:34.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:28:34.195 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:28:34.195 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:28:34.195 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:28:34.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:28:34.206 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:28:34.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:28:34.213 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:28:34.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:28:34.214 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:28:34.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:28:34.257 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:28:34.258 DEBUG:teuthology.parallel:result is None 2024-09-17T03:28:34.260 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:28:34.260 DEBUG:teuthology.parallel:result is None 2024-09-17T03:28:34.264 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:28:34.264 DEBUG:teuthology.parallel:result is None 2024-09-17T03:28:34.358 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:28:34.359 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:28:34.363 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:28:35.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-17T03:28:40.213 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:28:40.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:28:40.217 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:28:40.217 DEBUG:teuthology.parallel:result is None 2024-09-17T03:28:40.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:28:40.319 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 -- 172.21.15.46:0/3615605058 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c1527b0 msgr2=0x7fef0c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 --2- 172.21.15.46:0/3615605058 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c1527b0 0x7fef0c074e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feef4009840 tx=0x7feef401c760 comp rx=0 tx=0).stop 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 -- 172.21.15.46:0/3615605058 shutdown_connections 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 --2- 172.21.15.46:0/3615605058 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c1527b0 0x7fef0c074e30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 --2- 172.21.15.46:0/3615605058 >> v2:172.21.15.99:3300/0 conn(0x7fef0c0b5c70 0x7fef0c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 -- 172.21.15.46:0/3615605058 >> 172.21.15.46:0/3615605058 conn(0x7fef0c0b2040 msgr2=0x7fef0c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 -- 172.21.15.46:0/3615605058 shutdown_connections 2024-09-17T03:28:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.390+0000 7fef11000640 1 -- 172.21.15.46:0/3615605058 wait complete. 2024-09-17T03:28:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 Processor -- start 2024-09-17T03:28:40.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 -- start start 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c0b4de0 0x7fef0c154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 0x7fef0c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1527b0 0x7fef0c149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c168c50 con 0x7fef0c0b5c70 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c168ad0 con 0x7fef0c1527b0 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c168dd0 con 0x7fef0c0b4de0 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 0x7fef0c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 0x7fef0c0aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35804/0 (socket says 172.21.15.46:35804) 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 -- 172.21.15.46:0/3678248507 learned_addr learned my addr 172.21.15.46:0/3678248507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef09e00640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c0b4de0 0x7fef0c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef0a800640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1527b0 0x7fef0c149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 -- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c0b4de0 msgr2=0x7fef0c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c0b4de0 0x7fef0c154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 -- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1527b0 msgr2=0x7fef0c149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1527b0 0x7fef0c149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 -- 172.21.15.46:0/3678248507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c1f9110 con 0x7fef0c0b5c70 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef03e00640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 0x7fef0c0aeb70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feef401ae10 tx=0x7feef40a5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40bb070 con 0x7fef0c0b5c70 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feef4004470 con 0x7fef0c0b5c70 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40b2040 con 0x7fef0c0b5c70 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c1f93f0 con 0x7fef0c0b5c70 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.394+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c1f99b0 con 0x7fef0c0b5c70 2024-09-17T03:28:40.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.398+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c15bb60 con 0x7fef0c0b5c70 2024-09-17T03:28:40.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.398+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7feef4005290 con 0x7fef0c0b5c70 2024-09-17T03:28:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.398+0000 7fef02000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee8073f20 0x7feee8076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.398+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4943+0+0 (secure 0 0 0) 0x7feef412c570 con 0x7fef0c0b5c70 2024-09-17T03:28:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.398+0000 7fef09e00640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee8073f20 0x7feee8076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.402+0000 7fef09e00640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee8073f20 0x7feee8076430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feefc0096c0 tx=0x7feefc009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:40.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.406+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fef0c075580 con 0x7fef0c0b5c70 2024-09-17T03:28:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:40.590+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fef0c1660e0 con 0x7fef0c0b5c70 2024-09-17T03:28:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.750+0000 7fef02000640 1 -- 172.21.15.46:0/3678248507 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) ==== 97+0+0 (secure 0 0 0) 0x7feef4016410 con 0x7fef0c0b5c70 2024-09-17T03:28:41.752 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee8073f20 msgr2=0x7feee8076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee8073f20 0x7feee8076430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feefc0096c0 tx=0x7feefc009260 comp rx=0 tx=0).stop 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 msgr2=0x7fef0c0aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 0x7fef0c0aeb70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feef401ae10 tx=0x7feef40a5640 comp rx=0 tx=0).stop 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 shutdown_connections 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee8073f20 0x7feee8076430 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1527b0 0x7fef0c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef0c0b5c70 0x7fef0c0aeb70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 --2- 172.21.15.46:0/3678248507 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef0c0b4de0 0x7fef0c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 >> 172.21.15.46:0/3678248507 conn(0x7fef0c0b2040 msgr2=0x7fef0c159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 shutdown_connections 2024-09-17T03:28:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.754+0000 7fef11000640 1 -- 172.21.15.46:0/3678248507 wait complete. 2024-09-17T03:28:41.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 -- 172.21.15.46:0/1121254543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c152010 msgr2=0x7f221c15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 --2- 172.21.15.46:0/1121254543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c152010 0x7f221c15bd40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f221c0cf800 tx=0x7f220c01c8a0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 -- 172.21.15.46:0/1121254543 shutdown_connections 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 --2- 172.21.15.46:0/1121254543 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221c0d0ed0 0x7f221c0ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 --2- 172.21.15.46:0/1121254543 >> v2:172.21.15.99:3300/0 conn(0x7f221c0d0ac0 0x7f221c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 --2- 172.21.15.46:0/1121254543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c152010 0x7f221c15bd40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 -- 172.21.15.46:0/1121254543 >> 172.21.15.46:0/1121254543 conn(0x7f221c06d1f0 msgr2=0x7f221c06d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 -- 172.21.15.46:0/1121254543 shutdown_connections 2024-09-17T03:28:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.942+0000 7f221be00640 1 -- 172.21.15.46:0/1121254543 wait complete. 2024-09-17T03:28:41.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 Processor -- start 2024-09-17T03:28:41.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 -- start start 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221c0d0ac0 0x7f221c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 0x7f221c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221c152010 0x7f221c1fc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f221c0ce660 con 0x7f221c0d0ed0 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f221c0ce4e0 con 0x7f221c0d0ac0 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221be00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f221c0ce7e0 con 0x7f221c152010 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 0x7f221c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 0x7f221c0aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35820/0 (socket says 172.21.15.46:35820) 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 -- 172.21.15.46:0/1439042794 learned_addr learned my addr 172.21.15.46:0/1439042794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221b400640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221c152010 0x7f221c1fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f221aa00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221c0d0ac0 0x7f221c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 -- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221c152010 msgr2=0x7f221c1fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221c152010 0x7f221c1fc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 -- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221c0d0ac0 msgr2=0x7f221c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221c0d0ac0 0x7f221c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.946+0000 7f2213e00640 1 -- 172.21.15.46:0/1439042794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c14cc80 con 0x7f221c0d0ed0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f2213e00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 0x7f221c0aee20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f220400d6d0 tx=0x7f220400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204017070 con 0x7f221c0d0ed0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2204004310 con 0x7f221c0d0ed0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204005280 con 0x7f221c0d0ed0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c13e500 con 0x7f221c0d0ed0 2024-09-17T03:28:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c13edf0 con 0x7f221c0d0ed0 2024-09-17T03:28:41.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221c13e270 con 0x7f221c0d0ed0 2024-09-17T03:28:41.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f2204005420 con 0x7f221c0d0ed0 2024-09-17T03:28:41.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.950+0000 7f2212a00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f8073ef0 0x7f21f8076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:41.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.954+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7f22040930c0 con 0x7f221c0d0ed0 2024-09-17T03:28:41.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.954+0000 7f221aa00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f8073ef0 0x7f21f8076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:41.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.954+0000 7f221aa00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f8073ef0 0x7f21f8076400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f220c0af000 tx=0x7f220c01b0e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:41.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:41.958+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2204020020 con 0x7f221c0d0ed0 2024-09-17T03:28:42.007 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-17T03:28:42.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.190+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f221c0ccdf0 con 0x7f221c0d0ed0 2024-09-17T03:28:42.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.190+0000 7f2212a00640 1 -- 172.21.15.46:0/1439042794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) ==== 80+0+5304 (secure 0 0 0) 0x7f220405fb30 con 0x7f221c0d0ed0 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:28:42.196 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:28:42.197 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:28:42.198 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:28:42.200 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.202 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.203 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.204 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:28:42.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:28:42.208 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:28:42.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:28:42.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.208 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:28:42.208 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f8073ef0 msgr2=0x7f21f8076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f8073ef0 0x7f21f8076400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f220c0af000 tx=0x7f220c01b0e0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 msgr2=0x7f221c0aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 0x7f221c0aee20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f220400d6d0 tx=0x7f220400db90 comp rx=0 tx=0).stop 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 shutdown_connections 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f21f8073ef0 0x7f21f8076400 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f221c152010 0x7f221c1fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0d0ed0 0x7f221c0aee20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 --2- 172.21.15.46:0/1439042794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f221c0d0ac0 0x7f221c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 >> 172.21.15.46:0/1439042794 conn(0x7f221c06d1f0 msgr2=0x7f221c075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 shutdown_connections 2024-09-17T03:28:42.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.194+0000 7f221be00640 1 -- 172.21.15.46:0/1439042794 wait complete. 2024-09-17T03:28:42.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 -- 172.21.15.46:0/1564654561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400004810 msgr2=0x7f6400004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 --2- 172.21.15.46:0/1564654561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400004810 0x7f6400004c20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f640800ac20 tx=0x7f640801e070 comp rx=0 tx=0).stop 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f641be00640 1 -- 172.21.15.46:0/1564654561 reap_dead start 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 -- 172.21.15.46:0/1564654561 shutdown_connections 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 -- 172.21.15.46:0/1564654561 >> 172.21.15.46:0/1564654561 conn(0x7f641c06d1f0 msgr2=0x7f641c06d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 -- 172.21.15.46:0/1564654561 shutdown_connections 2024-09-17T03:28:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 -- 172.21.15.46:0/1564654561 wait complete. 2024-09-17T03:28:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 Processor -- start 2024-09-17T03:28:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.378+0000 7f6421200640 1 -- start start 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6400004810 0x7f641c0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f641c0b1d90 0x7f641c1f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 0x7f641c1f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f641c15ea60 con 0x7f641c1f7dd0 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f641c06a690 con 0x7f6400004810 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f641c15ebe0 con 0x7f641c0b1d90 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 0x7f641c1f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6400004810 0x7f641c0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 0x7f641c1f81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53206/0 (socket says 172.21.15.46:53206) 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 -- 172.21.15.46:0/2527851604 learned_addr learned my addr 172.21.15.46:0/2527851604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641aa00640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f641c0b1d90 0x7f641c1f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 -- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f641c0b1d90 msgr2=0x7f641c1f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f641c0b1d90 0x7f641c1f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 -- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6400004810 msgr2=0x7f641c0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6400004810 0x7f641c0b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 -- 172.21.15.46:0/2527851604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c1f8720 con 0x7f641c1f7dd0 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641b400640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6400004810 0x7f641c0b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641aa00640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f641c0b1d90 0x7f641c1f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f641be00640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 0x7f641c1f81e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6414000f90 tx=0x7f6414006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641400fc30 con 0x7f641c1f7dd0 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f641400fdd0 con 0x7f641c1f7dd0 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414011690 con 0x7f641c1f7dd0 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c0b25c0 con 0x7f641c1f7dd0 2024-09-17T03:28:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c0b2b80 con 0x7f641c1f7dd0 2024-09-17T03:28:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.382+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c0b2330 con 0x7f641c1f7dd0 2024-09-17T03:28:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.386+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f6414011830 con 0x7f641c1f7dd0 2024-09-17T03:28:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.386+0000 7f6413400640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63fc073ef0 0x7f63fc076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.386+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7f64140935b0 con 0x7f641c1f7dd0 2024-09-17T03:28:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.386+0000 7f641b400640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63fc073ef0 0x7f63fc076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.386+0000 7f641b400640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63fc073ef0 0x7f63fc076400 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f640c006930 tx=0x7f640c006b10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:42.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.390+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f641c0cf4f0 con 0x7f641c1f7dd0 2024-09-17T03:28:42.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f641c0ccf60 con 0x7f641c1f7dd0 2024-09-17T03:28:42.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6413400640 1 -- 172.21.15.46:0/2527851604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7f64140600b0 con 0x7f641c1f7dd0 2024-09-17T03:28:42.585 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:42.585 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:28:42.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:42.586 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:28:42.586 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:28:42.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:42.586 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:28:42.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63fc073ef0 msgr2=0x7f63fc076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63fc073ef0 0x7f63fc076400 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f640c006930 tx=0x7f640c006b10 comp rx=0 tx=0).stop 2024-09-17T03:28:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 msgr2=0x7f641c1f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 0x7f641c1f81e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6414000f90 tx=0x7f6414006720 comp rx=0 tx=0).stop 2024-09-17T03:28:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 shutdown_connections 2024-09-17T03:28:42.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63fc073ef0 0x7f63fc076400 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f641c1f7dd0 0x7f641c1f81e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f641c0b1d90 0x7f641c1f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 --2- 172.21.15.46:0/2527851604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6400004810 0x7f641c0b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 >> 172.21.15.46:0/2527851604 conn(0x7f641c06d1f0 msgr2=0x7f641c15e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 shutdown_connections 2024-09-17T03:28:42.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.582+0000 7f6421200640 1 -- 172.21.15.46:0/2527851604 wait complete. 2024-09-17T03:28:42.608 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:28:42.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:28:42.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 -- 172.21.15.46:0/2529882332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c159130 msgr2=0x7f444c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 --2- 172.21.15.46:0/2529882332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c159130 0x7f444c15bb90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4440009900 tx=0x7f444001c850 comp rx=0 tx=0).stop 2024-09-17T03:28:42.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 -- 172.21.15.46:0/2529882332 shutdown_connections 2024-09-17T03:28:42.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 --2- 172.21.15.46:0/2529882332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 0x7f444c06d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 --2- 172.21.15.46:0/2529882332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c159130 0x7f444c15bb90 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 -- 172.21.15.46:0/2529882332 >> 172.21.15.46:0/2529882332 conn(0x7f444c0b35f0 msgr2=0x7f444c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:42.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 -- 172.21.15.46:0/2529882332 shutdown_connections 2024-09-17T03:28:42.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.770+0000 7f4453e00640 1 -- 172.21.15.46:0/2529882332 wait complete. 2024-09-17T03:28:42.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 Processor -- start 2024-09-17T03:28:42.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 -- start start 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 0x7f444c0cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 0x7f444c1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444c159130 0x7f444c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f444c168df0 con 0x7f444c06cdf0 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f444c168c70 con 0x7f444c159130 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4453e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f444c168f70 con 0x7f444c0703a0 2024-09-17T03:28:42.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4451400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 0x7f444c0cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4451400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 0x7f444c0cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53212/0 (socket says 172.21.15.46:53212) 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.774+0000 7f4451400640 1 -- 172.21.15.46:0/2833746527 learned_addr learned my addr 172.21.15.46:0/2833746527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444c159130 0x7f444c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451400640 1 -- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 msgr2=0x7f444c1f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4450a00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 0x7f444c1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451400640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 0x7f444c1f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451400640 1 -- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444c159130 msgr2=0x7f444c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451400640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444c159130 0x7f444c15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451400640 1 -- 172.21.15.46:0/2833746527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c0b20f0 con 0x7f444c06cdf0 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444c159130 0x7f444c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4450a00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 0x7f444c1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4451400640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 0x7f444c0cfcf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4440009850 tx=0x7f4440005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:42.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44400b2040 con 0x7f444c06cdf0 2024-09-17T03:28:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4440004240 con 0x7f444c06cdf0 2024-09-17T03:28:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440005030 con 0x7f444c06cdf0 2024-09-17T03:28:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c0b23d0 con 0x7f444c06cdf0 2024-09-17T03:28:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c15e050 con 0x7f444c06cdf0 2024-09-17T03:28:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c0b28f0 con 0x7f444c06cdf0 2024-09-17T03:28:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.778+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f44400051d0 con 0x7f444c06cdf0 2024-09-17T03:28:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.782+0000 7f4449600640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c073ef0 0x7f442c076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.782+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7f444012c600 con 0x7f444c06cdf0 2024-09-17T03:28:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.782+0000 7f4450a00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c073ef0 0x7f442c076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.782+0000 7f4450a00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c073ef0 0x7f442c076400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f444c0d08e0 tx=0x7f4438009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.786+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f444c0b28f0 con 0x7f444c06cdf0 2024-09-17T03:28:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:42.958+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f444c06d200 con 0x7f444c06cdf0 2024-09-17T03:28:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.758+0000 7f4449600640 1 -- 172.21.15.46:0/2833746527 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7f44400164d0 con 0x7f444c06cdf0 2024-09-17T03:28:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c073ef0 msgr2=0x7f442c076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c073ef0 0x7f442c076400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f444c0d08e0 tx=0x7f4438009040 comp rx=0 tx=0).stop 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 msgr2=0x7f444c0cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 0x7f444c0cfcf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4440009850 tx=0x7f4440005de0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 shutdown_connections 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c073ef0 0x7f442c076400 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444c159130 0x7f444c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444c0703a0 0x7f444c1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 --2- 172.21.15.46:0/2833746527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c06cdf0 0x7f444c0cfcf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 >> 172.21.15.46:0/2833746527 conn(0x7f444c0b35f0 msgr2=0x7f444c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 shutdown_connections 2024-09-17T03:28:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.762+0000 7f4453e00640 1 -- 172.21.15.46:0/2833746527 wait complete. 2024-09-17T03:28:43.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:28:43.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 -- 172.21.15.46:0/2474764404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4000b54f0 msgr2=0x7ff40015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:43.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 --2- 172.21.15.46:0/2474764404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4000b54f0 0x7ff40015bb60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009bc0 tx=0x7ff3f801cae0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 -- 172.21.15.46:0/2474764404 shutdown_connections 2024-09-17T03:28:43.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 --2- 172.21.15.46:0/2474764404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4000b54f0 0x7ff40015bb60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 -- 172.21.15.46:0/2474764404 >> 172.21.15.46:0/2474764404 conn(0x7ff4000b2040 msgr2=0x7ff4000b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:43.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 -- 172.21.15.46:0/2474764404 shutdown_connections 2024-09-17T03:28:43.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 -- 172.21.15.46:0/2474764404 wait complete. 2024-09-17T03:28:43.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.962+0000 7ff407200640 1 Processor -- start 2024-09-17T03:28:43.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 -- start start 2024-09-17T03:28:43.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4000b5c70 0x7ff4000ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:43.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 0x7ff40013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff40013f030 0x7ff40013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff400169b30 con 0x7ff4001527b0 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001699b0 con 0x7ff40013f030 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff400169cb0 con 0x7ff4000b5c70 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 0x7ff40013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 0x7ff40013eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53224/0 (socket says 172.21.15.46:53224) 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 -- 172.21.15.46:0/2711723951 learned_addr learned my addr 172.21.15.46:0/2711723951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3ffe00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4000b5c70 0x7ff4000ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff405200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff40013f030 0x7ff40013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 -- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4000b5c70 msgr2=0x7ff4000ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:43.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4000b5c70 0x7ff4000ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 -- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff40013f030 msgr2=0x7ff40013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff40013f030 0x7ff40013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 -- 172.21.15.46:0/2711723951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400076500 con 0x7ff4001527b0 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3f7e00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 0x7ff40013eaf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800b440 tx=0x7ff3e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8015020 con 0x7ff4001527b0 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3e8011920 con 0x7ff4001527b0 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8011d70 con 0x7ff4001527b0 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001511c0 con 0x7ff4001527b0 2024-09-17T03:28:43.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.966+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff400151780 con 0x7ff4001527b0 2024-09-17T03:28:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.970+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff400150f30 con 0x7ff4001527b0 2024-09-17T03:28:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.970+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7ff3e8018020 con 0x7ff4001527b0 2024-09-17T03:28:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.970+0000 7ff3fcc00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3dc073ef0 0x7ff3dc076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.970+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5227+0+0 (secure 0 0 0) 0x7ff3e8092bf0 con 0x7ff4001527b0 2024-09-17T03:28:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.970+0000 7ff3ffe00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3dc073ef0 0x7ff3dc076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.970+0000 7ff3ffe00640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3dc073ef0 0x7ff3dc076400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0001870 tx=0x7ff3f0001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:43.974+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff3e801f030 con 0x7ff4001527b0 2024-09-17T03:28:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.150+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff40015bb60 con 0x7ff4001527b0 2024-09-17T03:28:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.770+0000 7ff3fcc00640 1 -- 172.21.15.46:0/2711723951 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) ==== 170+0+0 (secure 0 0 0) 0x7ff3e805f0a0 con 0x7ff4001527b0 2024-09-17T03:28:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:28:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3dc073ef0 msgr2=0x7ff3dc076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3dc073ef0 0x7ff3dc076400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0001870 tx=0x7ff3f0001380 comp rx=0 tx=0).stop 2024-09-17T03:28:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 msgr2=0x7ff40013eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 0x7ff40013eaf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800b440 tx=0x7ff3e800b900 comp rx=0 tx=0).stop 2024-09-17T03:28:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 shutdown_connections 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3dc073ef0 0x7ff3dc076400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff40013f030 0x7ff40013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4001527b0 0x7ff40013eaf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 --2- 172.21.15.46:0/2711723951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff4000b5c70 0x7ff4000ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 >> 172.21.15.46:0/2711723951 conn(0x7ff4000b2040 msgr2=0x7ff4000cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 shutdown_connections 2024-09-17T03:28:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.774+0000 7ff407200640 1 -- 172.21.15.46:0/2711723951 wait complete. 2024-09-17T03:28:44.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:28:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 -- 172.21.15.46:0/1383933289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 msgr2=0x7f6d98074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/1383933289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d98074240 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009840 tx=0x7f6d8001c4d0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 -- 172.21.15.46:0/1383933289 shutdown_connections 2024-09-17T03:28:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/1383933289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d98074240 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 -- 172.21.15.46:0/1383933289 >> 172.21.15.46:0/1383933289 conn(0x7f6d980b4560 msgr2=0x7f6d980d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 -- 172.21.15.46:0/1383933289 shutdown_connections 2024-09-17T03:28:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.986+0000 7f6d9cc00640 1 -- 172.21.15.46:0/1383933289 wait complete. 2024-09-17T03:28:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 Processor -- start 2024-09-17T03:28:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 -- start start 2024-09-17T03:28:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d981536f0 0x7f6d98143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d9806df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:44.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 0x7f6d9806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9816b4e0 con 0x7f6d98160470 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9816b360 con 0x7f6d98160060 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d9cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9816b660 con 0x7f6d981536f0 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 0x7f6d9806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 0x7f6d9806e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53232/0 (socket says 172.21.15.46:53232) 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 -- 172.21.15.46:0/4194573357 learned_addr learned my addr 172.21.15.46:0/4194573357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97400640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d981536f0 0x7f6d98143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d8fe00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d9806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 -- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d981536f0 msgr2=0x7f6d98143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d981536f0 0x7f6d98143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 -- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 msgr2=0x7f6d9806df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d9806df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 -- 172.21.15.46:0/4194573357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d980ab980 con 0x7f6d98160470 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d8fe00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d9806df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97400640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d981536f0 0x7f6d98143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.990+0000 7f6d97e00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 0x7f6d9806e470 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c730 tx=0x7f6d9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90003b80 con 0x7f6d98160470 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d981441c0 con 0x7f6d98160470 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d90003d20 con 0x7f6d98160470 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000aab0 con 0x7f6d98160470 2024-09-17T03:28:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d980ab6a0 con 0x7f6d98160470 2024-09-17T03:28:44.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d98074240 con 0x7f6d98160470 2024-09-17T03:28:44.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f6d9000ac50 con 0x7f6d98160470 2024-09-17T03:28:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.994+0000 7f6d8f400640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d74073f20 0x7f6d74076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.998+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5609+0+0 (secure 0 0 0) 0x7f6d90093530 con 0x7f6d98160470 2024-09-17T03:28:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.998+0000 7f6d97400640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d74073f20 0x7f6d74076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:45.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.998+0000 7f6d97400640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d74073f20 0x7f6d74076430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c10 tx=0x7f6d88009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:44.998+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d980ab980 con 0x7f6d98160470 2024-09-17T03:28:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.210+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6d980b5400 con 0x7f6d98160470 2024-09-17T03:28:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.782+0000 7f6d8f400640 1 -- 172.21.15.46:0/4194573357 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7f6d9005ffe0 con 0x7f6d98160470 2024-09-17T03:28:45.792 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 10 and data pool 11 2024-09-17T03:28:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.794+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d74073f20 msgr2=0x7f6d74076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.794+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d74073f20 0x7f6d74076430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c10 tx=0x7f6d88009040 comp rx=0 tx=0).stop 2024-09-17T03:28:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.794+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 msgr2=0x7f6d9806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.794+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 0x7f6d9806e470 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c730 tx=0x7f6d9000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:28:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.798+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 shutdown_connections 2024-09-17T03:28:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.798+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d74073f20 0x7f6d74076430 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.798+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d98160470 0x7f6d9806e470 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.798+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d98160060 0x7f6d9806df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.798+0000 7f6d9cc00640 1 --2- 172.21.15.46:0/4194573357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d981536f0 0x7f6d98143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.798+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 >> 172.21.15.46:0/4194573357 conn(0x7f6d980b4560 msgr2=0x7f6d9806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.802+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 shutdown_connections 2024-09-17T03:28:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:45.802+0000 7f6d9cc00640 1 -- 172.21.15.46:0/4194573357 wait complete. 2024-09-17T03:28:45.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 -- 172.21.15.46:0/2879731215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b5c70 msgr2=0x7f0b7415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 --2- 172.21.15.46:0/2879731215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b5c70 0x7f0b7415bb60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0b74072e30 tx=0x7f0b6401c800 comp rx=0 tx=0).stop 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 -- 172.21.15.46:0/2879731215 shutdown_connections 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 --2- 172.21.15.46:0/2879731215 >> v2:172.21.15.99:3300/0 conn(0x7f0b740b4de0 0x7f0b740b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 --2- 172.21.15.46:0/2879731215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b741527b0 0x7f0b74074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 --2- 172.21.15.46:0/2879731215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b5c70 0x7f0b7415bb60 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 -- 172.21.15.46:0/2879731215 >> 172.21.15.46:0/2879731215 conn(0x7f0b740b2040 msgr2=0x7f0b740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 -- 172.21.15.46:0/2879731215 shutdown_connections 2024-09-17T03:28:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 -- 172.21.15.46:0/2879731215 wait complete. 2024-09-17T03:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 Processor -- start 2024-09-17T03:28:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 -- start start 2024-09-17T03:28:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 0x7f0b740aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.002+0000 7f0b7a800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b740b5c70 0x7f0b74151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b741527b0 0x7f0b74158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7415f700 con 0x7f0b740b4de0 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7415f580 con 0x7f0b741527b0 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7415f880 con 0x7f0b740b5c70 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 0x7f0b740aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 0x7f0b740aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53252/0 (socket says 172.21.15.46:53252) 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 -- 172.21.15.46:0/3125628837 learned_addr learned my addr 172.21.15.46:0/3125628837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b6be00640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b740b5c70 0x7f0b74151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73e00640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b741527b0 0x7f0b74158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 -- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b740b5c70 msgr2=0x7f0b74151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b740b5c70 0x7f0b74151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 -- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b741527b0 msgr2=0x7f0b74158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b741527b0 0x7f0b74158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 -- 172.21.15.46:0/3125628837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b741fa880 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b73400640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 0x7f0b740aee20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0b640b0040 tx=0x7f0b640a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640b2040 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b64004340 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64005290 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b740cef00 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b740aa600 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b740cecf0 con 0x7f0b740b4de0 2024-09-17T03:28:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.006+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f0b64005430 con 0x7f0b740b4de0 2024-09-17T03:28:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.010+0000 7f0b6b400640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b50073ef0 0x7f0b50076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.010+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f0b6412c740 con 0x7f0b740b4de0 2024-09-17T03:28:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.010+0000 7f0b6be00640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b50073ef0 0x7f0b50076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.010+0000 7f0b6be00640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b50073ef0 0x7f0b50076400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0b7406ed70 tx=0x7f0b5c00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.014+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0b640164b0 con 0x7f0b740b4de0 2024-09-17T03:28:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.186+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b7415fdc0 con 0x7f0b740b4de0 2024-09-17T03:28:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b6b400640 1 -- 172.21.15.46:0/3125628837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16705 (secure 0 0 0) 0x7f0b640f8bf0 con 0x7f0b740b4de0 2024-09-17T03:28:46.192 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:46.192 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":71,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:28:45.214783+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:28:42.973730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T03:28:44.166208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:28:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b50073ef0 msgr2=0x7f0b50076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b50073ef0 0x7f0b50076400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0b7406ed70 tx=0x7f0b5c00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 msgr2=0x7f0b740aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 0x7f0b740aee20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0b640b0040 tx=0x7f0b640a5790 comp rx=0 tx=0).stop 2024-09-17T03:28:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 shutdown_connections 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b50073ef0 0x7f0b50076400 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b741527b0 0x7f0b74158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b740b5c70 0x7f0b74151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 --2- 172.21.15.46:0/3125628837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b740b4de0 0x7f0b740aee20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 >> 172.21.15.46:0/3125628837 conn(0x7f0b740b2040 msgr2=0x7f0b740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 shutdown_connections 2024-09-17T03:28:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.190+0000 7f0b7a800640 1 -- 172.21.15.46:0/3125628837 wait complete. 2024-09-17T03:28:46.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:28:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 -- 172.21.15.46:0/3912755713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800b4ae0 msgr2=0x7f2780073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 --2- 172.21.15.46:0/3912755713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800b4ae0 0x7f2780073480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2780075060 tx=0x7f277401c890 comp rx=0 tx=0).stop 2024-09-17T03:28:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 -- 172.21.15.46:0/3912755713 shutdown_connections 2024-09-17T03:28:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 --2- 172.21.15.46:0/3912755713 >> v2:172.21.15.99:3300/0 conn(0x7f27800afbd0 0x7f2780162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 --2- 172.21.15.46:0/3912755713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800b4ae0 0x7f2780073480 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 -- 172.21.15.46:0/3912755713 >> 172.21.15.46:0/3912755713 conn(0x7f27800d0cf0 msgr2=0x7f27800d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 -- 172.21.15.46:0/3912755713 shutdown_connections 2024-09-17T03:28:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.370+0000 7f2787a00640 1 -- 172.21.15.46:0/3912755713 wait complete. 2024-09-17T03:28:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 Processor -- start 2024-09-17T03:28:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 -- start start 2024-09-17T03:28:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 0x7f27801455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27800afbd0 0x7f27800b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27800b4ae0 0x7f27800b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f278016c290 con 0x7f27800739c0 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f278016c110 con 0x7f27800afbd0 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2787a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f278016c410 con 0x7f27800b4ae0 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 0x7f27801455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 0x7f27801455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53266/0 (socket says 172.21.15.46:53266) 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 -- 172.21.15.46:0/3609085324 learned_addr learned my addr 172.21.15.46:0/3609085324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27800b4ae0 0x7f27800b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f277fe00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27800afbd0 0x7f27800b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 -- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27800b4ae0 msgr2=0x7f27800b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27800b4ae0 0x7f27800b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 -- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27800afbd0 msgr2=0x7f27800b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27800afbd0 0x7f27800b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 -- 172.21.15.46:0/3609085324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2780162050 con 0x7f27800739c0 2024-09-17T03:28:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27800b4ae0 0x7f27800b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f277fe00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27800afbd0 0x7f27800b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.374+0000 7f2785000640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 0x7f27801455f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f27740194a0 tx=0x7f2774004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740bc070 con 0x7f27800739c0 2024-09-17T03:28:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2774005320 con 0x7f27800739c0 2024-09-17T03:28:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740b2040 con 0x7f27800739c0 2024-09-17T03:28:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2780145b30 con 0x7f27800739c0 2024-09-17T03:28:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27801fc540 con 0x7f27800739c0 2024-09-17T03:28:46.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780073480 con 0x7f27800739c0 2024-09-17T03:28:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f27740046d0 con 0x7f27800739c0 2024-09-17T03:28:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277cc00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2760073f20 0x7f2760076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f277412c9d0 con 0x7f27800739c0 2024-09-17T03:28:46.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.378+0000 7f277fe00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2760073f20 0x7f2760076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.382+0000 7f277fe00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2760073f20 0x7f2760076430 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2780162240 tx=0x7f276c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.382+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27740164b0 con 0x7f27800739c0 2024-09-17T03:28:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.558+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f278014f960 con 0x7f27800739c0 2024-09-17T03:28:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f277cc00640 1 -- 172.21.15.46:0/3609085324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16705 (secure 0 0 0) 0x7f27740f9450 con 0x7f27800739c0 2024-09-17T03:28:46.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:46.564 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":71,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:28:45.214783+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:28:42.973730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T03:28:44.166208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2760073f20 msgr2=0x7f2760076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2760073f20 0x7f2760076430 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2780162240 tx=0x7f276c009040 comp rx=0 tx=0).stop 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 msgr2=0x7f27801455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 0x7f27801455f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f27740194a0 tx=0x7f2774004600 comp rx=0 tx=0).stop 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 shutdown_connections 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2760073f20 0x7f2760076430 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27800b4ae0 0x7f27800b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27800afbd0 0x7f27800b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 --2- 172.21.15.46:0/3609085324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27800739c0 0x7f27801455f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 >> 172.21.15.46:0/3609085324 conn(0x7f27800d0cf0 msgr2=0x7f2780070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 shutdown_connections 2024-09-17T03:28:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.562+0000 7f2787a00640 1 -- 172.21.15.46:0/3609085324 wait complete. 2024-09-17T03:28:46.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 -- 172.21.15.46:0/197575382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f0073f90 msgr2=0x7f91f00743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 --2- 172.21.15.46:0/197575382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f0073f90 0x7f91f00743a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009930 tx=0x7f91d801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 -- 172.21.15.46:0/197575382 shutdown_connections 2024-09-17T03:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 --2- 172.21.15.46:0/197575382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f0073f90 0x7f91f00743a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 --2- 172.21.15.46:0/197575382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f91f015c730 0x7f91f00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 -- 172.21.15.46:0/197575382 >> 172.21.15.46:0/197575382 conn(0x7f91f00d06e0 msgr2=0x7f91f00d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 -- 172.21.15.46:0/197575382 shutdown_connections 2024-09-17T03:28:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 -- 172.21.15.46:0/197575382 wait complete. 2024-09-17T03:28:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 Processor -- start 2024-09-17T03:28:46.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.746+0000 7f91f7200640 1 -- start start 2024-09-17T03:28:46.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f91f0073f90 0x7f91f0153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f00af520 0x7f91f014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 0x7f91f00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91f006f740 con 0x7f91f015c730 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91f006f5c0 con 0x7f91f00af520 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f91f006f8c0 con 0x7f91f0073f90 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 0x7f91f00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 0x7f91f00ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53284/0 (socket says 172.21.15.46:53284) 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 -- 172.21.15.46:0/2901352599 learned_addr learned my addr 172.21.15.46:0/2901352599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91e7e00640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f00af520 0x7f91f014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91efe00640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f91f0073f90 0x7f91f0153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 -- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f91f0073f90 msgr2=0x7f91f0153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f91f0073f90 0x7f91f0153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 -- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f00af520 msgr2=0x7f91f014f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:46.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f00af520 0x7f91f014f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:46.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 -- 172.21.15.46:0/2901352599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f015ec90 con 0x7f91f015c730 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f5200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 0x7f91f00ae8e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c760 tx=0x7f91e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8003b80 con 0x7f91f015c730 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91e8003d20 con 0x7f91f015c730 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01fbff0 con 0x7f91f015c730 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e800aab0 con 0x7f91f015c730 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01fc5b0 con 0x7f91f015c730 2024-09-17T03:28:46.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.750+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f00743a0 con 0x7f91f015c730 2024-09-17T03:28:46.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.754+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f91e800ac50 con 0x7f91f015c730 2024-09-17T03:28:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.754+0000 7f91ecc00640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f91cc073ef0 0x7f91cc076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.754+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f91e8092830 con 0x7f91f015c730 2024-09-17T03:28:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.754+0000 7f91efe00640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f91cc073ef0 0x7f91cc076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:46.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.754+0000 7f91efe00640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f91cc073ef0 0x7f91cc076400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009b40 tx=0x7f91e00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.758+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f91e805ece0 con 0x7f91f015c730 2024-09-17T03:28:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:46.974+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f91f015ea20 con 0x7f91f015c730 2024-09-17T03:28:47.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.790+0000 7f91ecc00640 1 -- 172.21.15.46:0/2901352599 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f91e8063710 con 0x7f91f015c730 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f91cc073ef0 msgr2=0x7f91cc076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f91cc073ef0 0x7f91cc076400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009b40 tx=0x7f91e00091e0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 msgr2=0x7f91f00ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 0x7f91f00ae8e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c760 tx=0x7f91e800cc20 comp rx=0 tx=0).stop 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 shutdown_connections 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f91cc073ef0 0x7f91cc076400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f015c730 0x7f91f00ae8e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f00af520 0x7f91f014f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 --2- 172.21.15.46:0/2901352599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f91f0073f90 0x7f91f0153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 >> 172.21.15.46:0/2901352599 conn(0x7f91f00d06e0 msgr2=0x7f91f00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:47.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 shutdown_connections 2024-09-17T03:28:47.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.794+0000 7f91f7200640 1 -- 172.21.15.46:0/2901352599 wait complete. 2024-09-17T03:28:47.821 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:28:47.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:28:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 -- 172.21.15.46:0/1259607417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b5c70 msgr2=0x7f71f015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1259607417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b5c70 0x7f71f015bb60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f71f0070710 tx=0x7f71e001c890 comp rx=0 tx=0).stop 2024-09-17T03:28:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 -- 172.21.15.46:0/1259607417 shutdown_connections 2024-09-17T03:28:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1259607417 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f00b4de0 0x7f71f00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1259607417 >> v2:172.21.15.99:3300/0 conn(0x7f71f01527b0 0x7f71f0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1259607417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b5c70 0x7f71f015bb60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 -- 172.21.15.46:0/1259607417 >> 172.21.15.46:0/1259607417 conn(0x7f71f00b2040 msgr2=0x7f71f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 -- 172.21.15.46:0/1259607417 shutdown_connections 2024-09-17T03:28:47.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 -- 172.21.15.46:0/1259607417 wait complete. 2024-09-17T03:28:47.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 Processor -- start 2024-09-17T03:28:47.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.986+0000 7f71f5c00640 1 -- start start 2024-09-17T03:28:47.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 0x7f71f00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:47.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f00b5c70 0x7f71f0143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:47.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f01527b0 0x7f71f01f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71f015f6d0 con 0x7f71f00b4de0 2024-09-17T03:28:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71f015f550 con 0x7f71f00b5c70 2024-09-17T03:28:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f71f015f850 con 0x7f71f01527b0 2024-09-17T03:28:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 0x7f71f00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 0x7f71f00ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53290/0 (socket says 172.21.15.46:53290) 2024-09-17T03:28:47.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 -- 172.21.15.46:0/1396568595 learned_addr learned my addr 172.21.15.46:0/1396568595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71e7e00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f00b5c70 0x7f71f0143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71ef400640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f01527b0 0x7f71f01f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 -- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f01527b0 msgr2=0x7f71f01f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f01527b0 0x7f71f01f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 -- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f00b5c70 msgr2=0x7f71f0143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f00b5c70 0x7f71f0143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 -- 172.21.15.46:0/1396568595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f01514b0 con 0x7f71f00b4de0 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71ef400640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f01527b0 0x7f71f01f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71e7e00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f00b5c70 0x7f71f0143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:47.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71eea00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 0x7f71f00ccae0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f71e00b0040 tx=0x7f71e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e00b2040 con 0x7f71f00b4de0 2024-09-17T03:28:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e0005030 con 0x7f71f00b4de0 2024-09-17T03:28:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f01fc450 con 0x7f71f00b4de0 2024-09-17T03:28:47.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0005460 con 0x7f71f00b4de0 2024-09-17T03:28:47.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f01fca10 con 0x7f71f00b4de0 2024-09-17T03:28:47.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.990+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f015fdf0 con 0x7f71f00b4de0 2024-09-17T03:28:47.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.994+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f71e00046d0 con 0x7f71f00b4de0 2024-09-17T03:28:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.994+0000 7f71e6a00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f71cc073ef0 0x7f71cc076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.994+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f71e012c4c0 con 0x7f71f00b4de0 2024-09-17T03:28:47.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.994+0000 7f71e7e00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f71cc073ef0 0x7f71cc076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.994+0000 7f71e7e00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f71cc073ef0 0x7f71cc076400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d8005de0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:47.998+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f71e0132030 con 0x7f71f00b4de0 2024-09-17T03:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.210+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f71f00cc6d0 con 0x7f71f00b4de0 2024-09-17T03:28:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71e6a00640 1 -- 172.21.15.46:0/1396568595 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f71e00bc060 con 0x7f71f00b4de0 2024-09-17T03:28:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f71cc073ef0 msgr2=0x7f71cc076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f71cc073ef0 0x7f71cc076400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d8005de0 comp rx=0 tx=0).stop 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 msgr2=0x7f71f00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 0x7f71f00ccae0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f71e00b0040 tx=0x7f71e0004040 comp rx=0 tx=0).stop 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 shutdown_connections 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f71cc073ef0 0x7f71cc076400 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f71f01527b0 0x7f71f01f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71f00b5c70 0x7f71f0143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 --2- 172.21.15.46:0/1396568595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71f00b4de0 0x7f71f00ccae0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 >> 172.21.15.46:0/1396568595 conn(0x7f71f00b2040 msgr2=0x7f71f01516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 shutdown_connections 2024-09-17T03:28:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.794+0000 7f71f5c00640 1 -- 172.21.15.46:0/1396568595 wait complete. 2024-09-17T03:28:48.828 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:28:48.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d8800640 1 -- 172.21.15.46:0/1932764545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b8004810 msgr2=0x7f07b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d8800640 1 --2- 172.21.15.46:0/1932764545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b8004810 0x7f07b8004c20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f07d0071fd0 tx=0x7f07c001d1e0 comp rx=0 tx=0).stop 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d6800640 1 -- 172.21.15.46:0/1932764545 reap_dead start 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d8800640 1 -- 172.21.15.46:0/1932764545 shutdown_connections 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d8800640 1 -- 172.21.15.46:0/1932764545 >> 172.21.15.46:0/1932764545 conn(0x7f07d00b2040 msgr2=0x7f07d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d8800640 1 -- 172.21.15.46:0/1932764545 shutdown_connections 2024-09-17T03:28:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.990+0000 7f07d8800640 1 -- 172.21.15.46:0/1932764545 wait complete. 2024-09-17T03:28:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 Processor -- start 2024-09-17T03:28:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 -- start start 2024-09-17T03:28:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b8004810 0x7f07d00aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 0x7f07d0157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07d0154210 0x7f07d0154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0166900 con 0x7f07d0155640 2024-09-17T03:28:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07d0166780 con 0x7f07b8004810 2024-09-17T03:28:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d8800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f07d0166a80 con 0x7f07d0154210 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 0x7f07d0157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 0x7f07d0157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53310/0 (socket says 172.21.15.46:53310) 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d5400640 1 -- 172.21.15.46:0/2567979369 learned_addr learned my addr 172.21.15.46:0/2567979369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.994+0000 7f07d6800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07d0154210 0x7f07d0154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5e00640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b8004810 0x7f07d00aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5400640 1 -- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07d0154210 msgr2=0x7f07d0154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5400640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07d0154210 0x7f07d0154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5400640 1 -- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b8004810 msgr2=0x7f07d00aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5400640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b8004810 0x7f07d00aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5400640 1 -- 172.21.15.46:0/2567979369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d0154b60 con 0x7f07d0155640 2024-09-17T03:28:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d5400640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 0x7f07d0157a10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f07c00ae040 tx=0x7f07c0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00bb030 con 0x7f07d0155640 2024-09-17T03:28:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c0004880 con 0x7f07d0155640 2024-09-17T03:28:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d01f8b60 con 0x7f07d0155640 2024-09-17T03:28:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0004c90 con 0x7f07d0155640 2024-09-17T03:28:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d01f8e20 con 0x7f07d0155640 2024-09-17T03:28:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d00b51f0 con 0x7f07d0155640 2024-09-17T03:28:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:48.998+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f07c00be030 con 0x7f07d0155640 2024-09-17T03:28:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.002+0000 7f07ca000640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07b4073ef0 0x7f07b4076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.002+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f07c012e190 con 0x7f07d0155640 2024-09-17T03:28:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.002+0000 7f07d5e00640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07b4073ef0 0x7f07b4076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.002+0000 7f07d5e00640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07b4073ef0 0x7f07b4076400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07c4000fc0 tx=0x7f07c4001120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.006+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07d0154b60 con 0x7f07d0155640 2024-09-17T03:28:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07d00b0430 con 0x7f07d0155640 2024-09-17T03:28:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07ca000640 1 -- 172.21.15.46:0/2567979369 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5241 (secure 0 0 0) 0x7f07c0016f20 con 0x7f07d0155640 2024-09-17T03:28:49.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:49.216 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"btime":"2024-09-17T03:28:48:795549+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:28:45.214743+0000","modified":"2024-09-17T03:28:48.217828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246303":{"gid":246303,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6836/3671761014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3671761014},{"type":"v1","addr":"172.21.15.46:6836","nonce":3671761014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246303,"qdb_cluster":[246303]},"id":5}]} 2024-09-17T03:28:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-09-17T03:28:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07b4073ef0 msgr2=0x7f07b4076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07b4073ef0 0x7f07b4076400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f07c4000fc0 tx=0x7f07c4001120 comp rx=0 tx=0).stop 2024-09-17T03:28:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 msgr2=0x7f07d0157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 0x7f07d0157a10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f07c00ae040 tx=0x7f07c0005020 comp rx=0 tx=0).stop 2024-09-17T03:28:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 shutdown_connections 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07b4073ef0 0x7f07b4076400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07d0154210 0x7f07d0154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07d0155640 0x7f07d0157a10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 --2- 172.21.15.46:0/2567979369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b8004810 0x7f07d00aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 >> 172.21.15.46:0/2567979369 conn(0x7f07d00b2040 msgr2=0x7f07d015a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 shutdown_connections 2024-09-17T03:28:49.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.214+0000 7f07d8800640 1 -- 172.21.15.46:0/2567979369 wait complete. 2024-09-17T03:28:49.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:28:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 -- 172.21.15.46:0/3208802492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce50159130 msgr2=0x7fce5015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 --2- 172.21.15.46:0/3208802492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce50159130 0x7fce5015bb90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fce40009840 tx=0x7fce4001c700 comp rx=0 tx=0).stop 2024-09-17T03:28:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 -- 172.21.15.46:0/3208802492 shutdown_connections 2024-09-17T03:28:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 --2- 172.21.15.46:0/3208802492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce50159130 0x7fce5015bb90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 -- 172.21.15.46:0/3208802492 >> 172.21.15.46:0/3208802492 conn(0x7fce500b35f0 msgr2=0x7fce500b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 -- 172.21.15.46:0/3208802492 shutdown_connections 2024-09-17T03:28:49.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 -- 172.21.15.46:0/3208802492 wait complete. 2024-09-17T03:28:49.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 Processor -- start 2024-09-17T03:28:49.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.398+0000 7fce56c00640 1 -- start start 2024-09-17T03:28:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5006cdf0 0x7fce501555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 0x7fce5015e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce50159130 0x7fce5015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce50169cb0 con 0x7fce500703a0 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce50169b30 con 0x7fce5006cdf0 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fce50169e30 con 0x7fce50159130 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 0x7fce5015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 0x7fce5015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53336/0 (socket says 172.21.15.46:53336) 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 -- 172.21.15.46:0/2144455171 learned_addr learned my addr 172.21.15.46:0/2144455171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce4fe00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5006cdf0 0x7fce501555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce54c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce50159130 0x7fce5015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 -- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce50159130 msgr2=0x7fce5015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce50159130 0x7fce5015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 -- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5006cdf0 msgr2=0x7fce501555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5006cdf0 0x7fce501555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 -- 172.21.15.46:0/2144455171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce501fbe60 con 0x7fce500703a0 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce54c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce50159130 0x7fce5015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce47e00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 0x7fce5015e340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c3f0 tx=0x7fce3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38017070 con 0x7fce500703a0 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce38003600 con 0x7fce500703a0 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38003a50 con 0x7fce500703a0 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5015dd70 con 0x7fce500703a0 2024-09-17T03:28:49.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce501fc3b0 con 0x7fce500703a0 2024-09-17T03:28:49.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.402+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5015bb90 con 0x7fce500703a0 2024-09-17T03:28:49.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.406+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fce3801a070 con 0x7fce500703a0 2024-09-17T03:28:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.406+0000 7fce4cc00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce2c073ef0 0x7fce2c076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.406+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fce38091de0 con 0x7fce500703a0 2024-09-17T03:28:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.406+0000 7fce4fe00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce2c073ef0 0x7fce2c076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.406+0000 7fce4fe00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce2c073ef0 0x7fce2c076400 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fce400af000 tx=0x7fce400b2040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:49.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.410+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce5015c0d0 con 0x7fce500703a0 2024-09-17T03:28:49.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.586+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce500737c0 con 0x7fce500703a0 2024-09-17T03:28:49.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce4cc00640 1 -- 172.21.15.46:0/2144455171 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16692 (secure 0 0 0) 0x7fce3805e240 con 0x7fce500703a0 2024-09-17T03:28:49.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:49.592 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":72,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:28:46.778001+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:28:42.973730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T03:28:44.166208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:28:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce2c073ef0 msgr2=0x7fce2c076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce2c073ef0 0x7fce2c076400 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fce400af000 tx=0x7fce400b2040 comp rx=0 tx=0).stop 2024-09-17T03:28:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 msgr2=0x7fce5015e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 0x7fce5015e340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c3f0 tx=0x7fce3800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 shutdown_connections 2024-09-17T03:28:49.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce2c073ef0 0x7fce2c076400 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce50159130 0x7fce5015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce500703a0 0x7fce5015e340 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 --2- 172.21.15.46:0/2144455171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce5006cdf0 0x7fce501555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 >> 172.21.15.46:0/2144455171 conn(0x7fce500b35f0 msgr2=0x7fce50068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 shutdown_connections 2024-09-17T03:28:49.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.590+0000 7fce56c00640 1 -- 172.21.15.46:0/2144455171 wait complete. 2024-09-17T03:28:49.615 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:28:49.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:28:49.729 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-09-17T03:28:49.726+0000 7fad6d200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:28:49.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 -- 172.21.15.46:0/3047522438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018074b20 msgr2=0x7ff01815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 --2- 172.21.15.46:0/3047522438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018074b20 0x7ff01815cf90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff010009bf0 tx=0x7ff01001cb00 comp rx=0 tx=0).stop 2024-09-17T03:28:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 -- 172.21.15.46:0/3047522438 shutdown_connections 2024-09-17T03:28:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 --2- 172.21.15.46:0/3047522438 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018074b20 0x7ff01815cf90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 -- 172.21.15.46:0/3047522438 >> 172.21.15.46:0/3047522438 conn(0x7ff0180708c0 msgr2=0x7ff018070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 -- 172.21.15.46:0/3047522438 shutdown_connections 2024-09-17T03:28:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.770+0000 7ff020400640 1 -- 172.21.15.46:0/3047522438 wait complete. 2024-09-17T03:28:49.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 Processor -- start 2024-09-17T03:28:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 -- start start 2024-09-17T03:28:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0180b5a80 0x7ff01815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018157030 0x7ff01815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 0x7ff018150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff01816b590 con 0x7ff01815c000 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff01816b410 con 0x7ff0180b5a80 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff020400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff01816b710 con 0x7ff018157030 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 0x7ff018150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 0x7ff018150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53350/0 (socket says 172.21.15.46:53350) 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 -- 172.21.15.46:0/847756125 learned_addr learned my addr 172.21.15.46:0/847756125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01da00640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0180b5a80 0x7ff01815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01d000640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018157030 0x7ff01815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 -- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018157030 msgr2=0x7ff01815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018157030 0x7ff01815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 -- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0180b5a80 msgr2=0x7ff01815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0180b5a80 0x7ff01815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 -- 172.21.15.46:0/847756125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01815c410 con 0x7ff01815c000 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.774+0000 7ff01e400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 0x7ff018150ca0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff01001b1b0 tx=0x7ff010009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100b2040 con 0x7ff01815c000 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff010003d70 con 0x7ff01815c000 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010004cb0 con 0x7ff01815c000 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff018139e70 con 0x7ff01815c000 2024-09-17T03:28:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01813a430 con 0x7ff01815c000 2024-09-17T03:28:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0180d2260 con 0x7ff01815c000 2024-09-17T03:28:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7ff0100be060 con 0x7ff01815c000 2024-09-17T03:28:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff015600640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feff8073f20 0x7feff8076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7ff01012c640 con 0x7ff01815c000 2024-09-17T03:28:49.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.778+0000 7ff01da00640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feff8073f20 0x7feff8076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:49.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.782+0000 7ff01da00640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feff8073f20 0x7feff8076430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff00c001230 tx=0x7ff00c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:49.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.782+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff01815c410 con 0x7ff01815c000 2024-09-17T03:28:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.958+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff01806b540 con 0x7feff8073f20 2024-09-17T03:28:49.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.962+0000 7ff015600640 1 -- 172.21.15.46:0/847756125 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252052 (secure 0 0 0) 0x7ff00c001b40 con 0x7feff8073f20 2024-09-17T03:28:49.967 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:49.969 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:28:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feff8073f20 msgr2=0x7feff8076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feff8073f20 0x7feff8076430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff00c001230 tx=0x7ff00c009040 comp rx=0 tx=0).stop 2024-09-17T03:28:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 msgr2=0x7ff018150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 0x7ff018150ca0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff01001b1b0 tx=0x7ff010009280 comp rx=0 tx=0).stop 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 shutdown_connections 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feff8073f20 0x7feff8076430 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff01815c000 0x7ff018150ca0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff018157030 0x7ff01815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 --2- 172.21.15.46:0/847756125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0180b5a80 0x7ff01815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 >> 172.21.15.46:0/847756125 conn(0x7ff0180708c0 msgr2=0x7ff01815e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 shutdown_connections 2024-09-17T03:28:49.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:49.966+0000 7ff020400640 1 -- 172.21.15.46:0/847756125 wait complete. 2024-09-17T03:28:49.989 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-17T03:28:49.345998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430904,"kb_used_data":7864,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754543816,"statfs":{"total":773094113280,"available":772652867584,"internally_reserved":0,"allocated":8052736,"data_stored":4052520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.580512"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.830131+0000","last_change":"2024-09-17T03:28:45.827198+0000","last_active":"2024-09-17T03:28:45.830131+0000","last_peered":"2024-09-17T03:28:45.830131+0000","last_clean":"2024-09-17T03:28:45.830131+0000","last_became_active":"2024-09-17T03:28:45.826864+0000","last_became_peered":"2024-09-17T03:28:45.826864+0000","last_unstale":"2024-09-17T03:28:45.830131+0000","last_undegraded":"2024-09-17T03:28:45.830131+0000","last_fullsized":"2024-09-17T03:28:45.830131+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:18.477715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.813612+0000","last_change":"2024-09-17T03:28:45.810443+0000","last_active":"2024-09-17T03:28:45.813612+0000","last_peered":"2024-09-17T03:28:45.813612+0000","last_clean":"2024-09-17T03:28:45.813612+0000","last_became_active":"2024-09-17T03:28:45.810008+0000","last_became_peered":"2024-09-17T03:28:45.810008+0000","last_unstale":"2024-09-17T03:28:45.813612+0000","last_undegraded":"2024-09-17T03:28:45.813612+0000","last_fullsized":"2024-09-17T03:28:45.813612+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:16.225788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786257+0000","last_change":"2024-09-17T03:28:45.810022+0000","last_active":"2024-09-17T03:28:46.786257+0000","last_peered":"2024-09-17T03:28:46.786257+0000","last_clean":"2024-09-17T03:28:46.786257+0000","last_became_active":"2024-09-17T03:28:45.809662+0000","last_became_peered":"2024-09-17T03:28:45.809662+0000","last_unstale":"2024-09-17T03:28:46.786257+0000","last_undegraded":"2024-09-17T03:28:46.786257+0000","last_fullsized":"2024-09-17T03:28:46.786257+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:54.150105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816283+0000","last_change":"2024-09-17T03:28:45.814391+0000","last_active":"2024-09-17T03:28:45.816283+0000","last_peered":"2024-09-17T03:28:45.816283+0000","last_clean":"2024-09-17T03:28:45.816283+0000","last_became_active":"2024-09-17T03:28:45.814050+0000","last_became_peered":"2024-09-17T03:28:45.814050+0000","last_unstale":"2024-09-17T03:28:45.816283+0000","last_undegraded":"2024-09-17T03:28:45.816283+0000","last_fullsized":"2024-09-17T03:28:45.816283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:31.338300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.813913+0000","last_change":"2024-09-17T03:28:45.802646+0000","last_active":"2024-09-17T03:28:45.813913+0000","last_peered":"2024-09-17T03:28:45.813913+0000","last_clean":"2024-09-17T03:28:45.813913+0000","last_became_active":"2024-09-17T03:28:45.802278+0000","last_became_peered":"2024-09-17T03:28:45.802278+0000","last_unstale":"2024-09-17T03:28:45.813913+0000","last_undegraded":"2024-09-17T03:28:45.813913+0000","last_fullsized":"2024-09-17T03:28:45.813913+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:12.413427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819088+0000","last_change":"2024-09-17T03:28:45.814489+0000","last_active":"2024-09-17T03:28:45.819088+0000","last_peered":"2024-09-17T03:28:45.819088+0000","last_clean":"2024-09-17T03:28:45.819088+0000","last_became_active":"2024-09-17T03:28:45.813244+0000","last_became_peered":"2024-09-17T03:28:45.813244+0000","last_unstale":"2024-09-17T03:28:45.819088+0000","last_undegraded":"2024-09-17T03:28:45.819088+0000","last_fullsized":"2024-09-17T03:28:45.819088+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:34.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.811644+0000","last_change":"2024-09-17T03:28:45.806956+0000","last_active":"2024-09-17T03:28:45.811644+0000","last_peered":"2024-09-17T03:28:45.811644+0000","last_clean":"2024-09-17T03:28:45.811644+0000","last_became_active":"2024-09-17T03:28:45.806641+0000","last_became_peered":"2024-09-17T03:28:45.806641+0000","last_unstale":"2024-09-17T03:28:45.811644+0000","last_undegraded":"2024-09-17T03:28:45.811644+0000","last_fullsized":"2024-09-17T03:28:45.811644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:38.173247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816674+0000","last_change":"2024-09-17T03:28:45.811796+0000","last_active":"2024-09-17T03:28:45.816674+0000","last_peered":"2024-09-17T03:28:45.816674+0000","last_clean":"2024-09-17T03:28:45.816674+0000","last_became_active":"2024-09-17T03:28:45.811485+0000","last_became_peered":"2024-09-17T03:28:45.811485+0000","last_unstale":"2024-09-17T03:28:45.816674+0000","last_undegraded":"2024-09-17T03:28:45.816674+0000","last_fullsized":"2024-09-17T03:28:45.816674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:55.233796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.823726+0000","last_change":"2024-09-17T03:28:45.819619+0000","last_active":"2024-09-17T03:28:45.823726+0000","last_peered":"2024-09-17T03:28:45.823726+0000","last_clean":"2024-09-17T03:28:45.823726+0000","last_became_active":"2024-09-17T03:28:45.818422+0000","last_became_peered":"2024-09-17T03:28:45.818422+0000","last_unstale":"2024-09-17T03:28:45.823726+0000","last_undegraded":"2024-09-17T03:28:45.823726+0000","last_fullsized":"2024-09-17T03:28:45.823726+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:32.889342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793590+0000","last_change":"2024-09-17T03:28:45.818733+0000","last_active":"2024-09-17T03:28:46.793590+0000","last_peered":"2024-09-17T03:28:46.793590+0000","last_clean":"2024-09-17T03:28:46.793590+0000","last_became_active":"2024-09-17T03:28:45.818432+0000","last_became_peered":"2024-09-17T03:28:45.818432+0000","last_unstale":"2024-09-17T03:28:46.793590+0000","last_undegraded":"2024-09-17T03:28:46.793590+0000","last_fullsized":"2024-09-17T03:28:46.793590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:00.725496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.809480+0000","last_change":"2024-09-17T03:28:45.803215+0000","last_active":"2024-09-17T03:28:45.809480+0000","last_peered":"2024-09-17T03:28:45.809480+0000","last_clean":"2024-09-17T03:28:45.809480+0000","last_became_active":"2024-09-17T03:28:45.802851+0000","last_became_peered":"2024-09-17T03:28:45.802851+0000","last_unstale":"2024-09-17T03:28:45.809480+0000","last_undegraded":"2024-09-17T03:28:45.809480+0000","last_fullsized":"2024-09-17T03:28:45.809480+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:55.456639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812891+0000","last_change":"2024-09-17T03:28:45.808131+0000","last_active":"2024-09-17T03:28:45.812891+0000","last_peered":"2024-09-17T03:28:45.812891+0000","last_clean":"2024-09-17T03:28:45.812891+0000","last_became_active":"2024-09-17T03:28:45.807528+0000","last_became_peered":"2024-09-17T03:28:45.807528+0000","last_unstale":"2024-09-17T03:28:45.812891+0000","last_undegraded":"2024-09-17T03:28:45.812891+0000","last_fullsized":"2024-09-17T03:28:45.812891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:15.280804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810575+0000","last_change":"2024-09-17T03:28:45.807003+0000","last_active":"2024-09-17T03:28:46.810575+0000","last_peered":"2024-09-17T03:28:46.810575+0000","last_clean":"2024-09-17T03:28:46.810575+0000","last_became_active":"2024-09-17T03:28:45.806544+0000","last_became_peered":"2024-09-17T03:28:45.806544+0000","last_unstale":"2024-09-17T03:28:46.810575+0000","last_undegraded":"2024-09-17T03:28:46.810575+0000","last_fullsized":"2024-09-17T03:28:46.810575+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:07.918094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.817403+0000","last_change":"2024-09-17T03:28:45.807240+0000","last_active":"2024-09-17T03:28:45.817403+0000","last_peered":"2024-09-17T03:28:45.817403+0000","last_clean":"2024-09-17T03:28:45.817403+0000","last_became_active":"2024-09-17T03:28:45.806908+0000","last_became_peered":"2024-09-17T03:28:45.806908+0000","last_unstale":"2024-09-17T03:28:45.817403+0000","last_undegraded":"2024-09-17T03:28:45.817403+0000","last_fullsized":"2024-09-17T03:28:45.817403+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:55.797038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810391+0000","last_change":"2024-09-17T03:28:45.809314+0000","last_active":"2024-09-17T03:28:46.810391+0000","last_peered":"2024-09-17T03:28:46.810391+0000","last_clean":"2024-09-17T03:28:46.810391+0000","last_became_active":"2024-09-17T03:28:45.809007+0000","last_became_peered":"2024-09-17T03:28:45.809007+0000","last_unstale":"2024-09-17T03:28:46.810391+0000","last_undegraded":"2024-09-17T03:28:46.810391+0000","last_fullsized":"2024-09-17T03:28:46.810391+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:16.504679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.792967+0000","last_change":"2024-09-17T03:28:45.815770+0000","last_active":"2024-09-17T03:28:46.792967+0000","last_peered":"2024-09-17T03:28:46.792967+0000","last_clean":"2024-09-17T03:28:46.792967+0000","last_became_active":"2024-09-17T03:28:45.815447+0000","last_became_peered":"2024-09-17T03:28:45.815447+0000","last_unstale":"2024-09-17T03:28:46.792967+0000","last_undegraded":"2024-09-17T03:28:46.792967+0000","last_fullsized":"2024-09-17T03:28:46.792967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:02.877893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.817462+0000","last_change":"2024-09-17T03:28:45.810511+0000","last_active":"2024-09-17T03:28:45.817462+0000","last_peered":"2024-09-17T03:28:45.817462+0000","last_clean":"2024-09-17T03:28:45.817462+0000","last_became_active":"2024-09-17T03:28:45.810028+0000","last_became_peered":"2024-09-17T03:28:45.810028+0000","last_unstale":"2024-09-17T03:28:45.817462+0000","last_undegraded":"2024-09-17T03:28:45.817462+0000","last_fullsized":"2024-09-17T03:28:45.817462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.200674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816635+0000","last_change":"2024-09-17T03:28:45.815407+0000","last_active":"2024-09-17T03:28:45.816635+0000","last_peered":"2024-09-17T03:28:45.816635+0000","last_clean":"2024-09-17T03:28:45.816635+0000","last_became_active":"2024-09-17T03:28:45.813453+0000","last_became_peered":"2024-09-17T03:28:45.813453+0000","last_unstale":"2024-09-17T03:28:45.816635+0000","last_undegraded":"2024-09-17T03:28:45.816635+0000","last_fullsized":"2024-09-17T03:28:45.816635+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:25.249050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.814248+0000","last_change":"2024-09-17T03:28:45.810588+0000","last_active":"2024-09-17T03:28:45.814248+0000","last_peered":"2024-09-17T03:28:45.814248+0000","last_clean":"2024-09-17T03:28:45.814248+0000","last_became_active":"2024-09-17T03:28:45.810147+0000","last_became_peered":"2024-09-17T03:28:45.810147+0000","last_unstale":"2024-09-17T03:28:45.814248+0000","last_undegraded":"2024-09-17T03:28:45.814248+0000","last_fullsized":"2024-09-17T03:28:45.814248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:16.221026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810531+0000","last_change":"2024-09-17T03:28:45.809822+0000","last_active":"2024-09-17T03:28:46.810531+0000","last_peered":"2024-09-17T03:28:46.810531+0000","last_clean":"2024-09-17T03:28:46.810531+0000","last_became_active":"2024-09-17T03:28:45.807847+0000","last_became_peered":"2024-09-17T03:28:45.807847+0000","last_unstale":"2024-09-17T03:28:46.810531+0000","last_undegraded":"2024-09-17T03:28:46.810531+0000","last_fullsized":"2024-09-17T03:28:46.810531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:49.878675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793103+0000","last_change":"2024-09-17T03:28:45.810834+0000","last_active":"2024-09-17T03:28:46.793103+0000","last_peered":"2024-09-17T03:28:46.793103+0000","last_clean":"2024-09-17T03:28:46.793103+0000","last_became_active":"2024-09-17T03:28:45.810461+0000","last_became_peered":"2024-09-17T03:28:45.810461+0000","last_unstale":"2024-09-17T03:28:46.793103+0000","last_undegraded":"2024-09-17T03:28:46.793103+0000","last_fullsized":"2024-09-17T03:28:46.793103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:36.085989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793618+0000","last_change":"2024-09-17T03:28:45.819925+0000","last_active":"2024-09-17T03:28:46.793618+0000","last_peered":"2024-09-17T03:28:46.793618+0000","last_clean":"2024-09-17T03:28:46.793618+0000","last_became_active":"2024-09-17T03:28:45.817803+0000","last_became_peered":"2024-09-17T03:28:45.817803+0000","last_unstale":"2024-09-17T03:28:46.793618+0000","last_undegraded":"2024-09-17T03:28:46.793618+0000","last_fullsized":"2024-09-17T03:28:46.793618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:13.240432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.818719+0000","last_change":"2024-09-17T03:28:45.813660+0000","last_active":"2024-09-17T03:28:45.818719+0000","last_peered":"2024-09-17T03:28:45.818719+0000","last_clean":"2024-09-17T03:28:45.818719+0000","last_became_active":"2024-09-17T03:28:45.813291+0000","last_became_peered":"2024-09-17T03:28:45.813291+0000","last_unstale":"2024-09-17T03:28:45.818719+0000","last_undegraded":"2024-09-17T03:28:45.818719+0000","last_fullsized":"2024-09-17T03:28:45.818719+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:10.437453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819425+0000","last_change":"2024-09-17T03:28:45.815494+0000","last_active":"2024-09-17T03:28:45.819425+0000","last_peered":"2024-09-17T03:28:45.819425+0000","last_clean":"2024-09-17T03:28:45.819425+0000","last_became_active":"2024-09-17T03:28:45.815081+0000","last_became_peered":"2024-09-17T03:28:45.815081+0000","last_unstale":"2024-09-17T03:28:45.819425+0000","last_undegraded":"2024-09-17T03:28:45.819425+0000","last_fullsized":"2024-09-17T03:28:45.819425+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:02.031516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793547+0000","last_change":"2024-09-17T03:28:45.804852+0000","last_active":"2024-09-17T03:28:46.793547+0000","last_peered":"2024-09-17T03:28:46.793547+0000","last_clean":"2024-09-17T03:28:46.793547+0000","last_became_active":"2024-09-17T03:28:45.800852+0000","last_became_peered":"2024-09-17T03:28:45.800852+0000","last_unstale":"2024-09-17T03:28:46.793547+0000","last_undegraded":"2024-09-17T03:28:46.793547+0000","last_fullsized":"2024-09-17T03:28:46.793547+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:56.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.823923+0000","last_change":"2024-09-17T03:28:45.822985+0000","last_active":"2024-09-17T03:28:45.823923+0000","last_peered":"2024-09-17T03:28:45.823923+0000","last_clean":"2024-09-17T03:28:45.823923+0000","last_became_active":"2024-09-17T03:28:45.819193+0000","last_became_peered":"2024-09-17T03:28:45.819193+0000","last_unstale":"2024-09-17T03:28:45.823923+0000","last_undegraded":"2024-09-17T03:28:45.823923+0000","last_fullsized":"2024-09-17T03:28:45.823923+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:28:17.781870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786307+0000","last_change":"2024-09-17T03:28:45.798497+0000","last_active":"2024-09-17T03:28:46.786307+0000","last_peered":"2024-09-17T03:28:46.786307+0000","last_clean":"2024-09-17T03:28:46.786307+0000","last_became_active":"2024-09-17T03:28:45.797746+0000","last_became_peered":"2024-09-17T03:28:45.797746+0000","last_unstale":"2024-09-17T03:28:46.786307+0000","last_undegraded":"2024-09-17T03:28:46.786307+0000","last_fullsized":"2024-09-17T03:28:46.786307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:46.414153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793465+0000","last_change":"2024-09-17T03:28:45.813599+0000","last_active":"2024-09-17T03:28:46.793465+0000","last_peered":"2024-09-17T03:28:46.793465+0000","last_clean":"2024-09-17T03:28:46.793465+0000","last_became_active":"2024-09-17T03:28:45.813265+0000","last_became_peered":"2024-09-17T03:28:45.813265+0000","last_unstale":"2024-09-17T03:28:46.793465+0000","last_undegraded":"2024-09-17T03:28:46.793465+0000","last_fullsized":"2024-09-17T03:28:46.793465+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:06.373341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786176+0000","last_change":"2024-09-17T03:28:45.794834+0000","last_active":"2024-09-17T03:28:46.786176+0000","last_peered":"2024-09-17T03:28:46.786176+0000","last_clean":"2024-09-17T03:28:46.786176+0000","last_became_active":"2024-09-17T03:28:45.794542+0000","last_became_peered":"2024-09-17T03:28:45.794542+0000","last_unstale":"2024-09-17T03:28:46.786176+0000","last_undegraded":"2024-09-17T03:28:46.786176+0000","last_fullsized":"2024-09-17T03:28:46.786176+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:07.656159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.821847+0000","last_change":"2024-09-17T03:28:45.819685+0000","last_active":"2024-09-17T03:28:45.821847+0000","last_peered":"2024-09-17T03:28:45.821847+0000","last_clean":"2024-09-17T03:28:45.821847+0000","last_became_active":"2024-09-17T03:28:45.819024+0000","last_became_peered":"2024-09-17T03:28:45.819024+0000","last_unstale":"2024-09-17T03:28:45.821847+0000","last_undegraded":"2024-09-17T03:28:45.821847+0000","last_fullsized":"2024-09-17T03:28:45.821847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:14.645626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.818112+0000","last_change":"2024-09-17T03:28:45.816828+0000","last_active":"2024-09-17T03:28:45.818112+0000","last_peered":"2024-09-17T03:28:45.818112+0000","last_clean":"2024-09-17T03:28:45.818112+0000","last_became_active":"2024-09-17T03:28:45.814383+0000","last_became_peered":"2024-09-17T03:28:45.814383+0000","last_unstale":"2024-09-17T03:28:45.818112+0000","last_undegraded":"2024-09-17T03:28:45.818112+0000","last_fullsized":"2024-09-17T03:28:45.818112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:41.667034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.815866+0000","last_change":"2024-09-17T03:28:45.812548+0000","last_active":"2024-09-17T03:28:45.815866+0000","last_peered":"2024-09-17T03:28:45.815866+0000","last_clean":"2024-09-17T03:28:45.815866+0000","last_became_active":"2024-09-17T03:28:45.812213+0000","last_became_peered":"2024-09-17T03:28:45.812213+0000","last_unstale":"2024-09-17T03:28:45.815866+0000","last_undegraded":"2024-09-17T03:28:45.815866+0000","last_fullsized":"2024-09-17T03:28:45.815866+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:54.671107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819691+0000","last_change":"2024-09-17T03:28:45.815407+0000","last_active":"2024-09-17T03:28:45.819691+0000","last_peered":"2024-09-17T03:28:45.819691+0000","last_clean":"2024-09-17T03:28:45.819691+0000","last_became_active":"2024-09-17T03:28:45.815016+0000","last_became_peered":"2024-09-17T03:28:45.815016+0000","last_unstale":"2024-09-17T03:28:45.819691+0000","last_undegraded":"2024-09-17T03:28:45.819691+0000","last_fullsized":"2024-09-17T03:28:45.819691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.285014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.833264+0000","last_change":"2024-09-17T03:28:44.822691+0000","last_active":"2024-09-17T03:28:45.833264+0000","last_peered":"2024-09-17T03:28:45.833264+0000","last_clean":"2024-09-17T03:28:45.833264+0000","last_became_active":"2024-09-17T03:28:44.822375+0000","last_became_peered":"2024-09-17T03:28:44.822375+0000","last_unstale":"2024-09-17T03:28:45.833264+0000","last_undegraded":"2024-09-17T03:28:45.833264+0000","last_fullsized":"2024-09-17T03:28:45.833264+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:05.252640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.823697+0000","last_change":"2024-09-17T03:28:45.815695+0000","last_active":"2024-09-17T03:28:45.823697+0000","last_peered":"2024-09-17T03:28:45.823697+0000","last_clean":"2024-09-17T03:28:45.823697+0000","last_became_active":"2024-09-17T03:28:45.805928+0000","last_became_peered":"2024-09-17T03:28:45.805928+0000","last_unstale":"2024-09-17T03:28:45.823697+0000","last_undegraded":"2024-09-17T03:28:45.823697+0000","last_fullsized":"2024-09-17T03:28:45.823697+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:36.215624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810851+0000","last_change":"2024-09-17T03:28:44.794991+0000","last_active":"2024-09-17T03:28:46.810851+0000","last_peered":"2024-09-17T03:28:46.810851+0000","last_clean":"2024-09-17T03:28:46.810851+0000","last_became_active":"2024-09-17T03:28:44.794572+0000","last_became_peered":"2024-09-17T03:28:44.794572+0000","last_unstale":"2024-09-17T03:28:46.810851+0000","last_undegraded":"2024-09-17T03:28:46.810851+0000","last_fullsized":"2024-09-17T03:28:46.810851+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:57.701295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819181+0000","last_change":"2024-09-17T03:28:45.815390+0000","last_active":"2024-09-17T03:28:45.819181+0000","last_peered":"2024-09-17T03:28:45.819181+0000","last_clean":"2024-09-17T03:28:45.819181+0000","last_became_active":"2024-09-17T03:28:45.814121+0000","last_became_peered":"2024-09-17T03:28:45.814121+0000","last_unstale":"2024-09-17T03:28:45.819181+0000","last_undegraded":"2024-09-17T03:28:45.819181+0000","last_fullsized":"2024-09-17T03:28:45.819181+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:09.172192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793328+0000","last_change":"2024-09-17T03:28:44.809278+0000","last_active":"2024-09-17T03:28:46.793328+0000","last_peered":"2024-09-17T03:28:46.793328+0000","last_clean":"2024-09-17T03:28:46.793328+0000","last_became_active":"2024-09-17T03:28:44.808830+0000","last_became_peered":"2024-09-17T03:28:44.808830+0000","last_unstale":"2024-09-17T03:28:46.793328+0000","last_undegraded":"2024-09-17T03:28:46.793328+0000","last_fullsized":"2024-09-17T03:28:46.793328+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:23.662069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812771+0000","last_change":"2024-09-17T03:28:45.810066+0000","last_active":"2024-09-17T03:28:45.812771+0000","last_peered":"2024-09-17T03:28:45.812771+0000","last_clean":"2024-09-17T03:28:45.812771+0000","last_became_active":"2024-09-17T03:28:45.809733+0000","last_became_peered":"2024-09-17T03:28:45.809733+0000","last_unstale":"2024-09-17T03:28:45.812771+0000","last_undegraded":"2024-09-17T03:28:45.812771+0000","last_fullsized":"2024-09-17T03:28:45.812771+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:21.083296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.833166+0000","last_change":"2024-09-17T03:28:44.812929+0000","last_active":"2024-09-17T03:28:45.833166+0000","last_peered":"2024-09-17T03:28:45.833166+0000","last_clean":"2024-09-17T03:28:45.833166+0000","last_became_active":"2024-09-17T03:28:44.807116+0000","last_became_peered":"2024-09-17T03:28:44.807116+0000","last_unstale":"2024-09-17T03:28:45.833166+0000","last_undegraded":"2024-09-17T03:28:45.833166+0000","last_fullsized":"2024-09-17T03:28:45.833166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:02.724293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.811927+0000","last_change":"2024-09-17T03:28:45.802845+0000","last_active":"2024-09-17T03:28:45.811927+0000","last_peered":"2024-09-17T03:28:45.811927+0000","last_clean":"2024-09-17T03:28:45.811927+0000","last_became_active":"2024-09-17T03:28:45.802495+0000","last_became_peered":"2024-09-17T03:28:45.802495+0000","last_unstale":"2024-09-17T03:28:45.811927+0000","last_undegraded":"2024-09-17T03:28:45.811927+0000","last_fullsized":"2024-09-17T03:28:45.811927+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:47.634063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786310+0000","last_change":"2024-09-17T03:28:44.797569+0000","last_active":"2024-09-17T03:28:46.786310+0000","last_peered":"2024-09-17T03:28:46.786310+0000","last_clean":"2024-09-17T03:28:46.786310+0000","last_became_active":"2024-09-17T03:28:44.797089+0000","last_became_peered":"2024-09-17T03:28:44.797089+0000","last_unstale":"2024-09-17T03:28:46.786310+0000","last_undegraded":"2024-09-17T03:28:46.786310+0000","last_fullsized":"2024-09-17T03:28:46.786310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:57.895718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793120+0000","last_change":"2024-09-17T03:28:45.803393+0000","last_active":"2024-09-17T03:28:46.793120+0000","last_peered":"2024-09-17T03:28:46.793120+0000","last_clean":"2024-09-17T03:28:46.793120+0000","last_became_active":"2024-09-17T03:28:45.801974+0000","last_became_peered":"2024-09-17T03:28:45.801974+0000","last_unstale":"2024-09-17T03:28:46.793120+0000","last_undegraded":"2024-09-17T03:28:46.793120+0000","last_fullsized":"2024-09-17T03:28:46.793120+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:00.513379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.828446+0000","last_change":"2024-09-17T03:28:44.810924+0000","last_active":"2024-09-17T03:28:45.828446+0000","last_peered":"2024-09-17T03:28:45.828446+0000","last_clean":"2024-09-17T03:28:45.828446+0000","last_became_active":"2024-09-17T03:28:44.810597+0000","last_became_peered":"2024-09-17T03:28:44.810597+0000","last_unstale":"2024-09-17T03:28:45.828446+0000","last_undegraded":"2024-09-17T03:28:45.828446+0000","last_fullsized":"2024-09-17T03:28:45.828446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:04.869978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816175+0000","last_change":"2024-09-17T03:28:45.804065+0000","last_active":"2024-09-17T03:28:45.816175+0000","last_peered":"2024-09-17T03:28:45.816175+0000","last_clean":"2024-09-17T03:28:45.816175+0000","last_became_active":"2024-09-17T03:28:45.802283+0000","last_became_peered":"2024-09-17T03:28:45.802283+0000","last_unstale":"2024-09-17T03:28:45.816175+0000","last_undegraded":"2024-09-17T03:28:45.816175+0000","last_fullsized":"2024-09-17T03:28:45.816175+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:05.676121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.781475+0000","last_change":"2024-09-17T03:28:44.798788+0000","last_active":"2024-09-17T03:28:45.781475+0000","last_peered":"2024-09-17T03:28:45.781475+0000","last_clean":"2024-09-17T03:28:45.781475+0000","last_became_active":"2024-09-17T03:28:44.798431+0000","last_became_peered":"2024-09-17T03:28:44.798431+0000","last_unstale":"2024-09-17T03:28:45.781475+0000","last_undegraded":"2024-09-17T03:28:45.781475+0000","last_fullsized":"2024-09-17T03:28:45.781475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:35.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810618+0000","last_change":"2024-09-17T03:28:45.802762+0000","last_active":"2024-09-17T03:28:46.810618+0000","last_peered":"2024-09-17T03:28:46.810618+0000","last_clean":"2024-09-17T03:28:46.810618+0000","last_became_active":"2024-09-17T03:28:45.802326+0000","last_became_peered":"2024-09-17T03:28:45.802326+0000","last_unstale":"2024-09-17T03:28:46.810618+0000","last_undegraded":"2024-09-17T03:28:46.810618+0000","last_fullsized":"2024-09-17T03:28:46.810618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:33.916277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.797928+0000","last_change":"2024-09-17T03:28:44.801618+0000","last_active":"2024-09-17T03:28:45.797928+0000","last_peered":"2024-09-17T03:28:45.797928+0000","last_clean":"2024-09-17T03:28:45.797928+0000","last_became_active":"2024-09-17T03:28:44.793440+0000","last_became_peered":"2024-09-17T03:28:44.793440+0000","last_unstale":"2024-09-17T03:28:45.797928+0000","last_undegraded":"2024-09-17T03:28:45.797928+0000","last_fullsized":"2024-09-17T03:28:45.797928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:23.363356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812445+0000","last_change":"2024-09-17T03:28:45.801406+0000","last_active":"2024-09-17T03:28:45.812445+0000","last_peered":"2024-09-17T03:28:45.812445+0000","last_clean":"2024-09-17T03:28:45.812445+0000","last_became_active":"2024-09-17T03:28:45.801000+0000","last_became_peered":"2024-09-17T03:28:45.801000+0000","last_unstale":"2024-09-17T03:28:45.812445+0000","last_undegraded":"2024-09-17T03:28:45.812445+0000","last_fullsized":"2024-09-17T03:28:45.812445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:51.282149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793687+0000","last_change":"2024-09-17T03:28:44.803091+0000","last_active":"2024-09-17T03:28:46.793687+0000","last_peered":"2024-09-17T03:28:46.793687+0000","last_clean":"2024-09-17T03:28:46.793687+0000","last_became_active":"2024-09-17T03:28:44.802150+0000","last_became_peered":"2024-09-17T03:28:44.802150+0000","last_unstale":"2024-09-17T03:28:46.793687+0000","last_undegraded":"2024-09-17T03:28:46.793687+0000","last_fullsized":"2024-09-17T03:28:46.793687+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:12.508649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819691+0000","last_change":"2024-09-17T03:28:45.817735+0000","last_active":"2024-09-17T03:28:45.819691+0000","last_peered":"2024-09-17T03:28:45.819691+0000","last_clean":"2024-09-17T03:28:45.819691+0000","last_became_active":"2024-09-17T03:28:45.817435+0000","last_became_peered":"2024-09-17T03:28:45.817435+0000","last_unstale":"2024-09-17T03:28:45.819691+0000","last_undegraded":"2024-09-17T03:28:45.819691+0000","last_fullsized":"2024-09-17T03:28:45.819691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:22.779750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.799042+0000","last_change":"2024-09-17T03:28:44.813845+0000","last_active":"2024-09-17T03:28:45.799042+0000","last_peered":"2024-09-17T03:28:45.799042+0000","last_clean":"2024-09-17T03:28:45.799042+0000","last_became_active":"2024-09-17T03:28:44.813471+0000","last_became_peered":"2024-09-17T03:28:44.813471+0000","last_unstale":"2024-09-17T03:28:45.799042+0000","last_undegraded":"2024-09-17T03:28:45.799042+0000","last_fullsized":"2024-09-17T03:28:45.799042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:23.668468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819149+0000","last_change":"2024-09-17T03:28:45.814656+0000","last_active":"2024-09-17T03:28:45.819149+0000","last_peered":"2024-09-17T03:28:45.819149+0000","last_clean":"2024-09-17T03:28:45.819149+0000","last_became_active":"2024-09-17T03:28:45.813774+0000","last_became_peered":"2024-09-17T03:28:45.813774+0000","last_unstale":"2024-09-17T03:28:45.819149+0000","last_undegraded":"2024-09-17T03:28:45.819149+0000","last_fullsized":"2024-09-17T03:28:45.819149+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:16.920106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786419+0000","last_change":"2024-09-17T03:28:44.799439+0000","last_active":"2024-09-17T03:28:46.786419+0000","last_peered":"2024-09-17T03:28:46.786419+0000","last_clean":"2024-09-17T03:28:46.786419+0000","last_became_active":"2024-09-17T03:28:44.798699+0000","last_became_peered":"2024-09-17T03:28:44.798699+0000","last_unstale":"2024-09-17T03:28:46.786419+0000","last_undegraded":"2024-09-17T03:28:46.786419+0000","last_fullsized":"2024-09-17T03:28:46.786419+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:30.801287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812458+0000","last_change":"2024-09-17T03:28:45.805729+0000","last_active":"2024-09-17T03:28:45.812458+0000","last_peered":"2024-09-17T03:28:45.812458+0000","last_clean":"2024-09-17T03:28:45.812458+0000","last_became_active":"2024-09-17T03:28:45.805339+0000","last_became_peered":"2024-09-17T03:28:45.805339+0000","last_unstale":"2024-09-17T03:28:45.812458+0000","last_undegraded":"2024-09-17T03:28:45.812458+0000","last_fullsized":"2024-09-17T03:28:45.812458+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:48.246592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.826493+0000","last_change":"2024-09-17T03:28:44.802285+0000","last_active":"2024-09-17T03:28:45.826493+0000","last_peered":"2024-09-17T03:28:45.826493+0000","last_clean":"2024-09-17T03:28:45.826493+0000","last_became_active":"2024-09-17T03:28:44.800671+0000","last_became_peered":"2024-09-17T03:28:44.800671+0000","last_unstale":"2024-09-17T03:28:45.826493+0000","last_undegraded":"2024-09-17T03:28:45.826493+0000","last_fullsized":"2024-09-17T03:28:45.826493+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:34.008579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819485+0000","last_change":"2024-09-17T03:28:45.816753+0000","last_active":"2024-09-17T03:28:45.819485+0000","last_peered":"2024-09-17T03:28:45.819485+0000","last_clean":"2024-09-17T03:28:45.819485+0000","last_became_active":"2024-09-17T03:28:45.816429+0000","last_became_peered":"2024-09-17T03:28:45.816429+0000","last_unstale":"2024-09-17T03:28:45.819485+0000","last_undegraded":"2024-09-17T03:28:45.819485+0000","last_fullsized":"2024-09-17T03:28:45.819485+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:10.897535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.818473+0000","last_change":"2024-09-17T03:28:44.807188+0000","last_active":"2024-09-17T03:28:45.818473+0000","last_peered":"2024-09-17T03:28:45.818473+0000","last_clean":"2024-09-17T03:28:45.818473+0000","last_became_active":"2024-09-17T03:28:44.806356+0000","last_became_peered":"2024-09-17T03:28:44.806356+0000","last_unstale":"2024-09-17T03:28:45.818473+0000","last_undegraded":"2024-09-17T03:28:45.818473+0000","last_fullsized":"2024-09-17T03:28:45.818473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:53.498836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812273+0000","last_change":"2024-09-17T03:28:45.804863+0000","last_active":"2024-09-17T03:28:45.812273+0000","last_peered":"2024-09-17T03:28:45.812273+0000","last_clean":"2024-09-17T03:28:45.812273+0000","last_became_active":"2024-09-17T03:28:45.804493+0000","last_became_peered":"2024-09-17T03:28:45.804493+0000","last_unstale":"2024-09-17T03:28:45.812273+0000","last_undegraded":"2024-09-17T03:28:45.812273+0000","last_fullsized":"2024-09-17T03:28:45.812273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:58.261639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.803671+0000","last_change":"2024-09-17T03:28:44.811074+0000","last_active":"2024-09-17T03:28:45.803671+0000","last_peered":"2024-09-17T03:28:45.803671+0000","last_clean":"2024-09-17T03:28:45.803671+0000","last_became_active":"2024-09-17T03:28:44.809921+0000","last_became_peered":"2024-09-17T03:28:44.809921+0000","last_unstale":"2024-09-17T03:28:45.803671+0000","last_undegraded":"2024-09-17T03:28:45.803671+0000","last_fullsized":"2024-09-17T03:28:45.803671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:50.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810360+0000","last_change":"2024-09-17T03:28:45.808668+0000","last_active":"2024-09-17T03:28:46.810360+0000","last_peered":"2024-09-17T03:28:46.810360+0000","last_clean":"2024-09-17T03:28:46.810360+0000","last_became_active":"2024-09-17T03:28:45.808270+0000","last_became_peered":"2024-09-17T03:28:45.808270+0000","last_unstale":"2024-09-17T03:28:46.810360+0000","last_undegraded":"2024-09-17T03:28:46.810360+0000","last_fullsized":"2024-09-17T03:28:46.810360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:13.394497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.794243+0000","last_change":"2024-09-17T03:28:44.813876+0000","last_active":"2024-09-17T03:28:45.794243+0000","last_peered":"2024-09-17T03:28:45.794243+0000","last_clean":"2024-09-17T03:28:45.794243+0000","last_became_active":"2024-09-17T03:28:44.813556+0000","last_became_peered":"2024-09-17T03:28:44.813556+0000","last_unstale":"2024-09-17T03:28:45.794243+0000","last_undegraded":"2024-09-17T03:28:45.794243+0000","last_fullsized":"2024-09-17T03:28:45.794243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:21.183444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812229+0000","last_change":"2024-09-17T03:28:45.806848+0000","last_active":"2024-09-17T03:28:45.812229+0000","last_peered":"2024-09-17T03:28:45.812229+0000","last_clean":"2024-09-17T03:28:45.812229+0000","last_became_active":"2024-09-17T03:28:45.806501+0000","last_became_peered":"2024-09-17T03:28:45.806501+0000","last_unstale":"2024-09-17T03:28:45.812229+0000","last_undegraded":"2024-09-17T03:28:45.812229+0000","last_fullsized":"2024-09-17T03:28:45.812229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:30.914190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.792830+0000","last_change":"2024-09-17T03:28:44.807106+0000","last_active":"2024-09-17T03:28:46.792830+0000","last_peered":"2024-09-17T03:28:46.792830+0000","last_clean":"2024-09-17T03:28:46.792830+0000","last_became_active":"2024-09-17T03:28:44.806711+0000","last_became_peered":"2024-09-17T03:28:44.806711+0000","last_unstale":"2024-09-17T03:28:46.792830+0000","last_undegraded":"2024-09-17T03:28:46.792830+0000","last_fullsized":"2024-09-17T03:28:46.792830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:29.666865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793762+0000","last_change":"2024-09-17T03:28:45.797043+0000","last_active":"2024-09-17T03:28:46.793762+0000","last_peered":"2024-09-17T03:28:46.793762+0000","last_clean":"2024-09-17T03:28:46.793762+0000","last_became_active":"2024-09-17T03:28:45.795415+0000","last_became_peered":"2024-09-17T03:28:45.795415+0000","last_unstale":"2024-09-17T03:28:46.793762+0000","last_undegraded":"2024-09-17T03:28:46.793762+0000","last_fullsized":"2024-09-17T03:28:46.793762+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:51.911480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.831733+0000","last_change":"2024-09-17T03:28:44.809080+0000","last_active":"2024-09-17T03:28:45.831733+0000","last_peered":"2024-09-17T03:28:45.831733+0000","last_clean":"2024-09-17T03:28:45.831733+0000","last_became_active":"2024-09-17T03:28:44.808569+0000","last_became_peered":"2024-09-17T03:28:44.808569+0000","last_unstale":"2024-09-17T03:28:45.831733+0000","last_undegraded":"2024-09-17T03:28:45.831733+0000","last_fullsized":"2024-09-17T03:28:45.831733+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:53.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819275+0000","last_change":"2024-09-17T03:28:45.810038+0000","last_active":"2024-09-17T03:28:45.819275+0000","last_peered":"2024-09-17T03:28:45.819275+0000","last_clean":"2024-09-17T03:28:45.819275+0000","last_became_active":"2024-09-17T03:28:45.809694+0000","last_became_peered":"2024-09-17T03:28:45.809694+0000","last_unstale":"2024-09-17T03:28:45.819275+0000","last_undegraded":"2024-09-17T03:28:45.819275+0000","last_fullsized":"2024-09-17T03:28:45.819275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:00.053682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793743+0000","last_change":"2024-09-17T03:28:44.808811+0000","last_active":"2024-09-17T03:28:46.793743+0000","last_peered":"2024-09-17T03:28:46.793743+0000","last_clean":"2024-09-17T03:28:46.793743+0000","last_became_active":"2024-09-17T03:28:44.808496+0000","last_became_peered":"2024-09-17T03:28:44.808496+0000","last_unstale":"2024-09-17T03:28:46.793743+0000","last_undegraded":"2024-09-17T03:28:46.793743+0000","last_fullsized":"2024-09-17T03:28:46.793743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:19.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812861+0000","last_change":"2024-09-17T03:28:45.807830+0000","last_active":"2024-09-17T03:28:45.812861+0000","last_peered":"2024-09-17T03:28:45.812861+0000","last_clean":"2024-09-17T03:28:45.812861+0000","last_became_active":"2024-09-17T03:28:45.807227+0000","last_became_peered":"2024-09-17T03:28:45.807227+0000","last_unstale":"2024-09-17T03:28:45.812861+0000","last_undegraded":"2024-09-17T03:28:45.812861+0000","last_fullsized":"2024-09-17T03:28:45.812861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.829781+0000","last_change":"2024-09-17T03:28:44.808559+0000","last_active":"2024-09-17T03:28:45.829781+0000","last_peered":"2024-09-17T03:28:45.829781+0000","last_clean":"2024-09-17T03:28:45.829781+0000","last_became_active":"2024-09-17T03:28:44.808233+0000","last_became_peered":"2024-09-17T03:28:44.808233+0000","last_unstale":"2024-09-17T03:28:45.829781+0000","last_undegraded":"2024-09-17T03:28:45.829781+0000","last_fullsized":"2024-09-17T03:28:45.829781+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:23.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812486+0000","last_change":"2024-09-17T03:28:45.805787+0000","last_active":"2024-09-17T03:28:45.812486+0000","last_peered":"2024-09-17T03:28:45.812486+0000","last_clean":"2024-09-17T03:28:45.812486+0000","last_became_active":"2024-09-17T03:28:45.804996+0000","last_became_peered":"2024-09-17T03:28:45.804996+0000","last_unstale":"2024-09-17T03:28:45.812486+0000","last_undegraded":"2024-09-17T03:28:45.812486+0000","last_fullsized":"2024-09-17T03:28:45.812486+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:34.340461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.795166+0000","last_change":"2024-09-17T03:28:44.822755+0000","last_active":"2024-09-17T03:28:45.795166+0000","last_peered":"2024-09-17T03:28:45.795166+0000","last_clean":"2024-09-17T03:28:45.795166+0000","last_became_active":"2024-09-17T03:28:44.822350+0000","last_became_peered":"2024-09-17T03:28:44.822350+0000","last_unstale":"2024-09-17T03:28:45.795166+0000","last_undegraded":"2024-09-17T03:28:45.795166+0000","last_fullsized":"2024-09-17T03:28:45.795166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:47.661258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786501+0000","last_change":"2024-09-17T03:28:45.792116+0000","last_active":"2024-09-17T03:28:46.786501+0000","last_peered":"2024-09-17T03:28:46.786501+0000","last_clean":"2024-09-17T03:28:46.786501+0000","last_became_active":"2024-09-17T03:28:45.791120+0000","last_became_peered":"2024-09-17T03:28:45.791120+0000","last_unstale":"2024-09-17T03:28:46.786501+0000","last_undegraded":"2024-09-17T03:28:46.786501+0000","last_fullsized":"2024-09-17T03:28:46.786501+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:32.113237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.803913+0000","last_change":"2024-09-17T03:28:44.806357+0000","last_active":"2024-09-17T03:28:45.803913+0000","last_peered":"2024-09-17T03:28:45.803913+0000","last_clean":"2024-09-17T03:28:45.803913+0000","last_became_active":"2024-09-17T03:28:44.805862+0000","last_became_peered":"2024-09-17T03:28:44.805862+0000","last_unstale":"2024-09-17T03:28:45.803913+0000","last_undegraded":"2024-09-17T03:28:45.803913+0000","last_fullsized":"2024-09-17T03:28:45.803913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:24.205943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786099+0000","last_change":"2024-09-17T03:28:45.805240+0000","last_active":"2024-09-17T03:28:46.786099+0000","last_peered":"2024-09-17T03:28:46.786099+0000","last_clean":"2024-09-17T03:28:46.786099+0000","last_became_active":"2024-09-17T03:28:45.804932+0000","last_became_peered":"2024-09-17T03:28:45.804932+0000","last_unstale":"2024-09-17T03:28:46.786099+0000","last_undegraded":"2024-09-17T03:28:46.786099+0000","last_fullsized":"2024-09-17T03:28:46.786099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:10.255805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.794579+0000","last_change":"2024-09-17T03:28:44.813215+0000","last_active":"2024-09-17T03:28:45.794579+0000","last_peered":"2024-09-17T03:28:45.794579+0000","last_clean":"2024-09-17T03:28:45.794579+0000","last_became_active":"2024-09-17T03:28:44.812895+0000","last_became_peered":"2024-09-17T03:28:44.812895+0000","last_unstale":"2024-09-17T03:28:45.794579+0000","last_undegraded":"2024-09-17T03:28:45.794579+0000","last_fullsized":"2024-09-17T03:28:45.794579+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:10.875138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786133+0000","last_change":"2024-09-17T03:28:45.813236+0000","last_active":"2024-09-17T03:28:46.786133+0000","last_peered":"2024-09-17T03:28:46.786133+0000","last_clean":"2024-09-17T03:28:46.786133+0000","last_became_active":"2024-09-17T03:28:45.812868+0000","last_became_peered":"2024-09-17T03:28:45.812868+0000","last_unstale":"2024-09-17T03:28:46.786133+0000","last_undegraded":"2024-09-17T03:28:46.786133+0000","last_fullsized":"2024-09-17T03:28:46.786133+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:14.822118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810184+0000","last_change":"2024-09-17T03:28:44.801076+0000","last_active":"2024-09-17T03:28:46.810184+0000","last_peered":"2024-09-17T03:28:46.810184+0000","last_clean":"2024-09-17T03:28:46.810184+0000","last_became_active":"2024-09-17T03:28:44.800786+0000","last_became_peered":"2024-09-17T03:28:44.800786+0000","last_unstale":"2024-09-17T03:28:46.810184+0000","last_undegraded":"2024-09-17T03:28:46.810184+0000","last_fullsized":"2024-09-17T03:28:46.810184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:53.296535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810853+0000","last_change":"2024-09-17T03:28:45.810263+0000","last_active":"2024-09-17T03:28:46.810853+0000","last_peered":"2024-09-17T03:28:46.810853+0000","last_clean":"2024-09-17T03:28:46.810853+0000","last_became_active":"2024-09-17T03:28:45.809928+0000","last_became_peered":"2024-09-17T03:28:45.809928+0000","last_unstale":"2024-09-17T03:28:46.810853+0000","last_undegraded":"2024-09-17T03:28:46.810853+0000","last_fullsized":"2024-09-17T03:28:46.810853+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:25.475101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.832127+0000","last_change":"2024-09-17T03:28:44.811093+0000","last_active":"2024-09-17T03:28:45.832127+0000","last_peered":"2024-09-17T03:28:45.832127+0000","last_clean":"2024-09-17T03:28:45.832127+0000","last_became_active":"2024-09-17T03:28:44.810454+0000","last_became_peered":"2024-09-17T03:28:44.810454+0000","last_unstale":"2024-09-17T03:28:45.832127+0000","last_undegraded":"2024-09-17T03:28:45.832127+0000","last_fullsized":"2024-09-17T03:28:45.832127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:05.160027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.814001+0000","last_change":"2024-09-17T03:28:45.808653+0000","last_active":"2024-09-17T03:28:45.814001+0000","last_peered":"2024-09-17T03:28:45.814001+0000","last_clean":"2024-09-17T03:28:45.814001+0000","last_became_active":"2024-09-17T03:28:45.808357+0000","last_became_peered":"2024-09-17T03:28:45.808357+0000","last_unstale":"2024-09-17T03:28:45.814001+0000","last_undegraded":"2024-09-17T03:28:45.814001+0000","last_fullsized":"2024-09-17T03:28:45.814001+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:41.015864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.795846+0000","last_change":"2024-09-17T03:28:44.810924+0000","last_active":"2024-09-17T03:28:45.795846+0000","last_peered":"2024-09-17T03:28:45.795846+0000","last_clean":"2024-09-17T03:28:45.795846+0000","last_became_active":"2024-09-17T03:28:44.810524+0000","last_became_peered":"2024-09-17T03:28:44.810524+0000","last_unstale":"2024-09-17T03:28:45.795846+0000","last_undegraded":"2024-09-17T03:28:45.795846+0000","last_fullsized":"2024-09-17T03:28:45.795846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:02.741967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.820054+0000","last_change":"2024-09-17T03:28:45.818736+0000","last_active":"2024-09-17T03:28:45.820054+0000","last_peered":"2024-09-17T03:28:45.820054+0000","last_clean":"2024-09-17T03:28:45.820054+0000","last_became_active":"2024-09-17T03:28:45.818326+0000","last_became_peered":"2024-09-17T03:28:45.818326+0000","last_unstale":"2024-09-17T03:28:45.820054+0000","last_undegraded":"2024-09-17T03:28:45.820054+0000","last_fullsized":"2024-09-17T03:28:45.820054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:05.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.798453+0000","last_change":"2024-09-17T03:28:44.806886+0000","last_active":"2024-09-17T03:28:45.798453+0000","last_peered":"2024-09-17T03:28:45.798453+0000","last_clean":"2024-09-17T03:28:45.798453+0000","last_became_active":"2024-09-17T03:28:44.803126+0000","last_became_peered":"2024-09-17T03:28:44.803126+0000","last_unstale":"2024-09-17T03:28:45.798453+0000","last_undegraded":"2024-09-17T03:28:45.798453+0000","last_fullsized":"2024-09-17T03:28:45.798453+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:38.827779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816587+0000","last_change":"2024-09-17T03:28:45.811786+0000","last_active":"2024-09-17T03:28:45.816587+0000","last_peered":"2024-09-17T03:28:45.816587+0000","last_clean":"2024-09-17T03:28:45.816587+0000","last_became_active":"2024-09-17T03:28:45.811379+0000","last_became_peered":"2024-09-17T03:28:45.811379+0000","last_unstale":"2024-09-17T03:28:45.816587+0000","last_undegraded":"2024-09-17T03:28:45.816587+0000","last_fullsized":"2024-09-17T03:28:45.816587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:19.131335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810690+0000","last_change":"2024-09-17T03:28:44.799567+0000","last_active":"2024-09-17T03:28:46.810690+0000","last_peered":"2024-09-17T03:28:46.810690+0000","last_clean":"2024-09-17T03:28:46.810690+0000","last_became_active":"2024-09-17T03:28:44.799229+0000","last_became_peered":"2024-09-17T03:28:44.799229+0000","last_unstale":"2024-09-17T03:28:46.810690+0000","last_undegraded":"2024-09-17T03:28:46.810690+0000","last_fullsized":"2024-09-17T03:28:46.810690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:53.929276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.817370+0000","last_change":"2024-09-17T03:28:45.806154+0000","last_active":"2024-09-17T03:28:45.817370+0000","last_peered":"2024-09-17T03:28:45.817370+0000","last_clean":"2024-09-17T03:28:45.817370+0000","last_became_active":"2024-09-17T03:28:45.805825+0000","last_became_peered":"2024-09-17T03:28:45.805825+0000","last_unstale":"2024-09-17T03:28:45.817370+0000","last_undegraded":"2024-09-17T03:28:45.817370+0000","last_fullsized":"2024-09-17T03:28:45.817370+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:28.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.797551+0000","last_change":"2024-09-17T03:28:44.800382+0000","last_active":"2024-09-17T03:28:45.797551+0000","last_peered":"2024-09-17T03:28:45.797551+0000","last_clean":"2024-09-17T03:28:45.797551+0000","last_became_active":"2024-09-17T03:28:44.800060+0000","last_became_peered":"2024-09-17T03:28:44.800060+0000","last_unstale":"2024-09-17T03:28:45.797551+0000","last_undegraded":"2024-09-17T03:28:45.797551+0000","last_fullsized":"2024-09-17T03:28:45.797551+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:19.213426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819748+0000","last_change":"2024-09-17T03:28:45.814358+0000","last_active":"2024-09-17T03:28:45.819748+0000","last_peered":"2024-09-17T03:28:45.819748+0000","last_clean":"2024-09-17T03:28:45.819748+0000","last_became_active":"2024-09-17T03:28:45.812438+0000","last_became_peered":"2024-09-17T03:28:45.812438+0000","last_unstale":"2024-09-17T03:28:45.819748+0000","last_undegraded":"2024-09-17T03:28:45.819748+0000","last_fullsized":"2024-09-17T03:28:45.819748+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:22.696566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.781467+0000","last_change":"2024-09-17T03:28:44.799622+0000","last_active":"2024-09-17T03:28:45.781467+0000","last_peered":"2024-09-17T03:28:45.781467+0000","last_clean":"2024-09-17T03:28:45.781467+0000","last_became_active":"2024-09-17T03:28:44.799276+0000","last_became_peered":"2024-09-17T03:28:44.799276+0000","last_unstale":"2024-09-17T03:28:45.781467+0000","last_undegraded":"2024-09-17T03:28:45.781467+0000","last_fullsized":"2024-09-17T03:28:45.781467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:53.693967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.790673+0000","last_change":"2024-09-17T03:28:45.786439+0000","last_active":"2024-09-17T03:28:45.790673+0000","last_peered":"2024-09-17T03:28:45.790673+0000","last_clean":"2024-09-17T03:28:45.790673+0000","last_became_active":"2024-09-17T03:28:45.786027+0000","last_became_peered":"2024-09-17T03:28:45.786027+0000","last_unstale":"2024-09-17T03:28:45.790673+0000","last_undegraded":"2024-09-17T03:28:45.790673+0000","last_fullsized":"2024-09-17T03:28:45.790673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:16.351196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.792762+0000","last_change":"2024-09-17T03:28:44.814763+0000","last_active":"2024-09-17T03:28:46.792762+0000","last_peered":"2024-09-17T03:28:46.792762+0000","last_clean":"2024-09-17T03:28:46.792762+0000","last_became_active":"2024-09-17T03:28:44.814387+0000","last_became_peered":"2024-09-17T03:28:44.814387+0000","last_unstale":"2024-09-17T03:28:46.792762+0000","last_undegraded":"2024-09-17T03:28:46.792762+0000","last_fullsized":"2024-09-17T03:28:46.792762+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:28.760438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.831898+0000","last_change":"2024-09-17T03:28:45.831097+0000","last_active":"2024-09-17T03:28:45.831898+0000","last_peered":"2024-09-17T03:28:45.831898+0000","last_clean":"2024-09-17T03:28:45.831898+0000","last_became_active":"2024-09-17T03:28:45.830715+0000","last_became_peered":"2024-09-17T03:28:45.830715+0000","last_unstale":"2024-09-17T03:28:45.831898+0000","last_undegraded":"2024-09-17T03:28:45.831898+0000","last_fullsized":"2024-09-17T03:28:45.831898+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:45.415345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.799003+0000","last_change":"2024-09-17T03:28:44.808776+0000","last_active":"2024-09-17T03:28:45.799003+0000","last_peered":"2024-09-17T03:28:45.799003+0000","last_clean":"2024-09-17T03:28:45.799003+0000","last_became_active":"2024-09-17T03:28:44.808167+0000","last_became_peered":"2024-09-17T03:28:44.808167+0000","last_unstale":"2024-09-17T03:28:45.799003+0000","last_undegraded":"2024-09-17T03:28:45.799003+0000","last_fullsized":"2024-09-17T03:28:45.799003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:30.309465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786333+0000","last_change":"2024-09-17T03:28:45.813876+0000","last_active":"2024-09-17T03:28:46.786333+0000","last_peered":"2024-09-17T03:28:46.786333+0000","last_clean":"2024-09-17T03:28:46.786333+0000","last_became_active":"2024-09-17T03:28:45.813458+0000","last_became_peered":"2024-09-17T03:28:45.813458+0000","last_unstale":"2024-09-17T03:28:46.786333+0000","last_undegraded":"2024-09-17T03:28:46.786333+0000","last_fullsized":"2024-09-17T03:28:46.786333+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:10.364569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.831248+0000","last_change":"2024-09-17T03:28:44.808179+0000","last_active":"2024-09-17T03:28:45.831248+0000","last_peered":"2024-09-17T03:28:45.831248+0000","last_clean":"2024-09-17T03:28:45.831248+0000","last_became_active":"2024-09-17T03:28:44.805931+0000","last_became_peered":"2024-09-17T03:28:44.805931+0000","last_unstale":"2024-09-17T03:28:45.831248+0000","last_undegraded":"2024-09-17T03:28:45.831248+0000","last_fullsized":"2024-09-17T03:28:45.831248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:31.479083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":203,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.795715+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:28:45.795715+0000","last_peered":"2024-09-17T03:28:45.795715+0000","last_clean":"2024-09-17T03:28:45.795715+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:28:45.795715+0000","last_undegraded":"2024-09-17T03:28:45.795715+0000","last_fullsized":"2024-09-17T03:28:45.795715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1458176,"data_stored":951338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":864256,"data_stored":360446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509537,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54240,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317600,"statfs":{"total":96636764160,"available":96581222400,"internally_reserved":0,"allocated":860160,"data_stored":360424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58404,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313436,"statfs":{"total":96636764160,"available":96576958464,"internally_reserved":0,"allocated":864256,"data_stored":360466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":868352,"data_stored":361966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509537,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58980,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312860,"statfs":{"total":96636764160,"available":96576368640,"internally_reserved":0,"allocated":1449984,"data_stored":950722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542241,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":843776,"data_stored":353579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58380,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313460,"statfs":{"total":96636764160,"available":96576983040,"internally_reserved":0,"allocated":843776,"data_stored":353579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:28:49.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:28:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 -- 172.21.15.46:0/1342756511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc157030 msgr2=0x7f3afc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 --2- 172.21.15.46:0/1342756511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc157030 0x7f3afc0b5a80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0b6a50 tx=0x7f3aec01c830 comp rx=0 tx=0).stop 2024-09-17T03:28:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 -- 172.21.15.46:0/1342756511 shutdown_connections 2024-09-17T03:28:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 --2- 172.21.15.46:0/1342756511 >> v2:172.21.15.99:3300/0 conn(0x7f3afc074b20 0x7f3afc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 --2- 172.21.15.46:0/1342756511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc157030 0x7f3afc0b5a80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 -- 172.21.15.46:0/1342756511 >> 172.21.15.46:0/1342756511 conn(0x7f3afc0708c0 msgr2=0x7f3afc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 -- 172.21.15.46:0/1342756511 shutdown_connections 2024-09-17T03:28:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.186+0000 7f3b02600640 1 -- 172.21.15.46:0/1342756511 wait complete. 2024-09-17T03:28:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 Processor -- start 2024-09-17T03:28:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 -- start start 2024-09-17T03:28:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc157030 0x7f3afc14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 0x7f3afc154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3afc0b4510 0x7f3afc1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc16a740 con 0x7f3afc160d50 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc16a5c0 con 0x7f3afc157030 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3afc16a8c0 con 0x7f3afc0b4510 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 0x7f3afc154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 0x7f3afc154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53372/0 (socket says 172.21.15.46:53372) 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 -- 172.21.15.46:0/2889669178 learned_addr learned my addr 172.21.15.46:0/2889669178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3afb400640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc157030 0x7f3afc14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3afbe00640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3afc0b4510 0x7f3afc1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 -- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3afc0b4510 msgr2=0x7f3afc1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3afc0b4510 0x7f3afc1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 -- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc157030 msgr2=0x7f3afc14f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc157030 0x7f3afc14f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 -- 172.21.15.46:0/2889669178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc0b2470 con 0x7f3afc160d50 2024-09-17T03:28:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3e00640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 0x7f3afc154470 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c3f0 tx=0x7f3ae400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4017070 con 0x7f3afc160d50 2024-09-17T03:28:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ae40053c0 con 0x7f3afc160d50 2024-09-17T03:28:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc14a370 con 0x7f3afc160d50 2024-09-17T03:28:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4003860 con 0x7f3afc160d50 2024-09-17T03:28:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.190+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc14a650 con 0x7f3afc160d50 2024-09-17T03:28:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.194+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3afc15cf90 con 0x7f3afc160d50 2024-09-17T03:28:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.194+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f3ae401a070 con 0x7f3afc160d50 2024-09-17T03:28:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.194+0000 7f3af3400640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ad8073ef0 0x7f3ad8076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.194+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae4091e30 con 0x7f3afc160d50 2024-09-17T03:28:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.194+0000 7f3afb400640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ad8073ef0 0x7f3ad8076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.194+0000 7f3afb400640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ad8073ef0 0x7f3ad8076400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0af000 tx=0x7f3aec01b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.198+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ae405e2e0 con 0x7f3afc160d50 2024-09-17T03:28:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.370+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3afc1fb6b0 con 0x7f3ad8073ef0 2024-09-17T03:28:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.374+0000 7f3af3400640 1 -- 172.21.15.46:0/2889669178 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252052 (secure 0 0 0) 0x7f3afc1fb6b0 con 0x7f3ad8073ef0 2024-09-17T03:28:50.381 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ad8073ef0 msgr2=0x7f3ad8076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ad8073ef0 0x7f3ad8076400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3aec0af000 tx=0x7f3aec01b070 comp rx=0 tx=0).stop 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 msgr2=0x7f3afc154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 0x7f3afc154470 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c3f0 tx=0x7f3ae400c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 shutdown_connections 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ad8073ef0 0x7f3ad8076400 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3afc0b4510 0x7f3afc1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3afc160d50 0x7f3afc154470 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 --2- 172.21.15.46:0/2889669178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc157030 0x7f3afc14f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 >> 172.21.15.46:0/2889669178 conn(0x7f3afc0708c0 msgr2=0x7f3afc14cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 shutdown_connections 2024-09-17T03:28:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.378+0000 7f3b02600640 1 -- 172.21.15.46:0/2889669178 wait complete. 2024-09-17T03:28:50.403 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-17T03:28:49.345998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430904,"kb_used_data":7864,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754543816,"statfs":{"total":773094113280,"available":772652867584,"internally_reserved":0,"allocated":8052736,"data_stored":4052520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.580512"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.830131+0000","last_change":"2024-09-17T03:28:45.827198+0000","last_active":"2024-09-17T03:28:45.830131+0000","last_peered":"2024-09-17T03:28:45.830131+0000","last_clean":"2024-09-17T03:28:45.830131+0000","last_became_active":"2024-09-17T03:28:45.826864+0000","last_became_peered":"2024-09-17T03:28:45.826864+0000","last_unstale":"2024-09-17T03:28:45.830131+0000","last_undegraded":"2024-09-17T03:28:45.830131+0000","last_fullsized":"2024-09-17T03:28:45.830131+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:18.477715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.813612+0000","last_change":"2024-09-17T03:28:45.810443+0000","last_active":"2024-09-17T03:28:45.813612+0000","last_peered":"2024-09-17T03:28:45.813612+0000","last_clean":"2024-09-17T03:28:45.813612+0000","last_became_active":"2024-09-17T03:28:45.810008+0000","last_became_peered":"2024-09-17T03:28:45.810008+0000","last_unstale":"2024-09-17T03:28:45.813612+0000","last_undegraded":"2024-09-17T03:28:45.813612+0000","last_fullsized":"2024-09-17T03:28:45.813612+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:16.225788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786257+0000","last_change":"2024-09-17T03:28:45.810022+0000","last_active":"2024-09-17T03:28:46.786257+0000","last_peered":"2024-09-17T03:28:46.786257+0000","last_clean":"2024-09-17T03:28:46.786257+0000","last_became_active":"2024-09-17T03:28:45.809662+0000","last_became_peered":"2024-09-17T03:28:45.809662+0000","last_unstale":"2024-09-17T03:28:46.786257+0000","last_undegraded":"2024-09-17T03:28:46.786257+0000","last_fullsized":"2024-09-17T03:28:46.786257+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:54.150105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816283+0000","last_change":"2024-09-17T03:28:45.814391+0000","last_active":"2024-09-17T03:28:45.816283+0000","last_peered":"2024-09-17T03:28:45.816283+0000","last_clean":"2024-09-17T03:28:45.816283+0000","last_became_active":"2024-09-17T03:28:45.814050+0000","last_became_peered":"2024-09-17T03:28:45.814050+0000","last_unstale":"2024-09-17T03:28:45.816283+0000","last_undegraded":"2024-09-17T03:28:45.816283+0000","last_fullsized":"2024-09-17T03:28:45.816283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:31.338300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.813913+0000","last_change":"2024-09-17T03:28:45.802646+0000","last_active":"2024-09-17T03:28:45.813913+0000","last_peered":"2024-09-17T03:28:45.813913+0000","last_clean":"2024-09-17T03:28:45.813913+0000","last_became_active":"2024-09-17T03:28:45.802278+0000","last_became_peered":"2024-09-17T03:28:45.802278+0000","last_unstale":"2024-09-17T03:28:45.813913+0000","last_undegraded":"2024-09-17T03:28:45.813913+0000","last_fullsized":"2024-09-17T03:28:45.813913+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:12.413427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819088+0000","last_change":"2024-09-17T03:28:45.814489+0000","last_active":"2024-09-17T03:28:45.819088+0000","last_peered":"2024-09-17T03:28:45.819088+0000","last_clean":"2024-09-17T03:28:45.819088+0000","last_became_active":"2024-09-17T03:28:45.813244+0000","last_became_peered":"2024-09-17T03:28:45.813244+0000","last_unstale":"2024-09-17T03:28:45.819088+0000","last_undegraded":"2024-09-17T03:28:45.819088+0000","last_fullsized":"2024-09-17T03:28:45.819088+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:34.423661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.811644+0000","last_change":"2024-09-17T03:28:45.806956+0000","last_active":"2024-09-17T03:28:45.811644+0000","last_peered":"2024-09-17T03:28:45.811644+0000","last_clean":"2024-09-17T03:28:45.811644+0000","last_became_active":"2024-09-17T03:28:45.806641+0000","last_became_peered":"2024-09-17T03:28:45.806641+0000","last_unstale":"2024-09-17T03:28:45.811644+0000","last_undegraded":"2024-09-17T03:28:45.811644+0000","last_fullsized":"2024-09-17T03:28:45.811644+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:38.173247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816674+0000","last_change":"2024-09-17T03:28:45.811796+0000","last_active":"2024-09-17T03:28:45.816674+0000","last_peered":"2024-09-17T03:28:45.816674+0000","last_clean":"2024-09-17T03:28:45.816674+0000","last_became_active":"2024-09-17T03:28:45.811485+0000","last_became_peered":"2024-09-17T03:28:45.811485+0000","last_unstale":"2024-09-17T03:28:45.816674+0000","last_undegraded":"2024-09-17T03:28:45.816674+0000","last_fullsized":"2024-09-17T03:28:45.816674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:55.233796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.823726+0000","last_change":"2024-09-17T03:28:45.819619+0000","last_active":"2024-09-17T03:28:45.823726+0000","last_peered":"2024-09-17T03:28:45.823726+0000","last_clean":"2024-09-17T03:28:45.823726+0000","last_became_active":"2024-09-17T03:28:45.818422+0000","last_became_peered":"2024-09-17T03:28:45.818422+0000","last_unstale":"2024-09-17T03:28:45.823726+0000","last_undegraded":"2024-09-17T03:28:45.823726+0000","last_fullsized":"2024-09-17T03:28:45.823726+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:32.889342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793590+0000","last_change":"2024-09-17T03:28:45.818733+0000","last_active":"2024-09-17T03:28:46.793590+0000","last_peered":"2024-09-17T03:28:46.793590+0000","last_clean":"2024-09-17T03:28:46.793590+0000","last_became_active":"2024-09-17T03:28:45.818432+0000","last_became_peered":"2024-09-17T03:28:45.818432+0000","last_unstale":"2024-09-17T03:28:46.793590+0000","last_undegraded":"2024-09-17T03:28:46.793590+0000","last_fullsized":"2024-09-17T03:28:46.793590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:00.725496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.809480+0000","last_change":"2024-09-17T03:28:45.803215+0000","last_active":"2024-09-17T03:28:45.809480+0000","last_peered":"2024-09-17T03:28:45.809480+0000","last_clean":"2024-09-17T03:28:45.809480+0000","last_became_active":"2024-09-17T03:28:45.802851+0000","last_became_peered":"2024-09-17T03:28:45.802851+0000","last_unstale":"2024-09-17T03:28:45.809480+0000","last_undegraded":"2024-09-17T03:28:45.809480+0000","last_fullsized":"2024-09-17T03:28:45.809480+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:55.456639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812891+0000","last_change":"2024-09-17T03:28:45.808131+0000","last_active":"2024-09-17T03:28:45.812891+0000","last_peered":"2024-09-17T03:28:45.812891+0000","last_clean":"2024-09-17T03:28:45.812891+0000","last_became_active":"2024-09-17T03:28:45.807528+0000","last_became_peered":"2024-09-17T03:28:45.807528+0000","last_unstale":"2024-09-17T03:28:45.812891+0000","last_undegraded":"2024-09-17T03:28:45.812891+0000","last_fullsized":"2024-09-17T03:28:45.812891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:15.280804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810575+0000","last_change":"2024-09-17T03:28:45.807003+0000","last_active":"2024-09-17T03:28:46.810575+0000","last_peered":"2024-09-17T03:28:46.810575+0000","last_clean":"2024-09-17T03:28:46.810575+0000","last_became_active":"2024-09-17T03:28:45.806544+0000","last_became_peered":"2024-09-17T03:28:45.806544+0000","last_unstale":"2024-09-17T03:28:46.810575+0000","last_undegraded":"2024-09-17T03:28:46.810575+0000","last_fullsized":"2024-09-17T03:28:46.810575+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:07.918094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.817403+0000","last_change":"2024-09-17T03:28:45.807240+0000","last_active":"2024-09-17T03:28:45.817403+0000","last_peered":"2024-09-17T03:28:45.817403+0000","last_clean":"2024-09-17T03:28:45.817403+0000","last_became_active":"2024-09-17T03:28:45.806908+0000","last_became_peered":"2024-09-17T03:28:45.806908+0000","last_unstale":"2024-09-17T03:28:45.817403+0000","last_undegraded":"2024-09-17T03:28:45.817403+0000","last_fullsized":"2024-09-17T03:28:45.817403+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:55.797038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810391+0000","last_change":"2024-09-17T03:28:45.809314+0000","last_active":"2024-09-17T03:28:46.810391+0000","last_peered":"2024-09-17T03:28:46.810391+0000","last_clean":"2024-09-17T03:28:46.810391+0000","last_became_active":"2024-09-17T03:28:45.809007+0000","last_became_peered":"2024-09-17T03:28:45.809007+0000","last_unstale":"2024-09-17T03:28:46.810391+0000","last_undegraded":"2024-09-17T03:28:46.810391+0000","last_fullsized":"2024-09-17T03:28:46.810391+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:16.504679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.792967+0000","last_change":"2024-09-17T03:28:45.815770+0000","last_active":"2024-09-17T03:28:46.792967+0000","last_peered":"2024-09-17T03:28:46.792967+0000","last_clean":"2024-09-17T03:28:46.792967+0000","last_became_active":"2024-09-17T03:28:45.815447+0000","last_became_peered":"2024-09-17T03:28:45.815447+0000","last_unstale":"2024-09-17T03:28:46.792967+0000","last_undegraded":"2024-09-17T03:28:46.792967+0000","last_fullsized":"2024-09-17T03:28:46.792967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:02.877893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.817462+0000","last_change":"2024-09-17T03:28:45.810511+0000","last_active":"2024-09-17T03:28:45.817462+0000","last_peered":"2024-09-17T03:28:45.817462+0000","last_clean":"2024-09-17T03:28:45.817462+0000","last_became_active":"2024-09-17T03:28:45.810028+0000","last_became_peered":"2024-09-17T03:28:45.810028+0000","last_unstale":"2024-09-17T03:28:45.817462+0000","last_undegraded":"2024-09-17T03:28:45.817462+0000","last_fullsized":"2024-09-17T03:28:45.817462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:37.200674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816635+0000","last_change":"2024-09-17T03:28:45.815407+0000","last_active":"2024-09-17T03:28:45.816635+0000","last_peered":"2024-09-17T03:28:45.816635+0000","last_clean":"2024-09-17T03:28:45.816635+0000","last_became_active":"2024-09-17T03:28:45.813453+0000","last_became_peered":"2024-09-17T03:28:45.813453+0000","last_unstale":"2024-09-17T03:28:45.816635+0000","last_undegraded":"2024-09-17T03:28:45.816635+0000","last_fullsized":"2024-09-17T03:28:45.816635+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:25.249050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.814248+0000","last_change":"2024-09-17T03:28:45.810588+0000","last_active":"2024-09-17T03:28:45.814248+0000","last_peered":"2024-09-17T03:28:45.814248+0000","last_clean":"2024-09-17T03:28:45.814248+0000","last_became_active":"2024-09-17T03:28:45.810147+0000","last_became_peered":"2024-09-17T03:28:45.810147+0000","last_unstale":"2024-09-17T03:28:45.814248+0000","last_undegraded":"2024-09-17T03:28:45.814248+0000","last_fullsized":"2024-09-17T03:28:45.814248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:16.221026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810531+0000","last_change":"2024-09-17T03:28:45.809822+0000","last_active":"2024-09-17T03:28:46.810531+0000","last_peered":"2024-09-17T03:28:46.810531+0000","last_clean":"2024-09-17T03:28:46.810531+0000","last_became_active":"2024-09-17T03:28:45.807847+0000","last_became_peered":"2024-09-17T03:28:45.807847+0000","last_unstale":"2024-09-17T03:28:46.810531+0000","last_undegraded":"2024-09-17T03:28:46.810531+0000","last_fullsized":"2024-09-17T03:28:46.810531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:49.878675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793103+0000","last_change":"2024-09-17T03:28:45.810834+0000","last_active":"2024-09-17T03:28:46.793103+0000","last_peered":"2024-09-17T03:28:46.793103+0000","last_clean":"2024-09-17T03:28:46.793103+0000","last_became_active":"2024-09-17T03:28:45.810461+0000","last_became_peered":"2024-09-17T03:28:45.810461+0000","last_unstale":"2024-09-17T03:28:46.793103+0000","last_undegraded":"2024-09-17T03:28:46.793103+0000","last_fullsized":"2024-09-17T03:28:46.793103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:36.085989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793618+0000","last_change":"2024-09-17T03:28:45.819925+0000","last_active":"2024-09-17T03:28:46.793618+0000","last_peered":"2024-09-17T03:28:46.793618+0000","last_clean":"2024-09-17T03:28:46.793618+0000","last_became_active":"2024-09-17T03:28:45.817803+0000","last_became_peered":"2024-09-17T03:28:45.817803+0000","last_unstale":"2024-09-17T03:28:46.793618+0000","last_undegraded":"2024-09-17T03:28:46.793618+0000","last_fullsized":"2024-09-17T03:28:46.793618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:13.240432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.818719+0000","last_change":"2024-09-17T03:28:45.813660+0000","last_active":"2024-09-17T03:28:45.818719+0000","last_peered":"2024-09-17T03:28:45.818719+0000","last_clean":"2024-09-17T03:28:45.818719+0000","last_became_active":"2024-09-17T03:28:45.813291+0000","last_became_peered":"2024-09-17T03:28:45.813291+0000","last_unstale":"2024-09-17T03:28:45.818719+0000","last_undegraded":"2024-09-17T03:28:45.818719+0000","last_fullsized":"2024-09-17T03:28:45.818719+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:10.437453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819425+0000","last_change":"2024-09-17T03:28:45.815494+0000","last_active":"2024-09-17T03:28:45.819425+0000","last_peered":"2024-09-17T03:28:45.819425+0000","last_clean":"2024-09-17T03:28:45.819425+0000","last_became_active":"2024-09-17T03:28:45.815081+0000","last_became_peered":"2024-09-17T03:28:45.815081+0000","last_unstale":"2024-09-17T03:28:45.819425+0000","last_undegraded":"2024-09-17T03:28:45.819425+0000","last_fullsized":"2024-09-17T03:28:45.819425+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:02.031516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793547+0000","last_change":"2024-09-17T03:28:45.804852+0000","last_active":"2024-09-17T03:28:46.793547+0000","last_peered":"2024-09-17T03:28:46.793547+0000","last_clean":"2024-09-17T03:28:46.793547+0000","last_became_active":"2024-09-17T03:28:45.800852+0000","last_became_peered":"2024-09-17T03:28:45.800852+0000","last_unstale":"2024-09-17T03:28:46.793547+0000","last_undegraded":"2024-09-17T03:28:46.793547+0000","last_fullsized":"2024-09-17T03:28:46.793547+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:56.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.823923+0000","last_change":"2024-09-17T03:28:45.822985+0000","last_active":"2024-09-17T03:28:45.823923+0000","last_peered":"2024-09-17T03:28:45.823923+0000","last_clean":"2024-09-17T03:28:45.823923+0000","last_became_active":"2024-09-17T03:28:45.819193+0000","last_became_peered":"2024-09-17T03:28:45.819193+0000","last_unstale":"2024-09-17T03:28:45.823923+0000","last_undegraded":"2024-09-17T03:28:45.823923+0000","last_fullsized":"2024-09-17T03:28:45.823923+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:28:17.781870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786307+0000","last_change":"2024-09-17T03:28:45.798497+0000","last_active":"2024-09-17T03:28:46.786307+0000","last_peered":"2024-09-17T03:28:46.786307+0000","last_clean":"2024-09-17T03:28:46.786307+0000","last_became_active":"2024-09-17T03:28:45.797746+0000","last_became_peered":"2024-09-17T03:28:45.797746+0000","last_unstale":"2024-09-17T03:28:46.786307+0000","last_undegraded":"2024-09-17T03:28:46.786307+0000","last_fullsized":"2024-09-17T03:28:46.786307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:46.414153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793465+0000","last_change":"2024-09-17T03:28:45.813599+0000","last_active":"2024-09-17T03:28:46.793465+0000","last_peered":"2024-09-17T03:28:46.793465+0000","last_clean":"2024-09-17T03:28:46.793465+0000","last_became_active":"2024-09-17T03:28:45.813265+0000","last_became_peered":"2024-09-17T03:28:45.813265+0000","last_unstale":"2024-09-17T03:28:46.793465+0000","last_undegraded":"2024-09-17T03:28:46.793465+0000","last_fullsized":"2024-09-17T03:28:46.793465+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:06.373341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786176+0000","last_change":"2024-09-17T03:28:45.794834+0000","last_active":"2024-09-17T03:28:46.786176+0000","last_peered":"2024-09-17T03:28:46.786176+0000","last_clean":"2024-09-17T03:28:46.786176+0000","last_became_active":"2024-09-17T03:28:45.794542+0000","last_became_peered":"2024-09-17T03:28:45.794542+0000","last_unstale":"2024-09-17T03:28:46.786176+0000","last_undegraded":"2024-09-17T03:28:46.786176+0000","last_fullsized":"2024-09-17T03:28:46.786176+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:07.656159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.821847+0000","last_change":"2024-09-17T03:28:45.819685+0000","last_active":"2024-09-17T03:28:45.821847+0000","last_peered":"2024-09-17T03:28:45.821847+0000","last_clean":"2024-09-17T03:28:45.821847+0000","last_became_active":"2024-09-17T03:28:45.819024+0000","last_became_peered":"2024-09-17T03:28:45.819024+0000","last_unstale":"2024-09-17T03:28:45.821847+0000","last_undegraded":"2024-09-17T03:28:45.821847+0000","last_fullsized":"2024-09-17T03:28:45.821847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:14.645626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.818112+0000","last_change":"2024-09-17T03:28:45.816828+0000","last_active":"2024-09-17T03:28:45.818112+0000","last_peered":"2024-09-17T03:28:45.818112+0000","last_clean":"2024-09-17T03:28:45.818112+0000","last_became_active":"2024-09-17T03:28:45.814383+0000","last_became_peered":"2024-09-17T03:28:45.814383+0000","last_unstale":"2024-09-17T03:28:45.818112+0000","last_undegraded":"2024-09-17T03:28:45.818112+0000","last_fullsized":"2024-09-17T03:28:45.818112+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:41.667034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.815866+0000","last_change":"2024-09-17T03:28:45.812548+0000","last_active":"2024-09-17T03:28:45.815866+0000","last_peered":"2024-09-17T03:28:45.815866+0000","last_clean":"2024-09-17T03:28:45.815866+0000","last_became_active":"2024-09-17T03:28:45.812213+0000","last_became_peered":"2024-09-17T03:28:45.812213+0000","last_unstale":"2024-09-17T03:28:45.815866+0000","last_undegraded":"2024-09-17T03:28:45.815866+0000","last_fullsized":"2024-09-17T03:28:45.815866+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:54.671107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819691+0000","last_change":"2024-09-17T03:28:45.815407+0000","last_active":"2024-09-17T03:28:45.819691+0000","last_peered":"2024-09-17T03:28:45.819691+0000","last_clean":"2024-09-17T03:28:45.819691+0000","last_became_active":"2024-09-17T03:28:45.815016+0000","last_became_peered":"2024-09-17T03:28:45.815016+0000","last_unstale":"2024-09-17T03:28:45.819691+0000","last_undegraded":"2024-09-17T03:28:45.819691+0000","last_fullsized":"2024-09-17T03:28:45.819691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.285014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.833264+0000","last_change":"2024-09-17T03:28:44.822691+0000","last_active":"2024-09-17T03:28:45.833264+0000","last_peered":"2024-09-17T03:28:45.833264+0000","last_clean":"2024-09-17T03:28:45.833264+0000","last_became_active":"2024-09-17T03:28:44.822375+0000","last_became_peered":"2024-09-17T03:28:44.822375+0000","last_unstale":"2024-09-17T03:28:45.833264+0000","last_undegraded":"2024-09-17T03:28:45.833264+0000","last_fullsized":"2024-09-17T03:28:45.833264+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:05.252640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.823697+0000","last_change":"2024-09-17T03:28:45.815695+0000","last_active":"2024-09-17T03:28:45.823697+0000","last_peered":"2024-09-17T03:28:45.823697+0000","last_clean":"2024-09-17T03:28:45.823697+0000","last_became_active":"2024-09-17T03:28:45.805928+0000","last_became_peered":"2024-09-17T03:28:45.805928+0000","last_unstale":"2024-09-17T03:28:45.823697+0000","last_undegraded":"2024-09-17T03:28:45.823697+0000","last_fullsized":"2024-09-17T03:28:45.823697+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:36.215624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810851+0000","last_change":"2024-09-17T03:28:44.794991+0000","last_active":"2024-09-17T03:28:46.810851+0000","last_peered":"2024-09-17T03:28:46.810851+0000","last_clean":"2024-09-17T03:28:46.810851+0000","last_became_active":"2024-09-17T03:28:44.794572+0000","last_became_peered":"2024-09-17T03:28:44.794572+0000","last_unstale":"2024-09-17T03:28:46.810851+0000","last_undegraded":"2024-09-17T03:28:46.810851+0000","last_fullsized":"2024-09-17T03:28:46.810851+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:57.701295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819181+0000","last_change":"2024-09-17T03:28:45.815390+0000","last_active":"2024-09-17T03:28:45.819181+0000","last_peered":"2024-09-17T03:28:45.819181+0000","last_clean":"2024-09-17T03:28:45.819181+0000","last_became_active":"2024-09-17T03:28:45.814121+0000","last_became_peered":"2024-09-17T03:28:45.814121+0000","last_unstale":"2024-09-17T03:28:45.819181+0000","last_undegraded":"2024-09-17T03:28:45.819181+0000","last_fullsized":"2024-09-17T03:28:45.819181+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:09.172192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793328+0000","last_change":"2024-09-17T03:28:44.809278+0000","last_active":"2024-09-17T03:28:46.793328+0000","last_peered":"2024-09-17T03:28:46.793328+0000","last_clean":"2024-09-17T03:28:46.793328+0000","last_became_active":"2024-09-17T03:28:44.808830+0000","last_became_peered":"2024-09-17T03:28:44.808830+0000","last_unstale":"2024-09-17T03:28:46.793328+0000","last_undegraded":"2024-09-17T03:28:46.793328+0000","last_fullsized":"2024-09-17T03:28:46.793328+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:23.662069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812771+0000","last_change":"2024-09-17T03:28:45.810066+0000","last_active":"2024-09-17T03:28:45.812771+0000","last_peered":"2024-09-17T03:28:45.812771+0000","last_clean":"2024-09-17T03:28:45.812771+0000","last_became_active":"2024-09-17T03:28:45.809733+0000","last_became_peered":"2024-09-17T03:28:45.809733+0000","last_unstale":"2024-09-17T03:28:45.812771+0000","last_undegraded":"2024-09-17T03:28:45.812771+0000","last_fullsized":"2024-09-17T03:28:45.812771+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:21.083296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.833166+0000","last_change":"2024-09-17T03:28:44.812929+0000","last_active":"2024-09-17T03:28:45.833166+0000","last_peered":"2024-09-17T03:28:45.833166+0000","last_clean":"2024-09-17T03:28:45.833166+0000","last_became_active":"2024-09-17T03:28:44.807116+0000","last_became_peered":"2024-09-17T03:28:44.807116+0000","last_unstale":"2024-09-17T03:28:45.833166+0000","last_undegraded":"2024-09-17T03:28:45.833166+0000","last_fullsized":"2024-09-17T03:28:45.833166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:02.724293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.811927+0000","last_change":"2024-09-17T03:28:45.802845+0000","last_active":"2024-09-17T03:28:45.811927+0000","last_peered":"2024-09-17T03:28:45.811927+0000","last_clean":"2024-09-17T03:28:45.811927+0000","last_became_active":"2024-09-17T03:28:45.802495+0000","last_became_peered":"2024-09-17T03:28:45.802495+0000","last_unstale":"2024-09-17T03:28:45.811927+0000","last_undegraded":"2024-09-17T03:28:45.811927+0000","last_fullsized":"2024-09-17T03:28:45.811927+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:47.634063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786310+0000","last_change":"2024-09-17T03:28:44.797569+0000","last_active":"2024-09-17T03:28:46.786310+0000","last_peered":"2024-09-17T03:28:46.786310+0000","last_clean":"2024-09-17T03:28:46.786310+0000","last_became_active":"2024-09-17T03:28:44.797089+0000","last_became_peered":"2024-09-17T03:28:44.797089+0000","last_unstale":"2024-09-17T03:28:46.786310+0000","last_undegraded":"2024-09-17T03:28:46.786310+0000","last_fullsized":"2024-09-17T03:28:46.786310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:57.895718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793120+0000","last_change":"2024-09-17T03:28:45.803393+0000","last_active":"2024-09-17T03:28:46.793120+0000","last_peered":"2024-09-17T03:28:46.793120+0000","last_clean":"2024-09-17T03:28:46.793120+0000","last_became_active":"2024-09-17T03:28:45.801974+0000","last_became_peered":"2024-09-17T03:28:45.801974+0000","last_unstale":"2024-09-17T03:28:46.793120+0000","last_undegraded":"2024-09-17T03:28:46.793120+0000","last_fullsized":"2024-09-17T03:28:46.793120+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:00.513379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.828446+0000","last_change":"2024-09-17T03:28:44.810924+0000","last_active":"2024-09-17T03:28:45.828446+0000","last_peered":"2024-09-17T03:28:45.828446+0000","last_clean":"2024-09-17T03:28:45.828446+0000","last_became_active":"2024-09-17T03:28:44.810597+0000","last_became_peered":"2024-09-17T03:28:44.810597+0000","last_unstale":"2024-09-17T03:28:45.828446+0000","last_undegraded":"2024-09-17T03:28:45.828446+0000","last_fullsized":"2024-09-17T03:28:45.828446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:04.869978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816175+0000","last_change":"2024-09-17T03:28:45.804065+0000","last_active":"2024-09-17T03:28:45.816175+0000","last_peered":"2024-09-17T03:28:45.816175+0000","last_clean":"2024-09-17T03:28:45.816175+0000","last_became_active":"2024-09-17T03:28:45.802283+0000","last_became_peered":"2024-09-17T03:28:45.802283+0000","last_unstale":"2024-09-17T03:28:45.816175+0000","last_undegraded":"2024-09-17T03:28:45.816175+0000","last_fullsized":"2024-09-17T03:28:45.816175+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:05.676121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.781475+0000","last_change":"2024-09-17T03:28:44.798788+0000","last_active":"2024-09-17T03:28:45.781475+0000","last_peered":"2024-09-17T03:28:45.781475+0000","last_clean":"2024-09-17T03:28:45.781475+0000","last_became_active":"2024-09-17T03:28:44.798431+0000","last_became_peered":"2024-09-17T03:28:44.798431+0000","last_unstale":"2024-09-17T03:28:45.781475+0000","last_undegraded":"2024-09-17T03:28:45.781475+0000","last_fullsized":"2024-09-17T03:28:45.781475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:35.748400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810618+0000","last_change":"2024-09-17T03:28:45.802762+0000","last_active":"2024-09-17T03:28:46.810618+0000","last_peered":"2024-09-17T03:28:46.810618+0000","last_clean":"2024-09-17T03:28:46.810618+0000","last_became_active":"2024-09-17T03:28:45.802326+0000","last_became_peered":"2024-09-17T03:28:45.802326+0000","last_unstale":"2024-09-17T03:28:46.810618+0000","last_undegraded":"2024-09-17T03:28:46.810618+0000","last_fullsized":"2024-09-17T03:28:46.810618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:33.916277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.797928+0000","last_change":"2024-09-17T03:28:44.801618+0000","last_active":"2024-09-17T03:28:45.797928+0000","last_peered":"2024-09-17T03:28:45.797928+0000","last_clean":"2024-09-17T03:28:45.797928+0000","last_became_active":"2024-09-17T03:28:44.793440+0000","last_became_peered":"2024-09-17T03:28:44.793440+0000","last_unstale":"2024-09-17T03:28:45.797928+0000","last_undegraded":"2024-09-17T03:28:45.797928+0000","last_fullsized":"2024-09-17T03:28:45.797928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:23.363356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812445+0000","last_change":"2024-09-17T03:28:45.801406+0000","last_active":"2024-09-17T03:28:45.812445+0000","last_peered":"2024-09-17T03:28:45.812445+0000","last_clean":"2024-09-17T03:28:45.812445+0000","last_became_active":"2024-09-17T03:28:45.801000+0000","last_became_peered":"2024-09-17T03:28:45.801000+0000","last_unstale":"2024-09-17T03:28:45.812445+0000","last_undegraded":"2024-09-17T03:28:45.812445+0000","last_fullsized":"2024-09-17T03:28:45.812445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:51.282149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793687+0000","last_change":"2024-09-17T03:28:44.803091+0000","last_active":"2024-09-17T03:28:46.793687+0000","last_peered":"2024-09-17T03:28:46.793687+0000","last_clean":"2024-09-17T03:28:46.793687+0000","last_became_active":"2024-09-17T03:28:44.802150+0000","last_became_peered":"2024-09-17T03:28:44.802150+0000","last_unstale":"2024-09-17T03:28:46.793687+0000","last_undegraded":"2024-09-17T03:28:46.793687+0000","last_fullsized":"2024-09-17T03:28:46.793687+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:12.508649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819691+0000","last_change":"2024-09-17T03:28:45.817735+0000","last_active":"2024-09-17T03:28:45.819691+0000","last_peered":"2024-09-17T03:28:45.819691+0000","last_clean":"2024-09-17T03:28:45.819691+0000","last_became_active":"2024-09-17T03:28:45.817435+0000","last_became_peered":"2024-09-17T03:28:45.817435+0000","last_unstale":"2024-09-17T03:28:45.819691+0000","last_undegraded":"2024-09-17T03:28:45.819691+0000","last_fullsized":"2024-09-17T03:28:45.819691+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:22.779750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.799042+0000","last_change":"2024-09-17T03:28:44.813845+0000","last_active":"2024-09-17T03:28:45.799042+0000","last_peered":"2024-09-17T03:28:45.799042+0000","last_clean":"2024-09-17T03:28:45.799042+0000","last_became_active":"2024-09-17T03:28:44.813471+0000","last_became_peered":"2024-09-17T03:28:44.813471+0000","last_unstale":"2024-09-17T03:28:45.799042+0000","last_undegraded":"2024-09-17T03:28:45.799042+0000","last_fullsized":"2024-09-17T03:28:45.799042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:23.668468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819149+0000","last_change":"2024-09-17T03:28:45.814656+0000","last_active":"2024-09-17T03:28:45.819149+0000","last_peered":"2024-09-17T03:28:45.819149+0000","last_clean":"2024-09-17T03:28:45.819149+0000","last_became_active":"2024-09-17T03:28:45.813774+0000","last_became_peered":"2024-09-17T03:28:45.813774+0000","last_unstale":"2024-09-17T03:28:45.819149+0000","last_undegraded":"2024-09-17T03:28:45.819149+0000","last_fullsized":"2024-09-17T03:28:45.819149+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:16.920106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786419+0000","last_change":"2024-09-17T03:28:44.799439+0000","last_active":"2024-09-17T03:28:46.786419+0000","last_peered":"2024-09-17T03:28:46.786419+0000","last_clean":"2024-09-17T03:28:46.786419+0000","last_became_active":"2024-09-17T03:28:44.798699+0000","last_became_peered":"2024-09-17T03:28:44.798699+0000","last_unstale":"2024-09-17T03:28:46.786419+0000","last_undegraded":"2024-09-17T03:28:46.786419+0000","last_fullsized":"2024-09-17T03:28:46.786419+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:30.801287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812458+0000","last_change":"2024-09-17T03:28:45.805729+0000","last_active":"2024-09-17T03:28:45.812458+0000","last_peered":"2024-09-17T03:28:45.812458+0000","last_clean":"2024-09-17T03:28:45.812458+0000","last_became_active":"2024-09-17T03:28:45.805339+0000","last_became_peered":"2024-09-17T03:28:45.805339+0000","last_unstale":"2024-09-17T03:28:45.812458+0000","last_undegraded":"2024-09-17T03:28:45.812458+0000","last_fullsized":"2024-09-17T03:28:45.812458+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:48.246592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.826493+0000","last_change":"2024-09-17T03:28:44.802285+0000","last_active":"2024-09-17T03:28:45.826493+0000","last_peered":"2024-09-17T03:28:45.826493+0000","last_clean":"2024-09-17T03:28:45.826493+0000","last_became_active":"2024-09-17T03:28:44.800671+0000","last_became_peered":"2024-09-17T03:28:44.800671+0000","last_unstale":"2024-09-17T03:28:45.826493+0000","last_undegraded":"2024-09-17T03:28:45.826493+0000","last_fullsized":"2024-09-17T03:28:45.826493+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:34.008579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819485+0000","last_change":"2024-09-17T03:28:45.816753+0000","last_active":"2024-09-17T03:28:45.819485+0000","last_peered":"2024-09-17T03:28:45.819485+0000","last_clean":"2024-09-17T03:28:45.819485+0000","last_became_active":"2024-09-17T03:28:45.816429+0000","last_became_peered":"2024-09-17T03:28:45.816429+0000","last_unstale":"2024-09-17T03:28:45.819485+0000","last_undegraded":"2024-09-17T03:28:45.819485+0000","last_fullsized":"2024-09-17T03:28:45.819485+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:10.897535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.818473+0000","last_change":"2024-09-17T03:28:44.807188+0000","last_active":"2024-09-17T03:28:45.818473+0000","last_peered":"2024-09-17T03:28:45.818473+0000","last_clean":"2024-09-17T03:28:45.818473+0000","last_became_active":"2024-09-17T03:28:44.806356+0000","last_became_peered":"2024-09-17T03:28:44.806356+0000","last_unstale":"2024-09-17T03:28:45.818473+0000","last_undegraded":"2024-09-17T03:28:45.818473+0000","last_fullsized":"2024-09-17T03:28:45.818473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:53.498836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812273+0000","last_change":"2024-09-17T03:28:45.804863+0000","last_active":"2024-09-17T03:28:45.812273+0000","last_peered":"2024-09-17T03:28:45.812273+0000","last_clean":"2024-09-17T03:28:45.812273+0000","last_became_active":"2024-09-17T03:28:45.804493+0000","last_became_peered":"2024-09-17T03:28:45.804493+0000","last_unstale":"2024-09-17T03:28:45.812273+0000","last_undegraded":"2024-09-17T03:28:45.812273+0000","last_fullsized":"2024-09-17T03:28:45.812273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:58.261639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.803671+0000","last_change":"2024-09-17T03:28:44.811074+0000","last_active":"2024-09-17T03:28:45.803671+0000","last_peered":"2024-09-17T03:28:45.803671+0000","last_clean":"2024-09-17T03:28:45.803671+0000","last_became_active":"2024-09-17T03:28:44.809921+0000","last_became_peered":"2024-09-17T03:28:44.809921+0000","last_unstale":"2024-09-17T03:28:45.803671+0000","last_undegraded":"2024-09-17T03:28:45.803671+0000","last_fullsized":"2024-09-17T03:28:45.803671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:50.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810360+0000","last_change":"2024-09-17T03:28:45.808668+0000","last_active":"2024-09-17T03:28:46.810360+0000","last_peered":"2024-09-17T03:28:46.810360+0000","last_clean":"2024-09-17T03:28:46.810360+0000","last_became_active":"2024-09-17T03:28:45.808270+0000","last_became_peered":"2024-09-17T03:28:45.808270+0000","last_unstale":"2024-09-17T03:28:46.810360+0000","last_undegraded":"2024-09-17T03:28:46.810360+0000","last_fullsized":"2024-09-17T03:28:46.810360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:13.394497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.794243+0000","last_change":"2024-09-17T03:28:44.813876+0000","last_active":"2024-09-17T03:28:45.794243+0000","last_peered":"2024-09-17T03:28:45.794243+0000","last_clean":"2024-09-17T03:28:45.794243+0000","last_became_active":"2024-09-17T03:28:44.813556+0000","last_became_peered":"2024-09-17T03:28:44.813556+0000","last_unstale":"2024-09-17T03:28:45.794243+0000","last_undegraded":"2024-09-17T03:28:45.794243+0000","last_fullsized":"2024-09-17T03:28:45.794243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:21.183444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812229+0000","last_change":"2024-09-17T03:28:45.806848+0000","last_active":"2024-09-17T03:28:45.812229+0000","last_peered":"2024-09-17T03:28:45.812229+0000","last_clean":"2024-09-17T03:28:45.812229+0000","last_became_active":"2024-09-17T03:28:45.806501+0000","last_became_peered":"2024-09-17T03:28:45.806501+0000","last_unstale":"2024-09-17T03:28:45.812229+0000","last_undegraded":"2024-09-17T03:28:45.812229+0000","last_fullsized":"2024-09-17T03:28:45.812229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:30.914190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.792830+0000","last_change":"2024-09-17T03:28:44.807106+0000","last_active":"2024-09-17T03:28:46.792830+0000","last_peered":"2024-09-17T03:28:46.792830+0000","last_clean":"2024-09-17T03:28:46.792830+0000","last_became_active":"2024-09-17T03:28:44.806711+0000","last_became_peered":"2024-09-17T03:28:44.806711+0000","last_unstale":"2024-09-17T03:28:46.792830+0000","last_undegraded":"2024-09-17T03:28:46.792830+0000","last_fullsized":"2024-09-17T03:28:46.792830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:29.666865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793762+0000","last_change":"2024-09-17T03:28:45.797043+0000","last_active":"2024-09-17T03:28:46.793762+0000","last_peered":"2024-09-17T03:28:46.793762+0000","last_clean":"2024-09-17T03:28:46.793762+0000","last_became_active":"2024-09-17T03:28:45.795415+0000","last_became_peered":"2024-09-17T03:28:45.795415+0000","last_unstale":"2024-09-17T03:28:46.793762+0000","last_undegraded":"2024-09-17T03:28:46.793762+0000","last_fullsized":"2024-09-17T03:28:46.793762+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:51.911480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.831733+0000","last_change":"2024-09-17T03:28:44.809080+0000","last_active":"2024-09-17T03:28:45.831733+0000","last_peered":"2024-09-17T03:28:45.831733+0000","last_clean":"2024-09-17T03:28:45.831733+0000","last_became_active":"2024-09-17T03:28:44.808569+0000","last_became_peered":"2024-09-17T03:28:44.808569+0000","last_unstale":"2024-09-17T03:28:45.831733+0000","last_undegraded":"2024-09-17T03:28:45.831733+0000","last_fullsized":"2024-09-17T03:28:45.831733+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:53.841271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819275+0000","last_change":"2024-09-17T03:28:45.810038+0000","last_active":"2024-09-17T03:28:45.819275+0000","last_peered":"2024-09-17T03:28:45.819275+0000","last_clean":"2024-09-17T03:28:45.819275+0000","last_became_active":"2024-09-17T03:28:45.809694+0000","last_became_peered":"2024-09-17T03:28:45.809694+0000","last_unstale":"2024-09-17T03:28:45.819275+0000","last_undegraded":"2024-09-17T03:28:45.819275+0000","last_fullsized":"2024-09-17T03:28:45.819275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:00.053682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.793743+0000","last_change":"2024-09-17T03:28:44.808811+0000","last_active":"2024-09-17T03:28:46.793743+0000","last_peered":"2024-09-17T03:28:46.793743+0000","last_clean":"2024-09-17T03:28:46.793743+0000","last_became_active":"2024-09-17T03:28:44.808496+0000","last_became_peered":"2024-09-17T03:28:44.808496+0000","last_unstale":"2024-09-17T03:28:46.793743+0000","last_undegraded":"2024-09-17T03:28:46.793743+0000","last_fullsized":"2024-09-17T03:28:46.793743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:19.168406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812861+0000","last_change":"2024-09-17T03:28:45.807830+0000","last_active":"2024-09-17T03:28:45.812861+0000","last_peered":"2024-09-17T03:28:45.812861+0000","last_clean":"2024-09-17T03:28:45.812861+0000","last_became_active":"2024-09-17T03:28:45.807227+0000","last_became_peered":"2024-09-17T03:28:45.807227+0000","last_unstale":"2024-09-17T03:28:45.812861+0000","last_undegraded":"2024-09-17T03:28:45.812861+0000","last_fullsized":"2024-09-17T03:28:45.812861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.829781+0000","last_change":"2024-09-17T03:28:44.808559+0000","last_active":"2024-09-17T03:28:45.829781+0000","last_peered":"2024-09-17T03:28:45.829781+0000","last_clean":"2024-09-17T03:28:45.829781+0000","last_became_active":"2024-09-17T03:28:44.808233+0000","last_became_peered":"2024-09-17T03:28:44.808233+0000","last_unstale":"2024-09-17T03:28:45.829781+0000","last_undegraded":"2024-09-17T03:28:45.829781+0000","last_fullsized":"2024-09-17T03:28:45.829781+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:23.035284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.812486+0000","last_change":"2024-09-17T03:28:45.805787+0000","last_active":"2024-09-17T03:28:45.812486+0000","last_peered":"2024-09-17T03:28:45.812486+0000","last_clean":"2024-09-17T03:28:45.812486+0000","last_became_active":"2024-09-17T03:28:45.804996+0000","last_became_peered":"2024-09-17T03:28:45.804996+0000","last_unstale":"2024-09-17T03:28:45.812486+0000","last_undegraded":"2024-09-17T03:28:45.812486+0000","last_fullsized":"2024-09-17T03:28:45.812486+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:34.340461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.795166+0000","last_change":"2024-09-17T03:28:44.822755+0000","last_active":"2024-09-17T03:28:45.795166+0000","last_peered":"2024-09-17T03:28:45.795166+0000","last_clean":"2024-09-17T03:28:45.795166+0000","last_became_active":"2024-09-17T03:28:44.822350+0000","last_became_peered":"2024-09-17T03:28:44.822350+0000","last_unstale":"2024-09-17T03:28:45.795166+0000","last_undegraded":"2024-09-17T03:28:45.795166+0000","last_fullsized":"2024-09-17T03:28:45.795166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:47.661258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786501+0000","last_change":"2024-09-17T03:28:45.792116+0000","last_active":"2024-09-17T03:28:46.786501+0000","last_peered":"2024-09-17T03:28:46.786501+0000","last_clean":"2024-09-17T03:28:46.786501+0000","last_became_active":"2024-09-17T03:28:45.791120+0000","last_became_peered":"2024-09-17T03:28:45.791120+0000","last_unstale":"2024-09-17T03:28:46.786501+0000","last_undegraded":"2024-09-17T03:28:46.786501+0000","last_fullsized":"2024-09-17T03:28:46.786501+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:32.113237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.803913+0000","last_change":"2024-09-17T03:28:44.806357+0000","last_active":"2024-09-17T03:28:45.803913+0000","last_peered":"2024-09-17T03:28:45.803913+0000","last_clean":"2024-09-17T03:28:45.803913+0000","last_became_active":"2024-09-17T03:28:44.805862+0000","last_became_peered":"2024-09-17T03:28:44.805862+0000","last_unstale":"2024-09-17T03:28:45.803913+0000","last_undegraded":"2024-09-17T03:28:45.803913+0000","last_fullsized":"2024-09-17T03:28:45.803913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:24.205943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786099+0000","last_change":"2024-09-17T03:28:45.805240+0000","last_active":"2024-09-17T03:28:46.786099+0000","last_peered":"2024-09-17T03:28:46.786099+0000","last_clean":"2024-09-17T03:28:46.786099+0000","last_became_active":"2024-09-17T03:28:45.804932+0000","last_became_peered":"2024-09-17T03:28:45.804932+0000","last_unstale":"2024-09-17T03:28:46.786099+0000","last_undegraded":"2024-09-17T03:28:46.786099+0000","last_fullsized":"2024-09-17T03:28:46.786099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:10.255805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.794579+0000","last_change":"2024-09-17T03:28:44.813215+0000","last_active":"2024-09-17T03:28:45.794579+0000","last_peered":"2024-09-17T03:28:45.794579+0000","last_clean":"2024-09-17T03:28:45.794579+0000","last_became_active":"2024-09-17T03:28:44.812895+0000","last_became_peered":"2024-09-17T03:28:44.812895+0000","last_unstale":"2024-09-17T03:28:45.794579+0000","last_undegraded":"2024-09-17T03:28:45.794579+0000","last_fullsized":"2024-09-17T03:28:45.794579+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:10.875138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786133+0000","last_change":"2024-09-17T03:28:45.813236+0000","last_active":"2024-09-17T03:28:46.786133+0000","last_peered":"2024-09-17T03:28:46.786133+0000","last_clean":"2024-09-17T03:28:46.786133+0000","last_became_active":"2024-09-17T03:28:45.812868+0000","last_became_peered":"2024-09-17T03:28:45.812868+0000","last_unstale":"2024-09-17T03:28:46.786133+0000","last_undegraded":"2024-09-17T03:28:46.786133+0000","last_fullsized":"2024-09-17T03:28:46.786133+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:14.822118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810184+0000","last_change":"2024-09-17T03:28:44.801076+0000","last_active":"2024-09-17T03:28:46.810184+0000","last_peered":"2024-09-17T03:28:46.810184+0000","last_clean":"2024-09-17T03:28:46.810184+0000","last_became_active":"2024-09-17T03:28:44.800786+0000","last_became_peered":"2024-09-17T03:28:44.800786+0000","last_unstale":"2024-09-17T03:28:46.810184+0000","last_undegraded":"2024-09-17T03:28:46.810184+0000","last_fullsized":"2024-09-17T03:28:46.810184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:53.296535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810853+0000","last_change":"2024-09-17T03:28:45.810263+0000","last_active":"2024-09-17T03:28:46.810853+0000","last_peered":"2024-09-17T03:28:46.810853+0000","last_clean":"2024-09-17T03:28:46.810853+0000","last_became_active":"2024-09-17T03:28:45.809928+0000","last_became_peered":"2024-09-17T03:28:45.809928+0000","last_unstale":"2024-09-17T03:28:46.810853+0000","last_undegraded":"2024-09-17T03:28:46.810853+0000","last_fullsized":"2024-09-17T03:28:46.810853+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:25.475101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.832127+0000","last_change":"2024-09-17T03:28:44.811093+0000","last_active":"2024-09-17T03:28:45.832127+0000","last_peered":"2024-09-17T03:28:45.832127+0000","last_clean":"2024-09-17T03:28:45.832127+0000","last_became_active":"2024-09-17T03:28:44.810454+0000","last_became_peered":"2024-09-17T03:28:44.810454+0000","last_unstale":"2024-09-17T03:28:45.832127+0000","last_undegraded":"2024-09-17T03:28:45.832127+0000","last_fullsized":"2024-09-17T03:28:45.832127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:05.160027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.814001+0000","last_change":"2024-09-17T03:28:45.808653+0000","last_active":"2024-09-17T03:28:45.814001+0000","last_peered":"2024-09-17T03:28:45.814001+0000","last_clean":"2024-09-17T03:28:45.814001+0000","last_became_active":"2024-09-17T03:28:45.808357+0000","last_became_peered":"2024-09-17T03:28:45.808357+0000","last_unstale":"2024-09-17T03:28:45.814001+0000","last_undegraded":"2024-09-17T03:28:45.814001+0000","last_fullsized":"2024-09-17T03:28:45.814001+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:41.015864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.795846+0000","last_change":"2024-09-17T03:28:44.810924+0000","last_active":"2024-09-17T03:28:45.795846+0000","last_peered":"2024-09-17T03:28:45.795846+0000","last_clean":"2024-09-17T03:28:45.795846+0000","last_became_active":"2024-09-17T03:28:44.810524+0000","last_became_peered":"2024-09-17T03:28:44.810524+0000","last_unstale":"2024-09-17T03:28:45.795846+0000","last_undegraded":"2024-09-17T03:28:45.795846+0000","last_fullsized":"2024-09-17T03:28:45.795846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:02.741967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.820054+0000","last_change":"2024-09-17T03:28:45.818736+0000","last_active":"2024-09-17T03:28:45.820054+0000","last_peered":"2024-09-17T03:28:45.820054+0000","last_clean":"2024-09-17T03:28:45.820054+0000","last_became_active":"2024-09-17T03:28:45.818326+0000","last_became_peered":"2024-09-17T03:28:45.818326+0000","last_unstale":"2024-09-17T03:28:45.820054+0000","last_undegraded":"2024-09-17T03:28:45.820054+0000","last_fullsized":"2024-09-17T03:28:45.820054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:05.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.798453+0000","last_change":"2024-09-17T03:28:44.806886+0000","last_active":"2024-09-17T03:28:45.798453+0000","last_peered":"2024-09-17T03:28:45.798453+0000","last_clean":"2024-09-17T03:28:45.798453+0000","last_became_active":"2024-09-17T03:28:44.803126+0000","last_became_peered":"2024-09-17T03:28:44.803126+0000","last_unstale":"2024-09-17T03:28:45.798453+0000","last_undegraded":"2024-09-17T03:28:45.798453+0000","last_fullsized":"2024-09-17T03:28:45.798453+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:38.827779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.816587+0000","last_change":"2024-09-17T03:28:45.811786+0000","last_active":"2024-09-17T03:28:45.816587+0000","last_peered":"2024-09-17T03:28:45.816587+0000","last_clean":"2024-09-17T03:28:45.816587+0000","last_became_active":"2024-09-17T03:28:45.811379+0000","last_became_peered":"2024-09-17T03:28:45.811379+0000","last_unstale":"2024-09-17T03:28:45.816587+0000","last_undegraded":"2024-09-17T03:28:45.816587+0000","last_fullsized":"2024-09-17T03:28:45.816587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:19.131335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.810690+0000","last_change":"2024-09-17T03:28:44.799567+0000","last_active":"2024-09-17T03:28:46.810690+0000","last_peered":"2024-09-17T03:28:46.810690+0000","last_clean":"2024-09-17T03:28:46.810690+0000","last_became_active":"2024-09-17T03:28:44.799229+0000","last_became_peered":"2024-09-17T03:28:44.799229+0000","last_unstale":"2024-09-17T03:28:46.810690+0000","last_undegraded":"2024-09-17T03:28:46.810690+0000","last_fullsized":"2024-09-17T03:28:46.810690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:53.929276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.817370+0000","last_change":"2024-09-17T03:28:45.806154+0000","last_active":"2024-09-17T03:28:45.817370+0000","last_peered":"2024-09-17T03:28:45.817370+0000","last_clean":"2024-09-17T03:28:45.817370+0000","last_became_active":"2024-09-17T03:28:45.805825+0000","last_became_peered":"2024-09-17T03:28:45.805825+0000","last_unstale":"2024-09-17T03:28:45.817370+0000","last_undegraded":"2024-09-17T03:28:45.817370+0000","last_fullsized":"2024-09-17T03:28:45.817370+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:28.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.797551+0000","last_change":"2024-09-17T03:28:44.800382+0000","last_active":"2024-09-17T03:28:45.797551+0000","last_peered":"2024-09-17T03:28:45.797551+0000","last_clean":"2024-09-17T03:28:45.797551+0000","last_became_active":"2024-09-17T03:28:44.800060+0000","last_became_peered":"2024-09-17T03:28:44.800060+0000","last_unstale":"2024-09-17T03:28:45.797551+0000","last_undegraded":"2024-09-17T03:28:45.797551+0000","last_fullsized":"2024-09-17T03:28:45.797551+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:19.213426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.819748+0000","last_change":"2024-09-17T03:28:45.814358+0000","last_active":"2024-09-17T03:28:45.819748+0000","last_peered":"2024-09-17T03:28:45.819748+0000","last_clean":"2024-09-17T03:28:45.819748+0000","last_became_active":"2024-09-17T03:28:45.812438+0000","last_became_peered":"2024-09-17T03:28:45.812438+0000","last_unstale":"2024-09-17T03:28:45.819748+0000","last_undegraded":"2024-09-17T03:28:45.819748+0000","last_fullsized":"2024-09-17T03:28:45.819748+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:22.696566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.781467+0000","last_change":"2024-09-17T03:28:44.799622+0000","last_active":"2024-09-17T03:28:45.781467+0000","last_peered":"2024-09-17T03:28:45.781467+0000","last_clean":"2024-09-17T03:28:45.781467+0000","last_became_active":"2024-09-17T03:28:44.799276+0000","last_became_peered":"2024-09-17T03:28:44.799276+0000","last_unstale":"2024-09-17T03:28:45.781467+0000","last_undegraded":"2024-09-17T03:28:45.781467+0000","last_fullsized":"2024-09-17T03:28:45.781467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:53.693967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.790673+0000","last_change":"2024-09-17T03:28:45.786439+0000","last_active":"2024-09-17T03:28:45.790673+0000","last_peered":"2024-09-17T03:28:45.790673+0000","last_clean":"2024-09-17T03:28:45.790673+0000","last_became_active":"2024-09-17T03:28:45.786027+0000","last_became_peered":"2024-09-17T03:28:45.786027+0000","last_unstale":"2024-09-17T03:28:45.790673+0000","last_undegraded":"2024-09-17T03:28:45.790673+0000","last_fullsized":"2024-09-17T03:28:45.790673+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:16.351196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.792762+0000","last_change":"2024-09-17T03:28:44.814763+0000","last_active":"2024-09-17T03:28:46.792762+0000","last_peered":"2024-09-17T03:28:46.792762+0000","last_clean":"2024-09-17T03:28:46.792762+0000","last_became_active":"2024-09-17T03:28:44.814387+0000","last_became_peered":"2024-09-17T03:28:44.814387+0000","last_unstale":"2024-09-17T03:28:46.792762+0000","last_undegraded":"2024-09-17T03:28:46.792762+0000","last_fullsized":"2024-09-17T03:28:46.792762+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:28.760438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.831898+0000","last_change":"2024-09-17T03:28:45.831097+0000","last_active":"2024-09-17T03:28:45.831898+0000","last_peered":"2024-09-17T03:28:45.831898+0000","last_clean":"2024-09-17T03:28:45.831898+0000","last_became_active":"2024-09-17T03:28:45.830715+0000","last_became_peered":"2024-09-17T03:28:45.830715+0000","last_unstale":"2024-09-17T03:28:45.831898+0000","last_undegraded":"2024-09-17T03:28:45.831898+0000","last_fullsized":"2024-09-17T03:28:45.831898+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:45.415345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.799003+0000","last_change":"2024-09-17T03:28:44.808776+0000","last_active":"2024-09-17T03:28:45.799003+0000","last_peered":"2024-09-17T03:28:45.799003+0000","last_clean":"2024-09-17T03:28:45.799003+0000","last_became_active":"2024-09-17T03:28:44.808167+0000","last_became_peered":"2024-09-17T03:28:44.808167+0000","last_unstale":"2024-09-17T03:28:45.799003+0000","last_undegraded":"2024-09-17T03:28:45.799003+0000","last_fullsized":"2024-09-17T03:28:45.799003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:30.309465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-17T03:28:46.786333+0000","last_change":"2024-09-17T03:28:45.813876+0000","last_active":"2024-09-17T03:28:46.786333+0000","last_peered":"2024-09-17T03:28:46.786333+0000","last_clean":"2024-09-17T03:28:46.786333+0000","last_became_active":"2024-09-17T03:28:45.813458+0000","last_became_peered":"2024-09-17T03:28:45.813458+0000","last_unstale":"2024-09-17T03:28:46.786333+0000","last_undegraded":"2024-09-17T03:28:46.786333+0000","last_fullsized":"2024-09-17T03:28:46.786333+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:44.765103+0000","last_clean_scrub_stamp":"2024-09-17T03:28:44.765103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:10.364569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.831248+0000","last_change":"2024-09-17T03:28:44.808179+0000","last_active":"2024-09-17T03:28:45.831248+0000","last_peered":"2024-09-17T03:28:45.831248+0000","last_clean":"2024-09-17T03:28:45.831248+0000","last_became_active":"2024-09-17T03:28:44.805931+0000","last_became_peered":"2024-09-17T03:28:44.805931+0000","last_unstale":"2024-09-17T03:28:45.831248+0000","last_undegraded":"2024-09-17T03:28:45.831248+0000","last_fullsized":"2024-09-17T03:28:45.831248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:28:43.757217+0000","last_clean_scrub_stamp":"2024-09-17T03:28:43.757217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:31.479083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":203,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-17T03:28:45.795715+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:28:45.795715+0000","last_peered":"2024-09-17T03:28:45.795715+0000","last_clean":"2024-09-17T03:28:45.795715+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:28:45.795715+0000","last_undegraded":"2024-09-17T03:28:45.795715+0000","last_fullsized":"2024-09-17T03:28:45.795715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1458176,"data_stored":951338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50084,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":864256,"data_stored":360446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509537,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54240,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317600,"statfs":{"total":96636764160,"available":96581222400,"internally_reserved":0,"allocated":860160,"data_stored":360424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58404,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313436,"statfs":{"total":96636764160,"available":96576958464,"internally_reserved":0,"allocated":864256,"data_stored":360466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":868352,"data_stored":361966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509537,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58980,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312860,"statfs":{"total":96636764160,"available":96576368640,"internally_reserved":0,"allocated":1449984,"data_stored":950722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542241,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":843776,"data_stored":353579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58380,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313460,"statfs":{"total":96636764160,"available":96576983040,"internally_reserved":0,"allocated":843776,"data_stored":353579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:28:50.407 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:28:50.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:28:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 -- 172.21.15.46:0/3154709844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01340cc860 msgr2=0x7f013415e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 --2- 172.21.15.46:0/3154709844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01340cc860 0x7f013415e890 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f011c009930 tx=0x7f011c01c670 comp rx=0 tx=0).stop 2024-09-17T03:28:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 -- 172.21.15.46:0/3154709844 shutdown_connections 2024-09-17T03:28:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 --2- 172.21.15.46:0/3154709844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01340cc860 0x7f013415e890 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 --2- 172.21.15.46:0/3154709844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f013413e790 0x7f01340cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 -- 172.21.15.46:0/3154709844 >> 172.21.15.46:0/3154709844 conn(0x7f01340b2040 msgr2=0x7f01340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:50.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 -- 172.21.15.46:0/3154709844 shutdown_connections 2024-09-17T03:28:50.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 -- 172.21.15.46:0/3154709844 wait complete. 2024-09-17T03:28:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 Processor -- start 2024-09-17T03:28:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.614+0000 7f0138a00640 1 -- start start 2024-09-17T03:28:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01340cc860 0x7f013413d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013413e790 0x7f01340b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 0x7f01340b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0134140920 con 0x7f013415edd0 2024-09-17T03:28:50.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01341407a0 con 0x7f013413e790 2024-09-17T03:28:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0134140aa0 con 0x7f01340cc860 2024-09-17T03:28:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 0x7f01340b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 0x7f01340b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53384/0 (socket says 172.21.15.46:53384) 2024-09-17T03:28:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 -- 172.21.15.46:0/1280442205 learned_addr learned my addr 172.21.15.46:0/1280442205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f012be00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013413e790 0x7f01340b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0131e00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01340cc860 0x7f013413d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 -- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01340cc860 msgr2=0x7f013413d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01340cc860 0x7f013413d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 -- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013413e790 msgr2=0x7f01340b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013413e790 0x7f01340b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 -- 172.21.15.46:0/1280442205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013413d6c0 con 0x7f013415edd0 2024-09-17T03:28:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0132800640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 0x7f01340b10c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b9e0 tx=0x7f012c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c003b80 con 0x7f013415edd0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f012c003e10 con 0x7f013415edd0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013415dfb0 con 0x7f013415edd0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00cb80 con 0x7f013415edd0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01340aeff0 con 0x7f013415edd0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.618+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0134141000 con 0x7f013415edd0 2024-09-17T03:28:50.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.622+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f012c006d10 con 0x7f013415edd0 2024-09-17T03:28:50.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.622+0000 7f012a000640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0110073ef0 0x7f0110076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:50.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.622+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f012c092910 con 0x7f013415edd0 2024-09-17T03:28:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.622+0000 7f0131e00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0110073ef0 0x7f0110076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:50.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.622+0000 7f0131e00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0110073ef0 0x7f0110076400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0124004630 tx=0x7f012400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:50.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.626+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f012c05eb30 con 0x7f013415edd0 2024-09-17T03:28:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.862+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f013413e090 con 0x7f013415edd0 2024-09-17T03:28:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.866+0000 7f012a000640 1 -- 172.21.15.46:0/1280442205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) ==== 190+0+147 (secure 0 0 0) 0x7f012c063560 con 0x7f013415edd0 2024-09-17T03:28:50.870 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:28:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:28:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:28:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:28:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:28:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0110073ef0 msgr2=0x7f0110076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0110073ef0 0x7f0110076400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0124004630 tx=0x7f012400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 msgr2=0x7f01340b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 0x7f01340b10c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b9e0 tx=0x7f012c00bea0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 shutdown_connections 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0110073ef0 0x7f0110076400 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f013415edd0 0x7f01340b10c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013413e790 0x7f01340b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 --2- 172.21.15.46:0/1280442205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01340cc860 0x7f013413d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 >> 172.21.15.46:0/1280442205 conn(0x7f01340b2040 msgr2=0x7f0134074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 shutdown_connections 2024-09-17T03:28:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:50.870+0000 7f0138a00640 1 -- 172.21.15.46:0/1280442205 wait complete. 2024-09-17T03:28:50.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:28:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 -- 172.21.15.46:0/2489941602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445c003c20 msgr2=0x7f445c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 --2- 172.21.15.46:0/2489941602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445c003c20 0x7f445c004070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4468001860 tx=0x7f446801c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f4477e00640 1 -- 172.21.15.46:0/2489941602 reap_dead start 2024-09-17T03:28:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 -- 172.21.15.46:0/2489941602 shutdown_connections 2024-09-17T03:28:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 -- 172.21.15.46:0/2489941602 >> 172.21.15.46:0/2489941602 conn(0x7f44780b2040 msgr2=0x7f44780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 -- 172.21.15.46:0/2489941602 shutdown_connections 2024-09-17T03:28:51.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 -- 172.21.15.46:0/2489941602 wait complete. 2024-09-17T03:28:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 Processor -- start 2024-09-17T03:28:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.050+0000 7f447e400640 1 -- start start 2024-09-17T03:28:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445c003c20 0x7f447814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f447814fde0 0x7f4478158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 0x7f44781fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4478167bc0 con 0x7f44781588d0 2024-09-17T03:28:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4478167a40 con 0x7f445c003c20 2024-09-17T03:28:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4478167d40 con 0x7f447814fde0 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 0x7f44781fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 0x7f44781fbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53404/0 (socket says 172.21.15.46:53404) 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 -- 172.21.15.46:0/1820105529 learned_addr learned my addr 172.21.15.46:0/1820105529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f446ea00640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f447814fde0 0x7f4478158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445c003c20 0x7f447814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 -- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f447814fde0 msgr2=0x7f4478158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f447814fde0 0x7f4478158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 -- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445c003c20 msgr2=0x7f447814f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445c003c20 0x7f447814f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 -- 172.21.15.46:0/1820105529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4478158ce0 con 0x7f44781588d0 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f4477e00640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 0x7f44781fbf80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4470000f90 tx=0x7f447000e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470010890 con 0x7f44781588d0 2024-09-17T03:28:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4470018070 con 0x7f44781588d0 2024-09-17T03:28:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44780ccae0 con 0x7f44781588d0 2024-09-17T03:28:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470010e00 con 0x7f44781588d0 2024-09-17T03:28:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4478154550 con 0x7f44781588d0 2024-09-17T03:28:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.054+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44780cf820 con 0x7f44781588d0 2024-09-17T03:28:51.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.058+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f447001d460 con 0x7f44781588d0 2024-09-17T03:28:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.058+0000 7f446fe00640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4454073ef0 0x7f4454076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.058+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f4470092f10 con 0x7f44781588d0 2024-09-17T03:28:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.058+0000 7f4477400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4454073ef0 0x7f4454076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.058+0000 7f4477400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4454073ef0 0x7f4454076400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4468001830 tx=0x7f44680196f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.062+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4478158ce0 con 0x7f44781588d0 2024-09-17T03:28:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.270+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4478074e30 con 0x7f44781588d0 2024-09-17T03:28:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f446fe00640 1 -- 172.21.15.46:0/1820105529 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5241 (secure 0 0 0) 0x7f447005fa10 con 0x7f44781588d0 2024-09-17T03:28:51.276 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:28:51.276 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"btime":"2024-09-17T03:28:48:795549+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:28:45.214743+0000","modified":"2024-09-17T03:28:48.217828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246303":{"gid":246303,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6836/3671761014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3671761014},{"type":"v1","addr":"172.21.15.46:6836","nonce":3671761014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246303,"qdb_cluster":[246303]},"id":5}]} 2024-09-17T03:28:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4454073ef0 msgr2=0x7f4454076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4454073ef0 0x7f4454076400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4468001830 tx=0x7f44680196f0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 msgr2=0x7f44781fbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 0x7f44781fbf80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4470000f90 tx=0x7f447000e9d0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 shutdown_connections 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4454073ef0 0x7f4454076400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44781588d0 0x7f44781fbf80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f447814fde0 0x7f4478158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 --2- 172.21.15.46:0/1820105529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445c003c20 0x7f447814f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 >> 172.21.15.46:0/1820105529 conn(0x7f44780b2040 msgr2=0x7f44780b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 shutdown_connections 2024-09-17T03:28:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.274+0000 7f447e400640 1 -- 172.21.15.46:0/1820105529 wait complete. 2024-09-17T03:28:51.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:28:45.214743+0000', 'modified': '2024-09-17T03:28:48.217828+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 246303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_246303': {'gid': 246303, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6836/3671761014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3671761014}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 3671761014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 246303, 'qdb_cluster': [246303]} 2024-09-17T03:28:51.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:28:51.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 -- 172.21.15.46:0/1410208273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18140b5a80 msgr2=0x7f1814074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 --2- 172.21.15.46:0/1410208273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18140b5a80 0x7f1814074410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009900 tx=0x7f17fc01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 -- 172.21.15.46:0/1410208273 shutdown_connections 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 --2- 172.21.15.46:0/1410208273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 0x7f181415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 --2- 172.21.15.46:0/1410208273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18140b5a80 0x7f1814074410 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 -- 172.21.15.46:0/1410208273 >> 172.21.15.46:0/1410208273 conn(0x7f18140708c0 msgr2=0x7f1814070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 -- 172.21.15.46:0/1410208273 shutdown_connections 2024-09-17T03:28:51.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 -- 172.21.15.46:0/1410208273 wait complete. 2024-09-17T03:28:51.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 Processor -- start 2024-09-17T03:28:51.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.866+0000 7f181b200640 1 -- start start 2024-09-17T03:28:51.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 0x7f181414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 0x7f1814148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 0x7f18140b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f181416a310 con 0x7f18140b5a80 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f181416a190 con 0x7f1814157030 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f181416a490 con 0x7f1814074950 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 0x7f1814148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1813e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 0x7f181414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1819200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 0x7f18140b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1819200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 0x7f18140b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:36426/0 (socket says 172.21.15.46:36426) 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 0x7f1814148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53406/0 (socket says 172.21.15.46:53406) 2024-09-17T03:28:51.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1819200640 1 -- 172.21.15.46:0/1989027614 learned_addr learned my addr 172.21.15.46:0/1989027614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:51.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1813e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 0x7f181414dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:38082/0 (socket says 172.21.15.46:38082) 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 -- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 msgr2=0x7f181414dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 0x7f181414dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 -- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 msgr2=0x7f18140b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 0x7f18140b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 -- 172.21.15.46:0/1989027614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18141fa820 con 0x7f18140b5a80 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1819200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 0x7f18140b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f180be00640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 0x7f1814148b30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0ac000 tx=0x7f17fc0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1810c00640 1 -- 172.21.15.46:0/1989027614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0042c0 con 0x7f18140b5a80 2024-09-17T03:28:51.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1810c00640 1 -- 172.21.15.46:0/1989027614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17fc0b9070 con 0x7f18140b5a80 2024-09-17T03:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141fab00 con 0x7f18140b5a80 2024-09-17T03:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f1810c00640 1 -- 172.21.15.46:0/1989027614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc003860 con 0x7f18140b5a80 2024-09-17T03:28:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.870+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181414a120 con 0x7f18140b5a80 2024-09-17T03:28:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1810c00640 1 -- 172.21.15.46:0/1989027614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f17fc003a00 con 0x7f18140b5a80 2024-09-17T03:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1810c00640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f17f0073ef0 0x7f17f0076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1810c00640 1 -- 172.21.15.46:0/1989027614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f17fc130880 con 0x7f18140b5a80 2024-09-17T03:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1822a54000 1 Processor -- start 2024-09-17T03:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1813e00640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f17f0073ef0 0x7f17f0076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1822a54000 1 -- start start 2024-09-17T03:28:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.874+0000 7f1813e00640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f17f0073ef0 0x7f17f0076400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1804007fd0 tx=0x7f1804009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a1156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a115be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a116120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556709fea410 con 0x556709fc4950 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556709f27b40 con 0x556709bd4460 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x556709f16f70 con 0x556709fadca0 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53410/0 (socket says 172.21.15.46:53410) 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 -- 172.21.15.46:0/2718142748 learned_addr learned my addr 172.21.15.46:0/2718142748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1813e00640 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a1156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1819200640 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 -- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 msgr2=0x55670a116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a116120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 -- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 msgr2=0x55670a1156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a1156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 -- 172.21.15.46:0/2718142748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556709ede5b0 con 0x556709fc4950 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f180be00640 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a115be0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0fcdb0 tx=0x7f17fc0f8d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdb8d308eb7457f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1808c00640 1 -- 172.21.15.46:0/2718142748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0c4030 con 0x556709fc4950 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1808c00640 1 -- 172.21.15.46:0/2718142748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17fc0fc930 con 0x556709fc4950 2024-09-17T03:28:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1808c00640 1 -- 172.21.15.46:0/2718142748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0fc590 con 0x556709fc4950 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- 172.21.15.46:0/2718142748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 msgr2=0x55670a115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a115be0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0fcdb0 tx=0x7f17fc0f8d80 comp rx=0 tx=0).stop 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- 172.21.15.46:0/2718142748 shutdown_connections 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a115be0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 --2- 172.21.15.46:0/2718142748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a1156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- 172.21.15.46:0/2718142748 >> 172.21.15.46:0/2718142748 conn(0x55670a09ad90 msgr2=0x55670a0ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- 172.21.15.46:0/2718142748 shutdown_connections 2024-09-17T03:28:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- 172.21.15.46:0/2718142748 wait complete. 2024-09-17T03:28:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 -- start start 2024-09-17T03:28:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.878+0000 7f1822a54000 1 Processor -- start 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 0x55670a11c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a11ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fc4950 0x55670a11d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556709eb15f0 con 0x556709bd4460 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556709f1ad80 con 0x556709fadca0 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x556709ebd520 con 0x556709fc4950 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 10 client.0 _ll_register_callbacks cb 0x556709ec1830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:28:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 0x55670a11c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 0x55670a11c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53412/0 (socket says 172.21.15.46:53412) 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 -- 172.21.15.46:0/780443588 learned_addr learned my addr 172.21.15.46:0/780443588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f180be00640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a11ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1819200640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fc4950 0x55670a11d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 -- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fc4950 msgr2=0x55670a11d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fc4950 0x55670a11d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 -- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 msgr2=0x55670a11ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a11ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 -- 172.21.15.46:0/780443588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556709eb5190 con 0x556709bd4460 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1813e00640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 0x55670a11c920 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f18040012c0 tx=0x7f1804000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269b551b257415b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1801600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:28:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1801600640 1 -- 172.21.15.46:0/780443588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804025020 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556709eaf110 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1801600640 1 -- 172.21.15.46:0/780443588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f1804021610 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1801600640 1 -- 172.21.15.46:0/780443588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f180402f050 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 10 client.246423 fetch_fsmap learned FSMap version 45 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x556709f9bf10 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1801600640 1 -- 172.21.15.46:0/780443588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556709eaf110 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1801600640 1 -- 172.21.15.46:0/780443588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f1804032020 con 0x556709bd4460 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 10 client.246423 fetch_fsmap finished waiting for FSMap version 45 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 10 client.246423 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 20 client.246423 populate_metadata read hostname 'smithi046' 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709bb97d0 0x55670a126d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 4 client.246423 mds_command: new command op to 246303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:28:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 --> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556709f29a90 con 0x556709bb97d0 2024-09-17T03:28:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.882+0000 7f180be00640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709bb97d0 0x55670a126d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1801600640 10 client.246423 ms_handle_connect on v2:172.21.15.46:6834/3671761014 2024-09-17T03:28:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f180be00640 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709bb97d0 0x55670a126d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e53148447ac1dc server_cookie=c65de1aae1a0d6bf in_seq=0 out_seq=0 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1801600640 1 -- 172.21.15.46:0/780443588 <== mds.0 v2:172.21.15.46:6834/3671761014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556709f29a90 con 0x556709bb97d0 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1801600640 10 client.246423 handle_command_reply: tid=0 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 client.246423 shutdown 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 msgr2=0x55670a11c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 0x55670a11c920 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f18040012c0 tx=0x7f1804000ee0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 shutdown_connections 2024-09-17T03:28:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709bb97d0 0x55670a126d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fc4950 0x55670a11d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a11ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 --2- 172.21.15.46:0/780443588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709bd4460 0x55670a11c920 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1801600640 0 client.246423 ms_handle_reset on v2:172.21.15.46:6834/3671761014 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 >> 172.21.15.46:0/780443588 conn(0x55670a09ad90 msgr2=0x55670a0d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 shutdown_connections 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 1 -- 172.21.15.46:0/780443588 wait complete. 2024-09-17T03:28:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.886+0000 7f1822a54000 20 client.246423 trim_cache size 0 max 16384 2024-09-17T03:28:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 Processor -- start 2024-09-17T03:28:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 -- start start 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a114630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a114b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f181414b120 con 0x556709fc4950 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f181406eca0 con 0x556709bd4460 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f181406e000 con 0x556709fadca0 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53418/0 (socket says 172.21.15.46:53418) 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 -- 172.21.15.46:0/4268684590 learned_addr learned my addr 172.21.15.46:0/4268684590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1813e00640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a114630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f180be00640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 -- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 msgr2=0x55670a114b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a114b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 -- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 msgr2=0x55670a114630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a114630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 -- 172.21.15.46:0/4268684590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556709f72db0 con 0x556709fc4950 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1813e00640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a114630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f180be00640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1819200640 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a0c9cd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f180c011b90 tx=0x7f180c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4665a103dac6a451 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1803e00640 1 -- 172.21.15.46:0/4268684590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c011c80 con 0x556709fc4950 2024-09-17T03:28:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1803e00640 1 -- 172.21.15.46:0/4268684590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f180c011e20 con 0x556709fc4950 2024-09-17T03:28:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.906+0000 7f1803e00640 1 -- 172.21.15.46:0/4268684590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c004d70 con 0x556709fc4950 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- 172.21.15.46:0/4268684590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 msgr2=0x55670a0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a0c9cd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f180c011b90 tx=0x7f180c00f790 comp rx=0 tx=0).stop 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- 172.21.15.46:0/4268684590 shutdown_connections 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a0c9cd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709fadca0 0x55670a114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- 172.21.15.46:0/4268684590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709bd4460 0x55670a114630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- 172.21.15.46:0/4268684590 >> 172.21.15.46:0/4268684590 conn(0x55670a09ad90 msgr2=0x55670a0d70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- 172.21.15.46:0/4268684590 shutdown_connections 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- 172.21.15.46:0/4268684590 wait complete. 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- start start 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 Processor -- start 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709bd4460 0x55670a0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f181406e650 con 0x556709fc4950 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556709ee2ee0 con 0x556709fadca0 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x556709f1a090 con 0x556709bd4460 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1822a54000 10 client.0 _ll_register_callbacks cb 0x7f1804001410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:28:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1819200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1819200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53426/0 (socket says 172.21.15.46:53426) 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f180be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1819200640 1 -- 172.21.15.46:0/1951331309 learned_addr learned my addr 172.21.15.46:0/1951331309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1813e00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709bd4460 0x55670a0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1813e00640 1 -- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 msgr2=0x55670a0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1813e00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a0cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1813e00640 1 -- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 msgr2=0x55670a147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1813e00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1813e00640 1 -- 172.21.15.46:0/1951331309 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x556709eddab0 con 0x556709bd4460 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f1819200640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.910+0000 7f180be00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1813e00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709bd4460 0x55670a0cccd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1804001290 tx=0x7f180402aad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f812ead5e3a73475 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 10 client.0 ms_handle_connect on v2:172.21.15.99:3301/0 2024-09-17T03:28:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 1 -- 172.21.15.46:0/1951331309 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804037070 con 0x556709bd4460 2024-09-17T03:28:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c011e20 con 0x556709bd4460 2024-09-17T03:28:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 1 -- 172.21.15.46:0/1951331309 <== mon.2 v2:172.21.15.99:3301/0 2 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f180400f5a0 con 0x556709bd4460 2024-09-17T03:28:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 1 -- 172.21.15.46:0/1951331309 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f18040333f0 con 0x556709bd4460 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 10 client.245789 fetch_fsmap learned FSMap version 45 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x556709ebc070 con 0x556709bd4460 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 1 -- 172.21.15.46:0/1951331309 <== mon.2 v2:172.21.15.99:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f180c011e20 con 0x556709bd4460 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 1 -- 172.21.15.46:0/1951331309 <== mon.2 v2:172.21.15.99:3301/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f1804032040 con 0x556709bd4460 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 10 client.245789 fetch_fsmap finished waiting for FSMap version 45 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 10 client.245789 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:28:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 20 client.245789 populate_metadata read hostname 'smithi046' 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709ba7830 0x55670a18ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 4 client.245789 mds_command: new command op to 246303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 --> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] -- command(tid 0: {"prefix": "status"}) -- 0x556709f25540 con 0x556709ba7830 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f180be00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709ba7830 0x55670a18ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f180be00640 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709ba7830 0x55670a18ac80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0839beb7ed7ea89 server_cookie=92477bd45d09ea84 in_seq=0 out_seq=0 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 10 client.245789 ms_handle_connect on v2:172.21.15.46:6834/3671761014 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 1 -- 172.21.15.46:0/1951331309 <== mds.0 v2:172.21.15.46:6834/3671761014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556709f25540 con 0x556709ba7830 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 10 client.245789 handle_command_reply: tid=0 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 client.245789 shutdown 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709bd4460 msgr2=0x55670a0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709bd4460 0x55670a0cccd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1804001290 tx=0x7f180402aad0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 shutdown_connections 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:6834/3671761014,v1:172.21.15.46:6836/3671761014] conn(0x556709ba7830 0x55670a18ac80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556709fc4950 0x55670a147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556709fadca0 0x55670a0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 --2- 172.21.15.46:0/1951331309 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x556709bd4460 0x55670a0cccd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1801600640 0 client.245789 ms_handle_reset on v2:172.21.15.46:6834/3671761014 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 >> 172.21.15.46:0/1951331309 conn(0x55670a09ad90 msgr2=0x55670a1268d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 shutdown_connections 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 1 -- 172.21.15.46:0/1951331309 wait complete. 2024-09-17T03:28:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f1822a54000 20 client.245789 trim_cache size 0 max 16384 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 246303, 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.128719963, 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 45, 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 72, 2024-09-17T03:28:51.924 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 72, 2024-09-17T03:28:51.925 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 17.555309485999999 2024-09-17T03:28:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f17f0073ef0 msgr2=0x7f17f0076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f17f0073ef0 0x7f17f0076400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1804007fd0 tx=0x7f1804009040 comp rx=0 tx=0).stop 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 msgr2=0x7f1814148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 0x7f1814148b30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0ac000 tx=0x7f17fc0a5710 comp rx=0 tx=0).stop 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 shutdown_connections 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f17f0073ef0 0x7f17f0076400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1814157030 0x7f18140b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18140b5a80 0x7f1814148b30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 --2- 172.21.15.46:0/1989027614 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1814074950 0x7f181414dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 >> 172.21.15.46:0/1989027614 conn(0x7f18140708c0 msgr2=0x7f18140d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 shutdown_connections 2024-09-17T03:28:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:51.914+0000 7f181b200640 1 -- 172.21.15.46:0/1989027614 wait complete. 2024-09-17T03:28:51.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:28:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 -- 172.21.15.46:0/2538142744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 msgr2=0x7fe88815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 --2- 172.21.15.46:0/2538142744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe88815bb90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe888074b20 tx=0x7fe87801cb30 comp rx=0 tx=0).stop 2024-09-17T03:28:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 -- 172.21.15.46:0/2538142744 shutdown_connections 2024-09-17T03:28:52.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 --2- 172.21.15.46:0/2538142744 >> v2:172.21.15.99:3300/0 conn(0x7fe8881527b0 0x7fe888074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 --2- 172.21.15.46:0/2538142744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe88815bb90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 -- 172.21.15.46:0/2538142744 >> 172.21.15.46:0/2538142744 conn(0x7fe8880b2040 msgr2=0x7fe8880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:52.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 -- 172.21.15.46:0/2538142744 shutdown_connections 2024-09-17T03:28:52.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.094+0000 7fe88de00640 1 -- 172.21.15.46:0/2538142744 wait complete. 2024-09-17T03:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 Processor -- start 2024-09-17T03:28:52.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 -- start start 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8881fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8880b5c70 0x7fe8881fc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8881527b0 0x7fe888153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe888168a50 con 0x7fe8880b4de0 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8881688d0 con 0x7fe8880b5c70 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe888168bd0 con 0x7fe8881527b0 2024-09-17T03:28:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8881fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8881fbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53444/0 (socket says 172.21.15.46:53444) 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 -- 172.21.15.46:0/742804064 learned_addr learned my addr 172.21.15.46:0/742804064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886000640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8880b5c70 0x7fe8881fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe887400640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8881527b0 0x7fe888153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 -- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8881527b0 msgr2=0x7fe888153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8881527b0 0x7fe888153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 -- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8880b5c70 msgr2=0x7fe8881fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8880b5c70 0x7fe8881fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 -- 172.21.15.46:0/742804064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe888154a10 con 0x7fe8880b4de0 2024-09-17T03:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886000640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8880b5c70 0x7fe8881fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe886a00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8881fbcc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe87801c5a0 tx=0x7fe878005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780b2040 con 0x7fe8880b4de0 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe878004040 con 0x7fe8880b4de0 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878004470 con 0x7fe8880b4de0 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.098+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88814df50 con 0x7fe8880b4de0 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.102+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88814e510 con 0x7fe8880b4de0 2024-09-17T03:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.102+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888074e30 con 0x7fe8880b4de0 2024-09-17T03:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.102+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fe8780bd030 con 0x7fe8880b4de0 2024-09-17T03:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.102+0000 7fe87f400640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe868073ef0 0x7fe868076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.102+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fe87812d630 con 0x7fe8880b4de0 2024-09-17T03:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.102+0000 7fe886000640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe868073ef0 0x7fe868076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.106+0000 7fe886000640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe868073ef0 0x7fe868076400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe8880b2810 tx=0x7fe8740091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:52.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.106+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8780c4060 con 0x7fe8880b4de0 2024-09-17T03:28:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8880b2840 con 0x7fe8880b4de0 2024-09-17T03:28:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe87f400640 1 -- 172.21.15.46:0/742804064 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fe888075580 con 0x7fe8880b4de0 2024-09-17T03:28:52.321 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe868073ef0 msgr2=0x7fe868076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe868073ef0 0x7fe868076400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe8880b2810 tx=0x7fe8740091e0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 msgr2=0x7fe8881fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8881fbcc0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe87801c5a0 tx=0x7fe878005ae0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 shutdown_connections 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe868073ef0 0x7fe868076400 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8881527b0 0x7fe888153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8880b5c70 0x7fe8881fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 --2- 172.21.15.46:0/742804064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8881fbcc0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 >> 172.21.15.46:0/742804064 conn(0x7fe8880b2040 msgr2=0x7fe888151610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 shutdown_connections 2024-09-17T03:28:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52.318+0000 7fe88de00640 1 -- 172.21.15.46:0/742804064 wait complete. 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:28:52.344 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:28:52.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.345 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:28:52.356 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:28:52.356 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:28:52.356 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:28:52.357 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.357 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:28:52.357 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.357 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:28:52.357 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:28:52.357 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:28:52.358 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:28:52.358 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:28:52.358 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:28:52.358 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:28:52.358 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41956sec preferred_lft 41956sec 2024-09-17T03:28:52.358 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:28:52.359 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.359 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:28:52.359 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:28:52.359 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:28:52.359 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:28:52.359 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:28:52.360 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:28:52.360 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:28:52.360 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:28:52.360 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:28:52.360 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.361 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:28:52.361 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:28:52.361 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:28:52.361 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.361 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:28:52.361 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.362 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:28:52.362 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:28:52.362 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:28:52.362 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.362 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:28:52.362 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:28:52.363 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:28:52.363 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:28:52.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.363 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:28:52.420 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:28:52.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.423 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:28:52.481 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:28:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:28:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:28:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:28:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:28:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:28:52.483 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:28:52.484 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:28:52.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:28:52.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:28:52.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:28:52.484 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:28:52.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:28:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:28:52.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.620 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:28:52.671 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:28:52.671 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:28:52.672 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:28:52.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:28:52.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:28:52.739 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:28:52.796 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:28:52.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:28:52.796 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:28:52.852 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:28:52.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:28:52.957 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:28:52.957 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:28:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:28:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:28:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:28:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:28:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:28:52.973 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:28:52.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:52.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:28:52.994 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:28:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:28:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:28:53.100 INFO:teuthology.orchestra.run.smithi046.stdout:client246441 2024-09-17T03:28:53.100 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:53.100 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:53.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:28:53.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 -- 172.21.15.46:0/947414322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b4de0 msgr2=0x7f9bec0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/947414322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b4de0 0x7f9bec0b51f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9bec071cb0 tx=0x7f9be401ccc0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 -- 172.21.15.46:0/947414322 shutdown_connections 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/947414322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b4de0 0x7f9bec0b51f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/947414322 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bec1527b0 0x7f9bec074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/947414322 >> v2:172.21.15.99:3300/0 conn(0x7f9bec0b5c70 0x7f9bec15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 -- 172.21.15.46:0/947414322 >> 172.21.15.46:0/947414322 conn(0x7f9bec0b2040 msgr2=0x7f9bec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 -- 172.21.15.46:0/947414322 shutdown_connections 2024-09-17T03:28:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.310+0000 7f9bf2e00640 1 -- 172.21.15.46:0/947414322 wait complete. 2024-09-17T03:28:53.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 Processor -- start 2024-09-17T03:28:53.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 -- start start 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bec0b4de0 0x7f9bec0cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 0x7f9bec1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bec1527b0 0x7f9bec158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec15f730 con 0x7f9bec0b5c70 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec15f5b0 con 0x7f9bec0b4de0 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec15f8b0 con 0x7f9bec1527b0 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 0x7f9bec1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 0x7f9bec1517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37232/0 (socket says 172.21.15.46:37232) 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 -- 172.21.15.46:0/963198928 learned_addr learned my addr 172.21.15.46:0/963198928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bebe00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bec0b4de0 0x7f9bec0cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf0e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bec1527b0 0x7f9bec158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 -- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bec1527b0 msgr2=0x7f9bec158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bec1527b0 0x7f9bec158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 -- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bec0b4de0 msgr2=0x7f9bec0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bec0b4de0 0x7f9bec0cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 -- 172.21.15.46:0/963198928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec06f030 con 0x7f9bec0b5c70 2024-09-17T03:28:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be3e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 0x7f9bec1517a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400d6a0 tx=0x7f9bd400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4017070 con 0x7f9bec0b5c70 2024-09-17T03:28:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bd4004310 con 0x7f9bec0b5c70 2024-09-17T03:28:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4005280 con 0x7f9bec0b5c70 2024-09-17T03:28:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec1fc230 con 0x7f9bec0b5c70 2024-09-17T03:28:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.314+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bec1fc7f0 con 0x7f9bec0b5c70 2024-09-17T03:28:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.318+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bec15fdd0 con 0x7f9bec0b5c70 2024-09-17T03:28:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.318+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f9bd4005420 con 0x7f9bec0b5c70 2024-09-17T03:28:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.318+0000 7f9be8c00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bc8073f20 0x7f9bc8076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.318+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f9bd4092ea0 con 0x7f9bec0b5c70 2024-09-17T03:28:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.318+0000 7f9bebe00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bc8073f20 0x7f9bc8076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.318+0000 7f9bebe00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bc8073f20 0x7f9bc8076430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00bad0 tx=0x7f9bdc00b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.322+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9bd4020020 con 0x7f9bec0b5c70 2024-09-17T03:28:53.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.538+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9bec15a6b0 con 0x7f9bec0b5c70 2024-09-17T03:28:53.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9be8c00640 1 -- 172.21.15.46:0/963198928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v46) ==== 136+0+0 (secure 0 0 0) 0x7f9bd405f320 con 0x7f9bec0b5c70 2024-09-17T03:28:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bc8073f20 msgr2=0x7f9bc8076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bc8073f20 0x7f9bc8076430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00bad0 tx=0x7f9bdc00b260 comp rx=0 tx=0).stop 2024-09-17T03:28:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 msgr2=0x7f9bec1517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 0x7f9bec1517a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400d6a0 tx=0x7f9bd400db60 comp rx=0 tx=0).stop 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 shutdown_connections 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bc8073f20 0x7f9bc8076430 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bec1527b0 0x7f9bec158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bec0b5c70 0x7f9bec1517a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 --2- 172.21.15.46:0/963198928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bec0b4de0 0x7f9bec0cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 >> 172.21.15.46:0/963198928 conn(0x7f9bec0b2040 msgr2=0x7f9bec06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 shutdown_connections 2024-09-17T03:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:53.878+0000 7f9bf2e00640 1 -- 172.21.15.46:0/963198928 wait complete. 2024-09-17T03:28:53.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:28:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 -- 172.21.15.46:0/412952618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b5c70 msgr2=0x7f1ee815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 --2- 172.21.15.46:0/412952618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b5c70 0x7f1ee815bb60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8070710 tx=0x7f1ed801c890 comp rx=0 tx=0).stop 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 -- 172.21.15.46:0/412952618 shutdown_connections 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 --2- 172.21.15.46:0/412952618 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee80b4de0 0x7f1ee80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 --2- 172.21.15.46:0/412952618 >> v2:172.21.15.99:3300/0 conn(0x7f1ee81527b0 0x7f1ee8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 --2- 172.21.15.46:0/412952618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b5c70 0x7f1ee815bb60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 -- 172.21.15.46:0/412952618 >> 172.21.15.46:0/412952618 conn(0x7f1ee80b2040 msgr2=0x7f1ee80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 -- 172.21.15.46:0/412952618 shutdown_connections 2024-09-17T03:28:54.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 -- 172.21.15.46:0/412952618 wait complete. 2024-09-17T03:28:54.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 Processor -- start 2024-09-17T03:28:54.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.062+0000 7f1eece00640 1 -- start start 2024-09-17T03:28:54.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 0x7f1ee8167700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ee80b5c70 0x7f1ee813bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee81527b0 0x7f1ee80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee815f6d0 con 0x7f1ee80b4de0 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee815f550 con 0x7f1ee80b5c70 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee815f850 con 0x7f1ee81527b0 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 0x7f1ee8167700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 0x7f1ee8167700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37254/0 (socket says 172.21.15.46:37254) 2024-09-17T03:28:54.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 -- 172.21.15.46:0/2570851187 learned_addr learned my addr 172.21.15.46:0/2570851187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee6800640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee81527b0 0x7f1ee80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 -- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee81527b0 msgr2=0x7f1ee80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1edfe00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ee80b5c70 0x7f1ee813bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee81527b0 0x7f1ee80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 -- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ee80b5c70 msgr2=0x7f1ee813bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ee80b5c70 0x7f1ee813bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 -- 172.21.15.46:0/2570851187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee8139320 con 0x7f1ee80b4de0 2024-09-17T03:28:54.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee6800640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee81527b0 0x7f1ee80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1edfe00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ee80b5c70 0x7f1ee813bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ee5e00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 0x7f1ee8167700 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1ed801af40 tx=0x7f1ed8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80b2040 con 0x7f1ee80b4de0 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed8005030 con 0x7f1ee80b4de0 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8005460 con 0x7f1ee80b4de0 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee81682c0 con 0x7f1ee80b4de0 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee814ce80 con 0x7f1ee80b4de0 2024-09-17T03:28:54.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee81394f0 con 0x7f1ee80b4de0 2024-09-17T03:28:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.066+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f1ed80be060 con 0x7f1ee80b4de0 2024-09-17T03:28:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.070+0000 7f1ede000640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ec4073ef0 0x7f1ec4076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.070+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f1ed812c250 con 0x7f1ee80b4de0 2024-09-17T03:28:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.070+0000 7f1edfe00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ec4073ef0 0x7f1ec4076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.070+0000 7f1edfe00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ec4073ef0 0x7f1ec4076400 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0005fd0 tx=0x7f1ed0005de0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.074+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1ed80164b0 con 0x7f1ee80b4de0 2024-09-17T03:28:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.270+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1ee806b540 con 0x7f1ec4073ef0 2024-09-17T03:28:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.270+0000 7f1ede000640 1 -- 172.21.15.46:0/2570851187 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1ed00096f0 con 0x7f1ec4073ef0 2024-09-17T03:28:54.274 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:28:54.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:28:54.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:28:54.274 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:28:54.274 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:28:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ec4073ef0 msgr2=0x7f1ec4076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ec4073ef0 0x7f1ec4076400 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0005fd0 tx=0x7f1ed0005de0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 msgr2=0x7f1ee8167700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 0x7f1ee8167700 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1ed801af40 tx=0x7f1ed8004040 comp rx=0 tx=0).stop 2024-09-17T03:28:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 shutdown_connections 2024-09-17T03:28:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ec4073ef0 0x7f1ec4076400 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1ee81527b0 0x7f1ee80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ee80b5c70 0x7f1ee813bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 --2- 172.21.15.46:0/2570851187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ee80b4de0 0x7f1ee8167700 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 >> 172.21.15.46:0/2570851187 conn(0x7f1ee80b2040 msgr2=0x7f1ee815d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 shutdown_connections 2024-09-17T03:28:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.274+0000 7f1eece00640 1 -- 172.21.15.46:0/2570851187 wait complete. 2024-09-17T03:28:54.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:28:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd6b400640 1 -- 172.21.15.46:0/2983334850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd4c005220 msgr2=0x7fbd4c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2983334850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd4c005220 0x7fbd4c00b9d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbd640cca90 tx=0x7fbd5c01d630 comp rx=0 tx=0).stop 2024-09-17T03:28:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd69400640 1 -- 172.21.15.46:0/2983334850 reap_dead start 2024-09-17T03:28:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd6b400640 1 -- 172.21.15.46:0/2983334850 shutdown_connections 2024-09-17T03:28:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd6b400640 1 -- 172.21.15.46:0/2983334850 >> 172.21.15.46:0/2983334850 conn(0x7fbd640b2040 msgr2=0x7fbd640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd6b400640 1 -- 172.21.15.46:0/2983334850 shutdown_connections 2024-09-17T03:28:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.458+0000 7fbd6b400640 1 -- 172.21.15.46:0/2983334850 wait complete. 2024-09-17T03:28:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 Processor -- start 2024-09-17T03:28:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 -- start start 2024-09-17T03:28:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4c005220 0x7fbd6414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 0x7fbd6415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd640ac080 0x7fbd641f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6415cba0 con 0x7fbd640adf10 2024-09-17T03:28:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6415bf00 con 0x7fbd4c005220 2024-09-17T03:28:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6415cd20 con 0x7fbd640ac080 2024-09-17T03:28:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 0x7fbd6415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 0x7fbd6415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37264/0 (socket says 172.21.15.46:37264) 2024-09-17T03:28:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 -- 172.21.15.46:0/2755447357 learned_addr learned my addr 172.21.15.46:0/2755447357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd68a00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4c005220 0x7fbd6414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd69400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd640ac080 0x7fbd641f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 -- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd640ac080 msgr2=0x7fbd641f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd640ac080 0x7fbd641f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 -- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4c005220 msgr2=0x7fbd6414b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4c005220 0x7fbd6414b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 -- 172.21.15.46:0/2755447357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6414c090 con 0x7fbd640adf10 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd63e00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 0x7fbd6415a860 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001870 tx=0x7fbd50012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50013980 con 0x7fbd640adf10 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd50017030 con 0x7fbd640adf10 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50013da0 con 0x7fbd640adf10 2024-09-17T03:28:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6414c370 con 0x7fbd640adf10 2024-09-17T03:28:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.462+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd640ae350 con 0x7fbd640adf10 2024-09-17T03:28:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.466+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6414be00 con 0x7fbd640adf10 2024-09-17T03:28:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.466+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fbd500103e0 con 0x7fbd640adf10 2024-09-17T03:28:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.466+0000 7fbd60c00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd44073ef0 0x7fbd44076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.466+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fbd500967b0 con 0x7fbd640adf10 2024-09-17T03:28:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.466+0000 7fbd68a00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd44073ef0 0x7fbd44076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.466+0000 7fbd68a00640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd44073ef0 0x7fbd44076400 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbd58006fd0 tx=0x7fbd58006b20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:54.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.470+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd50062c60 con 0x7fbd640adf10 2024-09-17T03:28:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.646+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbd64073cf0 con 0x7fbd640adf10 2024-09-17T03:28:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.646+0000 7fbd60c00640 1 -- 172.21.15.46:0/2755447357 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fbd50067690 con 0x7fbd640adf10 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd44073ef0 msgr2=0x7fbd44076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd44073ef0 0x7fbd44076400 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbd58006fd0 tx=0x7fbd58006b20 comp rx=0 tx=0).stop 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 msgr2=0x7fbd6415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 0x7fbd6415a860 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001870 tx=0x7fbd50012aa0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 shutdown_connections 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd44073ef0 0x7fbd44076400 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd640ac080 0x7fbd641f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd640adf10 0x7fbd6415a860 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 --2- 172.21.15.46:0/2755447357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd4c005220 0x7fbd6414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 >> 172.21.15.46:0/2755447357 conn(0x7fbd640b2040 msgr2=0x7fbd64150bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 shutdown_connections 2024-09-17T03:28:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.650+0000 7fbd6b400640 1 -- 172.21.15.46:0/2755447357 wait complete. 2024-09-17T03:28:54.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:28:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 -- 172.21.15.46:0/1793425538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c0b4de0 msgr2=0x7fce7c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 --2- 172.21.15.46:0/1793425538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c0b4de0 0x7fce7c0b51f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce78009c80 tx=0x7fce7801cce0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 -- 172.21.15.46:0/1793425538 shutdown_connections 2024-09-17T03:28:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 --2- 172.21.15.46:0/1793425538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c0b4de0 0x7fce7c0b51f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 --2- 172.21.15.46:0/1793425538 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce7c1527b0 0x7fce7c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 -- 172.21.15.46:0/1793425538 >> 172.21.15.46:0/1793425538 conn(0x7fce7c0b2040 msgr2=0x7fce7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 -- 172.21.15.46:0/1793425538 shutdown_connections 2024-09-17T03:28:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 -- 172.21.15.46:0/1793425538 wait complete. 2024-09-17T03:28:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 Processor -- start 2024-09-17T03:28:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.834+0000 7fce84800640 1 -- start start 2024-09-17T03:28:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c0b4de0 0x7fce7c140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce7c0b5c70 0x7fce7c1fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 0x7fce7c13ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c168ab0 con 0x7fce7c1527b0 2024-09-17T03:28:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c168930 con 0x7fce7c0b4de0 2024-09-17T03:28:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c168c30 con 0x7fce7c0b5c70 2024-09-17T03:28:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 0x7fce7c13ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 0x7fce7c13ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37282/0 (socket says 172.21.15.46:37282) 2024-09-17T03:28:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 -- 172.21.15.46:0/2590885000 learned_addr learned my addr 172.21.15.46:0/2590885000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce81400640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce7c0b5c70 0x7fce7c1fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce81e00640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c0b4de0 0x7fce7c140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 -- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce7c0b5c70 msgr2=0x7fce7c1fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce7c0b5c70 0x7fce7c1fc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 -- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c0b4de0 msgr2=0x7fce7c140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c0b4de0 0x7fce7c140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 -- 172.21.15.46:0/2590885000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c0cfaf0 con 0x7fce7c1527b0 2024-09-17T03:28:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce82800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 0x7fce7c13ee50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce7801b390 tx=0x7fce78009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780bb070 con 0x7fce7c1527b0 2024-09-17T03:28:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce78004d40 con 0x7fce7c1527b0 2024-09-17T03:28:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c1fbd30 con 0x7fce7c1527b0 2024-09-17T03:28:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780b2040 con 0x7fce7c1527b0 2024-09-17T03:28:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c141a30 con 0x7fce7c1527b0 2024-09-17T03:28:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.838+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7c0b51f0 con 0x7fce7c1527b0 2024-09-17T03:28:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.842+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fce780be070 con 0x7fce7c1527b0 2024-09-17T03:28:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.842+0000 7fce76000640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce60073ef0 0x7fce60076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.842+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fce7812cdc0 con 0x7fce7c1527b0 2024-09-17T03:28:54.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.842+0000 7fce81e00640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce60073ef0 0x7fce60076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:54.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.842+0000 7fce81e00640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce60073ef0 0x7fce60076400 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:54.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:54.846+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce78016990 con 0x7fce7c1527b0 2024-09-17T03:28:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.018+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fce7c0b2860 con 0x7fce7c1527b0 2024-09-17T03:28:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.018+0000 7fce76000640 1 -- 172.21.15.46:0/2590885000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7fce780f9270 con 0x7fce7c1527b0 2024-09-17T03:28:55.023 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce60073ef0 msgr2=0x7fce60076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce60073ef0 0x7fce60076400 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).stop 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 msgr2=0x7fce7c13ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 0x7fce7c13ee50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce7801b390 tx=0x7fce78009350 comp rx=0 tx=0).stop 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 shutdown_connections 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fce60073ef0 0x7fce60076400 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c1527b0 0x7fce7c13ee50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce7c0b5c70 0x7fce7c1fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 --2- 172.21.15.46:0/2590885000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c0b4de0 0x7fce7c140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 >> 172.21.15.46:0/2590885000 conn(0x7fce7c0b2040 msgr2=0x7fce7c1581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 shutdown_connections 2024-09-17T03:28:55.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.022+0000 7fce84800640 1 -- 172.21.15.46:0/2590885000 wait complete. 2024-09-17T03:28:55.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:28:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 -- 172.21.15.46:0/777451247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534073a20 msgr2=0x7f75340b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 --2- 172.21.15.46:0/777451247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534073a20 0x7f75340b56d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f751c009840 tx=0x7f751c01c820 comp rx=0 tx=0).stop 2024-09-17T03:28:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 -- 172.21.15.46:0/777451247 shutdown_connections 2024-09-17T03:28:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 --2- 172.21.15.46:0/777451247 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7534070430 0x7f75340b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 --2- 172.21.15.46:0/777451247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534073a20 0x7f75340b56d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 -- 172.21.15.46:0/777451247 >> 172.21.15.46:0/777451247 conn(0x7f753406cfc0 msgr2=0x7f753406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 -- 172.21.15.46:0/777451247 shutdown_connections 2024-09-17T03:28:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.206+0000 7f7539200640 1 -- 172.21.15.46:0/777451247 wait complete. 2024-09-17T03:28:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 Processor -- start 2024-09-17T03:28:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 -- start start 2024-09-17T03:28:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7534070020 0x7f753413be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534070430 0x7f753413c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 0x7f7534151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f753416bba0 con 0x7f7534158810 2024-09-17T03:28:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f753416ba20 con 0x7f7534070430 2024-09-17T03:28:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7539200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f753416bd20 con 0x7f7534070020 2024-09-17T03:28:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 0x7f7534151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 0x7f7534151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37290/0 (socket says 172.21.15.46:37290) 2024-09-17T03:28:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 -- 172.21.15.46:0/3058687034 learned_addr learned my addr 172.21.15.46:0/3058687034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7531e00640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7534070020 0x7f753413be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7529400640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534070430 0x7f753413c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 -- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7534070020 msgr2=0x7f753413be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7534070020 0x7f753413be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 -- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534070430 msgr2=0x7f753413c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534070430 0x7f753413c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 -- 172.21.15.46:0/3058687034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7534152980 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.210+0000 7f7532800640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 0x7f7534151150 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f752c00eeb0 tx=0x7f752c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c003980 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f752c003b40 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c004020 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753413ca00 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7534150670 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75340b56d0 con 0x7f7534158810 2024-09-17T03:28:55.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f752c0041c0 con 0x7f7534158810 2024-09-17T03:28:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f752aa00640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7510073ef0 0x7f7510076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f752c0935f0 con 0x7f7534158810 2024-09-17T03:28:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.214+0000 7f7531e00640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7510073ef0 0x7f7510076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.218+0000 7f7531e00640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7510073ef0 0x7f7510076400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7524007fd0 tx=0x7f7524009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:55.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.218+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f752c05f780 con 0x7f7534158810 2024-09-17T03:28:55.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.394+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f75340afb70 con 0x7f7534158810 2024-09-17T03:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.398+0000 7f752aa00640 1 -- 172.21.15.46:0/3058687034 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7f752c0641b0 con 0x7f7534158810 2024-09-17T03:28:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7510073ef0 msgr2=0x7f7510076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7510073ef0 0x7f7510076400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7524007fd0 tx=0x7f7524009040 comp rx=0 tx=0).stop 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 msgr2=0x7f7534151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 0x7f7534151150 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f752c00eeb0 tx=0x7f752c00c740 comp rx=0 tx=0).stop 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 shutdown_connections 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7510073ef0 0x7f7510076400 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7534158810 0x7f7534151150 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7534070430 0x7f753413c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 --2- 172.21.15.46:0/3058687034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7534070020 0x7f753413be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 >> 172.21.15.46:0/3058687034 conn(0x7f753406cfc0 msgr2=0x7f753415c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 shutdown_connections 2024-09-17T03:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.402+0000 7f7539200640 1 -- 172.21.15.46:0/3058687034 wait complete. 2024-09-17T03:28:55.420 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:55.418+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:55.420 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:55.418+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:55.420 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:55.418+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:55.420 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:55.418+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:55.421 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:55.418+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:55.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-17T03:28:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 -- 172.21.15.46:0/2770201504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc1527b0 msgr2=0x7f0fbc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 --2- 172.21.15.46:0/2770201504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc1527b0 0x7f0fbc074e30 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc072e30 tx=0x7f0fac01c800 comp rx=0 tx=0).stop 2024-09-17T03:28:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 -- 172.21.15.46:0/2770201504 shutdown_connections 2024-09-17T03:28:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 --2- 172.21.15.46:0/2770201504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc1527b0 0x7f0fbc074e30 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 --2- 172.21.15.46:0/2770201504 >> v2:172.21.15.99:3300/0 conn(0x7f0fbc0b4de0 0x7f0fbc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 --2- 172.21.15.46:0/2770201504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fbc0b5c70 0x7f0fbc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 -- 172.21.15.46:0/2770201504 >> 172.21.15.46:0/2770201504 conn(0x7f0fbc0b2040 msgr2=0x7f0fbc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 -- 172.21.15.46:0/2770201504 shutdown_connections 2024-09-17T03:28:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 -- 172.21.15.46:0/2770201504 wait complete. 2024-09-17T03:28:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 Processor -- start 2024-09-17T03:28:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.590+0000 7f0fc4800640 1 -- start start 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 0x7f0fbc1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fbc0b5c70 0x7f0fbc148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fbc1527b0 0x7f0fbc1fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc15f700 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc15f580 con 0x7f0fbc0b5c70 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc15f880 con 0x7f0fbc1527b0 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 0x7f0fbc1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 0x7f0fbc1561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37334/0 (socket says 172.21.15.46:37334) 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 -- 172.21.15.46:0/868214300 learned_addr learned my addr 172.21.15.46:0/868214300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc2800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fbc1527b0 0x7f0fbc1fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1400640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fbc0b5c70 0x7f0fbc148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 -- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fbc1527b0 msgr2=0x7f0fbc1fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fbc1527b0 0x7f0fbc1fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 -- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fbc0b5c70 msgr2=0x7f0fbc148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fbc0b5c70 0x7f0fbc148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 -- 172.21.15.46:0/868214300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc156840 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc1e00640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 0x7f0fbc1561c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000d780 tx=0x7f0fb000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0017070 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fb000de90 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc154690 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0005020 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc0ae8e0 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.594+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc15fda0 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.598+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f0fb00051c0 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.598+0000 7f0fb6000640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fa0073f20 0x7f0fa0076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.598+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f0fb0092c90 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.598+0000 7f0fc1400640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fa0073f20 0x7f0fa0076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.598+0000 7f0fc1400640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fa0073f20 0x7f0fa0076430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0af000 tx=0x7f0fac0a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:55.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.602+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0fb005f140 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.778+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f0fbc15c660 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.782+0000 7f0fb6000640 1 -- 172.21.15.46:0/868214300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7f0fb0063b70 con 0x7f0fbc0b4de0 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fa0073f20 msgr2=0x7f0fa0076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fa0073f20 0x7f0fa0076430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0af000 tx=0x7f0fac0a7040 comp rx=0 tx=0).stop 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 msgr2=0x7f0fbc1561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 0x7f0fbc1561c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000d780 tx=0x7f0fb000dc40 comp rx=0 tx=0).stop 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 shutdown_connections 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fa0073f20 0x7f0fa0076430 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fbc1527b0 0x7f0fbc1fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fbc0b5c70 0x7f0fbc148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 --2- 172.21.15.46:0/868214300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc0b4de0 0x7f0fbc1561c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 >> 172.21.15.46:0/868214300 conn(0x7f0fbc0b2040 msgr2=0x7f0fbc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 shutdown_connections 2024-09-17T03:28:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.786+0000 7f0fc4800640 1 -- 172.21.15.46:0/868214300 wait complete. 2024-09-17T03:28:55.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:28:55.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 -- 172.21.15.46:0/2710758167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5640b5c70 msgr2=0x7fa56415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 --2- 172.21.15.46:0/2710758167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5640b5c70 0x7fa56415bb60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa554009900 tx=0x7fa55401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 -- 172.21.15.46:0/2710758167 shutdown_connections 2024-09-17T03:28:55.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 --2- 172.21.15.46:0/2710758167 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5640b4de0 0x7fa5640b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 --2- 172.21.15.46:0/2710758167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5640b5c70 0x7fa56415bb60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 -- 172.21.15.46:0/2710758167 >> 172.21.15.46:0/2710758167 conn(0x7fa5640b2040 msgr2=0x7fa5640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:55.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 -- 172.21.15.46:0/2710758167 shutdown_connections 2024-09-17T03:28:55.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 -- 172.21.15.46:0/2710758167 wait complete. 2024-09-17T03:28:55.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 Processor -- start 2024-09-17T03:28:55.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.974+0000 7fa56ac00640 1 -- start start 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5640b4de0 0x7fa5640abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5640b5c70 0x7fa5641fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 0x7fa5641417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa564168a20 con 0x7fa5641527b0 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641688a0 con 0x7fa5640b4de0 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa564168ba0 con 0x7fa5640b5c70 2024-09-17T03:28:55.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 0x7fa5641417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 0x7fa5641417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37342/0 (socket says 172.21.15.46:37342) 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 -- 172.21.15.46:0/701720244 learned_addr learned my addr 172.21.15.46:0/701720244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa563e00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5640b4de0 0x7fa5640abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa55be00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5640b5c70 0x7fa5641fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 -- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5640b5c70 msgr2=0x7fa5641fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5640b5c70 0x7fa5641fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 -- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5640b4de0 msgr2=0x7fa5640abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5640b4de0 0x7fa5640abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 -- 172.21.15.46:0/701720244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5641faa20 con 0x7fa5641527b0 2024-09-17T03:28:55.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa568c00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 0x7fa5641417f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ea80 tx=0x7fa55c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00cea0 con 0x7fa5641527b0 2024-09-17T03:28:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa55c004b30 con 0x7fa5641527b0 2024-09-17T03:28:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c005020 con 0x7fa5641527b0 2024-09-17T03:28:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5641fad00 con 0x7fa5641527b0 2024-09-17T03:28:55.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5640aa4d0 con 0x7fa5641527b0 2024-09-17T03:28:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.978+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5641fa790 con 0x7fa5641527b0 2024-09-17T03:28:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.982+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7fa55c0051c0 con 0x7fa5641527b0 2024-09-17T03:28:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.982+0000 7fa560c00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa540073ef0 0x7fa540076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.982+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fa55c093bd0 con 0x7fa5641527b0 2024-09-17T03:28:55.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.982+0000 7fa563e00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa540073ef0 0x7fa540076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:55.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.982+0000 7fa563e00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa540073ef0 0x7fa540076400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa5540af000 tx=0x7fa55401b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:55.986+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa55c060080 con 0x7fa5641527b0 2024-09-17T03:28:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.182+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fa56406b540 con 0x7fa540073ef0 2024-09-17T03:28:56.195 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:56.194+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:56.195 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:56.194+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:56.196 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:56.194+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:56.196 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:56.194+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:56.196 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:28:56.194+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:28:56.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa560c00640 1 -- 172.21.15.46:0/701720244 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa5540b3040 con 0x7fa540073ef0 2024-09-17T03:28:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa540073ef0 msgr2=0x7fa540076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa540073ef0 0x7fa540076400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa5540af000 tx=0x7fa55401b0f0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 msgr2=0x7fa5641417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 0x7fa5641417f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ea80 tx=0x7fa55c00ef40 comp rx=0 tx=0).stop 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 shutdown_connections 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa540073ef0 0x7fa540076400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5641527b0 0x7fa5641417f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa5640b5c70 0x7fa5641fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 --2- 172.21.15.46:0/701720244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5640b4de0 0x7fa5640abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 >> 172.21.15.46:0/701720244 conn(0x7fa5640b2040 msgr2=0x7fa5640cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 shutdown_connections 2024-09-17T03:28:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.226+0000 7fa56ac00640 1 -- 172.21.15.46:0/701720244 wait complete. 2024-09-17T03:28:56.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 -- 172.21.15.46:0/3684472419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500b5a80 msgr2=0x7f3550074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 --2- 172.21.15.46:0/3684472419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500b5a80 0x7f3550074410 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3538009900 tx=0x7f353801c700 comp rx=0 tx=0).stop 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 -- 172.21.15.46:0/3684472419 shutdown_connections 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 --2- 172.21.15.46:0/3684472419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3550074950 0x7f355015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 --2- 172.21.15.46:0/3684472419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35500b5a80 0x7f3550074410 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 -- 172.21.15.46:0/3684472419 >> 172.21.15.46:0/3684472419 conn(0x7f35500708c0 msgr2=0x7f3550070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 -- 172.21.15.46:0/3684472419 shutdown_connections 2024-09-17T03:28:56.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.410+0000 7f3556400640 1 -- 172.21.15.46:0/3684472419 wait complete. 2024-09-17T03:28:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 Processor -- start 2024-09-17T03:28:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 -- start start 2024-09-17T03:28:56.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 0x7f35501fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:56.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35500b5a80 0x7f3550152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:56.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3550157030 0x7f3550154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f355016a310 con 0x7f3550074950 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f355016a190 con 0x7f35500b5a80 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3556400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f355016a490 con 0x7f3550157030 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 0x7f35501fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 0x7f35501fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37394/0 (socket says 172.21.15.46:37394) 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 -- 172.21.15.46:0/3696152752 learned_addr learned my addr 172.21.15.46:0/3696152752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354fe00640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3550157030 0x7f3550154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f3547e00640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35500b5a80 0x7f3550152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 -- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3550157030 msgr2=0x7f3550154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3550157030 0x7f3550154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 -- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35500b5a80 msgr2=0x7f3550152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35500b5a80 0x7f3550152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 -- 172.21.15.46:0/3696152752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3550153b30 con 0x7f3550074950 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.414+0000 7f354f400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 0x7f35501fbd50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f354000b410 tx=0x7f354000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540015030 con 0x7f3550074950 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3540004020 con 0x7f3550074950 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35400044b0 con 0x7f3550074950 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3550153e10 con 0x7f3550074950 2024-09-17T03:28:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35500b24a0 con 0x7f3550074950 2024-09-17T03:28:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355015cf90 con 0x7f3550074950 2024-09-17T03:28:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f3540018070 con 0x7f3550074950 2024-09-17T03:28:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3547400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f352c073ef0 0x7f352c076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:28:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.418+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f3540092690 con 0x7f3550074950 2024-09-17T03:28:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.422+0000 7f3547e00640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f352c073ef0 0x7f352c076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:28:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.422+0000 7f3547e00640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f352c073ef0 0x7f352c076400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f35380ae000 tx=0x7f353801af40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:28:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.426+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f354005eb40 con 0x7f3550074950 2024-09-17T03:28:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.626+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f355006b540 con 0x7f352c073ef0 2024-09-17T03:28:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.630+0000 7f3547400640 1 -- 172.21.15.46:0/3696152752 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f3538004050 con 0x7f352c073ef0 2024-09-17T03:28:56.633 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:28:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.630+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f352c073ef0 msgr2=0x7f352c076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.630+0000 7f3556400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f352c073ef0 0x7f352c076400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f35380ae000 tx=0x7f353801af40 comp rx=0 tx=0).stop 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.630+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 msgr2=0x7f35501fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.630+0000 7f3556400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 0x7f35501fbd50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f354000b410 tx=0x7f354000b8d0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 shutdown_connections 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f352c073ef0 0x7f352c076400 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3550157030 0x7f3550154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35500b5a80 0x7f3550152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 --2- 172.21.15.46:0/3696152752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3550074950 0x7f35501fbd50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 >> 172.21.15.46:0/3696152752 conn(0x7f35500708c0 msgr2=0x7f3550151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 shutdown_connections 2024-09-17T03:28:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:28:56.634+0000 7f3556400640 1 -- 172.21.15.46:0/3696152752 wait complete. 2024-09-17T03:28:56.656 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:28:56.656 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:28:56.656 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:28:56.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:28:56.665 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:28:59.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:28:59.766 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:29:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:29:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:29:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.32874 s, 129 MB/s 2024-09-17T03:29:05.057 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:29:05.057 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:29:05.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:29:05.067 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:29:13.647 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:29:13.647 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:29:13.647 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.52498 s, 126 MB/s 2024-09-17T03:29:13.648 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:29:13.648 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:29:13.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:29:13.658 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:29:21.983 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.27403 s, 130 MB/s 2024-09-17T03:29:21.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:29:22.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 -- 172.21.15.46:0/1835037767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3594004810 msgr2=0x7f3594004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 --2- 172.21.15.46:0/1835037767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3594004810 0x7f3594004c40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3598009b60 tx=0x7f359801d240 comp rx=0 tx=0).stop 2024-09-17T03:29:22.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 -- 172.21.15.46:0/1835037767 shutdown_connections 2024-09-17T03:29:22.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 --2- 172.21.15.46:0/1835037767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3594004810 0x7f3594004c40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 -- 172.21.15.46:0/1835037767 >> 172.21.15.46:0/1835037767 conn(0x7f35b00b2040 msgr2=0x7f35b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:22.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 -- 172.21.15.46:0/1835037767 shutdown_connections 2024-09-17T03:29:22.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.150+0000 7f35b5000640 1 -- 172.21.15.46:0/1835037767 wait complete. 2024-09-17T03:29:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 Processor -- start 2024-09-17T03:29:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 -- start start 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3594004810 0x7f35b01e7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 0x7f35b0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35b0141e00 0x7f35b01588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35b00b55d0 con 0x7f35b01419f0 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35b00b5450 con 0x7f3594004810 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f35b00b5750 con 0x7f35b0141e00 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 0x7f35b0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 0x7f35b0158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32872/0 (socket says 172.21.15.46:32872) 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 -- 172.21.15.46:0/3999539633 learned_addr learned my addr 172.21.15.46:0/3999539633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35ae800640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35b0141e00 0x7f35b01588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35ade00640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3594004810 0x7f35b01e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 -- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35b0141e00 msgr2=0x7f35b01588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35b0141e00 0x7f35b01588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 -- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3594004810 msgr2=0x7f35b01e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3594004810 0x7f35b01e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 -- 172.21.15.46:0/3999539633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b01e7a40 con 0x7f35b01419f0 2024-09-17T03:29:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a7e00640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 0x7f35b0158360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f35980af040 tx=0x7f3598005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980bd070 con 0x7f35b01419f0 2024-09-17T03:29:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3598004120 con 0x7f35b01419f0 2024-09-17T03:29:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b0142600 con 0x7f35b01419f0 2024-09-17T03:29:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980b1040 con 0x7f35b01419f0 2024-09-17T03:29:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.154+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b013fa00 con 0x7f35b01419f0 2024-09-17T03:29:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.158+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b0142370 con 0x7f35b01419f0 2024-09-17T03:29:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.158+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) ==== 98509+0+0 (secure 0 0 0) 0x7f35980c0070 con 0x7f35b01419f0 2024-09-17T03:29:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.158+0000 7f35a6000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f358c074010 0x7f358c076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.158+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f359812d190 con 0x7f35b01419f0 2024-09-17T03:29:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.158+0000 7f35ade00640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f358c074010 0x7f358c076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.158+0000 7f35ade00640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f358c074010 0x7f358c076520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f35b006a420 tx=0x7f35a0006730 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:22.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.162+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f35980a5e20 con 0x7f35b01419f0 2024-09-17T03:29:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.366+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f35b006b540 con 0x7f358c074010 2024-09-17T03:29:22.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35a6000640 1 -- 172.21.15.46:0/3999539633 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f35a0001600 con 0x7f358c074010 2024-09-17T03:29:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f358c074010 msgr2=0x7f358c076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f358c074010 0x7f358c076520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f35b006a420 tx=0x7f35a0006730 comp rx=0 tx=0).stop 2024-09-17T03:29:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 msgr2=0x7f35b0158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 0x7f35b0158360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f35980af040 tx=0x7f3598005520 comp rx=0 tx=0).stop 2024-09-17T03:29:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 shutdown_connections 2024-09-17T03:29:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f358c074010 0x7f358c076520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35b0141e00 0x7f35b01588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35b01419f0 0x7f35b0158360 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 --2- 172.21.15.46:0/3999539633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3594004810 0x7f35b01e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 >> 172.21.15.46:0/3999539633 conn(0x7f35b00b2040 msgr2=0x7f35b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 shutdown_connections 2024-09-17T03:29:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.382+0000 7f35b5000640 1 -- 172.21.15.46:0/3999539633 wait complete. 2024-09-17T03:29:22.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:29:22.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.566+0000 7fd7c4a00640 1 -- 172.21.15.46:0/3327079773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b5c70 msgr2=0x7fd7c015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.566+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/3327079773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b5c70 0x7fd7c015bb60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0070710 tx=0x7fd7b001c740 comp rx=0 tx=0).stop 2024-09-17T03:29:22.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- 172.21.15.46:0/3327079773 shutdown_connections 2024-09-17T03:29:22.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/3327079773 >> v2:172.21.15.99:3300/0 conn(0x7fd7c01527b0 0x7fd7c0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/3327079773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b5c70 0x7fd7c015bb60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- 172.21.15.46:0/3327079773 >> 172.21.15.46:0/3327079773 conn(0x7fd7c00b2040 msgr2=0x7fd7c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:22.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- 172.21.15.46:0/3327079773 shutdown_connections 2024-09-17T03:29:22.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- 172.21.15.46:0/3327079773 wait complete. 2024-09-17T03:29:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 Processor -- start 2024-09-17T03:29:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- start start 2024-09-17T03:29:22.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 0x7fd7c00af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7c00b5c70 0x7fd7c00cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7c01527b0 0x7fd7c01518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0168a80 con 0x7fd7c00b4de0 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0168900 con 0x7fd7c01527b0 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.570+0000 7fd7c4a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c0168c00 con 0x7fd7c00b5c70 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 0x7fd7c00af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 0x7fd7c00af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57476/0 (socket says 172.21.15.46:57476) 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 -- 172.21.15.46:0/1214012145 learned_addr learned my addr 172.21.15.46:0/1214012145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7b7e00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7c00b5c70 0x7fd7c00cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7be800640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7c01527b0 0x7fd7c01518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 -- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7c00b5c70 msgr2=0x7fd7c00cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7c00b5c70 0x7fd7c00cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 -- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7c01527b0 msgr2=0x7fd7c01518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7c01527b0 0x7fd7c01518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 -- 172.21.15.46:0/1214012145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c01f6a90 con 0x7fd7c00b4de0 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7be800640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7c01527b0 0x7fd7c01518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7bde00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 0x7fd7c00af0f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd7b001cc20 tx=0x7fd7b0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00b2040 con 0x7fd7c00b4de0 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7b0004130 con 0x7fd7c00b4de0 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0003600 con 0x7fd7c00b4de0 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c01f6d70 con 0x7fd7c00b4de0 2024-09-17T03:29:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c01f7330 con 0x7fd7c00b4de0 2024-09-17T03:29:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c015bb60 con 0x7fd7c00b4de0 2024-09-17T03:29:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.574+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) ==== 98509+0+0 (secure 0 0 0) 0x7fd7b00bd030 con 0x7fd7c00b4de0 2024-09-17T03:29:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.578+0000 7fd7b6000640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd79c074010 0x7fd79c076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.578+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fd7b012c8e0 con 0x7fd7c00b4de0 2024-09-17T03:29:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.578+0000 7fd7b7e00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd79c074010 0x7fd79c076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.578+0000 7fd7b7e00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd79c074010 0x7fd79c076520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0151100 tx=0x7fd7a80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.582+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd7b00c4060 con 0x7fd7c00b4de0 2024-09-17T03:29:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.782+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd7c006b540 con 0x7fd79c074010 2024-09-17T03:29:22.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.786+0000 7fd7b6000640 1 -- 172.21.15.46:0/1214012145 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd7a8011af0 con 0x7fd79c074010 2024-09-17T03:29:22.786 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd79c074010 msgr2=0x7fd79c076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd79c074010 0x7fd79c076520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0151100 tx=0x7fd7a80091e0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 msgr2=0x7fd7c00af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 0x7fd7c00af0f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd7b001cc20 tx=0x7fd7b0005db0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 shutdown_connections 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd79c074010 0x7fd79c076520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7c01527b0 0x7fd7c01518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7c00b5c70 0x7fd7c00cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 --2- 172.21.15.46:0/1214012145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7c00b4de0 0x7fd7c00af0f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 >> 172.21.15.46:0/1214012145 conn(0x7fd7c00b2040 msgr2=0x7fd7c0152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 shutdown_connections 2024-09-17T03:29:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:22.790+0000 7fd7c4a00640 1 -- 172.21.15.46:0/1214012145 wait complete. 2024-09-17T03:29:22.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:29:22.811 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:29:22.811 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:29:22.821 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:29:22.821 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:29:22.821 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:22.822 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:29:24.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:29:24.823 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:29:24.824 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:29:24.834 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/c4bb0623-ca62-4e78-aafc-64ead74c1923 2024-09-17T03:29:24.834 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T03:29:24.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:24.835 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T03:29:24.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone94810 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 -- 172.21.15.46:0/2855884115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc80b5c70 msgr2=0x7effc815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 --2- 172.21.15.46:0/2855884115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc80b5c70 0x7effc815bb60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7effb8009840 tx=0x7effb801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 -- 172.21.15.46:0/2855884115 shutdown_connections 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 --2- 172.21.15.46:0/2855884115 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effc80b4de0 0x7effc80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 --2- 172.21.15.46:0/2855884115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc80b5c70 0x7effc815bb60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 -- 172.21.15.46:0/2855884115 >> 172.21.15.46:0/2855884115 conn(0x7effc80b2040 msgr2=0x7effc80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 -- 172.21.15.46:0/2855884115 shutdown_connections 2024-09-17T03:29:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 -- 172.21.15.46:0/2855884115 wait complete. 2024-09-17T03:29:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 Processor -- start 2024-09-17T03:29:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.058+0000 7effce000640 1 -- start start 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effc80b4de0 0x7effc814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc80b5c70 0x7effc81fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 0x7effc80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7effc8168970 con 0x7effc81527b0 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effc81687f0 con 0x7effc80b5c70 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7effc8168af0 con 0x7effc80b4de0 2024-09-17T03:29:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 0x7effc80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 0x7effc80cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57492/0 (socket says 172.21.15.46:57492) 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 -- 172.21.15.46:0/3386614081 learned_addr learned my addr 172.21.15.46:0/3386614081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7400640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effc80b4de0 0x7effc814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effbea00640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc80b5c70 0x7effc81fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 -- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effc80b4de0 msgr2=0x7effc814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effc80b4de0 0x7effc814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 -- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc80b5c70 msgr2=0x7effc81fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc80b5c70 0x7effc81fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 -- 172.21.15.46:0/3386614081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc814fce0 con 0x7effc81527b0 2024-09-17T03:29:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effc7e00640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 0x7effc80cd4f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7effc000e940 tx=0x7effc000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0004d30 con 0x7effc81527b0 2024-09-17T03:29:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effc000cac0 con 0x7effc81527b0 2024-09-17T03:29:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0005020 con 0x7effc81527b0 2024-09-17T03:29:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc814ffc0 con 0x7effc81527b0 2024-09-17T03:29:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81f9df0 con 0x7effc81527b0 2024-09-17T03:29:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.062+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc80b51f0 con 0x7effc81527b0 2024-09-17T03:29:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.066+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) ==== 98509+0+0 (secure 0 0 0) 0x7effc00051c0 con 0x7effc81527b0 2024-09-17T03:29:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.066+0000 7effbfe00640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effa4074010 0x7effa4076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.066+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7effc0093840 con 0x7effc81527b0 2024-09-17T03:29:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.066+0000 7effc7400640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effa4074010 0x7effa4076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.066+0000 7effc7400640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effa4074010 0x7effa4076520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7effb80af000 tx=0x7effb801b030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:25.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.070+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7effc0016030 con 0x7effc81527b0 2024-09-17T03:29:25.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.278+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone94810", "target": ["mon-mgr", ""]}) -- 0x7effc806b540 con 0x7effa4074010 2024-09-17T03:29:25.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effbfe00640 1 -- 172.21.15.46:0/3386614081 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7effb80040c0 con 0x7effa4074010 2024-09-17T03:29:25.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effa4074010 msgr2=0x7effa4076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effa4074010 0x7effa4076520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7effb80af000 tx=0x7effb801b030 comp rx=0 tx=0).stop 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 msgr2=0x7effc80cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 0x7effc80cd4f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7effc000e940 tx=0x7effc000ee00 comp rx=0 tx=0).stop 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 shutdown_connections 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7effa4074010 0x7effa4076520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effc81527b0 0x7effc80cd4f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc80b5c70 0x7effc81fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 --2- 172.21.15.46:0/3386614081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effc80b4de0 0x7effc814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.334+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 >> 172.21.15.46:0/3386614081 conn(0x7effc80b2040 msgr2=0x7effc815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.338+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 shutdown_connections 2024-09-17T03:29:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.338+0000 7effce000640 1 -- 172.21.15.46:0/3386614081 wait complete. 2024-09-17T03:29:25.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.338+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.338+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.338+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.338+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.338+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.357 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.358+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.357 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.358+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.357 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.358+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.357 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.358+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.357 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:25.358+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:25.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone64201 2024-09-17T03:29:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 -- 172.21.15.46:0/2243027553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558155b50 msgr2=0x7f95580741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 --2- 172.21.15.46:0/2243027553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558155b50 0x7f95580741a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9550009bc0 tx=0x7f955001cb10 comp rx=0 tx=0).stop 2024-09-17T03:29:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 -- 172.21.15.46:0/2243027553 shutdown_connections 2024-09-17T03:29:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 --2- 172.21.15.46:0/2243027553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558155b50 0x7f95580741a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 -- 172.21.15.46:0/2243027553 >> 172.21.15.46:0/2243027553 conn(0x7f95580b2040 msgr2=0x7f95580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 -- 172.21.15.46:0/2243027553 shutdown_connections 2024-09-17T03:29:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.534+0000 7f955d200640 1 -- 172.21.15.46:0/2243027553 wait complete. 2024-09-17T03:29:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 Processor -- start 2024-09-17T03:29:25.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 -- start start 2024-09-17T03:29:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 0x7f95581353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558155b50 0x7f9558149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f955814a040 0x7f955814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9558169ed0 con 0x7f9558072100 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9558169d50 con 0x7f9558155b50 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f955816a050 con 0x7f955814a040 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 0x7f95581353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 0x7f95581353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57554/0 (socket says 172.21.15.46:57554) 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 -- 172.21.15.46:0/1262000370 learned_addr learned my addr 172.21.15.46:0/1262000370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9556800640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f955814a040 0x7f955814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f954fe00640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558155b50 0x7f9558149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 -- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f955814a040 msgr2=0x7f955814c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f955814a040 0x7f955814c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 -- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558155b50 msgr2=0x7f9558149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558155b50 0x7f9558149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:25.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 -- 172.21.15.46:0/1262000370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9558153c60 con 0x7f9558072100 2024-09-17T03:29:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f9555e00640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 0x7f95581353b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f954800c3f0 tx=0x7f954800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548017070 con 0x7f9558072100 2024-09-17T03:29:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9548003600 con 0x7f9558072100 2024-09-17T03:29:25.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955814a780 con 0x7f9558072100 2024-09-17T03:29:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548003a50 con 0x7f9558072100 2024-09-17T03:29:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.538+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955814ad40 con 0x7f9558072100 2024-09-17T03:29:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.542+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95580b4de0 con 0x7f9558072100 2024-09-17T03:29:25.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.542+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) ==== 98509+0+0 (secure 0 0 0) 0x7f954801a070 con 0x7f9558072100 2024-09-17T03:29:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.542+0000 7f954e000640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9534074040 0x7f9534076550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.542+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f9548092120 con 0x7f9558072100 2024-09-17T03:29:25.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.542+0000 7f954fe00640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9534074040 0x7f9534076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:25.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.542+0000 7f954fe00640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9534074040 0x7f9534076550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9540007fd0 tx=0x7f9540009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:25.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.546+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f954805e2c0 con 0x7f9558072100 2024-09-17T03:29:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.754+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone64201", "target": ["mon-mgr", ""]}) -- 0x7f955806b540 con 0x7f9534074040 2024-09-17T03:29:25.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:25.978+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 29) ==== 98611+0+0 (secure 0 0 0) 0x7f954805a530 con 0x7f9558072100 2024-09-17T03:29:26.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:26.338+0000 7fbe3b000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:26.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:26.338+0000 7fbe3b000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:26.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:26.338+0000 7fbe3b000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:26.338 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:26.338+0000 7fbe3b000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:26.338 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:26.338+0000 7fbe3b000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.434+0000 7f954e000640 1 -- 172.21.15.46:0/1262000370 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f954000f7d0 con 0x7f9534074040 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9534074040 msgr2=0x7f9534076550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9534074040 0x7f9534076550 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9540007fd0 tx=0x7f9540009040 comp rx=0 tx=0).stop 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 msgr2=0x7f95581353b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 0x7f95581353b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f954800c3f0 tx=0x7f954800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 shutdown_connections 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9534074040 0x7f9534076550 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f955814a040 0x7f955814c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558155b50 0x7f9558149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 --2- 172.21.15.46:0/1262000370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9558072100 0x7f95581353b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.438+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 >> 172.21.15.46:0/1262000370 conn(0x7f95580b2040 msgr2=0x7f955815ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.442+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 shutdown_connections 2024-09-17T03:29:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.442+0000 7f955d200640 1 -- 172.21.15.46:0/1262000370 wait complete. 2024-09-17T03:29:26.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone14462 2024-09-17T03:29:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 -- 172.21.15.46:0/2026493725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 msgr2=0x7fef1015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:26.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 --2- 172.21.15.46:0/2026493725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef1015e8c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fef00009900 tx=0x7fef0001c700 comp rx=0 tx=0).stop 2024-09-17T03:29:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 -- 172.21.15.46:0/2026493725 shutdown_connections 2024-09-17T03:29:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 --2- 172.21.15.46:0/2026493725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef1015ee00 0x7fef1015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 --2- 172.21.15.46:0/2026493725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef1015e8c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 -- 172.21.15.46:0/2026493725 >> 172.21.15.46:0/2026493725 conn(0x7fef100b2040 msgr2=0x7fef100b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 -- 172.21.15.46:0/2026493725 shutdown_connections 2024-09-17T03:29:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 -- 172.21.15.46:0/2026493725 wait complete. 2024-09-17T03:29:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 Processor -- start 2024-09-17T03:29:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.794+0000 7fef17600640 1 -- start start 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef10166380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1015b7b0 0x7fef1015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef1015ee00 0x7fef10168870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef1016a100 con 0x7fef100cf900 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef10169f80 con 0x7fef1015b7b0 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fef1016a280 con 0x7fef1015ee00 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef10166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef10166380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57596/0 (socket says 172.21.15.46:57596) 2024-09-17T03:29:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 -- 172.21.15.46:0/1060056819 learned_addr learned my addr 172.21.15.46:0/1060056819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:26.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 -- 172.21.15.46:0/1060056819 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef1015ee00 msgr2=0x7fef10168870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:26.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef1015ee00 0x7fef10168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 -- 172.21.15.46:0/1060056819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1015b7b0 msgr2=0x7fef1015f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:26.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1015b7b0 0x7fef1015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:26.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 -- 172.21.15.46:0/1060056819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef10160450 con 0x7fef100cf900 2024-09-17T03:29:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef14c00640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef10166380 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fef00009850 tx=0x7fef000aa5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef000aad10 con 0x7fef100cf900 2024-09-17T03:29:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef00003600 con 0x7fef100cf900 2024-09-17T03:29:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef101f9540 con 0x7fef100cf900 2024-09-17T03:29:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00003a10 con 0x7fef100cf900 2024-09-17T03:29:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.798+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef101f9b00 con 0x7fef100cf900 2024-09-17T03:29:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.802+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1015e8c0 con 0x7fef100cf900 2024-09-17T03:29:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.802+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 29) ==== 98611+0+0 (secure 0 0 0) 0x7fef00003bb0 con 0x7fef100cf900 2024-09-17T03:29:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.806+0000 7fef0d600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feeec074130 0x7feeec076640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.806+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fef0012e6d0 con 0x7fef100cf900 2024-09-17T03:29:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.806+0000 7fef07e00640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feeec074130 0x7feeec076640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.806+0000 7fef07e00640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feeec074130 0x7feeec076640 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef10169780 tx=0x7feef8006c00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:26.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:26.810+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fef1015f210 con 0x7fef100cf900 2024-09-17T03:29:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.286+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone14462", "target": ["mon-mgr", ""]}) -- 0x7fef1006b540 con 0x7feeec074130 2024-09-17T03:29:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.770+0000 7fef0d600640 1 -- 172.21.15.46:0/1060056819 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feef80092a0 con 0x7feeec074130 2024-09-17T03:29:27.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feeec074130 msgr2=0x7feeec076640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:27.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feeec074130 0x7feeec076640 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef10169780 tx=0x7feef8006c00 comp rx=0 tx=0).stop 2024-09-17T03:29:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 msgr2=0x7fef10166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef10166380 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fef00009850 tx=0x7fef000aa5d0 comp rx=0 tx=0).stop 2024-09-17T03:29:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 shutdown_connections 2024-09-17T03:29:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feeec074130 0x7feeec076640 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef1015ee00 0x7fef10168870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1015b7b0 0x7fef1015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 --2- 172.21.15.46:0/1060056819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef100cf900 0x7fef10166380 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 >> 172.21.15.46:0/1060056819 conn(0x7fef100b2040 msgr2=0x7fef1015bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 shutdown_connections 2024-09-17T03:29:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:27.774+0000 7fef17600640 1 -- 172.21.15.46:0/1060056819 wait complete. 2024-09-17T03:29:27.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28183 2024-09-17T03:29:27.856 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:27.854+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:27.856 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:27.854+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:27.856 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:27.854+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:27.856 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:27.854+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:27.856 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:27.854+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:28.000 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:28.002+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:28.001 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:28.002+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:28.001 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:28.002+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:28.001 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:28.002+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:28.001 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:28.002+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.202+0000 7ffb96e00640 1 -- 172.21.15.46:0/1427017473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006dbc0 msgr2=0x7ffb9006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.202+0000 7ffb96e00640 1 --2- 172.21.15.46:0/1427017473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006dbc0 0x7ffb9006dfd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ffb900ced20 tx=0x7ffb7c01c770 comp rx=0 tx=0).stop 2024-09-17T03:29:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.206+0000 7ffb96e00640 1 -- 172.21.15.46:0/1427017473 shutdown_connections 2024-09-17T03:29:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.206+0000 7ffb96e00640 1 --2- 172.21.15.46:0/1427017473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006dbc0 0x7ffb9006dfd0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.206+0000 7ffb96e00640 1 --2- 172.21.15.46:0/1427017473 >> v2:172.21.15.99:3300/0 conn(0x7ffb900d1140 0x7ffb900b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.206+0000 7ffb96e00640 1 -- 172.21.15.46:0/1427017473 >> 172.21.15.46:0/1427017473 conn(0x7ffb900b41f0 msgr2=0x7ffb900b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.206+0000 7ffb96e00640 1 -- 172.21.15.46:0/1427017473 shutdown_connections 2024-09-17T03:29:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.206+0000 7ffb96e00640 1 -- 172.21.15.46:0/1427017473 wait complete. 2024-09-17T03:29:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.214+0000 7ffb96e00640 1 Processor -- start 2024-09-17T03:29:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.214+0000 7ffb96e00640 1 -- start start 2024-09-17T03:29:28.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb9006dbc0 0x7ffb90155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:28.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 0x7ffb900cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9015f5c0 0x7ffb900cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90168c60 con 0x7ffb900d1140 2024-09-17T03:29:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90160250 con 0x7ffb9015f5c0 2024-09-17T03:29:28.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ffb90168de0 con 0x7ffb9006dbc0 2024-09-17T03:29:28.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 0x7ffb900cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:28.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 0x7ffb900cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57638/0 (socket says 172.21.15.46:57638) 2024-09-17T03:29:28.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 -- 172.21.15.46:0/327380845 learned_addr learned my addr 172.21.15.46:0/327380845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:28.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb96400640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9015f5c0 0x7ffb900cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:28.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 -- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb9006dbc0 msgr2=0x7ffb90155240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:29:28.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb9006dbc0 0x7ffb90155240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:28.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 -- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9015f5c0 msgr2=0x7ffb900cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:28.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9015f5c0 0x7ffb900cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:28.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 -- 172.21.15.46:0/327380845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb900ae200 con 0x7ffb900d1140 2024-09-17T03:29:28.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.222+0000 7ffb95000640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 0x7ffb900cc090 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009790 tx=0x7ffb7c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:28.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.226+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0bb070 con 0x7ffb900d1140 2024-09-17T03:29:28.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.226+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb7c004320 con 0x7ffb900d1140 2024-09-17T03:29:28.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.226+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0b2040 con 0x7ffb900d1140 2024-09-17T03:29:28.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.226+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb900ae4e0 con 0x7ffb900d1140 2024-09-17T03:29:28.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.226+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb900b0040 con 0x7ffb900d1140 2024-09-17T03:29:28.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.226+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90201230 con 0x7ffb900d1140 2024-09-17T03:29:28.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.230+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 30) ==== 98662+0+0 (secure 0 0 0) 0x7ffb7c0be070 con 0x7ffb900d1140 2024-09-17T03:29:28.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.230+0000 7ffb8d600640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffb70074240 0x7ffb70076750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:28.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.230+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7ffb7c12c900 con 0x7ffb900d1140 2024-09-17T03:29:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.242+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffb7c0163f0 con 0x7ffb900d1140 2024-09-17T03:29:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.254+0000 7ffb95a00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffb70074240 0x7ffb70076750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.262+0000 7ffb95a00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffb70074240 0x7ffb70076750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb840099d0 tx=0x7ffb840091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:28.726+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28183", "target": ["mon-mgr", ""]}) -- 0x7ffb902014e0 con 0x7ffb70074240 2024-09-17T03:29:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.662+0000 7ffb8d600640 1 -- 172.21.15.46:0/327380845 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb902014e0 con 0x7ffb70074240 2024-09-17T03:29:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffb70074240 msgr2=0x7ffb70076750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffb70074240 0x7ffb70076750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb840099d0 tx=0x7ffb840091e0 comp rx=0 tx=0).stop 2024-09-17T03:29:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 msgr2=0x7ffb900cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 0x7ffb900cc090 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009790 tx=0x7ffb7c005d40 comp rx=0 tx=0).stop 2024-09-17T03:29:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 shutdown_connections 2024-09-17T03:29:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffb70074240 0x7ffb70076750 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9015f5c0 0x7ffb900cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb900d1140 0x7ffb900cc090 secure :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009790 tx=0x7ffb7c005d40 comp rx=0 tx=0).stop 2024-09-17T03:29:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 --2- 172.21.15.46:0/327380845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffb9006dbc0 0x7ffb90155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:29.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.670+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 >> 172.21.15.46:0/327380845 conn(0x7ffb900b41f0 msgr2=0x7ffb900b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.674+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 shutdown_connections 2024-09-17T03:29:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:29.674+0000 7ffb96e00640 1 -- 172.21.15.46:0/327380845 wait complete. 2024-09-17T03:29:29.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone59644 2024-09-17T03:29:29.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:29:29.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:29:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 -- 172.21.15.46:0/2999268259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 msgr2=0x7f7f38068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/2999268259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 0x7f7f38068df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009900 tx=0x7f7f2801c820 comp rx=0 tx=0).stop 2024-09-17T03:29:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 -- 172.21.15.46:0/2999268259 shutdown_connections 2024-09-17T03:29:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/2999268259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 0x7f7f38068df0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 -- 172.21.15.46:0/2999268259 >> 172.21.15.46:0/2999268259 conn(0x7f7f380b35f0 msgr2=0x7f7f380b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 -- 172.21.15.46:0/2999268259 shutdown_connections 2024-09-17T03:29:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.202+0000 7f7f3dc00640 1 -- 172.21.15.46:0/2999268259 wait complete. 2024-09-17T03:29:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.206+0000 7f7f3dc00640 1 Processor -- start 2024-09-17T03:29:30.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.206+0000 7f7f3dc00640 1 -- start start 2024-09-17T03:29:30.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f3dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f38069330 0x7f7f3815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f3dc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 0x7f7f38155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f3dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f3815d630 0x7f7f3814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f3dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3816b060 con 0x7f7f3815d630 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f3dc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3816aee0 con 0x7f7f38071940 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f3dc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3816b1e0 con 0x7f7f38069330 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f3815d630 0x7f7f3814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f3815d630 0x7f7f3814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57640/0 (socket says 172.21.15.46:57640) 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f37400640 1 -- 172.21.15.46:0/994877064 learned_addr learned my addr 172.21.15.46:0/994877064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36a00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f38069330 0x7f7f3815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:30.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36000640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 0x7f7f38155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36a00640 1 -- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 msgr2=0x7f7f38155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36a00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 0x7f7f38155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36a00640 1 -- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f3815d630 msgr2=0x7f7f3814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36a00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f3815d630 0x7f7f3814fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:30.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.210+0000 7f7f36a00640 1 -- 172.21.15.46:0/994877064 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f381556d0 con 0x7f7f38069330 2024-09-17T03:29:30.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.214+0000 7f7f36a00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f38069330 0x7f7f3815b8b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f280ac000 tx=0x7f7f280040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.214+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280b2040 con 0x7f7f38069330 2024-09-17T03:29:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.214+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f381543d0 con 0x7f7f38069330 2024-09-17T03:29:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.214+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f280044d0 con 0x7f7f38069330 2024-09-17T03:29:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.214+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28003860 con 0x7f7f38069330 2024-09-17T03:29:30.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.214+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f38154990 con 0x7f7f38069330 2024-09-17T03:29:30.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.218+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f38068df0 con 0x7f7f38069330 2024-09-17T03:29:30.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.222+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f7f28003a00 con 0x7f7f38069330 2024-09-17T03:29:30.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.222+0000 7f7f2ea00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f140742e0 0x7f7f140767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:30.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.222+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f7f2812ceb0 con 0x7f7f38069330 2024-09-17T03:29:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.226+0000 7f7f36000640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f140742e0 0x7f7f140767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.226+0000 7f7f36000640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f140742e0 0x7f7f140767f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f380b64c0 tx=0x7f7f20009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:30.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.230+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f280164d0 con 0x7f7f38069330 2024-09-17T03:29:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:30.666+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone59644", "target": ["mon-mgr", ""]}) -- 0x7f7f381562d0 con 0x7f7f140742e0 2024-09-17T03:29:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.450+0000 7f7f2ea00640 1 -- 172.21.15.46:0/994877064 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f381562d0 con 0x7f7f140742e0 2024-09-17T03:29:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f140742e0 msgr2=0x7f7f140767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f140742e0 0x7f7f140767f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f380b64c0 tx=0x7f7f20009040 comp rx=0 tx=0).stop 2024-09-17T03:29:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f38069330 msgr2=0x7f7f3815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f38069330 0x7f7f3815b8b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f280ac000 tx=0x7f7f280040c0 comp rx=0 tx=0).stop 2024-09-17T03:29:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 shutdown_connections 2024-09-17T03:29:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f140742e0 0x7f7f140767f0 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f380b64c0 tx=0x7f7f20009040 comp rx=0 tx=0).stop 2024-09-17T03:29:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f3815d630 0x7f7f3814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f38071940 0x7f7f38155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 --2- 172.21.15.46:0/994877064 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f38069330 0x7f7f3815b8b0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f280ac000 tx=0x7f7f280040c0 comp rx=0 tx=0).stop 2024-09-17T03:29:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 >> 172.21.15.46:0/994877064 conn(0x7f7f380b35f0 msgr2=0x7f7f38153830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 shutdown_connections 2024-09-17T03:29:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:31.462+0000 7f7f3dc00640 1 -- 172.21.15.46:0/994877064 wait complete. 2024-09-17T03:29:31.559 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone96045 2024-09-17T03:29:32.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.026+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3518676232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e40b3130 msgr2=0x7fb9e40b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:32.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.026+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3518676232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e40b3130 0x7fb9e40b3540 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40afd10 tx=0x7fb9d401c8c0 comp rx=0 tx=0).stop 2024-09-17T03:29:32.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.030+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3518676232 shutdown_connections 2024-09-17T03:29:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.030+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3518676232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e40b3130 0x7fb9e40b3540 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.030+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3518676232 >> v2:172.21.15.99:3300/0 conn(0x7fb9e40d0500 0x7fb9e40d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.030+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3518676232 >> 172.21.15.46:0/3518676232 conn(0x7fb9e406e4a0 msgr2=0x7fb9e406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.030+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3518676232 shutdown_connections 2024-09-17T03:29:32.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.030+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3518676232 wait complete. 2024-09-17T03:29:32.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.042+0000 7fb9e8a00640 1 Processor -- start 2024-09-17T03:29:32.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.046+0000 7fb9e8a00640 1 -- start start 2024-09-17T03:29:32.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e406fd70 0x7fb9e416be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e8a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9e40b3130 0x7fb9e416da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e8a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e40d0500 0x7fb9e414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e4157750 con 0x7fb9e406fd70 2024-09-17T03:29:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e8a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e41575d0 con 0x7fb9e40d0500 2024-09-17T03:29:32.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e8a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e41578d0 con 0x7fb9e40b3130 2024-09-17T03:29:32.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9e40b3130 0x7fb9e416da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:32.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9e40b3130 0x7fb9e416da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:56522/0 (socket says 172.21.15.46:56522) 2024-09-17T03:29:32.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e2a00640 1 -- 172.21.15.46:0/3535371528 learned_addr learned my addr 172.21.15.46:0/3535371528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:32.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e3400640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e406fd70 0x7fb9e416be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e3400640 1 -- 172.21.15.46:0/3535371528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9e40b3130 msgr2=0x7fb9e416da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e3400640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9e40b3130 0x7fb9e416da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e3400640 1 -- 172.21.15.46:0/3535371528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e40d0500 msgr2=0x7fb9e414cce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e3400640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e40d0500 0x7fb9e414cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.050+0000 7fb9e3400640 1 -- 172.21.15.46:0/3535371528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e4161de0 con 0x7fb9e406fd70 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.054+0000 7fb9e3400640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e406fd70 0x7fb9e416be50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b440 tx=0x7fb9d000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.054+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0015020 con 0x7fb9e406fd70 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.054+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9d0011920 con 0x7fb9e406fd70 2024-09-17T03:29:32.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.054+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0011d70 con 0x7fb9e406fd70 2024-09-17T03:29:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.054+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e416df50 con 0x7fb9e406fd70 2024-09-17T03:29:32.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.054+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e414d220 con 0x7fb9e406fd70 2024-09-17T03:29:32.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.062+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fb9d0018020 con 0x7fb9e406fd70 2024-09-17T03:29:32.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.066+0000 7fb9db400640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9c40743b0 0x7fb9c40768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:32.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.066+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fb9d0092b00 con 0x7fb9e406fd70 2024-09-17T03:29:32.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.066+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc0052a0 con 0x7fb9e406fd70 2024-09-17T03:29:32.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.078+0000 7fb9e2a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9c40743b0 0x7fb9c40768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:32.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.082+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb9d001f020 con 0x7fb9e406fd70 2024-09-17T03:29:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.082+0000 7fb9e2a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9c40743b0 0x7fb9c40768c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb9e416ee50 tx=0x7fb9dc00b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:32.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:32.558+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone96045", "target": ["mon-mgr", ""]}) -- 0x7fb9bc000c90 con 0x7fb9c40743b0 2024-09-17T03:29:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9db400640 1 -- 172.21.15.46:0/3535371528 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9bc000c90 con 0x7fb9c40743b0 2024-09-17T03:29:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9c40743b0 msgr2=0x7fb9c40768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9c40743b0 0x7fb9c40768c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb9e416ee50 tx=0x7fb9dc00b040 comp rx=0 tx=0).stop 2024-09-17T03:29:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e406fd70 msgr2=0x7fb9e416be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e406fd70 0x7fb9e416be50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b440 tx=0x7fb9d000b900 comp rx=0 tx=0).stop 2024-09-17T03:29:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 shutdown_connections 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9c40743b0 0x7fb9c40768c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9e40d0500 0x7fb9e414cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9e40b3130 0x7fb9e416da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 --2- 172.21.15.46:0/3535371528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e406fd70 0x7fb9e416be50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.365+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 >> 172.21.15.46:0/3535371528 conn(0x7fb9e406e4a0 msgr2=0x7fb9e4076060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.369+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 shutdown_connections 2024-09-17T03:29:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.369+0000 7fb9e8a00640 1 -- 172.21.15.46:0/3535371528 wait complete. 2024-09-17T03:29:33.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92176 2024-09-17T03:29:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 -- 172.21.15.46:0/3833751125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 msgr2=0x7f786815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 --2- 172.21.15.46:0/3833751125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 0x7f786815ebf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f786806f210 tx=0x7f786001cda0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 -- 172.21.15.46:0/3833751125 shutdown_connections 2024-09-17T03:29:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 --2- 172.21.15.46:0/3833751125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 0x7f786815ebf0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 --2- 172.21.15.46:0/3833751125 >> v2:172.21.15.99:3300/0 conn(0x7f78680ce4c0 0x7f78680cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 -- 172.21.15.46:0/3833751125 >> 172.21.15.46:0/3833751125 conn(0x7f78680b2040 msgr2=0x7f78680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.825+0000 7f786cc00640 1 -- 172.21.15.46:0/3833751125 shutdown_connections 2024-09-17T03:29:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.829+0000 7f786cc00640 1 -- 172.21.15.46:0/3833751125 wait complete. 2024-09-17T03:29:33.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.833+0000 7f786cc00640 1 Processor -- start 2024-09-17T03:29:33.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.833+0000 7f786cc00640 1 -- start start 2024-09-17T03:29:33.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f786cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78680cd7a0 0x7f78681532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:33.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f786cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 0x7f7868151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:33.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f7865e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78680cd7a0 0x7f78681532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f786cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78680ce4c0 0x7f7868152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f786cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7868167b10 con 0x7f78680cdbb0 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f786cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7868167990 con 0x7f78680ce4c0 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f786cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7868167c90 con 0x7f78680cd7a0 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f7865e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78680cd7a0 0x7f78681532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:34612/0 (socket says 172.21.15.46:34612) 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.837+0000 7f7865e00640 1 -- 172.21.15.46:0/1523720313 learned_addr learned my addr 172.21.15.46:0/1523720313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78680ce4c0 0x7f7868152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7865400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 0x7f7868151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 -- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78680cd7a0 msgr2=0x7f78681532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78680cd7a0 0x7f78681532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 -- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 msgr2=0x7f7868151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 0x7f7868151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 -- 172.21.15.46:0/1523720313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78681463c0 con 0x7f78680ce4c0 2024-09-17T03:29:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.841+0000 7f7866800640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78680ce4c0 0x7f7868152180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78600ac000 tx=0x7f78600a5b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.845+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600b2040 con 0x7f78680ce4c0 2024-09-17T03:29:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.845+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7860003b50 con 0x7f78680ce4c0 2024-09-17T03:29:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.845+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860004030 con 0x7f78680ce4c0 2024-09-17T03:29:33.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.845+0000 7f786cc00640 1 -- 172.21.15.46:0/1523720313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7868154e90 con 0x7f78680ce4c0 2024-09-17T03:29:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.845+0000 7f786cc00640 1 -- 172.21.15.46:0/1523720313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7868159270 con 0x7f78680ce4c0 2024-09-17T03:29:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.849+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78681579a0 con 0x7f78680ce4c0 2024-09-17T03:29:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.849+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f78600bd030 con 0x7f78680ce4c0 2024-09-17T03:29:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.853+0000 7f785e000640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78480743b0 0x7f78480768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.853+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f786012db10 con 0x7f78680ce4c0 2024-09-17T03:29:33.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.857+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7860016990 con 0x7f78680ce4c0 2024-09-17T03:29:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.889+0000 7f7865e00640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78480743b0 0x7f78480768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:33.925+0000 7f7865e00640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78480743b0 0x7f78480768c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7854005fd0 tx=0x7f7854005f00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:34.381+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92176", "target": ["mon-mgr", ""]}) -- 0x7f7868154c40 con 0x7f78480743b0 2024-09-17T03:29:35.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.745+0000 7f785e000640 1 -- 172.21.15.46:0/1523720313 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7868154c40 con 0x7f78480743b0 2024-09-17T03:29:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78480743b0 msgr2=0x7f78480768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78480743b0 0x7f78480768c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7854005fd0 tx=0x7f7854005f00 comp rx=0 tx=0).stop 2024-09-17T03:29:35.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78680ce4c0 msgr2=0x7f7868152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78680ce4c0 0x7f7868152180 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78600ac000 tx=0x7f78600a5b70 comp rx=0 tx=0).stop 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 shutdown_connections 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78480743b0 0x7f78480768c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78680ce4c0 0x7f7868152180 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78680cdbb0 0x7f7868151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 --2- 172.21.15.46:0/1523720313 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78680cd7a0 0x7f78681532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 >> 172.21.15.46:0/1523720313 conn(0x7f78680b2040 msgr2=0x7f78680b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:35.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 shutdown_connections 2024-09-17T03:29:35.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:35.757+0000 7f7853400640 1 -- 172.21.15.46:0/1523720313 wait complete. 2024-09-17T03:29:35.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:29:36.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.277+0000 7fbe3f200640 1 -- 172.21.15.46:0/4055278756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe24005330 msgr2=0x7fbe2400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:36.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.277+0000 7fbe3f200640 1 --2- 172.21.15.46:0/4055278756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe24005330 0x7fbe2400b9d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbe40074120 tx=0x7fbe3001c870 comp rx=0 tx=0).stop 2024-09-17T03:29:36.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.281+0000 7fbe3f200640 1 -- 172.21.15.46:0/4055278756 shutdown_connections 2024-09-17T03:29:36.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.281+0000 7fbe3f200640 1 --2- 172.21.15.46:0/4055278756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe24005330 0x7fbe2400b9d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.281+0000 7fbe3f200640 1 -- 172.21.15.46:0/4055278756 >> 172.21.15.46:0/4055278756 conn(0x7fbe4006cdf0 msgr2=0x7fbe4006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.281+0000 7fbe3f200640 1 -- 172.21.15.46:0/4055278756 shutdown_connections 2024-09-17T03:29:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.281+0000 7fbe3f200640 1 -- 172.21.15.46:0/4055278756 wait complete. 2024-09-17T03:29:36.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.293+0000 7fbe3f200640 1 Processor -- start 2024-09-17T03:29:36.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.293+0000 7fbe3f200640 1 -- start start 2024-09-17T03:29:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.305+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 0x7fbe401fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe40147bb0 0x7fbe40157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 0x7fbe401fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 0x7fbe401fdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34558/0 (socket says 172.21.15.46:34558) 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3de00640 1 -- 172.21.15.46:0/1516971698 learned_addr learned my addr 172.21.15.46:0/1516971698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe40157a50 0x7fbe40164710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbe400b4220 con 0x7fbe24005330 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe400b5000 con 0x7fbe40147bb0 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbe400b43a0 con 0x7fbe40157a50 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3d400640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe40147bb0 0x7fbe40157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.309+0000 7fbe3e800640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe40157a50 0x7fbe40164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3d400640 1 -- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe40157a50 msgr2=0x7fbe40164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3d400640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe40157a50 0x7fbe40164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3d400640 1 -- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 msgr2=0x7fbe401fdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3d400640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 0x7fbe401fdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3d400640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe40157e60 con 0x7fbe40147bb0 2024-09-17T03:29:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3de00640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 0x7fbe401fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:29:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.313+0000 7fbe3d400640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe40147bb0 0x7fbe40157510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe400cc580 tx=0x7fbe38006430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.317+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38015070 con 0x7fbe40147bb0 2024-09-17T03:29:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.317+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe38011900 con 0x7fbe40147bb0 2024-09-17T03:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.317+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38011d30 con 0x7fbe40147bb0 2024-09-17T03:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.317+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4014ef90 con 0x7fbe40147bb0 2024-09-17T03:29:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.321+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401543a0 con 0x7fbe40147bb0 2024-09-17T03:29:36.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.325+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe400b1dc0 con 0x7fbe40147bb0 2024-09-17T03:29:36.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.333+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fbe38018070 con 0x7fbe40147bb0 2024-09-17T03:29:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.333+0000 7fbe36000640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe20074330 0x7fbe20076840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.333+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fbe38094010 con 0x7fbe40147bb0 2024-09-17T03:29:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.337+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe40157e60 con 0x7fbe40147bb0 2024-09-17T03:29:36.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.357+0000 7fbe3de00640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe20074330 0x7fbe20076840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.361+0000 7fbe3de00640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe20074330 0x7fbe20076840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001d20 tx=0x7fbe2c001cb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.913+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbe4014ece0 con 0x7fbe40147bb0 2024-09-17T03:29:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.917+0000 7fbe36000640 1 -- 172.21.15.46:0/1516971698 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2184 (secure 0 0 0) 0x7fbe380601c0 con 0x7fbe40147bb0 2024-09-17T03:29:36.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:36.924 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:29:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:29:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:29:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:29:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:29:36.924 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:29:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:29:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:29:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:29:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:29:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:29:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:29:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:29:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:29:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:29:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 534, 2024-09-17T03:29:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:29:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:29:36.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:29:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:29:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:29:36.928 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 72, 2024-09-17T03:29:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:29:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:29:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:29:36.929 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:29:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:29:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:29:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:29:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:29:36.930 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:29:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:29:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:36.931 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:29:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:29:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 1609, 2024-09-17T03:29:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 6607210075, 2024-09-17T03:29:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 13256376320, 2024-09-17T03:29:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 759837736960, 2024-09-17T03:29:36.932 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:29:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 288946666, 2024-09-17T03:29:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 288287039, 2024-09-17T03:29:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 70, 2024-09-17T03:29:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 80 2024-09-17T03:29:36.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:29:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 46, 2024-09-17T03:29:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:28:53:876203+0000", 2024-09-17T03:29:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5, 2024-09-17T03:29:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:29:36.934 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:29:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:29:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:29:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 5, 2024-09-17T03:29:36.935 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:29:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "a", 2024-09-17T03:29:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:29:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246303 2024-09-17T03:29:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:36.936 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:29:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:29:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:29:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:29:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:29:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:29:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:29:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:29:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:29:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:29:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 10, 2024-09-17T03:29:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:27:27.320954+0000", 2024-09-17T03:29:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:29:36.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:29:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:29:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 31.12% (6s)\n [========....................] (remaining: 13s)", 2024-09-17T03:29:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.31119999289512634, 2024-09-17T03:29:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:29:36.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:36.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:36.941 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe20074330 msgr2=0x7fbe20076840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe20074330 0x7fbe20076840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001d20 tx=0x7fbe2c001cb0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe40147bb0 msgr2=0x7fbe40157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe40147bb0 0x7fbe40157510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe400cc580 tx=0x7fbe38006430 comp rx=0 tx=0).stop 2024-09-17T03:29:36.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 shutdown_connections 2024-09-17T03:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe20074330 0x7fbe20076840 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c001d20 tx=0x7fbe2c001cb0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbe40157a50 0x7fbe40164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe40147bb0 0x7fbe40157510 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe400cc580 tx=0x7fbe38006430 comp rx=0 tx=0).stop 2024-09-17T03:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 --2- 172.21.15.46:0/1516971698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe24005330 0x7fbe401fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:36.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 >> 172.21.15.46:0/1516971698 conn(0x7fbe4006cdf0 msgr2=0x7fbe4014ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:36.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 shutdown_connections 2024-09-17T03:29:36.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:36.941+0000 7fbe3f200640 1 -- 172.21.15.46:0/1516971698 wait complete. 2024-09-17T03:29:38.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:29:38.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 -- 172.21.15.46:0/1181582677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30069500 msgr2=0x7f6d300b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:38.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 --2- 172.21.15.46:0/1181582677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30069500 0x7f6d300b07c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6d30073ce0 tx=0x7f6d2801cd80 comp rx=0 tx=0).stop 2024-09-17T03:29:38.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 -- 172.21.15.46:0/1181582677 shutdown_connections 2024-09-17T03:29:38.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 --2- 172.21.15.46:0/1181582677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30069500 0x7f6d300b07c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:38.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 --2- 172.21.15.46:0/1181582677 >> v2:172.21.15.99:3300/0 conn(0x7f6d30075160 0x7f6d3015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:38.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 -- 172.21.15.46:0/1181582677 >> 172.21.15.46:0/1181582677 conn(0x7f6d300b41f0 msgr2=0x7f6d300b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:38.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.489+0000 7f6d36400640 1 -- 172.21.15.46:0/1181582677 shutdown_connections 2024-09-17T03:29:38.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.493+0000 7f6d36400640 1 -- 172.21.15.46:0/1181582677 wait complete. 2024-09-17T03:29:38.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.493+0000 7f6d36400640 1 Processor -- start 2024-09-17T03:29:38.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.493+0000 7f6d36400640 1 -- start start 2024-09-17T03:29:38.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d36400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 0x7f6d3009a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d36400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d300ce600 0x7f6d300af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d36400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d30151b80 0x7f6d300afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d36400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3016c220 con 0x7f6d30075160 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d36400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3016c0a0 con 0x7f6d300ce600 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d36400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3016c3a0 con 0x7f6d30151b80 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.497+0000 7f6d35000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 0x7f6d3009a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 0x7f6d3009a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34578/0 (socket says 172.21.15.46:34578) 2024-09-17T03:29:38.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 -- 172.21.15.46:0/1811282710 learned_addr learned my addr 172.21.15.46:0/1811282710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35a00640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d30151b80 0x7f6d300afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:38.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d27e00640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d300ce600 0x7f6d300af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 -- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d30151b80 msgr2=0x7f6d300afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d30151b80 0x7f6d300afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 -- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d300ce600 msgr2=0x7f6d300af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d300ce600 0x7f6d300af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 -- 172.21.15.46:0/1811282710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3015a980 con 0x7f6d30075160 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d27e00640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d300ce600 0x7f6d300af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d35000640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 0x7f6d3009a990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000d750 tx=0x7f6d2000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:38.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20017070 con 0x7f6d30075160 2024-09-17T03:29:38.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d20009d20 con 0x7f6d30075160 2024-09-17T03:29:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.501+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20005280 con 0x7f6d30075160 2024-09-17T03:29:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.505+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3009aed0 con 0x7f6d30075160 2024-09-17T03:29:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.505+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d30158e80 con 0x7f6d30075160 2024-09-17T03:29:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.509+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d300b07c0 con 0x7f6d30075160 2024-09-17T03:29:38.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.525+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f6d20005420 con 0x7f6d30075160 2024-09-17T03:29:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.533+0000 7f6d2d600640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d0c0743e0 0x7f6d0c0768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:38.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.533+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f6d20093330 con 0x7f6d30075160 2024-09-17T03:29:38.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.541+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d3015a980 con 0x7f6d30075160 2024-09-17T03:29:38.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.565+0000 7f6d27e00640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d0c0743e0 0x7f6d0c0768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:38.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:38.573+0000 7f6d27e00640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d0c0743e0 0x7f6d0c0768f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6d30159d30 tx=0x7f6d18009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:39.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.129+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6d30069920 con 0x7f6d30075160 2024-09-17T03:29:39.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.133+0000 7f6d2d600640 1 -- 172.21.15.46:0/1811282710 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2968 (secure 0 0 0) 0x7f6d2000de60 con 0x7f6d30075160 2024-09-17T03:29:39.132 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:29:39.133 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 536, 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 72, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:29:39.135 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2443, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10097103159, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 20110307328, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 752983805952, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 579453616, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 578816032, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 140, 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 157 2024-09-17T03:29:39.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 46, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:28:53:876203+0000", 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 5, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:29:39.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "a", 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246303 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:29:39.138 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 11, 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:29:37.365264+0000", 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "daemons": { 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "summary": "", 2024-09-17T03:29:39.139 INFO:teuthology.orchestra.run.smithi046.stdout: "a": { 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "d": { 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:29:39.140 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 31.12% (6s)\n [========....................] (remaining: 13s)", 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.31119999289512634, 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.141+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d0c0743e0 msgr2=0x7f6d0c0768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.141+0000 7f6d36400640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d0c0743e0 0x7f6d0c0768f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6d30159d30 tx=0x7f6d18009040 comp rx=0 tx=0).stop 2024-09-17T03:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.141+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 msgr2=0x7f6d3009a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.141+0000 7f6d36400640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 0x7f6d3009a990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000d750 tx=0x7f6d2000dc10 comp rx=0 tx=0).stop 2024-09-17T03:29:39.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 shutdown_connections 2024-09-17T03:29:39.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6d0c0743e0 0x7f6d0c0768f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:39.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6d30151b80 0x7f6d300afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:39.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d300ce600 0x7f6d300af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:39.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 --2- 172.21.15.46:0/1811282710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d30075160 0x7f6d3009a990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:39.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 >> 172.21.15.46:0/1811282710 conn(0x7f6d300b41f0 msgr2=0x7f6d30155590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:39.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 shutdown_connections 2024-09-17T03:29:39.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:39.149+0000 7f6d36400640 1 -- 172.21.15.46:0/1811282710 wait complete. 2024-09-17T03:29:40.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 -- 172.21.15.46:0/2505998904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740cf4a0 msgr2=0x7f307415b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 --2- 172.21.15.46:0/2505998904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740cf4a0 0x7f307415b570 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3074073380 tx=0x7f306001c600 comp rx=0 tx=0).stop 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 -- 172.21.15.46:0/2505998904 shutdown_connections 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 --2- 172.21.15.46:0/2505998904 >> v2:172.21.15.99:3300/0 conn(0x7f3074074040 0x7f307406f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 --2- 172.21.15.46:0/2505998904 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30740b57c0 0x7f3074073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 --2- 172.21.15.46:0/2505998904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740cf4a0 0x7f307415b570 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 -- 172.21.15.46:0/2505998904 >> 172.21.15.46:0/2505998904 conn(0x7f30740b3b30 msgr2=0x7f30740733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 -- 172.21.15.46:0/2505998904 shutdown_connections 2024-09-17T03:29:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.649+0000 7f307b400640 1 -- 172.21.15.46:0/2505998904 wait complete. 2024-09-17T03:29:40.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 Processor -- start 2024-09-17T03:29:40.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 -- start start 2024-09-17T03:29:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3074074040 0x7f3074165be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740b57c0 0x7f30741614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30740cf4a0 0x7f3074158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f307415fa80 con 0x7f30740b57c0 2024-09-17T03:29:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f307415f900 con 0x7f30740cf4a0 2024-09-17T03:29:40.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f307b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f307415fc00 con 0x7f3074074040 2024-09-17T03:29:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f3073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740b57c0 0x7f30741614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f3078a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3074074040 0x7f3074165be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f3078a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3074074040 0x7f3074165be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:34652/0 (socket says 172.21.15.46:34652) 2024-09-17T03:29:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f3078a00640 1 -- 172.21.15.46:0/4173076793 learned_addr learned my addr 172.21.15.46:0/4173076793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.653+0000 7f3079400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30740cf4a0 0x7f3074158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:40.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3079400640 1 -- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3074074040 msgr2=0x7f3074165be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3079400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3074074040 0x7f3074165be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3079400640 1 -- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740b57c0 msgr2=0x7f30741614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3079400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740b57c0 0x7f30741614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3079400640 1 -- 172.21.15.46:0/4173076793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3074166120 con 0x7f30740cf4a0 2024-09-17T03:29:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3073e00640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740b57c0 0x7f30741614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:29:40.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3079400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30740cf4a0 0x7f3074158b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f306c00d950 tx=0x7f306c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:40.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c017030 con 0x7f30740cf4a0 2024-09-17T03:29:40.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f306c004c20 con 0x7f30740cf4a0 2024-09-17T03:29:40.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3074164b40 con 0x7f30740cf4a0 2024-09-17T03:29:40.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00bd70 con 0x7f30740cf4a0 2024-09-17T03:29:40.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.657+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3074165100 con 0x7f30740cf4a0 2024-09-17T03:29:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.661+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074073b00 con 0x7f30740cf4a0 2024-09-17T03:29:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.665+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f306c003980 con 0x7f30740cf4a0 2024-09-17T03:29:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.665+0000 7f3070c00640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3054074310 0x7f3054076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.669+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f306c093020 con 0x7f30740cf4a0 2024-09-17T03:29:40.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.669+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f306c003c70 con 0x7f30740cf4a0 2024-09-17T03:29:40.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.689+0000 7f3078a00640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3054074310 0x7f3054076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:40.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:40.721+0000 7f3078a00640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3054074310 0x7f3054076820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f306001cea0 tx=0x7f306001ae40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.249+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f30740cdd50 con 0x7f30740cf4a0 2024-09-17T03:29:41.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.249+0000 7f3070c00640 1 -- 172.21.15.46:0/4173076793 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+3209 (secure 0 0 0) 0x7f306c05f440 con 0x7f30740cf4a0 2024-09-17T03:29:41.252 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:41.252 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:41.253 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 539, 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 72, 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:29:41.254 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:29:41.255 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 2443, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 10097103159, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 20110307328, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 752983805952, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 559985644, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 556896606, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 135, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 149 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 46, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:28:53:876203+0000", 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5, 2024-09-17T03:29:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 5, 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "a", 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246303 2024-09-17T03:29:41.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.258 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:41.258 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:29:41.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.259 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:29:41.259 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:29:41.259 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:29:41.259 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 11, 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:29:37.365264+0000", 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "daemons": { 2024-09-17T03:29:41.260 INFO:teuthology.orchestra.run.smithi046.stdout: "summary": "", 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "a": { 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "d": { 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:29:41.261 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 48.438% (10s)\n [=============...............] (remaining: 11s)", 2024-09-17T03:29:41.262 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.48438000679016113, 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-total-clones": { 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Total 6 clones - average progress is 36.285% (0s)\n [==========..................] ", 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.36285001039505005, 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:41.263 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.257+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3054074310 msgr2=0x7f3054076820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.257+0000 7f307b400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3054074310 0x7f3054076820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f306001cea0 tx=0x7f306001ae40 comp rx=0 tx=0).stop 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.257+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30740cf4a0 msgr2=0x7f3074158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.257+0000 7f307b400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30740cf4a0 0x7f3074158b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f306c00d950 tx=0x7f306c00de10 comp rx=0 tx=0).stop 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 shutdown_connections 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3054074310 0x7f3054076820 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30740cf4a0 0x7f3074158b40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30740b57c0 0x7f30741614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 --2- 172.21.15.46:0/4173076793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3074074040 0x7f3074165be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 >> 172.21.15.46:0/4173076793 conn(0x7f30740b3b30 msgr2=0x7f3074156d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 shutdown_connections 2024-09-17T03:29:41.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.261+0000 7f307b400640 1 -- 172.21.15.46:0/4173076793 wait complete. 2024-09-17T03:29:41.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone94810 2024-09-17T03:29:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.857+0000 7f0f33400640 1 -- 172.21.15.46:0/1505704787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f14003a60 msgr2=0x7f0f14003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.857+0000 7f0f33400640 1 --2- 172.21.15.46:0/1505704787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f14003a60 0x7f0f14003ef0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000bb40 tx=0x7f0f2001cdb0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.861+0000 7f0f31400640 1 -- 172.21.15.46:0/1505704787 reap_dead start 2024-09-17T03:29:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.861+0000 7f0f33400640 1 -- 172.21.15.46:0/1505704787 shutdown_connections 2024-09-17T03:29:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.861+0000 7f0f33400640 1 -- 172.21.15.46:0/1505704787 >> 172.21.15.46:0/1505704787 conn(0x7f0f2c0b2040 msgr2=0x7f0f2c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:41.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.865+0000 7f0f33400640 1 -- 172.21.15.46:0/1505704787 shutdown_connections 2024-09-17T03:29:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.869+0000 7f0f33400640 1 -- 172.21.15.46:0/1505704787 wait complete. 2024-09-17T03:29:41.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.873+0000 7f0f33400640 1 Processor -- start 2024-09-17T03:29:41.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.873+0000 7f0f33400640 1 -- start start 2024-09-17T03:29:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f33400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c1f5670 0x7f0f2c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:41.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c1558b0 0x7f0f2c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:41.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f33400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c155cc0 0x7f0f2c0b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f33400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c0b45b0 con 0x7f0f2c1558b0 2024-09-17T03:29:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f33400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c0b4430 con 0x7f0f2c1f5670 2024-09-17T03:29:41.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f33400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c0b4730 con 0x7f0f2c155cc0 2024-09-17T03:29:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f31400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c155cc0 0x7f0f2c0b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f31400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c155cc0 0x7f0f2c0b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:34684/0 (socket says 172.21.15.46:34684) 2024-09-17T03:29:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f31400640 1 -- 172.21.15.46:0/2433666707 learned_addr learned my addr 172.21.15.46:0/2433666707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.877+0000 7f0f2be00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c1558b0 0x7f0f2c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:41.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f30a00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c1f5670 0x7f0f2c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:41.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f2be00640 1 -- 172.21.15.46:0/2433666707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c155cc0 msgr2=0x7f0f2c0b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:41.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f2be00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c155cc0 0x7f0f2c0b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f2be00640 1 -- 172.21.15.46:0/2433666707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c1f5670 msgr2=0x7f0f2c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:41.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f2be00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c1f5670 0x7f0f2c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:41.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f2be00640 1 -- 172.21.15.46:0/2433666707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c0cc9f0 con 0x7f0f2c1558b0 2024-09-17T03:29:41.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.881+0000 7f0f2be00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c1558b0 0x7f0f2c0affb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c000c10 tx=0x7f0f1c00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:41.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.885+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c016020 con 0x7f0f2c1558b0 2024-09-17T03:29:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.885+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f1c012630 con 0x7f0f2c1558b0 2024-09-17T03:29:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.885+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c012a20 con 0x7f0f2c1558b0 2024-09-17T03:29:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.885+0000 7f0f33400640 1 -- 172.21.15.46:0/2433666707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c0ccc60 con 0x7f0f2c1558b0 2024-09-17T03:29:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.885+0000 7f0f33400640 1 -- 172.21.15.46:0/2433666707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c0cd220 con 0x7f0f2c1558b0 2024-09-17T03:29:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.889+0000 7f0f33400640 1 -- 172.21.15.46:0/2433666707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f080052a0 con 0x7f0f2c1558b0 2024-09-17T03:29:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.893+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f0f1c019020 con 0x7f0f2c1558b0 2024-09-17T03:29:41.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.901+0000 7f0f28c00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f10074360 0x7f0f10076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.901+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f0f1c092880 con 0x7f0f2c1558b0 2024-09-17T03:29:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.913+0000 7f0f30a00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f10074360 0x7f0f10076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.921+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0f1c020020 con 0x7f0f2c1558b0 2024-09-17T03:29:41.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:41.949+0000 7f0f30a00640 1 --2- 172.21.15.46:0/2433666707 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f10074360 0x7f0f10076870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0f200b0000 tx=0x7f0f200a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:42.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:42.485+0000 7f0f33400640 1 -- 172.21.15.46:0/2433666707 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone94810", "target": ["mon-mgr", ""]}) -- 0x7f0f08000bd0 con 0x7f0f10074360 2024-09-17T03:29:42.612 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:42.609+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:29:42.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:42.613+0000 7f0f28c00640 1 -- 172.21.15.46:0/2433666707 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f0f08000bd0 con 0x7f0f10074360 2024-09-17T03:29:42.612 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:29:42.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:29:42.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone64201 2024-09-17T03:29:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 -- 172.21.15.46:0/4156881641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd640b5730 msgr2=0x7fcd6415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 --2- 172.21.15.46:0/4156881641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd640b5730 0x7fcd6415b380 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400aa10 tx=0x7fcd5401d920 comp rx=0 tx=0).stop 2024-09-17T03:29:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 -- 172.21.15.46:0/4156881641 shutdown_connections 2024-09-17T03:29:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 --2- 172.21.15.46:0/4156881641 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd6415b8c0 0x7fcd6415e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 --2- 172.21.15.46:0/4156881641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd640b5730 0x7fcd6415b380 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 -- 172.21.15.46:0/4156881641 >> 172.21.15.46:0/4156881641 conn(0x7fcd640b2c40 msgr2=0x7fcd640b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.049+0000 7fcd68e00640 1 -- 172.21.15.46:0/4156881641 shutdown_connections 2024-09-17T03:29:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.053+0000 7fcd68e00640 1 -- 172.21.15.46:0/4156881641 wait complete. 2024-09-17T03:29:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 Processor -- start 2024-09-17T03:29:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 -- start start 2024-09-17T03:29:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd641446d0 0x7fcd64159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6415b8c0 0x7fcd64140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd6415a260 0x7fcd64140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6416a0e0 con 0x7fcd6415b8c0 2024-09-17T03:29:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64169f60 con 0x7fcd6415a260 2024-09-17T03:29:43.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd68e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6416a260 con 0x7fcd641446d0 2024-09-17T03:29:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd61e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd641446d0 0x7fcd64159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd61e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd641446d0 0x7fcd64159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:48836/0 (socket says 172.21.15.46:48836) 2024-09-17T03:29:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd61e00640 1 -- 172.21.15.46:0/2878365989 learned_addr learned my addr 172.21.15.46:0/2878365989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd62800640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd6415a260 0x7fcd64140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.057+0000 7fcd61400640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6415b8c0 0x7fcd64140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd62800640 1 -- 172.21.15.46:0/2878365989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd641446d0 msgr2=0x7fcd64159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd62800640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd641446d0 0x7fcd64159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd62800640 1 -- 172.21.15.46:0/2878365989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6415b8c0 msgr2=0x7fcd64140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd62800640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6415b8c0 0x7fcd64140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd62800640 1 -- 172.21.15.46:0/2878365989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6415a9d0 con 0x7fcd6415a260 2024-09-17T03:29:43.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd62800640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd6415a260 0x7fcd64140a60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00e990 tx=0x7fcd5c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c004d30 con 0x7fcd6415a260 2024-09-17T03:29:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd5c00ca40 con 0x7fcd6415a260 2024-09-17T03:29:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c005020 con 0x7fcd6415a260 2024-09-17T03:29:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd68e00640 1 -- 172.21.15.46:0/2878365989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd64164570 con 0x7fcd6415a260 2024-09-17T03:29:43.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.061+0000 7fcd68e00640 1 -- 172.21.15.46:0/2878365989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6415c9f0 con 0x7fcd6415a260 2024-09-17T03:29:43.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.065+0000 7fcd68e00640 1 -- 172.21.15.46:0/2878365989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6415b380 con 0x7fcd6415a260 2024-09-17T03:29:43.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.065+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fcd5c0051c0 con 0x7fcd6415a260 2024-09-17T03:29:43.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.069+0000 7fcd5a000640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd440743e0 0x7fcd440768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:43.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.069+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fcd5c093580 con 0x7fcd6415a260 2024-09-17T03:29:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.073+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd5c016030 con 0x7fcd6415a260 2024-09-17T03:29:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.101+0000 7fcd61e00640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd440743e0 0x7fcd440768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.101+0000 7fcd61e00640 1 --2- 172.21.15.46:0/2878365989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd440743e0 0x7fcd440768f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098b0 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:43.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.553+0000 7fcd68e00640 1 -- 172.21.15.46:0/2878365989 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone64201", "target": ["mon-mgr", ""]}) -- 0x7fcd6415cf10 con 0x7fcd440743e0 2024-09-17T03:29:43.851 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:43.853+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:29:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:43.853+0000 7fcd5a000640 1 -- 172.21.15.46:0/2878365989 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fcd500094a0 con 0x7fcd440743e0 2024-09-17T03:29:43.854 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:29:43.877 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:29:43.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone14462 2024-09-17T03:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87e00640 1 -- 172.21.15.46:0/2600567164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6c0039f0 msgr2=0x7f0b6c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87e00640 1 --2- 172.21.15.46:0/2600567164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6c0039f0 0x7f0b6c003e60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0b880d2250 tx=0x7f0b7801cd60 comp rx=0 tx=0).stop 2024-09-17T03:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87400640 1 -- 172.21.15.46:0/2600567164 reap_dead start 2024-09-17T03:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87e00640 1 -- 172.21.15.46:0/2600567164 shutdown_connections 2024-09-17T03:29:44.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87e00640 1 -- 172.21.15.46:0/2600567164 >> 172.21.15.46:0/2600567164 conn(0x7f0b88073470 msgr2=0x7f0b880b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:44.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87e00640 1 -- 172.21.15.46:0/2600567164 shutdown_connections 2024-09-17T03:29:44.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.233+0000 7f0b87e00640 1 -- 172.21.15.46:0/2600567164 wait complete. 2024-09-17T03:29:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.241+0000 7f0b87e00640 1 Processor -- start 2024-09-17T03:29:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.241+0000 7f0b87e00640 1 -- start start 2024-09-17T03:29:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b6c0039f0 0x7f0b88162970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b881610a0 0x7f0b881614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b881619f0 0x7f0b8809d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8816b010 con 0x7f0b881619f0 2024-09-17T03:29:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8816ae90 con 0x7f0b881610a0 2024-09-17T03:29:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8816b190 con 0x7f0b6c0039f0 2024-09-17T03:29:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b881619f0 0x7f0b8809d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b881619f0 0x7f0b8809d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53678/0 (socket says 172.21.15.46:53678) 2024-09-17T03:29:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 -- 172.21.15.46:0/3338921674 learned_addr learned my addr 172.21.15.46:0/3338921674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b7e000640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b881610a0 0x7f0b881614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 -- 172.21.15.46:0/3338921674 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b6c0039f0 msgr2=0x7f0b88162970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:29:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b6c0039f0 0x7f0b88162970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:44.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 -- 172.21.15.46:0/3338921674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b881610a0 msgr2=0x7f0b881614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:44.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b881610a0 0x7f0b881614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:44.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 -- 172.21.15.46:0/3338921674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b88159290 con 0x7f0b881619f0 2024-09-17T03:29:44.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b87400640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b881619f0 0x7f0b8809d4a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0b880b2f50 tx=0x7f0b8000b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:44.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.245+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80003980 con 0x7f0b881619f0 2024-09-17T03:29:44.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.249+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b80003c50 con 0x7f0b881619f0 2024-09-17T03:29:44.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.249+0000 7f0b87e00640 1 -- 172.21.15.46:0/3338921674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8809ca00 con 0x7f0b881619f0 2024-09-17T03:29:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.249+0000 7f0b87e00640 1 -- 172.21.15.46:0/3338921674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b88158fb0 con 0x7f0b881619f0 2024-09-17T03:29:44.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.249+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800062d0 con 0x7f0b881619f0 2024-09-17T03:29:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.257+0000 7f0b87e00640 1 -- 172.21.15.46:0/3338921674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b600052a0 con 0x7f0b881619f0 2024-09-17T03:29:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.257+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f0b80006470 con 0x7f0b881619f0 2024-09-17T03:29:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.257+0000 7f0b7f400640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b640743e0 0x7f0b640768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.257+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f0b800934e0 con 0x7f0b881619f0 2024-09-17T03:29:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.269+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0b88069740 con 0x7f0b881619f0 2024-09-17T03:29:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.269+0000 7f0b86a00640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b640743e0 0x7f0b640768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.277+0000 7f0b86a00640 1 --2- 172.21.15.46:0/3338921674 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b640743e0 0x7f0b640768f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b780b0000 tx=0x7f0b78019b00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:44.781+0000 7f0b87e00640 1 -- 172.21.15.46:0/3338921674 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone14462", "target": ["mon-mgr", ""]}) -- 0x7f0b60000bd0 con 0x7f0b640743e0 2024-09-17T03:29:45.583 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:45.581+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:29:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:45.613+0000 7f0b7f400640 1 -- 172.21.15.46:0/3338921674 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f0b60000bd0 con 0x7f0b640743e0 2024-09-17T03:29:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:29:45.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:29:45.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28183 2024-09-17T03:29:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.017+0000 7f0d25a00640 1 -- 172.21.15.46:0/478610137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d200afdd0 msgr2=0x7f0d20163270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.017+0000 7f0d25a00640 1 --2- 172.21.15.46:0/478610137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d200afdd0 0x7f0d20163270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009900 tx=0x7f0d1001c920 comp rx=0 tx=0).stop 2024-09-17T03:29:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.021+0000 7f0d25a00640 1 -- 172.21.15.46:0/478610137 shutdown_connections 2024-09-17T03:29:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.021+0000 7f0d25a00640 1 --2- 172.21.15.46:0/478610137 >> v2:172.21.15.99:3300/0 conn(0x7f0d200b4530 0x7f0d200b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.021+0000 7f0d25a00640 1 --2- 172.21.15.46:0/478610137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d200afdd0 0x7f0d20163270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.021+0000 7f0d25a00640 1 -- 172.21.15.46:0/478610137 >> 172.21.15.46:0/478610137 conn(0x7f0d200d0cf0 msgr2=0x7f0d200d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.021+0000 7f0d25a00640 1 -- 172.21.15.46:0/478610137 shutdown_connections 2024-09-17T03:29:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.021+0000 7f0d25a00640 1 -- 172.21.15.46:0/478610137 wait complete. 2024-09-17T03:29:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.025+0000 7f0d25a00640 1 Processor -- start 2024-09-17T03:29:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.025+0000 7f0d25a00640 1 -- start start 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 0x7f0d201396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d200afdd0 0x7f0d20160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d200b4530 0x7f0d200aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2016c590 con 0x7f0d200b4530 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2016c410 con 0x7f0d200afdd0 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2016c710 con 0x7f0d2006efd0 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 0x7f0d201396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 0x7f0d201396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:48882/0 (socket says 172.21.15.46:48882) 2024-09-17T03:29:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 -- 172.21.15.46:0/467380000 learned_addr learned my addr 172.21.15.46:0/467380000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1f400640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d200b4530 0x7f0d200aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1e000640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d200afdd0 0x7f0d20160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 -- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d200afdd0 msgr2=0x7f0d20160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d200afdd0 0x7f0d20160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 -- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d200b4530 msgr2=0x7f0d200aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d200b4530 0x7f0d200aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 -- 172.21.15.46:0/467380000 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d20139c10 con 0x7f0d2006efd0 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d1ea00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 0x7f0d201396d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d1001a550 tx=0x7f0d100aeb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100bb070 con 0x7f0d2006efd0 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d100aee30 con 0x7f0d2006efd0 2024-09-17T03:29:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100b2040 con 0x7f0d2006efd0 2024-09-17T03:29:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.029+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2015fb20 con 0x7f0d2006efd0 2024-09-17T03:29:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.033+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201600e0 con 0x7f0d2006efd0 2024-09-17T03:29:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.033+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2015f050 con 0x7f0d2006efd0 2024-09-17T03:29:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.033+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f0d10004290 con 0x7f0d2006efd0 2024-09-17T03:29:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.037+0000 7f0d16a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d00074310 0x7f0d00076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.037+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f0d1012cd70 con 0x7f0d2006efd0 2024-09-17T03:29:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.037+0000 7f0d1e000640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d00074310 0x7f0d00076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:46.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.037+0000 7f0d1e000640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d00074310 0x7f0d00076820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d20161b30 tx=0x7f0d0c009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.041+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0d100c4080 con 0x7f0d2006efd0 2024-09-17T03:29:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:46.505+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28183", "target": ["mon-mgr", ""]}) -- 0x7f0d2006b540 con 0x7f0d00074310 2024-09-17T03:29:47.294 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:47.293+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:47.294 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:47.293+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:47.294 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:47.293+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:47.294 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:47.293+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:47.294 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:47.293+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:48.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.465+0000 7f0d16a00640 1 -- 172.21.15.46:0/467380000 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d0c009550 con 0x7f0d00074310 2024-09-17T03:29:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.469+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d00074310 msgr2=0x7f0d00076820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.473+0000 7f0d25a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d00074310 0x7f0d00076820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d20161b30 tx=0x7f0d0c009060 comp rx=0 tx=0).stop 2024-09-17T03:29:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.473+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 msgr2=0x7f0d201396d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.473+0000 7f0d25a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 0x7f0d201396d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d1001a550 tx=0x7f0d100aeb50 comp rx=0 tx=0).stop 2024-09-17T03:29:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 shutdown_connections 2024-09-17T03:29:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d00074310 0x7f0d00076820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d200b4530 0x7f0d200aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d200afdd0 0x7f0d20160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 --2- 172.21.15.46:0/467380000 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0d2006efd0 0x7f0d201396d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 >> 172.21.15.46:0/467380000 conn(0x7f0d200d0cf0 msgr2=0x7f0d200d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 shutdown_connections 2024-09-17T03:29:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.477+0000 7f0d25a00640 1 -- 172.21.15.46:0/467380000 wait complete. 2024-09-17T03:29:48.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone59644 2024-09-17T03:29:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 -- 172.21.15.46:0/2936080685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688074b20 msgr2=0x7f968815cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 --2- 172.21.15.46:0/2936080685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688074b20 0x7f968815cf90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9680009bc0 tx=0x7f968001cba0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 -- 172.21.15.46:0/2936080685 shutdown_connections 2024-09-17T03:29:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 --2- 172.21.15.46:0/2936080685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688074b20 0x7f968815cf90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 --2- 172.21.15.46:0/2936080685 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688157030 0x7f96880b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 -- 172.21.15.46:0/2936080685 >> 172.21.15.46:0/2936080685 conn(0x7f96880708c0 msgr2=0x7f9688070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 -- 172.21.15.46:0/2936080685 shutdown_connections 2024-09-17T03:29:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 -- 172.21.15.46:0/2936080685 wait complete. 2024-09-17T03:29:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 Processor -- start 2024-09-17T03:29:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.709+0000 7f968ea00640 1 -- start start 2024-09-17T03:29:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 0x7f9688154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688160d50 0x7f968813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968814cf60 0x7f968815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f968816a800 con 0x7f9688157030 2024-09-17T03:29:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f968816a680 con 0x7f968814cf60 2024-09-17T03:29:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f968816a980 con 0x7f9688160d50 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 0x7f9688154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 0x7f9688154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53716/0 (socket says 172.21.15.46:53716) 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 -- 172.21.15.46:0/1435335811 learned_addr learned my addr 172.21.15.46:0/1435335811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f967fe00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688160d50 0x7f968813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ca00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968814cf60 0x7f968815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 -- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688160d50 msgr2=0x7f968813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688160d50 0x7f968813fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 -- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968814cf60 msgr2=0x7f968815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968814cf60 0x7f968815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 -- 172.21.15.46:0/1435335811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96881568d0 con 0x7f9688157030 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f967fe00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688160d50 0x7f968813fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ca00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968814cf60 0x7f968815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:29:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9687e00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 0x7f9688154fa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f967800e7c0 tx=0x7f967800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967800ccc0 con 0x7f9688157030 2024-09-17T03:29:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f967800ce80 con 0x7f9688157030 2024-09-17T03:29:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9688155830 con 0x7f9688157030 2024-09-17T03:29:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678005280 con 0x7f9688157030 2024-09-17T03:29:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9688155df0 con 0x7f9688157030 2024-09-17T03:29:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.713+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968815cf90 con 0x7f9688157030 2024-09-17T03:29:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.717+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f9678005420 con 0x7f9688157030 2024-09-17T03:29:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.717+0000 7f9684c00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96640742e0 0x7f96640767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.717+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5767+0+0 (secure 0 0 0) 0x7f9678093af0 con 0x7f9688157030 2024-09-17T03:29:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.717+0000 7f967fe00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96640742e0 0x7f96640767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.717+0000 7f967fe00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96640742e0 0x7f96640767f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f96881541f0 tx=0x7f9670009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:48.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.721+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9678060220 con 0x7f9688157030 2024-09-17T03:29:48.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:48.933+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone59644", "target": ["mon-mgr", ""]}) -- 0x7f968806b540 con 0x7f96640742e0 2024-09-17T03:29:49.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.977+0000 7f9684c00640 1 -- 172.21.15.46:0/1435335811 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f96700013b0 con 0x7f96640742e0 2024-09-17T03:29:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.985+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96640742e0 msgr2=0x7f96640767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.985+0000 7f968ea00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96640742e0 0x7f96640767f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f96881541f0 tx=0x7f9670009040 comp rx=0 tx=0).stop 2024-09-17T03:29:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 msgr2=0x7f9688154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 0x7f9688154fa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f967800e7c0 tx=0x7f967800ec80 comp rx=0 tx=0).stop 2024-09-17T03:29:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 shutdown_connections 2024-09-17T03:29:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96640742e0 0x7f96640767f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968814cf60 0x7f968815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9688160d50 0x7f968813fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 --2- 172.21.15.46:0/1435335811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9688157030 0x7f9688154fa0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 >> 172.21.15.46:0/1435335811 conn(0x7f96880708c0 msgr2=0x7f968815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 shutdown_connections 2024-09-17T03:29:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:49.989+0000 7f968ea00640 1 -- 172.21.15.46:0/1435335811 wait complete. 2024-09-17T03:29:50.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone96045 2024-09-17T03:29:50.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.485+0000 7fb75f000640 1 -- 172.21.15.46:0/1149922589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 msgr2=0x7fb7580cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:50.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.485+0000 7fb75f000640 1 --2- 172.21.15.46:0/1149922589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 0x7fb7580cdaf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb748009c80 tx=0x7fb74801cda0 comp rx=0 tx=0).stop 2024-09-17T03:29:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.489+0000 7fb75f000640 1 -- 172.21.15.46:0/1149922589 shutdown_connections 2024-09-17T03:29:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.489+0000 7fb75f000640 1 --2- 172.21.15.46:0/1149922589 >> v2:172.21.15.46:3300/0 conn(0x7fb758163650 0x7fb758165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.489+0000 7fb75f000640 1 --2- 172.21.15.46:0/1149922589 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 0x7fb7580cdaf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.489+0000 7fb75f000640 1 -- 172.21.15.46:0/1149922589 >> 172.21.15.46:0/1149922589 conn(0x7fb7580b2040 msgr2=0x7fb7580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.493+0000 7fb75f000640 1 -- 172.21.15.46:0/1149922589 shutdown_connections 2024-09-17T03:29:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.493+0000 7fb75f000640 1 -- 172.21.15.46:0/1149922589 wait complete. 2024-09-17T03:29:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.509+0000 7fb75f000640 1 Processor -- start 2024-09-17T03:29:50.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.513+0000 7fb75f000640 1 -- start start 2024-09-17T03:29:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.517+0000 7fb75f000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 0x7fb758152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb75f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 0x7fb758164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb75f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb758163650 0x7fb758164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb75f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580b3fc0 con 0x7fb758163650 2024-09-17T03:29:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb75f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580b3e40 con 0x7fb75815aa50 2024-09-17T03:29:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb75f000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580b4140 con 0x7fb7580ce4c0 2024-09-17T03:29:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 0x7fb758164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 0x7fb758164ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:60382/0 (socket says 172.21.15.46:60382) 2024-09-17T03:29:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 -- 172.21.15.46:0/2300931063 learned_addr learned my addr 172.21.15.46:0/2300931063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb757e00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 0x7fb758152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb75d000640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb758163650 0x7fb758164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 -- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 msgr2=0x7fb758152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 0x7fb758152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 -- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb758163650 msgr2=0x7fb758164ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb758163650 0x7fb758164ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.521+0000 7fb74fe00640 1 -- 172.21.15.46:0/2300931063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb758200d50 con 0x7fb75815aa50 2024-09-17T03:29:50.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.525+0000 7fb74fe00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 0x7fb758164ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c430 tx=0x7fb74000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:50.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.525+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740017070 con 0x7fb75815aa50 2024-09-17T03:29:50.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.525+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb740003600 con 0x7fb75815aa50 2024-09-17T03:29:50.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.525+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740003a90 con 0x7fb75815aa50 2024-09-17T03:29:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.525+0000 7fb75f000640 1 -- 172.21.15.46:0/2300931063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb758201030 con 0x7fb75815aa50 2024-09-17T03:29:50.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.525+0000 7fb75f000640 1 -- 172.21.15.46:0/2300931063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb758156020 con 0x7fb75815aa50 2024-09-17T03:29:50.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.529+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758200ac0 con 0x7fb75815aa50 2024-09-17T03:29:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.533+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fb74001a070 con 0x7fb75815aa50 2024-09-17T03:29:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.533+0000 7fb754c00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb734074370 0x7fb734076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.533+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7fb74005a9a0 con 0x7fb75815aa50 2024-09-17T03:29:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.537+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb74005ee30 con 0x7fb75815aa50 2024-09-17T03:29:50.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.565+0000 7fb757e00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb734074370 0x7fb734076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:50.573+0000 7fb757e00640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb734074370 0x7fb734076880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7480af000 tx=0x7fb74801b5e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:51.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.013+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone96045", "target": ["mon-mgr", ""]}) -- 0x7fb75806b540 con 0x7fb734074370 2024-09-17T03:29:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.493+0000 7fb754c00640 1 -- 172.21.15.46:0/2300931063 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb7480098a0 con 0x7fb734074370 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb734074370 msgr2=0x7fb734076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb734074370 0x7fb734076880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7480af000 tx=0x7fb74801b5e0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 msgr2=0x7fb758164ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 0x7fb758164ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c430 tx=0x7fb74000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 shutdown_connections 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb734074370 0x7fb734076880 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb758163650 0x7fb758164ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb75815aa50 0x7fb758164ab0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 --2- 172.21.15.46:0/2300931063 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb7580ce4c0 0x7fb758152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.497+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 >> 172.21.15.46:0/2300931063 conn(0x7fb7580b2040 msgr2=0x7fb758150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.501+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 shutdown_connections 2024-09-17T03:29:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.501+0000 7fb74d600640 1 -- 172.21.15.46:0/2300931063 wait complete. 2024-09-17T03:29:51.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92176 2024-09-17T03:29:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 -- 172.21.15.46:0/1361254093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cd970 msgr2=0x7fae680b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 --2- 172.21.15.46:0/1361254093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cd970 0x7fae680b3340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fae5000afd0 tx=0x7fae5001d4b0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 -- 172.21.15.46:0/1361254093 shutdown_connections 2024-09-17T03:29:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 --2- 172.21.15.46:0/1361254093 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae680cdd80 0x7fae680b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 --2- 172.21.15.46:0/1361254093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cd970 0x7fae680b3340 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 -- 172.21.15.46:0/1361254093 >> 172.21.15.46:0/1361254093 conn(0x7fae680b2040 msgr2=0x7fae680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 -- 172.21.15.46:0/1361254093 shutdown_connections 2024-09-17T03:29:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.905+0000 7fae6f600640 1 -- 172.21.15.46:0/1361254093 wait complete. 2024-09-17T03:29:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.909+0000 7fae6f600640 1 Processor -- start 2024-09-17T03:29:51.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.909+0000 7fae6f600640 1 -- start start 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 0x7fae68164f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cdd80 0x7fae6814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae6815aa50 0x7fae68152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae681680f0 con 0x7fae680cd970 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae68167f70 con 0x7fae680cdd80 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fae68168270 con 0x7fae6815aa50 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 0x7fae68164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 0x7fae68164f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53732/0 (socket says 172.21.15.46:53732) 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 -- 172.21.15.46:0/41579752 learned_addr learned my addr 172.21.15.46:0/41579752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6d600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae6815aa50 0x7fae68152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae5fe00640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cdd80 0x7fae6814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 -- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae6815aa50 msgr2=0x7fae68152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae6815aa50 0x7fae68152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 -- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cdd80 msgr2=0x7fae6814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cdd80 0x7fae6814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 -- 172.21.15.46:0/41579752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae681f9de0 con 0x7fae680cd970 2024-09-17T03:29:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6cc00640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 0x7fae68164f80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c430 tx=0x7fae5800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58011790 con 0x7fae680cd970 2024-09-17T03:29:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae58017070 con 0x7fae680cd970 2024-09-17T03:29:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae68150bf0 con 0x7fae680cd970 2024-09-17T03:29:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58011ce0 con 0x7fae680cd970 2024-09-17T03:29:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.913+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae681516b0 con 0x7fae680cd970 2024-09-17T03:29:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.917+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae681f98f0 con 0x7fae680cd970 2024-09-17T03:29:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.917+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fae5801c460 con 0x7fae680cd970 2024-09-17T03:29:51.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.921+0000 7fae65600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074370 0x7fae44076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.921+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7fae58092eb0 con 0x7fae680cd970 2024-09-17T03:29:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.921+0000 7fae5fe00640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074370 0x7fae44076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.921+0000 7fae5fe00640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074370 0x7fae44076880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae500af000 tx=0x7fae5001bcf0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:51.925+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fae5805f690 con 0x7fae680cd970 2024-09-17T03:29:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.313+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92176", "target": ["mon-mgr", ""]}) -- 0x7fae6806b540 con 0x7fae44074370 2024-09-17T03:29:52.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.493+0000 7fae65600640 1 -- 172.21.15.46:0/41579752 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae50008e50 con 0x7fae44074370 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074370 msgr2=0x7fae44076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074370 0x7fae44076880 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae500af000 tx=0x7fae5001bcf0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 msgr2=0x7fae68164f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 0x7fae68164f80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c430 tx=0x7fae5800c830 comp rx=0 tx=0).stop 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 shutdown_connections 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074370 0x7fae44076880 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae6815aa50 0x7fae68152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae680cdd80 0x7fae6814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 --2- 172.21.15.46:0/41579752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae680cd970 0x7fae68164f80 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 >> 172.21.15.46:0/41579752 conn(0x7fae680b2040 msgr2=0x7fae6815c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 shutdown_connections 2024-09-17T03:29:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.497+0000 7fae6f600640 1 -- 172.21.15.46:0/41579752 wait complete. 2024-09-17T03:29:52.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:29:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 -- 172.21.15.46:0/3303932041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb80cefa0 msgr2=0x7fcdb80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 --2- 172.21.15.46:0/3303932041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb80cefa0 0x7fcdb80cf3b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009900 tx=0x7fcda401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 -- 172.21.15.46:0/3303932041 shutdown_connections 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 --2- 172.21.15.46:0/3303932041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb80cefa0 0x7fcdb80cf3b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 --2- 172.21.15.46:0/3303932041 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb80749a0 0x7fcdb8074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 -- 172.21.15.46:0/3303932041 >> 172.21.15.46:0/3303932041 conn(0x7fcdb80b2040 msgr2=0x7fcdb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 -- 172.21.15.46:0/3303932041 shutdown_connections 2024-09-17T03:29:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.713+0000 7fcdc0400640 1 -- 172.21.15.46:0/3303932041 wait complete. 2024-09-17T03:29:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 Processor -- start 2024-09-17T03:29:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 -- start start 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb80749a0 0x7fcdb806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb80cefa0 0x7fcdb80aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 0x7fcdb80af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb816a060 con 0x7fcdb8160330 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8169ee0 con 0x7fcdb80749a0 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.717+0000 7fcdc0400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb816a1e0 con 0x7fcdb80cefa0 2024-09-17T03:29:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 0x7fcdb80af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 0x7fcdb80af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58038/0 (socket says 172.21.15.46:58038) 2024-09-17T03:29:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 -- 172.21.15.46:0/1753778603 learned_addr learned my addr 172.21.15.46:0/1753778603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbda00640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb80749a0 0x7fcdb806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbd000640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb80cefa0 0x7fcdb80aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 -- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb80cefa0 msgr2=0x7fcdb80aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb80cefa0 0x7fcdb80aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 -- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb80749a0 msgr2=0x7fcdb806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb80749a0 0x7fcdb806e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 -- 172.21.15.46:0/1753778603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb813cbb0 con 0x7fcdb8160330 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbda00640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb80749a0 0x7fcdb806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdbe400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 0x7fcdb80af010 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c6e0 tx=0x7fcdb000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0003b80 con 0x7fcdb8160330 2024-09-17T03:29:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdb0003d20 con 0x7fcdb8160330 2024-09-17T03:29:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb813ce90 con 0x7fcdb8160330 2024-09-17T03:29:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0005330 con 0x7fcdb8160330 2024-09-17T03:29:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb81434d0 con 0x7fcdb8160330 2024-09-17T03:29:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.721+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb81432c0 con 0x7fcdb8160330 2024-09-17T03:29:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.725+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fcdb00054d0 con 0x7fcdb8160330 2024-09-17T03:29:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.725+0000 7fcdb5600640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98074370 0x7fcd98076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.725+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7fcdb0092d60 con 0x7fcdb8160330 2024-09-17T03:29:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.725+0000 7fcdbda00640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98074370 0x7fcd98076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:52.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.725+0000 7fcdbda00640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98074370 0x7fcd98076880 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009f30 tx=0x7fcdac0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.729+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcdb005ee40 con 0x7fcdb8160330 2024-09-17T03:29:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.933+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fcdb8074db0 con 0x7fcd98074370 2024-09-17T03:29:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdb5600640 1 -- 172.21.15.46:0/1753778603 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fcdb8074db0 con 0x7fcd98074370 2024-09-17T03:29:52.937 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:29:52.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:52.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:29:52.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:52.937 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:29:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98074370 msgr2=0x7fcd98076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98074370 0x7fcd98076880 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009f30 tx=0x7fcdac0091e0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 msgr2=0x7fcdb80af010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 0x7fcdb80af010 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c6e0 tx=0x7fcdb000cba0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 shutdown_connections 2024-09-17T03:29:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd98074370 0x7fcd98076880 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdb8160330 0x7fcdb80af010 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcdb80cefa0 0x7fcdb80aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 --2- 172.21.15.46:0/1753778603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcdb80749a0 0x7fcdb806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.937+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 >> 172.21.15.46:0/1753778603 conn(0x7fcdb80b2040 msgr2=0x7fcdb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.941+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 shutdown_connections 2024-09-17T03:29:52.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:52.941+0000 7fcdc0400640 1 -- 172.21.15.46:0/1753778603 wait complete. 2024-09-17T03:29:52.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:29:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 -- 172.21.15.46:0/4044488870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec1527b0 msgr2=0x7f31ec074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 --2- 172.21.15.46:0/4044488870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec1527b0 0x7f31ec074e30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f31ec072e30 tx=0x7f31d401c800 comp rx=0 tx=0).stop 2024-09-17T03:29:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 -- 172.21.15.46:0/4044488870 shutdown_connections 2024-09-17T03:29:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 --2- 172.21.15.46:0/4044488870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec1527b0 0x7f31ec074e30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 --2- 172.21.15.46:0/4044488870 >> v2:172.21.15.99:3300/0 conn(0x7f31ec0b4de0 0x7f31ec0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 --2- 172.21.15.46:0/4044488870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec0b5c70 0x7f31ec15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 -- 172.21.15.46:0/4044488870 >> 172.21.15.46:0/4044488870 conn(0x7f31ec0b2040 msgr2=0x7f31ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 -- 172.21.15.46:0/4044488870 shutdown_connections 2024-09-17T03:29:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.121+0000 7f31f2800640 1 -- 172.21.15.46:0/4044488870 wait complete. 2024-09-17T03:29:53.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 Processor -- start 2024-09-17T03:29:53.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 -- start start 2024-09-17T03:29:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31ec0b4de0 0x7f31ec153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 0x7f31ec1fc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec1527b0 0x7f31ec146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec15f6b0 con 0x7f31ec0b5c70 2024-09-17T03:29:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec15f530 con 0x7f31ec0b4de0 2024-09-17T03:29:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec15f830 con 0x7f31ec1527b0 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 0x7f31ec1fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 0x7f31ec1fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58048/0 (socket says 172.21.15.46:58048) 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 -- 172.21.15.46:0/899618084 learned_addr learned my addr 172.21.15.46:0/899618084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31eb400640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31ec0b4de0 0x7f31ec153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31ebe00640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec1527b0 0x7f31ec146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 -- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec1527b0 msgr2=0x7f31ec146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec1527b0 0x7f31ec146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 -- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31ec0b4de0 msgr2=0x7f31ec153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31ec0b4de0 0x7f31ec153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 -- 172.21.15.46:0/899618084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec154650 con 0x7f31ec0b5c70 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31ebe00640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec1527b0 0x7f31ec146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e2a00640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 0x7f31ec1fc570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f31d40b0040 tx=0x7f31d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40b2040 con 0x7f31ec0b5c70 2024-09-17T03:29:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31d4004320 con 0x7f31ec0b5c70 2024-09-17T03:29:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4003860 con 0x7f31ec0b5c70 2024-09-17T03:29:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec154930 con 0x7f31ec0b5c70 2024-09-17T03:29:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.125+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec146ff0 con 0x7f31ec0b5c70 2024-09-17T03:29:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.129+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec15fce0 con 0x7f31ec0b5c70 2024-09-17T03:29:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.129+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f31d40bf020 con 0x7f31ec0b5c70 2024-09-17T03:29:53.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.129+0000 7f31e3e00640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31c8074370 0x7f31c8076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.129+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f31d412d4b0 con 0x7f31ec0b5c70 2024-09-17T03:29:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.129+0000 7f31eb400640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31c8074370 0x7f31c8076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.129+0000 7f31eb400640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31c8074370 0x7f31c8076880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005fd0 tx=0x7f31dc00a590 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:53.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.133+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31d40c6020 con 0x7f31ec0b5c70 2024-09-17T03:29:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.333+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31ec06b540 con 0x7f31c8074370 2024-09-17T03:29:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31e3e00640 1 -- 172.21.15.46:0/899618084 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f31dc014070 con 0x7f31c8074370 2024-09-17T03:29:53.340 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:53.340 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:29:53.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone96045" 2024-09-17T03:29:53.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone94810" 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone59644" 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone28183" 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone92176" 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone14462" 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone64201" 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:53.342 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:29:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31c8074370 msgr2=0x7f31c8076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31c8074370 0x7f31c8076880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f31dc005fd0 tx=0x7f31dc00a590 comp rx=0 tx=0).stop 2024-09-17T03:29:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 msgr2=0x7f31ec1fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 0x7f31ec1fc570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f31d40b0040 tx=0x7f31d4004040 comp rx=0 tx=0).stop 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 shutdown_connections 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31c8074370 0x7f31c8076880 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31ec1527b0 0x7f31ec146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31ec0b5c70 0x7f31ec1fc570 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 --2- 172.21.15.46:0/899618084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31ec0b4de0 0x7f31ec153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 >> 172.21.15.46:0/899618084 conn(0x7f31ec0b2040 msgr2=0x7f31ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 shutdown_connections 2024-09-17T03:29:53.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.341+0000 7f31f2800640 1 -- 172.21.15.46:0/899618084 wait complete. 2024-09-17T03:29:53.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone96045 2024-09-17T03:29:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 -- 172.21.15.46:0/3416482974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b5c70 msgr2=0x7f53e815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 --2- 172.21.15.46:0/3416482974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b5c70 0x7f53e815bb60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53e8074b20 tx=0x7f53dc01c760 comp rx=0 tx=0).stop 2024-09-17T03:29:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 -- 172.21.15.46:0/3416482974 shutdown_connections 2024-09-17T03:29:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 --2- 172.21.15.46:0/3416482974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b5c70 0x7f53e815bb60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 -- 172.21.15.46:0/3416482974 >> 172.21.15.46:0/3416482974 conn(0x7f53e80b2040 msgr2=0x7f53e80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 -- 172.21.15.46:0/3416482974 shutdown_connections 2024-09-17T03:29:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 -- 172.21.15.46:0/3416482974 wait complete. 2024-09-17T03:29:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 Processor -- start 2024-09-17T03:29:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.529+0000 7f53f0400640 1 -- start start 2024-09-17T03:29:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 0x7f53e80aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53e80b5c70 0x7f53e813a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e81527b0 0x7f53e813ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8169bd0 con 0x7f53e80b4de0 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8169a50 con 0x7f53e81527b0 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f53e8169d50 con 0x7f53e80b5c70 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 0x7f53e80aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 0x7f53e80aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58054/0 (socket says 172.21.15.46:58054) 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 -- 172.21.15.46:0/688685219 learned_addr learned my addr 172.21.15.46:0/688685219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53ee400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e81527b0 0x7f53e813ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53ed000640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53e80b5c70 0x7f53e813a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 -- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53e80b5c70 msgr2=0x7f53e813a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53e80b5c70 0x7f53e813a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 -- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e81527b0 msgr2=0x7f53e813ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e81527b0 0x7f53e813ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 -- 172.21.15.46:0/688685219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e80ab040 con 0x7f53e80b4de0 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53eda00640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 0x7f53e80aa4d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f53dc000c10 tx=0x7f53dc0a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0bb070 con 0x7f53e80b4de0 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53dc004470 con 0x7f53e80b4de0 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e80ab320 con 0x7f53e80b4de0 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0b2040 con 0x7f53e80b4de0 2024-09-17T03:29:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e80ab8e0 con 0x7f53e80b4de0 2024-09-17T03:29:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e815bb60 con 0x7f53e80b4de0 2024-09-17T03:29:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.533+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f53dc0be070 con 0x7f53e80b4de0 2024-09-17T03:29:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.537+0000 7f53e5600640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53c80743a0 0x7f53c80768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.537+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f53dc12cad0 con 0x7f53e80b4de0 2024-09-17T03:29:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.537+0000 7f53ed000640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53c80743a0 0x7f53c80768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.537+0000 7f53ed000640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53c80743a0 0x7f53c80768b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f53d40046c0 tx=0x7f53d4007cd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.541+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53dc016410 con 0x7f53e80b4de0 2024-09-17T03:29:53.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:53.737+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone96045", "target": ["mon-mgr", ""]}) -- 0x7f53e8076490 con 0x7f53c80743a0 2024-09-17T03:29:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.397+0000 7f53e5600640 1 -- 172.21.15.46:0/688685219 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53e8076490 con 0x7f53c80743a0 2024-09-17T03:29:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53c80743a0 msgr2=0x7f53c80768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53c80743a0 0x7f53c80768b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f53d40046c0 tx=0x7f53d4007cd0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 msgr2=0x7f53e80aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 0x7f53e80aa4d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f53dc000c10 tx=0x7f53dc0a56a0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 shutdown_connections 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53c80743a0 0x7f53c80768b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e81527b0 0x7f53e813ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53e80b5c70 0x7f53e813a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 --2- 172.21.15.46:0/688685219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e80b4de0 0x7f53e80aa4d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 >> 172.21.15.46:0/688685219 conn(0x7f53e80b2040 msgr2=0x7f53e80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 shutdown_connections 2024-09-17T03:29:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.401+0000 7f53f0400640 1 -- 172.21.15.46:0/688685219 wait complete. 2024-09-17T03:29:54.415 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:54.417+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:54.415 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:54.417+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:54.416 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:54.417+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:54.416 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:54.417+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:54.416 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:29:54.417+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:29:54.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:29:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 -- 172.21.15.46:0/4147745970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb828004d50 msgr2=0x7fb8280051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 --2- 172.21.15.46:0/4147745970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb828004d50 0x7fb8280051a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb834001860 tx=0x7fb83401c880 comp rx=0 tx=0).stop 2024-09-17T03:29:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 -- 172.21.15.46:0/4147745970 shutdown_connections 2024-09-17T03:29:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 --2- 172.21.15.46:0/4147745970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb828004d50 0x7fb8280051a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 -- 172.21.15.46:0/4147745970 >> 172.21.15.46:0/4147745970 conn(0x7fb8440b2040 msgr2=0x7fb8440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 -- 172.21.15.46:0/4147745970 shutdown_connections 2024-09-17T03:29:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 -- 172.21.15.46:0/4147745970 wait complete. 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 Processor -- start 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.601+0000 7fb84a600640 1 -- start start 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb828004d50 0x7fb8440b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8441fd1e0 0x7fb844158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 0x7fb8441fc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb84415e500 con 0x7fb8441fc520 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb84415e380 con 0x7fb828004d50 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb84415bae0 con 0x7fb8441fd1e0 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 0x7fb8441fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 0x7fb8441fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58094/0 (socket says 172.21.15.46:58094) 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 -- 172.21.15.46:0/2099673701 learned_addr learned my addr 172.21.15.46:0/2099673701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb83be00640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8441fd1e0 0x7fb844158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843400640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb828004d50 0x7fb8440b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 -- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8441fd1e0 msgr2=0x7fb844158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8441fd1e0 0x7fb844158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 -- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb828004d50 msgr2=0x7fb8440b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb828004d50 0x7fb8440b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 -- 172.21.15.46:0/2099673701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb844158bd0 con 0x7fb8441fc520 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb843e00640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 0x7fb8441fc930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb8440b5ee0 tx=0x7fb83c006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c00fa20 con 0x7fb8441fc520 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb83c017070 con 0x7fb8441fc520 2024-09-17T03:29:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8441fce70 con 0x7fb8441fc520 2024-09-17T03:29:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c00fe80 con 0x7fb8441fc520 2024-09-17T03:29:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8440acb30 con 0x7fb8441fc520 2024-09-17T03:29:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.605+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84415c040 con 0x7fb8441fc520 2024-09-17T03:29:54.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.609+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fb83c01c460 con 0x7fb8441fc520 2024-09-17T03:29:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.609+0000 7fb83b400640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb820074370 0x7fb820076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.609+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7fb83c094380 con 0x7fb8441fc520 2024-09-17T03:29:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.609+0000 7fb843400640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb820074370 0x7fb820076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.609+0000 7fb843400640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb820074370 0x7fb820076880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb834001830 tx=0x7fb8340a8040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:54.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.613+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb8441fce70 con 0x7fb8441fc520 2024-09-17T03:29:54.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.813+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb844159bc0 con 0x7fb820074370 2024-09-17T03:29:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.829+0000 7fb83b400640 1 -- 172.21.15.46:0/2099673701 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb844159bc0 con 0x7fb820074370 2024-09-17T03:29:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:54.831 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:29:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:29:54.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:29:54.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:29:54.832 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:29:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb820074370 msgr2=0x7fb820076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb820074370 0x7fb820076880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb834001830 tx=0x7fb8340a8040 comp rx=0 tx=0).stop 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 msgr2=0x7fb8441fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 0x7fb8441fc930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb8440b5ee0 tx=0x7fb83c006660 comp rx=0 tx=0).stop 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 shutdown_connections 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb820074370 0x7fb820076880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8441fc520 0x7fb8441fc930 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb8441fd1e0 0x7fb844158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 --2- 172.21.15.46:0/2099673701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb828004d50 0x7fb8440b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 >> 172.21.15.46:0/2099673701 conn(0x7fb8440b2040 msgr2=0x7fb8440cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 shutdown_connections 2024-09-17T03:29:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:54.833+0000 7fb84a600640 1 -- 172.21.15.46:0/2099673701 wait complete. 2024-09-17T03:29:54.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:29:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f44a0200640 1 -- 172.21.15.46:0/2773953232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4480004810 msgr2=0x7f4480004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f44a0200640 1 --2- 172.21.15.46:0/2773953232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4480004810 0x7f4480004c20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4498071fd0 tx=0x7f448401c850 comp rx=0 tx=0).stop 2024-09-17T03:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f449e200640 1 -- 172.21.15.46:0/2773953232 reap_dead start 2024-09-17T03:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f44a0200640 1 -- 172.21.15.46:0/2773953232 shutdown_connections 2024-09-17T03:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f44a0200640 1 -- 172.21.15.46:0/2773953232 >> 172.21.15.46:0/2773953232 conn(0x7f44980b2040 msgr2=0x7f44980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f44a0200640 1 -- 172.21.15.46:0/2773953232 shutdown_connections 2024-09-17T03:29:55.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.021+0000 7f44a0200640 1 -- 172.21.15.46:0/2773953232 wait complete. 2024-09-17T03:29:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 Processor -- start 2024-09-17T03:29:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 -- start start 2024-09-17T03:29:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4480004810 0x7f4498152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f449814fa10 0x7f4498156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 0x7f44981fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4498167c10 con 0x7f4498157280 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4498167a90 con 0x7f4480004810 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f44a0200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4498167d90 con 0x7f449814fa10 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 0x7f44981fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 0x7f44981fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58110/0 (socket says 172.21.15.46:58110) 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449ce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f449814fa10 0x7f4498156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 -- 172.21.15.46:0/2993221696 learned_addr learned my addr 172.21.15.46:0/2993221696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449d800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4480004810 0x7f4498152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 -- 172.21.15.46:0/2993221696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f449814fa10 msgr2=0x7f4498156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f449814fa10 0x7f4498156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 -- 172.21.15.46:0/2993221696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4480004810 msgr2=0x7f4498152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4480004810 0x7f4498152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 -- 172.21.15.46:0/2993221696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980b3d50 con 0x7f4498157280 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.025+0000 7f449e200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 0x7f44981fc260 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4490000f90 tx=0x7f4490006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490015070 con 0x7f4498157280 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f449000fa80 con 0x7f4498157280 2024-09-17T03:29:55.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490011430 con 0x7f4498157280 2024-09-17T03:29:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4498158460 con 0x7f4498157280 2024-09-17T03:29:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4498156260 con 0x7f4498157280 2024-09-17T03:29:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4498074e30 con 0x7f4498157280 2024-09-17T03:29:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.029+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f44900115d0 con 0x7f4498157280 2024-09-17T03:29:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.033+0000 7f4495600640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4478074370 0x7f4478076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.033+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f4490093e30 con 0x7f4498157280 2024-09-17T03:29:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.033+0000 7f449d800640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4478074370 0x7f4478076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.033+0000 7f449d800640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4478074370 0x7f4478076880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f448c005ec0 tx=0x7f448c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:55.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.037+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f449005ff90 con 0x7f4498157280 2024-09-17T03:29:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.245+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44980b42a0 con 0x7f4478074370 2024-09-17T03:29:55.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.273+0000 7f4495600640 1 -- 172.21.15.46:0/2993221696 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44980b42a0 con 0x7f4478074370 2024-09-17T03:29:55.275 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:29:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4478074370 msgr2=0x7f4478076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4478074370 0x7f4478076880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f448c005ec0 tx=0x7f448c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 msgr2=0x7f44981fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 0x7f44981fc260 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4490000f90 tx=0x7f4490006720 comp rx=0 tx=0).stop 2024-09-17T03:29:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 shutdown_connections 2024-09-17T03:29:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4478074370 0x7f4478076880 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4498157280 0x7f44981fc260 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f449814fa10 0x7f4498156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 --2- 172.21.15.46:0/2993221696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4480004810 0x7f4498152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 >> 172.21.15.46:0/2993221696 conn(0x7f44980b2040 msgr2=0x7f44980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 shutdown_connections 2024-09-17T03:29:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.277+0000 7f44a0200640 1 -- 172.21.15.46:0/2993221696 wait complete. 2024-09-17T03:29:55.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 -- 172.21.15.46:0/2487034118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 msgr2=0x7f696815f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 --2- 172.21.15.46:0/2487034118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f696815f170 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f695001c8b0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 -- 172.21.15.46:0/2487034118 shutdown_connections 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 --2- 172.21.15.46:0/2487034118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f696815f170 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 --2- 172.21.15.46:0/2487034118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f696815c5d0 0x7f6968073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 -- 172.21.15.46:0/2487034118 >> 172.21.15.46:0/2487034118 conn(0x7f69680b2c40 msgr2=0x7f69680b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 -- 172.21.15.46:0/2487034118 shutdown_connections 2024-09-17T03:29:55.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.461+0000 7f696e800640 1 -- 172.21.15.46:0/2487034118 wait complete. 2024-09-17T03:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 Processor -- start 2024-09-17T03:29:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 -- start start 2024-09-17T03:29:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f69681fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f696815c5d0 0x7f6968154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f696815f6b0 0x7f6968151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f696806f980 con 0x7f6968074300 2024-09-17T03:29:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69680b3d50 con 0x7f696815f6b0 2024-09-17T03:29:55.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f696806fb00 con 0x7f696815c5d0 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f69681fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f69681fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58134/0 (socket says 172.21.15.46:58134) 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 -- 172.21.15.46:0/1519023533 learned_addr learned my addr 172.21.15.46:0/1519023533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f695fe00640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f696815c5d0 0x7f6968154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967e00640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f696815f6b0 0x7f6968151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 -- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f696815c5d0 msgr2=0x7f6968154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f696815c5d0 0x7f6968154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 -- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f696815f6b0 msgr2=0x7f6968151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f696815f6b0 0x7f6968151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 -- 172.21.15.46:0/1519023533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6968153a40 con 0x7f6968074300 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967e00640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f696815f6b0 0x7f6968151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:29:55.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f6967400640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f69681fcca0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f695800e7c0 tx=0x7f695800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800ccc0 con 0x7f6968074300 2024-09-17T03:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f695800ce80 con 0x7f6968074300 2024-09-17T03:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958005280 con 0x7f6968074300 2024-09-17T03:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6968153d20 con 0x7f6968074300 2024-09-17T03:29:55.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.465+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681542e0 con 0x7f6968074300 2024-09-17T03:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.469+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696815f170 con 0x7f6968074300 2024-09-17T03:29:55.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.469+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f6958005420 con 0x7f6968074300 2024-09-17T03:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.469+0000 7f695f400640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69440743a0 0x7f69440768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.469+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f6958093af0 con 0x7f6968074300 2024-09-17T03:29:55.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.469+0000 7f695fe00640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69440743a0 0x7f69440768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.469+0000 7f695fe00640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69440743a0 0x7f69440768b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f69500af000 tx=0x7f695001b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.473+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6958060210 con 0x7f6968074300 2024-09-17T03:29:55.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.677+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f69681462a0 con 0x7f69440743a0 2024-09-17T03:29:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.697+0000 7f695f400640 1 -- 172.21.15.46:0/1519023533 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69500b3040 con 0x7f69440743a0 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69440743a0 msgr2=0x7f69440768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69440743a0 0x7f69440768b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f69500af000 tx=0x7f695001b0f0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 msgr2=0x7f69681fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f69681fcca0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f695800e7c0 tx=0x7f695800ec80 comp rx=0 tx=0).stop 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 shutdown_connections 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69440743a0 0x7f69440768b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f696815f6b0 0x7f6968151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f696815c5d0 0x7f6968154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 --2- 172.21.15.46:0/1519023533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6968074300 0x7f69681fcca0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 >> 172.21.15.46:0/1519023533 conn(0x7f69680b2c40 msgr2=0x7f696814c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 shutdown_connections 2024-09-17T03:29:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.701+0000 7f696e800640 1 -- 172.21.15.46:0/1519023533 wait complete. 2024-09-17T03:29:55.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone94810 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 -- 172.21.15.46:0/2784081725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0cefa0 msgr2=0x7f3f2c0cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 --2- 172.21.15.46:0/2784081725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0cefa0 0x7f3f2c0cf3b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f180098a0 tx=0x7f3f1801c6d0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 -- 172.21.15.46:0/2784081725 shutdown_connections 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 --2- 172.21.15.46:0/2784081725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0cefa0 0x7f3f2c0cf3b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 -- 172.21.15.46:0/2784081725 >> 172.21.15.46:0/2784081725 conn(0x7f3f2c0b2040 msgr2=0x7f3f2c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 -- 172.21.15.46:0/2784081725 shutdown_connections 2024-09-17T03:29:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.893+0000 7f3f32400640 1 -- 172.21.15.46:0/2784081725 wait complete. 2024-09-17T03:29:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 Processor -- start 2024-09-17T03:29:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 -- start start 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 0x7f3f2c15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0b3340 0x7f3f2c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f2c0cefa0 0x7f3f2c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c0b41f0 con 0x7f3f2c0749a0 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c0b4070 con 0x7f3f2c0cefa0 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f32400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c0b4370 con 0x7f3f2c0b3340 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 0x7f3f2c15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 0x7f3f2c15bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58140/0 (socket says 172.21.15.46:58140) 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 -- 172.21.15.46:0/3321495003 learned_addr learned my addr 172.21.15.46:0/3321495003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2aa00640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0b3340 0x7f3f2c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 -- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0b3340 msgr2=0x7f3f2c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0b3340 0x7f3f2c0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2be00640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f2c0cefa0 0x7f3f2c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 -- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f2c0cefa0 msgr2=0x7f3f2c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f2c0cefa0 0x7f3f2c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2b400640 1 -- 172.21.15.46:0/3321495003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c0ce220 con 0x7f3f2c0749a0 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2aa00640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0b3340 0x7f3f2c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.897+0000 7f3f2be00640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f2c0cefa0 0x7f3f2c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:29:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f2b400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 0x7f3f2c15bd90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c460 tx=0x7f3f1c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c017070 con 0x7f3f2c0749a0 2024-09-17T03:29:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f1c003600 con 0x7f3f2c0749a0 2024-09-17T03:29:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c003a50 con 0x7f3f2c0749a0 2024-09-17T03:29:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c0cd980 con 0x7f3f2c0749a0 2024-09-17T03:29:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c0cdf40 con 0x7f3f2c0749a0 2024-09-17T03:29:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c0cf3b0 con 0x7f3f2c0749a0 2024-09-17T03:29:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f3f1c01a070 con 0x7f3f2c0749a0 2024-09-17T03:29:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.901+0000 7f3f23400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f0c074370 0x7f3f0c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.905+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f3f1c0923c0 con 0x7f3f2c0749a0 2024-09-17T03:29:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.905+0000 7f3f2aa00640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f0c074370 0x7f3f0c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.905+0000 7f3f2aa00640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f0c074370 0x7f3f0c076880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f180af000 tx=0x7f3f1801af10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:55.909+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f1c05eb70 con 0x7f3f2c0749a0 2024-09-17T03:29:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.113+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone94810", "target": ["mon-mgr", ""]}) -- 0x7f3f2c06b540 con 0x7f3f0c074370 2024-09-17T03:29:56.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.133+0000 7f3f23400640 1 -- 172.21.15.46:0/3321495003 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f180041b0 con 0x7f3f0c074370 2024-09-17T03:29:56.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f0c074370 msgr2=0x7f3f0c076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f0c074370 0x7f3f0c076880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f180af000 tx=0x7f3f1801af10 comp rx=0 tx=0).stop 2024-09-17T03:29:56.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 msgr2=0x7f3f2c15bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 0x7f3f2c15bd90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c460 tx=0x7f3f1c00c920 comp rx=0 tx=0).stop 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 shutdown_connections 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f0c074370 0x7f3f0c076880 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f2c0cefa0 0x7f3f2c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f2c0b3340 0x7f3f2c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 --2- 172.21.15.46:0/3321495003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c0749a0 0x7f3f2c15bd90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 >> 172.21.15.46:0/3321495003 conn(0x7f3f2c0b2040 msgr2=0x7f3f2c0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 shutdown_connections 2024-09-17T03:29:56.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.137+0000 7f3f32400640 1 -- 172.21.15.46:0/3321495003 wait complete. 2024-09-17T03:29:56.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone59644 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 -- 172.21.15.46:0/3679711175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43900b4de0 msgr2=0x7f43900b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 --2- 172.21.15.46:0/3679711175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43900b4de0 0x7f43900b51f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f438c009c80 tx=0x7f438c01cb20 comp rx=0 tx=0).stop 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 -- 172.21.15.46:0/3679711175 shutdown_connections 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 --2- 172.21.15.46:0/3679711175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43900b4de0 0x7f43900b51f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 --2- 172.21.15.46:0/3679711175 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43901527b0 0x7f439015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 -- 172.21.15.46:0/3679711175 >> 172.21.15.46:0/3679711175 conn(0x7f43900b2040 msgr2=0x7f43900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 -- 172.21.15.46:0/3679711175 shutdown_connections 2024-09-17T03:29:56.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 -- 172.21.15.46:0/3679711175 wait complete. 2024-09-17T03:29:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.329+0000 7f4398800640 1 Processor -- start 2024-09-17T03:29:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- start start 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43900b4de0 0x7f43900ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43900b5c70 0x7f43900cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 0x7f439013d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4390168ac0 con 0x7f43901527b0 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4390168940 con 0x7f43900b5c70 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4390168c40 con 0x7f43900b4de0 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 0x7f439013d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 0x7f439013d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58152/0 (socket says 172.21.15.46:58152) 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 -- 172.21.15.46:0/986400777 learned_addr learned my addr 172.21.15.46:0/986400777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4395400640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43900b5c70 0x7f43900cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4395e00640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43900b4de0 0x7f43900ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 -- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43900b4de0 msgr2=0x7f43900ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43900b4de0 0x7f43900ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 -- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43900b5c70 msgr2=0x7f43900cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43900b5c70 0x7f43900cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 -- 172.21.15.46:0/986400777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43900ae680 con 0x7f43901527b0 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4395e00640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43900b4de0 0x7f43900ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4396800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 0x7f439013d260 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f438c01c950 tx=0x7f438c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0bc070 con 0x7f43901527b0 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f438c0042f0 con 0x7f43901527b0 2024-09-17T03:29:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0b2040 con 0x7f43901527b0 2024-09-17T03:29:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43901402a0 con 0x7f43901527b0 2024-09-17T03:29:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4390140860 con 0x7f43901527b0 2024-09-17T03:29:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.333+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43900ae8f0 con 0x7f43901527b0 2024-09-17T03:29:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.337+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f438c004720 con 0x7f43901527b0 2024-09-17T03:29:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.337+0000 7f438a000640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4374074370 0x7f4374076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.337+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f438c12d2d0 con 0x7f43901527b0 2024-09-17T03:29:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.337+0000 7f4395e00640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4374074370 0x7f4374076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.337+0000 7f4395e00640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4374074370 0x7f4374076880 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4384009890 tx=0x7f4384008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.341+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f438c0c5080 con 0x7f43901527b0 2024-09-17T03:29:56.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.365+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f438c0f5610 con 0x7f43901527b0 2024-09-17T03:29:56.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.545+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone59644", "target": ["mon-mgr", ""]}) -- 0x7f439006b540 con 0x7f4374074370 2024-09-17T03:29:56.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.561+0000 7f438a000640 1 -- 172.21.15.46:0/986400777 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4384009480 con 0x7f4374074370 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4374074370 msgr2=0x7f4374076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4374074370 0x7f4374076880 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4384009890 tx=0x7f4384008040 comp rx=0 tx=0).stop 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 msgr2=0x7f439013d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 0x7f439013d260 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f438c01c950 tx=0x7f438c0093d0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 shutdown_connections 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4374074370 0x7f4374076880 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901527b0 0x7f439013d260 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43900b5c70 0x7f43900cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 --2- 172.21.15.46:0/986400777 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43900b4de0 0x7f43900ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 >> 172.21.15.46:0/986400777 conn(0x7f43900b2040 msgr2=0x7f439015a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 shutdown_connections 2024-09-17T03:29:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.565+0000 7f4398800640 1 -- 172.21.15.46:0/986400777 wait complete. 2024-09-17T03:29:56.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28183 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 -- 172.21.15.46:0/1365929356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac0b4de0 msgr2=0x7f52ac0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1365929356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac0b4de0 0x7f52ac0b51f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009c80 tx=0x7f52a401cce0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 -- 172.21.15.46:0/1365929356 shutdown_connections 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1365929356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac0b4de0 0x7f52ac0b51f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1365929356 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52ac1527b0 0x7f52ac15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 -- 172.21.15.46:0/1365929356 >> 172.21.15.46:0/1365929356 conn(0x7f52ac0b2040 msgr2=0x7f52ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 -- 172.21.15.46:0/1365929356 shutdown_connections 2024-09-17T03:29:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.753+0000 7f52b0c00640 1 -- 172.21.15.46:0/1365929356 wait complete. 2024-09-17T03:29:56.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 Processor -- start 2024-09-17T03:29:56.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 -- start start 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52ac0b4de0 0x7f52ac1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52ac0b5c70 0x7f52ac154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 0x7f52ac15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac168ab0 con 0x7f52ac1527b0 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac168930 con 0x7f52ac0b5c70 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac168c30 con 0x7f52ac0b4de0 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 0x7f52ac15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 0x7f52ac15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58176/0 (socket says 172.21.15.46:58176) 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 -- 172.21.15.46:0/1293649122 learned_addr learned my addr 172.21.15.46:0/1293649122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52a9e00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52ac0b4de0 0x7f52ac1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52a9400640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52ac0b5c70 0x7f52ac154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 -- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52ac0b4de0 msgr2=0x7f52ac1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52ac0b4de0 0x7f52ac1498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 -- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52ac0b5c70 msgr2=0x7f52ac154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52ac0b5c70 0x7f52ac154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 -- 172.21.15.46:0/1293649122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac149df0 con 0x7f52ac1527b0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52aa800640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 0x7f52ac15aaa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009bd0 tx=0x7f52a4009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40bb070 con 0x7f52ac1527b0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a4004d40 con 0x7f52ac1527b0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1fbda0 con 0x7f52ac1527b0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.757+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40b2040 con 0x7f52ac1527b0 2024-09-17T03:29:56.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1fc360 con 0x7f52ac1527b0 2024-09-17T03:29:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac0b51f0 con 0x7f52ac1527b0 2024-09-17T03:29:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f52a40be070 con 0x7f52ac1527b0 2024-09-17T03:29:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52a2000640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f528c074430 0x7f528c076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7f52a412ccc0 con 0x7f52ac1527b0 2024-09-17T03:29:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52a9e00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f528c074430 0x7f528c076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:56.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.761+0000 7f52a9e00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f528c074430 0x7f528c076940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f529c007fd0 tx=0x7f529c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:56.765+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52a4016990 con 0x7f52ac1527b0 2024-09-17T03:29:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.025+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28183", "target": ["mon-mgr", ""]}) -- 0x7f52ac0b2910 con 0x7f528c074430 2024-09-17T03:29:57.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.057+0000 7f52a2000640 1 -- 172.21.15.46:0/1293649122 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52ac0b2910 con 0x7f528c074430 2024-09-17T03:29:57.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f528c074430 msgr2=0x7f528c076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:57.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f528c074430 0x7f528c076940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f529c007fd0 tx=0x7f529c009040 comp rx=0 tx=0).stop 2024-09-17T03:29:57.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 msgr2=0x7f52ac15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:57.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 0x7f52ac15aaa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009bd0 tx=0x7f52a4009350 comp rx=0 tx=0).stop 2024-09-17T03:29:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 shutdown_connections 2024-09-17T03:29:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f528c074430 0x7f528c076940 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52ac1527b0 0x7f52ac15aaa0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52ac0b5c70 0x7f52ac154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 --2- 172.21.15.46:0/1293649122 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52ac0b4de0 0x7f52ac1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.061+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 >> 172.21.15.46:0/1293649122 conn(0x7f52ac0b2040 msgr2=0x7f52ac154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.065+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 shutdown_connections 2024-09-17T03:29:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.065+0000 7f52b0c00640 1 -- 172.21.15.46:0/1293649122 wait complete. 2024-09-17T03:29:57.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92176 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 -- 172.21.15.46:0/3017014724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 msgr2=0x7ff19815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 --2- 172.21.15.46:0/3017014724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff19815bb60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff188009840 tx=0x7ff18801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 -- 172.21.15.46:0/3017014724 shutdown_connections 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 --2- 172.21.15.46:0/3017014724 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1980b4de0 0x7ff1980b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 --2- 172.21.15.46:0/3017014724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff19815bb60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 -- 172.21.15.46:0/3017014724 >> 172.21.15.46:0/3017014724 conn(0x7ff1980b2040 msgr2=0x7ff1980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 -- 172.21.15.46:0/3017014724 shutdown_connections 2024-09-17T03:29:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 -- 172.21.15.46:0/3017014724 wait complete. 2024-09-17T03:29:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.277+0000 7ff19da00640 1 Processor -- start 2024-09-17T03:29:57.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 -- start start 2024-09-17T03:29:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1980b4de0 0x7ff19813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff1981f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:57.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 0x7ff198153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff198168a20 con 0x7ff1981527b0 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981688a0 con 0x7ff1980b5c70 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff198168ba0 con 0x7ff1980b4de0 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 0x7ff198153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 0x7ff198153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58188/0 (socket says 172.21.15.46:58188) 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 -- 172.21.15.46:0/312263793 learned_addr learned my addr 172.21.15.46:0/312263793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff18fe00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff1981f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff196a00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1980b4de0 0x7ff19813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 -- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1980b4de0 msgr2=0x7ff19813cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1980b4de0 0x7ff19813cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 -- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 msgr2=0x7ff1981f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff1981f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 -- 172.21.15.46:0/312263793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19813d100 con 0x7ff1981527b0 2024-09-17T03:29:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff18fe00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff1981f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:29:57.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff197400640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 0x7ff198153530 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff19000ea80 tx=0x7ff19000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19000ce40 con 0x7ff1981527b0 2024-09-17T03:29:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff190004b30 con 0x7ff1981527b0 2024-09-17T03:29:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981f94c0 con 0x7ff1981527b0 2024-09-17T03:29:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190005020 con 0x7ff1981527b0 2024-09-17T03:29:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.281+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff198152d30 con 0x7ff1981527b0 2024-09-17T03:29:57.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.285+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1980b51f0 con 0x7ff1981527b0 2024-09-17T03:29:57.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.285+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7ff1900051c0 con 0x7ff1981527b0 2024-09-17T03:29:57.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.285+0000 7ff18ea00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff174074400 0x7ff174076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:57.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.285+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5767+0+0 (secure 0 0 0) 0x7ff190093cb0 con 0x7ff1981527b0 2024-09-17T03:29:57.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.285+0000 7ff196a00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff174074400 0x7ff174076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.285+0000 7ff196a00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff174074400 0x7ff174076910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff1880af000 tx=0x7ff18801b030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.289+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff1900601c0 con 0x7ff1981527b0 2024-09-17T03:29:57.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:57.489+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92176", "target": ["mon-mgr", ""]}) -- 0x7ff1981511f0 con 0x7ff174074400 2024-09-17T03:29:59.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff18ea00640 1 -- 172.21.15.46:0/312263793 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff1981511f0 con 0x7ff174074400 2024-09-17T03:29:59.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff174074400 msgr2=0x7ff174076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff174074400 0x7ff174076910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff1880af000 tx=0x7ff18801b030 comp rx=0 tx=0).stop 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 msgr2=0x7ff198153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 0x7ff198153530 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff19000ea80 tx=0x7ff19000ef40 comp rx=0 tx=0).stop 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 shutdown_connections 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff174074400 0x7ff174076910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1981527b0 0x7ff198153530 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1980b5c70 0x7ff1981f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 --2- 172.21.15.46:0/312263793 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1980b4de0 0x7ff19813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 >> 172.21.15.46:0/312263793 conn(0x7ff1980b2040 msgr2=0x7ff198154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 shutdown_connections 2024-09-17T03:29:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.401+0000 7ff19da00640 1 -- 172.21.15.46:0/312263793 wait complete. 2024-09-17T03:29:59.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone14462 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 -- 172.21.15.46:0/2696400352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015e8c0 msgr2=0x7fb4a015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 --2- 172.21.15.46:0/2696400352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015e8c0 0x7fb4a015ecd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb4a015cd60 tx=0x7fb49001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 -- 172.21.15.46:0/2696400352 shutdown_connections 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 --2- 172.21.15.46:0/2696400352 >> v2:172.21.15.99:3300/0 conn(0x7fb4a015bdf0 0x7fb4a0165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 --2- 172.21.15.46:0/2696400352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015e8c0 0x7fb4a015ecd0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 -- 172.21.15.46:0/2696400352 >> 172.21.15.46:0/2696400352 conn(0x7fb4a00b2040 msgr2=0x7fb4a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 -- 172.21.15.46:0/2696400352 shutdown_connections 2024-09-17T03:29:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 -- 172.21.15.46:0/2696400352 wait complete. 2024-09-17T03:29:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 Processor -- start 2024-09-17T03:29:59.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.605+0000 7fb4a5400640 1 -- start start 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 0x7fb4a0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a015bdf0 0x7fb4a01584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a015e8c0 0x7fb4a01f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0168c00 con 0x7fb4a015b2d0 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0168a80 con 0x7fb4a015e8c0 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a0168d80 con 0x7fb4a015bdf0 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 0x7fb4a0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 0x7fb4a0154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58210/0 (socket says 172.21.15.46:58210) 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 -- 172.21.15.46:0/394509474 learned_addr learned my addr 172.21.15.46:0/394509474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49de00640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a015bdf0 0x7fb4a01584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49f200640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a015e8c0 0x7fb4a01f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 -- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a015bdf0 msgr2=0x7fb4a01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a015bdf0 0x7fb4a01584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 -- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a015e8c0 msgr2=0x7fb4a01f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a015e8c0 0x7fb4a01f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 -- 172.21.15.46:0/394509474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a0157a40 con 0x7fb4a015b2d0 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb49e800640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 0x7fb4a0154000 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb4900194d0 tx=0x7fb490005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900bb070 con 0x7fb4a015b2d0 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb490004320 con 0x7fb4a015b2d0 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a015ffd0 con 0x7fb4a015b2d0 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900b2040 con 0x7fb4a015b2d0 2024-09-17T03:29:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a0157f90 con 0x7fb4a015b2d0 2024-09-17T03:29:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.609+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a015ecd0 con 0x7fb4a015b2d0 2024-09-17T03:29:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.613+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fb4900052d0 con 0x7fb4a015b2d0 2024-09-17T03:29:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.613+0000 7fb496a00640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb480074430 0x7fb480076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:29:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.613+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7fb49012c460 con 0x7fb4a015b2d0 2024-09-17T03:29:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.613+0000 7fb49de00640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb480074430 0x7fb480076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:29:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.613+0000 7fb49de00640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb480074430 0x7fb480076940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00cc460 tx=0x7fb48c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:29:59.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.617+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb4900164b0 con 0x7fb4a015b2d0 2024-09-17T03:29:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.817+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone14462", "target": ["mon-mgr", ""]}) -- 0x7fb4a006b540 con 0x7fb480074430 2024-09-17T03:29:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.837+0000 7fb496a00640 1 -- 172.21.15.46:0/394509474 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb48c0050d0 con 0x7fb480074430 2024-09-17T03:29:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb480074430 msgr2=0x7fb480076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb480074430 0x7fb480076940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00cc460 tx=0x7fb48c009040 comp rx=0 tx=0).stop 2024-09-17T03:29:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 msgr2=0x7fb4a0154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:29:59.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 0x7fb4a0154000 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb4900194d0 tx=0x7fb490005e70 comp rx=0 tx=0).stop 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 shutdown_connections 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb480074430 0x7fb480076940 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a015e8c0 0x7fb4a01f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4a015bdf0 0x7fb4a01584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 --2- 172.21.15.46:0/394509474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4a015b2d0 0x7fb4a0154000 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 >> 172.21.15.46:0/394509474 conn(0x7fb4a00b2040 msgr2=0x7fb4a006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 shutdown_connections 2024-09-17T03:29:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:29:59.841+0000 7fb4a5400640 1 -- 172.21.15.46:0/394509474 wait complete. 2024-09-17T03:29:59.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone64201 2024-09-17T03:29:59.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:29:59.969 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:30:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 -- 172.21.15.46:0/3939409220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051006dbc0 msgr2=0x7f051006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 --2- 172.21.15.46:0/3939409220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051006dbc0 0x7f051006dfd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc01c850 comp rx=0 tx=0).stop 2024-09-17T03:30:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 -- 172.21.15.46:0/3939409220 shutdown_connections 2024-09-17T03:30:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 --2- 172.21.15.46:0/3939409220 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f051015f5c0 0x7f051015f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 --2- 172.21.15.46:0/3939409220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051006dbc0 0x7f051006dfd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 -- 172.21.15.46:0/3939409220 >> 172.21.15.46:0/3939409220 conn(0x7f05100b41f0 msgr2=0x7f05100b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:00.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 -- 172.21.15.46:0/3939409220 shutdown_connections 2024-09-17T03:30:00.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 -- 172.21.15.46:0/3939409220 wait complete. 2024-09-17T03:30:00.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.101+0000 7f0514a00640 1 Processor -- start 2024-09-17T03:30:00.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 -- start start 2024-09-17T03:30:00.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f051006dbc0 0x7f05100c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05100d1140 0x7f051009a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 0x7f0510154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 0x7f0510154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 0x7f0510154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58228/0 (socket says 172.21.15.46:58228) 2024-09-17T03:30:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 -- 172.21.15.46:0/1437458057 learned_addr learned my addr 172.21.15.46:0/1437458057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0510069990 con 0x7f051015f5c0 2024-09-17T03:30:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0510069810 con 0x7f051006dbc0 2024-09-17T03:30:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0510069b10 con 0x7f05100d1140 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050f400640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f051006dbc0 0x7f05100c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050ea00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05100d1140 0x7f051009a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 -- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05100d1140 msgr2=0x7f051009a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05100d1140 0x7f051009a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 -- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f051006dbc0 msgr2=0x7f05100c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f051006dbc0 0x7f05100c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050fe00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0510167580 con 0x7f051015f5c0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.105+0000 7f050f400640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f051006dbc0 0x7f05100c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f050fe00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 0x7f0510154cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f050800ea80 tx=0x7f050800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050800cea0 con 0x7f051015f5c0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0508004b30 con 0x7f051015f5c0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508005020 con 0x7f051015f5c0 2024-09-17T03:30:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0510200bd0 con 0x7f051015f5c0 2024-09-17T03:30:00.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0510201190 con 0x7f051015f5c0 2024-09-17T03:30:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.109+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051006dfd0 con 0x7f051015f5c0 2024-09-17T03:30:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.113+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f05080051c0 con 0x7f051015f5c0 2024-09-17T03:30:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.113+0000 7f0507400640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04f0074430 0x7f04f0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.113+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7f0508093f50 con 0x7f051015f5c0 2024-09-17T03:30:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.117+0000 7f050f400640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04f0074430 0x7f04f0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.117+0000 7f050f400640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04f0074430 0x7f04f0076940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0500000f70 tx=0x7f0500009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.121+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f05080609c0 con 0x7f051015f5c0 2024-09-17T03:30:00.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.401+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone64201", "target": ["mon-mgr", ""]}) -- 0x7f05100cbb30 con 0x7f04f0074430 2024-09-17T03:30:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.421+0000 7f0507400640 1 -- 172.21.15.46:0/1437458057 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f05100cbb30 con 0x7f04f0074430 2024-09-17T03:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04f0074430 msgr2=0x7f04f0076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04f0074430 0x7f04f0076940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0500000f70 tx=0x7f0500009040 comp rx=0 tx=0).stop 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 msgr2=0x7f0510154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 0x7f0510154cd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f050800ea80 tx=0x7f050800ef40 comp rx=0 tx=0).stop 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 shutdown_connections 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04f0074430 0x7f04f0076940 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f051015f5c0 0x7f0510154cd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05100d1140 0x7f051009a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 --2- 172.21.15.46:0/1437458057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f051006dbc0 0x7f05100c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 >> 172.21.15.46:0/1437458057 conn(0x7f05100b41f0 msgr2=0x7f0510166640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 shutdown_connections 2024-09-17T03:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.425+0000 7f0514a00640 1 -- 172.21.15.46:0/1437458057 wait complete. 2024-09-17T03:30:00.453 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:30:00.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:30:00.510 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:30:00.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:30:00.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:30:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 -- 172.21.15.46:0/3503665551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc1527b0 msgr2=0x7f4cdc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 --2- 172.21.15.46:0/3503665551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc1527b0 0x7f4cdc074e30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009900 tx=0x7f4cc801c820 comp rx=0 tx=0).stop 2024-09-17T03:30:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 -- 172.21.15.46:0/3503665551 shutdown_connections 2024-09-17T03:30:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 --2- 172.21.15.46:0/3503665551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc1527b0 0x7f4cdc074e30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 --2- 172.21.15.46:0/3503665551 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 0x7f4cdc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 -- 172.21.15.46:0/3503665551 >> 172.21.15.46:0/3503665551 conn(0x7f4cdc0b2040 msgr2=0x7f4cdc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 -- 172.21.15.46:0/3503665551 shutdown_connections 2024-09-17T03:30:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.677+0000 7f4ce4000640 1 -- 172.21.15.46:0/3503665551 wait complete. 2024-09-17T03:30:00.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 Processor -- start 2024-09-17T03:30:00.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 -- start start 2024-09-17T03:30:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 0x7f4cdc1f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 0x7f4cdc14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc1527b0 0x7f4cdc15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc168c50 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc168ad0 con 0x7f4cdc1527b0 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce4000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc168dd0 con 0x7f4cdc0b5c70 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 0x7f4cdc1f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 0x7f4cdc1f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58246/0 (socket says 172.21.15.46:58246) 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce1600640 1 -- 172.21.15.46:0/2036929294 learned_addr learned my addr 172.21.15.46:0/2036929294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce0c00640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 0x7f4cdc14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.681+0000 7f4ce2000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc1527b0 0x7f4cdc15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce1600640 1 -- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 msgr2=0x7f4cdc14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce1600640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 0x7f4cdc14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce1600640 1 -- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc1527b0 msgr2=0x7f4cdc15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce1600640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc1527b0 0x7f4cdc15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce1600640 1 -- 172.21.15.46:0/2036929294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc0cf990 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce0c00640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 0x7f4cdc14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce1600640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 0x7f4cdc1f8bb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000e7c0 tx=0x7f4cd000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000ccc0 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cd000ce80 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0005280 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc0cfc70 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc154420 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc15bb60 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.685+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f4cd0005420 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.689+0000 7f4cd9600640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cbc074400 0x7f4cbc076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.689+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7f4cd0093730 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.689+0000 7f4ce0c00640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cbc074400 0x7f4cbc076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.689+0000 7f4ce0c00640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cbc074400 0x7f4cbc076910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80af000 tx=0x7f4cc801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.693+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4cd005fb50 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4cdc0cfe40 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4cd9600640 1 -- 172.21.15.46:0/2036929294 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7f4cd0064580 con 0x7f4cdc0b4de0 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cbc074400 msgr2=0x7f4cbc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cbc074400 0x7f4cbc076910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80af000 tx=0x7f4cc801b060 comp rx=0 tx=0).stop 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 msgr2=0x7f4cdc1f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 0x7f4cdc1f8bb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000e7c0 tx=0x7f4cd000ec80 comp rx=0 tx=0).stop 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 shutdown_connections 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cbc074400 0x7f4cbc076910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cdc1527b0 0x7f4cdc15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cdc0b5c70 0x7f4cdc14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 --2- 172.21.15.46:0/2036929294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0b4de0 0x7f4cdc1f8bb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 >> 172.21.15.46:0/2036929294 conn(0x7f4cdc0b2040 msgr2=0x7f4cdc151a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:00.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 shutdown_connections 2024-09-17T03:30:00.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:00.865+0000 7f4ce4000640 1 -- 172.21.15.46:0/2036929294 wait complete. 2024-09-17T03:30:00.888 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:30:00.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:30:00.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:30:01.096 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:30:01.096 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:30:01.096 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:01.096 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:01.106 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:01.106 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:01.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:30:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 -- 172.21.15.46:0/736128007 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c00b4de0 msgr2=0x7f10c00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 --2- 172.21.15.46:0/736128007 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c00b4de0 0x7f10c00b51f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009bc0 tx=0x7f10b801cb70 comp rx=0 tx=0).stop 2024-09-17T03:30:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 -- 172.21.15.46:0/736128007 shutdown_connections 2024-09-17T03:30:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 --2- 172.21.15.46:0/736128007 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c00b4de0 0x7f10c00b51f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 --2- 172.21.15.46:0/736128007 >> v2:172.21.15.99:3300/0 conn(0x7f10c01527b0 0x7f10c015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 -- 172.21.15.46:0/736128007 >> 172.21.15.46:0/736128007 conn(0x7f10c00b2040 msgr2=0x7f10c00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 -- 172.21.15.46:0/736128007 shutdown_connections 2024-09-17T03:30:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.357+0000 7f10c4a00640 1 -- 172.21.15.46:0/736128007 wait complete. 2024-09-17T03:30:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 Processor -- start 2024-09-17T03:30:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 -- start start 2024-09-17T03:30:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10c00b4de0 0x7f10c01f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 0x7f10c014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c01527b0 0x7f10c015ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01689d0 con 0x7f10c00b5c70 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0168850 con 0x7f10c00b4de0 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10c4a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f10c0168b50 con 0x7f10c01527b0 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 0x7f10c014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 0x7f10c014dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58258/0 (socket says 172.21.15.46:58258) 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 -- 172.21.15.46:0/2850887980 learned_addr learned my addr 172.21.15.46:0/2850887980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10be800640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c01527b0 0x7f10c015ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10bde00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10c00b4de0 0x7f10c01f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 -- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c01527b0 msgr2=0x7f10c015ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c01527b0 0x7f10c015ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 -- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10c00b4de0 msgr2=0x7f10c01f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10c00b4de0 0x7f10c01f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 -- 172.21.15.46:0/2850887980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c0158200 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10be800640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c01527b0 0x7f10c015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.361+0000 7f10b7e00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 0x7f10c014dcc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f10a800e9a0 tx=0x7f10a800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a800cd80 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c015b070 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10a8004020 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004450 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c014d890 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c00b51f0 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f10a8003600 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10b6000640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f109c074400 0x7f109c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7f10a80937e0 con 0x7f10c00b5c70 2024-09-17T03:30:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.365+0000 7f10bde00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f109c074400 0x7f109c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.369+0000 7f10bde00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f109c074400 0x7f109c076910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f10b0007d70 tx=0x7f10b0007d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.373+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f10c0158200 con 0x7f10c00b5c70 2024-09-17T03:30:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.557+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10c00b1d20 con 0x7f10c00b5c70 2024-09-17T03:30:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.557+0000 7f10b6000640 1 -- 172.21.15.46:0/2850887980 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7f10a805fb80 con 0x7f10c00b5c70 2024-09-17T03:30:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f109c074400 msgr2=0x7f109c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f109c074400 0x7f109c076910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f10b0007d70 tx=0x7f10b0007d00 comp rx=0 tx=0).stop 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 msgr2=0x7f10c014dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 0x7f10c014dcc0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f10a800e9a0 tx=0x7f10a800ee60 comp rx=0 tx=0).stop 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 shutdown_connections 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f109c074400 0x7f109c076910 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10c01527b0 0x7f10c015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c00b5c70 0x7f10c014dcc0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 --2- 172.21.15.46:0/2850887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10c00b4de0 0x7f10c01f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 >> 172.21.15.46:0/2850887980 conn(0x7f10c00b2040 msgr2=0x7f10c00b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 shutdown_connections 2024-09-17T03:30:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.561+0000 7f10c4a00640 1 -- 172.21.15.46:0/2850887980 wait complete. 2024-09-17T03:30:01.580 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:30:01.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:01.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:30:01.626 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:30:01.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:01.627 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:30:01.676 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:30:01.676 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:30:01.676 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:01.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:30:01.685 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:30:01.685 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:01.685 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:01.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:30:01.727 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:30:01.727 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:30:01.728 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:01.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:30:01.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 -- 172.21.15.46:0/3709784243 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9700b5c70 msgr2=0x7fd97015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 --2- 172.21.15.46:0/3709784243 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9700b5c70 0x7fd97015bb60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd968009bf0 tx=0x7fd96801cb60 comp rx=0 tx=0).stop 2024-09-17T03:30:02.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 -- 172.21.15.46:0/3709784243 shutdown_connections 2024-09-17T03:30:02.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 --2- 172.21.15.46:0/3709784243 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9700b5c70 0x7fd97015bb60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 -- 172.21.15.46:0/3709784243 >> 172.21.15.46:0/3709784243 conn(0x7fd9700b2040 msgr2=0x7fd9700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:02.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 -- 172.21.15.46:0/3709784243 shutdown_connections 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.945+0000 7fd975400640 1 -- 172.21.15.46:0/3709784243 wait complete. 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 Processor -- start 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 -- start start 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9700b4de0 0x7fd970141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 0x7fd970158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9701527b0 0x7fd9701fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9700b4700 con 0x7fd9700b4de0 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9700b4580 con 0x7fd9700b5c70 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.949+0000 7fd975400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd9700b4880 con 0x7fd9701527b0 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 0x7fd970158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9700b4de0 0x7fd970141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 0x7fd970158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:38450/0 (socket says 172.21.15.46:38450) 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 -- 172.21.15.46:0/2599814780 learned_addr learned my addr 172.21.15.46:0/2599814780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96f200640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9701527b0 0x7fd9701fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 -- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9701527b0 msgr2=0x7fd9701fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9701527b0 0x7fd9701fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 -- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9700b4de0 msgr2=0x7fd970141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9700b4de0 0x7fd970141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 -- 172.21.15.46:0/2599814780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9701420a0 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd96de00640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 0x7fd970158080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c390 tx=0x7fd95c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0052d0 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd95c007020 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c007470 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd970158b50 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9700cd640 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.953+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9700ceec0 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.957+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fd95c0067e0 con 0x7fd9700b5c70 2024-09-17T03:30:02.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.957+0000 7fd966a00640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd950074400 0x7fd950076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.957+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7fd95c092610 con 0x7fd9700b5c70 2024-09-17T03:30:02.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.957+0000 7fd96e800640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd950074400 0x7fd950076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.961+0000 7fd96e800640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd950074400 0x7fd950076910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd9680af000 tx=0x7fd96801b3a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:02.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:01.965+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd95c05e6c0 con 0x7fd9700b5c70 2024-09-17T03:30:02.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.341+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97015bb60 con 0x7fd9700b5c70 2024-09-17T03:30:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd966a00640 1 -- 172.21.15.46:0/2599814780 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5241 (secure 0 0 0) 0x7fd95c0630f0 con 0x7fd9700b5c70 2024-09-17T03:30:02.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:02.344 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"btime":"2024-09-17T03:28:53:876203+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:28:45.214743+0000","modified":"2024-09-17T03:28:48.217828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246303":{"gid":246303,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6836/3671761014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3671761014},{"type":"v1","addr":"172.21.15.46:6836","nonce":3671761014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246303,"qdb_cluster":[246303]},"id":5}]} 2024-09-17T03:30:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd950074400 msgr2=0x7fd950076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd950074400 0x7fd950076910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd9680af000 tx=0x7fd96801b3a0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 msgr2=0x7fd970158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 0x7fd970158080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c390 tx=0x7fd95c00c850 comp rx=0 tx=0).stop 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 shutdown_connections 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd950074400 0x7fd950076910 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd9701527b0 0x7fd9701fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9700b5c70 0x7fd970158080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 --2- 172.21.15.46:0/2599814780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9700b4de0 0x7fd970141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 >> 172.21.15.46:0/2599814780 conn(0x7fd9700b2040 msgr2=0x7fd9700b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 shutdown_connections 2024-09-17T03:30:02.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.345+0000 7fd975400640 1 -- 172.21.15.46:0/2599814780 wait complete. 2024-09-17T03:30:02.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 -- 172.21.15.46:0/3758871493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e40b5c70 msgr2=0x7f01e415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 --2- 172.21.15.46:0/3758871493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e40b5c70 0x7f01e415bb60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009870 tx=0x7f01d401c7a0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 -- 172.21.15.46:0/3758871493 shutdown_connections 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 --2- 172.21.15.46:0/3758871493 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e40b5c70 0x7f01e415bb60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 -- 172.21.15.46:0/3758871493 >> 172.21.15.46:0/3758871493 conn(0x7f01e40b2040 msgr2=0x7f01e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 -- 172.21.15.46:0/3758871493 shutdown_connections 2024-09-17T03:30:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.525+0000 7f01eae00640 1 -- 172.21.15.46:0/3758871493 wait complete. 2024-09-17T03:30:02.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 Processor -- start 2024-09-17T03:30:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 -- start start 2024-09-17T03:30:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01e40b4de0 0x7f01e41f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 0x7f01e4146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e41527b0 0x7f01e41f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4169d00 con 0x7f01e40b5c70 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4169b80 con 0x7f01e40b4de0 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f01e4169e80 con 0x7f01e41527b0 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 0x7f01e4146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 0x7f01e4146880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46674/0 (socket says 172.21.15.46:46674) 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 -- 172.21.15.46:0/2628328325 learned_addr learned my addr 172.21.15.46:0/2628328325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01e3e00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01e40b4de0 0x7f01e41f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01e8e00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e41527b0 0x7f01e41f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 -- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e41527b0 msgr2=0x7f01e41f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e41527b0 0x7f01e41f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 -- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01e40b4de0 msgr2=0x7f01e41f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01e40b4de0 0x7f01e41f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 -- 172.21.15.46:0/2628328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e4146dc0 con 0x7f01e40b5c70 2024-09-17T03:30:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01e3e00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01e40b4de0 0x7f01e41f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01dbe00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 0x7f01e4146880 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c390 tx=0x7f01cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc011790 con 0x7f01e40b5c70 2024-09-17T03:30:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01cc017070 con 0x7f01e40b5c70 2024-09-17T03:30:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e40cdf50 con 0x7f01e40b5c70 2024-09-17T03:30:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.529+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc011ce0 con 0x7f01e40b5c70 2024-09-17T03:30:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e40ce510 con 0x7f01e40b5c70 2024-09-17T03:30:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e415bb60 con 0x7f01e40b5c70 2024-09-17T03:30:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f01cc01c460 con 0x7f01e40b5c70 2024-09-17T03:30:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01e0c00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01c0074430 0x7f01c0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7f01cc092de0 con 0x7f01e40b5c70 2024-09-17T03:30:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01e3e00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01c0074430 0x7f01c0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.533+0000 7f01e3e00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01c0074430 0x7f01c0076940 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f01d40af000 tx=0x7f01d401afe0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.537+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f01e4146dc0 con 0x7f01e40b5c70 2024-09-17T03:30:02.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.749+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01e40b2560 con 0x7f01e40b5c70 2024-09-17T03:30:02.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.749+0000 7f01e0c00640 1 -- 172.21.15.46:0/2628328325 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5241 (secure 0 0 0) 0x7f01cc05ef20 con 0x7f01e40b5c70 2024-09-17T03:30:02.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:02.751 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"btime":"2024-09-17T03:28:53:876203+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:28:45.214743+0000","modified":"2024-09-17T03:28:48.217828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246303":{"gid":246303,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6836/3671761014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3671761014},{"type":"v1","addr":"172.21.15.46:6836","nonce":3671761014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246303,"qdb_cluster":[246303]},"id":5}]} 2024-09-17T03:30:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-09-17T03:30:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01c0074430 msgr2=0x7f01c0076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01c0074430 0x7f01c0076940 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f01d40af000 tx=0x7f01d401afe0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 msgr2=0x7f01e4146880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 0x7f01e4146880 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c390 tx=0x7f01cc00c850 comp rx=0 tx=0).stop 2024-09-17T03:30:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 shutdown_connections 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01c0074430 0x7f01c0076940 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01e41527b0 0x7f01e41f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e40b5c70 0x7f01e4146880 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 --2- 172.21.15.46:0/2628328325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01e40b4de0 0x7f01e41f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 >> 172.21.15.46:0/2628328325 conn(0x7f01e40b2040 msgr2=0x7f01e4154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 shutdown_connections 2024-09-17T03:30:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.753+0000 7f01eae00640 1 -- 172.21.15.46:0/2628328325 wait complete. 2024-09-17T03:30:02.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:30:02.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 -- 172.21.15.46:0/3036098112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 msgr2=0x7ff30006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 --2- 172.21.15.46:0/3036098112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff30006d200 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009840 tx=0x7ff2ec01c790 comp rx=0 tx=0).stop 2024-09-17T03:30:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 -- 172.21.15.46:0/3036098112 shutdown_connections 2024-09-17T03:30:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 --2- 172.21.15.46:0/3036098112 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff30006fc90 0x7ff3000700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 --2- 172.21.15.46:0/3036098112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff30006d200 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 -- 172.21.15.46:0/3036098112 >> 172.21.15.46:0/3036098112 conn(0x7ff3000b35f0 msgr2=0x7ff3000b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:02.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 -- 172.21.15.46:0/3036098112 shutdown_connections 2024-09-17T03:30:02.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.945+0000 7ff306000640 1 -- 172.21.15.46:0/3036098112 wait complete. 2024-09-17T03:30:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.949+0000 7ff306000640 1 Processor -- start 2024-09-17T03:30:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.949+0000 7ff306000640 1 -- start start 2024-09-17T03:30:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff306000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff3001fab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff306000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30006fc90 0x7ff3002015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff306000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff300159130 0x7ff30015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff306000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff300166e40 con 0x7ff30006cdf0 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff306000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff300166cc0 con 0x7ff30006fc90 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff306000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff300166fc0 con 0x7ff300159130 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2ffe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff300159130 0x7ff30015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2ffe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff300159130 0x7ff30015af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:46298/0 (socket says 172.21.15.46:46298) 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2ffe00640 1 -- 172.21.15.46:0/3912357731 learned_addr learned my addr 172.21.15.46:0/3912357731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2ff400640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff3001fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30006fc90 0x7ff3002015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 -- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff300159130 msgr2=0x7ff30015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff300159130 0x7ff30015af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 -- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 msgr2=0x7ff3001fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff3001fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 -- 172.21.15.46:0/3912357731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff300200b50 con 0x7ff30006fc90 2024-09-17T03:30:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2ff400640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff3001fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.953+0000 7ff2fea00640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30006fc90 0x7ff3002015c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff3000b5440 tx=0x7ff2ec0a6ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.957+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0ac070 con 0x7ff30006fc90 2024-09-17T03:30:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.957+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2ec0043e0 con 0x7ff30006fc90 2024-09-17T03:30:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.957+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0052d0 con 0x7ff30006fc90 2024-09-17T03:30:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.957+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3000cf8f0 con 0x7ff30006fc90 2024-09-17T03:30:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.957+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001fdf90 con 0x7ff30006fc90 2024-09-17T03:30:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.957+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3000cf660 con 0x7ff30006fc90 2024-09-17T03:30:02.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.961+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7ff2ec0af070 con 0x7ff30006fc90 2024-09-17T03:30:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.961+0000 7ff2f7400640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2e0074450 0x7ff2e0076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.961+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7ff2ec12cc50 con 0x7ff30006fc90 2024-09-17T03:30:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.961+0000 7ff2ff400640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2e0074450 0x7ff2e0076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.965+0000 7ff2ff400640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2e0074450 0x7ff2e0076960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0007f30 tx=0x7ff2f0007ec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:02.965+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff2ec0b6070 con 0x7ff30006fc90 2024-09-17T03:30:03.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.293+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff30006d200 con 0x7ff30006fc90 2024-09-17T03:30:03.297 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:03.298 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:29:59.377916+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:28:42.973730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T03:28:44.166208+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:30:03.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.297+0000 7ff2f7400640 1 -- 172.21.15.46:0/3912357731 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16900 (secure 0 0 0) 0x7ff2ec016430 con 0x7ff30006fc90 2024-09-17T03:30:03.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2e0074450 msgr2=0x7ff2e0076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2e0074450 0x7ff2e0076960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0007f30 tx=0x7ff2f0007ec0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30006fc90 msgr2=0x7ff3002015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30006fc90 0x7ff3002015c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff3000b5440 tx=0x7ff2ec0a6ff0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 shutdown_connections 2024-09-17T03:30:03.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff2e0074450 0x7ff2e0076960 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0007f30 tx=0x7ff2f0007ec0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff300159130 0x7ff30015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30006fc90 0x7ff3002015c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 --2- 172.21.15.46:0/3912357731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30006cdf0 0x7ff3001fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.301+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 >> 172.21.15.46:0/3912357731 conn(0x7ff3000b35f0 msgr2=0x7ff300070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:03.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.305+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 shutdown_connections 2024-09-17T03:30:03.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.305+0000 7ff306000640 1 -- 172.21.15.46:0/3912357731 wait complete. 2024-09-17T03:30:03.330 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:30:03.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 -- 172.21.15.46:0/3096173607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 msgr2=0x7fa634074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 --2- 172.21.15.46:0/3096173607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa634074e30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009840 tx=0x7fa61c01c760 comp rx=0 tx=0).stop 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 -- 172.21.15.46:0/3096173607 shutdown_connections 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 --2- 172.21.15.46:0/3096173607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa634074e30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 --2- 172.21.15.46:0/3096173607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6340b5c70 0x7fa63415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 -- 172.21.15.46:0/3096173607 >> 172.21.15.46:0/3096173607 conn(0x7fa6340b2040 msgr2=0x7fa6340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 -- 172.21.15.46:0/3096173607 shutdown_connections 2024-09-17T03:30:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 -- 172.21.15.46:0/3096173607 wait complete. 2024-09-17T03:30:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 Processor -- start 2024-09-17T03:30:03.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.493+0000 7fa639c00640 1 -- start start 2024-09-17T03:30:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6340b4de0 0x7fa63414bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6340b5c70 0x7fa6341f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa6340cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa634168c00 con 0x7fa6341527b0 2024-09-17T03:30:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa634168a80 con 0x7fa6340b4de0 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa634168d80 con 0x7fa6340b5c70 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa6340cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa632a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6340b4de0 0x7fa63414bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa6340cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46704/0 (socket says 172.21.15.46:46704) 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 -- 172.21.15.46:0/1764650276 learned_addr learned my addr 172.21.15.46:0/1764650276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa62be00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6340b5c70 0x7fa6341f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 -- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6340b5c70 msgr2=0x7fa6341f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6340b5c70 0x7fa6341f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 -- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6340b4de0 msgr2=0x7fa63414bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6340b4de0 0x7fa63414bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 -- 172.21.15.46:0/1764650276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341f6b10 con 0x7fa6341527b0 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa632a00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6340b4de0 0x7fa63414bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa633400640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa6340cc950 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c760 tx=0x7fa62c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c003db0 con 0x7fa6341527b0 2024-09-17T03:30:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa62c00ce10 con 0x7fa6341527b0 2024-09-17T03:30:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa634156340 con 0x7fa6341527b0 2024-09-17T03:30:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0052d0 con 0x7fa6341527b0 2024-09-17T03:30:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341f6d90 con 0x7fa6341527b0 2024-09-17T03:30:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.497+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63415bb60 con 0x7fa6341527b0 2024-09-17T03:30:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.501+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fa62c005470 con 0x7fa6341527b0 2024-09-17T03:30:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.501+0000 7fa62aa00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa610074430 0x7fa610076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.501+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7fa62c092dc0 con 0x7fa6341527b0 2024-09-17T03:30:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.501+0000 7fa632a00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa610074430 0x7fa610076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.501+0000 7fa632a00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa610074430 0x7fa610076940 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa62400af10 tx=0x7fa62400a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:03.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.505+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6341f6b10 con 0x7fa6341527b0 2024-09-17T03:30:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa63415c1c0 con 0x7fa6341527b0 2024-09-17T03:30:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa62aa00640 1 -- 172.21.15.46:0/1764650276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5241 (secure 0 0 0) 0x7fa62c05ef00 con 0x7fa6341527b0 2024-09-17T03:30:03.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:03.713 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"btime":"2024-09-17T03:28:53:876203+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:28:45.214743+0000","modified":"2024-09-17T03:28:48.217828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246303":{"gid":246303,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6836/3671761014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3671761014},{"type":"v1","addr":"172.21.15.46:6836","nonce":3671761014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246303,"qdb_cluster":[246303]},"id":5}]} 2024-09-17T03:30:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-09-17T03:30:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa610074430 msgr2=0x7fa610076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa610074430 0x7fa610076940 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa62400af10 tx=0x7fa62400a250 comp rx=0 tx=0).stop 2024-09-17T03:30:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 msgr2=0x7fa6340cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa6340cc950 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c760 tx=0x7fa62c00cc20 comp rx=0 tx=0).stop 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 shutdown_connections 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa610074430 0x7fa610076940 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341527b0 0x7fa6340cc950 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa6340b5c70 0x7fa6341f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 --2- 172.21.15.46:0/1764650276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6340b4de0 0x7fa63414bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.713+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 >> 172.21.15.46:0/1764650276 conn(0x7fa6340b2040 msgr2=0x7fa6340b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.717+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 shutdown_connections 2024-09-17T03:30:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.717+0000 7fa639c00640 1 -- 172.21.15.46:0/1764650276 wait complete. 2024-09-17T03:30:03.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 -- 172.21.15.46:0/3965730618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1c005350 msgr2=0x7f8f1c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/3965730618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1c005350 0x7f8f1c00ba10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8f38074950 tx=0x7f8f3001d1e0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 -- 172.21.15.46:0/3965730618 shutdown_connections 2024-09-17T03:30:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/3965730618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1c005350 0x7f8f1c00ba10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 -- 172.21.15.46:0/3965730618 >> 172.21.15.46:0/3965730618 conn(0x7f8f380b2040 msgr2=0x7f8f380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 -- 172.21.15.46:0/3965730618 shutdown_connections 2024-09-17T03:30:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 -- 172.21.15.46:0/3965730618 wait complete. 2024-09-17T03:30:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 Processor -- start 2024-09-17T03:30:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.897+0000 7f8f3dc00640 1 -- start start 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f1c005350 0x7f8f3813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380af760 0x7f8f3813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f38140410 0x7f8f3813e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380751c0 con 0x7f8f380af760 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f38075040 con 0x7f8f38140410 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8f38075340 con 0x7f8f1c005350 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380af760 0x7f8f3813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f36a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f1c005350 0x7f8f3813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f36a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f1c005350 0x7f8f3813d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:46338/0 (socket says 172.21.15.46:46338) 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f36a00640 1 -- 172.21.15.46:0/564601556 learned_addr learned my addr 172.21.15.46:0/564601556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f37400640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f38140410 0x7f8f3813e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 -- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f1c005350 msgr2=0x7f8f3813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f1c005350 0x7f8f3813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 -- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f38140410 msgr2=0x7f8f3813e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f38140410 0x7f8f3813e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 -- 172.21.15.46:0/564601556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f38140820 con 0x7f8f380af760 2024-09-17T03:30:03.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2fe00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380af760 0x7f8f3813fed0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006fd0 tx=0x7f8f20011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20011e50 con 0x7f8f380af760 2024-09-17T03:30:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f20018020 con 0x7f8f380af760 2024-09-17T03:30:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20003600 con 0x7f8f380af760 2024-09-17T03:30:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f38158b30 con 0x7f8f380af760 2024-09-17T03:30:03.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3813e1c0 con 0x7f8f380af760 2024-09-17T03:30:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.901+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3806f0e0 con 0x7f8f380af760 2024-09-17T03:30:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.905+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f8f200037a0 con 0x7f8f380af760 2024-09-17T03:30:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.905+0000 7f8f2ea00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f14074400 0x7f8f14076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.905+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7f8f20096760 con 0x7f8f380af760 2024-09-17T03:30:03.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.905+0000 7f8f36a00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f14074400 0x7f8f14076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.905+0000 7f8f36a00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f14074400 0x7f8f14076910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005f70 tx=0x7f8f280091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:03.909+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8f20022020 con 0x7f8f380af760 2024-09-17T03:30:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.121+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f381588c0 con 0x7f8f380af760 2024-09-17T03:30:04.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.121+0000 7f8f2ea00640 1 -- 172.21.15.46:0/564601556 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5241 (secure 0 0 0) 0x7f8f38140820 con 0x7f8f380af760 2024-09-17T03:30:04.121 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:04.121 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"btime":"2024-09-17T03:28:53:876203+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:28:45.214743+0000","modified":"2024-09-17T03:28:48.217828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246303":{"gid":246303,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6836/3671761014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3671761014},{"type":"v1","addr":"172.21.15.46:6836","nonce":3671761014}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246303,"qdb_cluster":[246303]},"id":5}]} 2024-09-17T03:30:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f14074400 msgr2=0x7f8f14076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f14074400 0x7f8f14076910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005f70 tx=0x7f8f280091e0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380af760 msgr2=0x7f8f3813fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380af760 0x7f8f3813fed0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006fd0 tx=0x7f8f20011740 comp rx=0 tx=0).stop 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 shutdown_connections 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f14074400 0x7f8f14076910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f38140410 0x7f8f3813e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380af760 0x7f8f3813fed0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 --2- 172.21.15.46:0/564601556 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f1c005350 0x7f8f3813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 >> 172.21.15.46:0/564601556 conn(0x7f8f380b2040 msgr2=0x7f8f381516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 shutdown_connections 2024-09-17T03:30:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.125+0000 7f8f3dc00640 1 -- 172.21.15.46:0/564601556 wait complete. 2024-09-17T03:30:04.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:30:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 -- 172.21.15.46:0/2648915444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 msgr2=0x7f293015f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 --2- 172.21.15.46:0/2648915444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f293015f9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2928009840 tx=0x7f292801c790 comp rx=0 tx=0).stop 2024-09-17T03:30:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 -- 172.21.15.46:0/2648915444 shutdown_connections 2024-09-17T03:30:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 --2- 172.21.15.46:0/2648915444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f293015f9d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 -- 172.21.15.46:0/2648915444 >> 172.21.15.46:0/2648915444 conn(0x7f29300b41f0 msgr2=0x7f29300b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 -- 172.21.15.46:0/2648915444 shutdown_connections 2024-09-17T03:30:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.305+0000 7f2934a00640 1 -- 172.21.15.46:0/2648915444 wait complete. 2024-09-17T03:30:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 Processor -- start 2024-09-17T03:30:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 -- start start 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f293006dbc0 0x7f293009a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29300d1140 0x7f2930155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f2930155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f293016a0c0 con 0x7f293015f5c0 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2930169f40 con 0x7f29300d1140 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f293016a240 con 0x7f293006dbc0 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f2930155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f2930155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46736/0 (socket says 172.21.15.46:46736) 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 -- 172.21.15.46:0/2093724136 learned_addr learned my addr 172.21.15.46:0/2093724136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2926a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29300d1140 0x7f2930155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292f400640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f293006dbc0 0x7f293009a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 -- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f293006dbc0 msgr2=0x7f293009a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f293006dbc0 0x7f293009a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 -- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29300d1140 msgr2=0x7f2930155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29300d1140 0x7f2930155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 -- 172.21.15.46:0/2093724136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29300ce690 con 0x7f293015f5c0 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292f400640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f293006dbc0 0x7f293009a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f292fe00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f2930155b60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f29280b0040 tx=0x7f2928005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280b2040 con 0x7f293015f5c0 2024-09-17T03:30:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29280bb070 con 0x7f293015f5c0 2024-09-17T03:30:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928003600 con 0x7f293015f5c0 2024-09-17T03:30:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2930166520 con 0x7f293015f5c0 2024-09-17T03:30:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.309+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293009ae60 con 0x7f293015f5c0 2024-09-17T03:30:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.313+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293015f9d0 con 0x7f293015f5c0 2024-09-17T03:30:04.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.313+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f29280037a0 con 0x7f293015f5c0 2024-09-17T03:30:04.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.313+0000 7f2927e00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f290c074430 0x7f290c076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.313+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7f292812d140 con 0x7f293015f5c0 2024-09-17T03:30:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.313+0000 7f292f400640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f290c074430 0x7f290c076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.313+0000 7f292f400640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f290c074430 0x7f290c076940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2920009fc0 tx=0x7f2920009550 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:04.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.317+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2928016410 con 0x7f293015f5c0 2024-09-17T03:30:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.505+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f293006d270 con 0x7f293015f5c0 2024-09-17T03:30:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.505+0000 7f2927e00640 1 -- 172.21.15.46:0/2093724136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16900 (secure 0 0 0) 0x7f29280f9bb0 con 0x7f293015f5c0 2024-09-17T03:30:04.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:04.506 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:29:59.377916+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:28:42.973730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T03:28:44.166208+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:30:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f290c074430 msgr2=0x7f290c076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f290c074430 0x7f290c076940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2920009fc0 tx=0x7f2920009550 comp rx=0 tx=0).stop 2024-09-17T03:30:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 msgr2=0x7f2930155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f2930155b60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f29280b0040 tx=0x7f2928005760 comp rx=0 tx=0).stop 2024-09-17T03:30:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 shutdown_connections 2024-09-17T03:30:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f290c074430 0x7f290c076940 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293015f5c0 0x7f2930155b60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29300d1140 0x7f2930155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 --2- 172.21.15.46:0/2093724136 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f293006dbc0 0x7f293009a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 >> 172.21.15.46:0/2093724136 conn(0x7f29300b41f0 msgr2=0x7f29300d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 shutdown_connections 2024-09-17T03:30:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.509+0000 7f2934a00640 1 -- 172.21.15.46:0/2093724136 wait complete. 2024-09-17T03:30:04.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 -- 172.21.15.46:0/762272122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415b2d0 msgr2=0x7ff16415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 --2- 172.21.15.46:0/762272122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415b2d0 0x7ff16415b6e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009900 tx=0x7ff14c01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 -- 172.21.15.46:0/762272122 shutdown_connections 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 --2- 172.21.15.46:0/762272122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415b2d0 0x7ff16415b6e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 --2- 172.21.15.46:0/762272122 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff16415e8c0 0x7ff16415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 -- 172.21.15.46:0/762272122 >> 172.21.15.46:0/762272122 conn(0x7ff1640b2040 msgr2=0x7ff1640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 -- 172.21.15.46:0/762272122 shutdown_connections 2024-09-17T03:30:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 -- 172.21.15.46:0/762272122 wait complete. 2024-09-17T03:30:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 Processor -- start 2024-09-17T03:30:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.689+0000 7ff16b200640 1 -- start start 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff16415b2d0 0x7ff16413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16415bc20 0x7ff16413e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 0x7ff16413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640b54e0 con 0x7ff16415e8c0 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640b5360 con 0x7ff16415bc20 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff16415d990 con 0x7ff16415b2d0 2024-09-17T03:30:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 0x7ff16413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 0x7ff16413e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46762/0 (socket says 172.21.15.46:46762) 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 -- 172.21.15.46:0/1135412285 learned_addr learned my addr 172.21.15.46:0/1135412285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff163e00640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff16415b2d0 0x7ff16413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff15be00640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16415bc20 0x7ff16413e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 -- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff16415b2d0 msgr2=0x7ff16413ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff16415b2d0 0x7ff16413ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 -- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16415bc20 msgr2=0x7ff16413e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16415bc20 0x7ff16413e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 -- 172.21.15.46:0/1135412285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16413d390 con 0x7ff16415e8c0 2024-09-17T03:30:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff169200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 0x7ff16413e940 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00c6b0 tx=0x7ff15c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c003b80 con 0x7ff16415e8c0 2024-09-17T03:30:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff15c003d20 con 0x7ff16415e8c0 2024-09-17T03:30:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff164164db0 con 0x7ff16415e8c0 2024-09-17T03:30:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c005330 con 0x7ff16415e8c0 2024-09-17T03:30:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641647f0 con 0x7ff16415e8c0 2024-09-17T03:30:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.693+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16415b6e0 con 0x7ff16415e8c0 2024-09-17T03:30:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.697+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7ff15c0054d0 con 0x7ff16415e8c0 2024-09-17T03:30:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.697+0000 7ff160c00640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff140074400 0x7ff140076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.697+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5823+0+0 (secure 0 0 0) 0x7ff15c093000 con 0x7ff16415e8c0 2024-09-17T03:30:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.697+0000 7ff163e00640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff140074400 0x7ff140076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.697+0000 7ff163e00640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff140074400 0x7ff140076910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff154005fd0 tx=0x7ff154005e40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.701+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff15c05f700 con 0x7ff16415e8c0 2024-09-17T03:30:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.913+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff16415a3f0 con 0x7ff16415e8c0 2024-09-17T03:30:04.914 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:30:04.913+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:30:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.917+0000 7ff160c00640 1 -- 172.21.15.46:0/1135412285 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7ff15c05f000 con 0x7ff16415e8c0 2024-09-17T03:30:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.917+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff140074400 msgr2=0x7ff140076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.917+0000 7ff16b200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff140074400 0x7ff140076910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff154005fd0 tx=0x7ff154005e40 comp rx=0 tx=0).stop 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.917+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 msgr2=0x7ff16413e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.917+0000 7ff16b200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 0x7ff16413e940 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00c6b0 tx=0x7ff15c00cb70 comp rx=0 tx=0).stop 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 shutdown_connections 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff140074400 0x7ff140076910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16415e8c0 0x7ff16413e940 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff16415bc20 0x7ff16413e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 --2- 172.21.15.46:0/1135412285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff16415b2d0 0x7ff16413ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 >> 172.21.15.46:0/1135412285 conn(0x7ff1640b2040 msgr2=0x7ff164073dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 shutdown_connections 2024-09-17T03:30:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:04.921+0000 7ff16b200640 1 -- 172.21.15.46:0/1135412285 wait complete. 2024-09-17T03:30:04.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:30:05.180 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:30:05.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 -- 172.21.15.46:0/2724109788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc073cf0 msgr2=0x7f3ccc074100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:05.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/2724109788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc073cf0 0x7f3ccc074100 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc071cb0 tx=0x7f3cc401d4d0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 -- 172.21.15.46:0/2724109788 shutdown_connections 2024-09-17T03:30:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/2724109788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc073cf0 0x7f3ccc074100 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/2724109788 >> v2:172.21.15.99:3300/0 conn(0x7f3ccc16b7c0 0x7f3ccc16f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 -- 172.21.15.46:0/2724109788 >> 172.21.15.46:0/2724109788 conn(0x7f3ccc06e1d0 msgr2=0x7f3ccc06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 -- 172.21.15.46:0/2724109788 shutdown_connections 2024-09-17T03:30:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 -- 172.21.15.46:0/2724109788 wait complete. 2024-09-17T03:30:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.185+0000 7f3cd0e00640 1 Processor -- start 2024-09-17T03:30:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 -- start start 2024-09-17T03:30:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ccc073cf0 0x7f3ccc1627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ccc0ccca0 0x7f3ccc164050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 0x7f3ccc16a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc179040 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc178ec0 con 0x7f3ccc073cf0 2024-09-17T03:30:05.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3ccc1791c0 con 0x7f3ccc0ccca0 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 0x7f3ccc16a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 0x7f3ccc16a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46808/0 (socket says 172.21.15.46:46808) 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 -- 172.21.15.46:0/1074451443 learned_addr learned my addr 172.21.15.46:0/1074451443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cc9e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ccc073cf0 0x7f3ccc1627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cc1400640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ccc0ccca0 0x7f3ccc164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 -- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ccc0ccca0 msgr2=0x7f3ccc164050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ccc0ccca0 0x7f3ccc164050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 -- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ccc073cf0 msgr2=0x7f3ccc1627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ccc073cf0 0x7f3ccc1627c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 -- 172.21.15.46:0/1074451443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc164590 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cca800640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 0x7f3ccc16a5b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3cc401cf40 tx=0x7f3cc4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4003b80 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cc4003e10 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40abaf0 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc164b60 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.189+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc16beb0 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.193+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ccc1648c0 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.193+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f3cc40abc90 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.193+0000 7f3cc2a00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ca8074400 0x7f3ca8076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.193+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5881+0+0 (secure 0 0 0) 0x7f3cc412e6f0 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.193+0000 7f3cc9e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ca8074400 0x7f3ca8076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.193+0000 7f3cc9e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ca8074400 0x7f3ca8076910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009b40 tx=0x7f3cbc009630 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:05.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.197+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ccc1648c0 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.409+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3ccc074100 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.921+0000 7f3cc2a00640 1 -- 172.21.15.46:0/1074451443 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7f3cc4017210 con 0x7f3ccc16b7c0 2024-09-17T03:30:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ca8074400 msgr2=0x7f3ca8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ca8074400 0x7f3ca8076910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009b40 tx=0x7f3cbc009630 comp rx=0 tx=0).stop 2024-09-17T03:30:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 msgr2=0x7f3ccc16a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 0x7f3ccc16a5b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3cc401cf40 tx=0x7f3cc4005c90 comp rx=0 tx=0).stop 2024-09-17T03:30:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 shutdown_connections 2024-09-17T03:30:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3ca8074400 0x7f3ca8076910 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc16b7c0 0x7f3ccc16a5b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3ccc0ccca0 0x7f3ccc164050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 --2- 172.21.15.46:0/1074451443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ccc073cf0 0x7f3ccc1627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 >> 172.21.15.46:0/1074451443 conn(0x7f3ccc06e1d0 msgr2=0x7f3ccc0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 shutdown_connections 2024-09-17T03:30:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:05.925+0000 7f3cd0e00640 1 -- 172.21.15.46:0/1074451443 wait complete. 2024-09-17T03:30:05.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:30:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 -- 172.21.15.46:0/2812325299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a80cd7a0 msgr2=0x7f50a80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 --2- 172.21.15.46:0/2812325299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a80cd7a0 0x7f50a80cdbb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f50900093d0 tx=0x7f509001eaf0 comp rx=0 tx=0).stop 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 -- 172.21.15.46:0/2812325299 shutdown_connections 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 --2- 172.21.15.46:0/2812325299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a80cd7a0 0x7f50a80cdbb0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 --2- 172.21.15.46:0/2812325299 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50a815aa50 0x7f50a80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 -- 172.21.15.46:0/2812325299 >> 172.21.15.46:0/2812325299 conn(0x7f50a80b2040 msgr2=0x7f50a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 -- 172.21.15.46:0/2812325299 shutdown_connections 2024-09-17T03:30:06.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 -- 172.21.15.46:0/2812325299 wait complete. 2024-09-17T03:30:06.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 Processor -- start 2024-09-17T03:30:06.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.125+0000 7f50aee00640 1 -- start start 2024-09-17T03:30:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50a80cd7a0 0x7f50a80aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 0x7f50a8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50a815e4e0 0x7f50a814da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81683e0 con 0x7f50a815aa50 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8168260 con 0x7f50a815e4e0 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f50a8168560 con 0x7f50a80cd7a0 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 0x7f50a8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 0x7f50a8158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46820/0 (socket says 172.21.15.46:46820) 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 -- 172.21.15.46:0/2388184219 learned_addr learned my addr 172.21.15.46:0/2388184219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50a7e00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50a80cd7a0 0x7f50a80aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50ace00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50a815e4e0 0x7f50a814da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 -- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50a80cd7a0 msgr2=0x7f50a80aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50a80cd7a0 0x7f50a80aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 -- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50a815e4e0 msgr2=0x7f50a814da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50a815e4e0 0x7f50a814da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 -- 172.21.15.46:0/2388184219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a8156d70 con 0x7f50a815aa50 2024-09-17T03:30:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f509fe00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 0x7f50a8158050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f509001c720 tx=0x7f5090005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50900be070 con 0x7f50a815aa50 2024-09-17T03:30:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50900038f0 con 0x7f50a815aa50 2024-09-17T03:30:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50900b4040 con 0x7f50a815aa50 2024-09-17T03:30:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a8157050 con 0x7f50a815aa50 2024-09-17T03:30:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a8153020 con 0x7f50a815aa50 2024-09-17T03:30:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.129+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a8152b30 con 0x7f50a815aa50 2024-09-17T03:30:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.133+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f50900c1070 con 0x7f50a815aa50 2024-09-17T03:30:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.133+0000 7f50a4c00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5084074400 0x7f5084076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.133+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5881+0+0 (secure 0 0 0) 0x7f509012e910 con 0x7f50a815aa50 2024-09-17T03:30:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.133+0000 7f50a7e00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5084074400 0x7f5084076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.133+0000 7f50a7e00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5084074400 0x7f5084076910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f509800bc50 tx=0x7f509800b3e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:06.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.137+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f50900186e0 con 0x7f50a815aa50 2024-09-17T03:30:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:06.313+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50a80cdbb0 con 0x7f50a815aa50 2024-09-17T03:30:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.017+0000 7f50a4c00640 1 -- 172.21.15.46:0/2388184219 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) ==== 181+0+0 (secure 0 0 0) 0x7f50900fadc0 con 0x7f50a815aa50 2024-09-17T03:30:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:30:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.021+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5084074400 msgr2=0x7f5084076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.021+0000 7f50aee00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5084074400 0x7f5084076910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f509800bc50 tx=0x7f509800b3e0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.021+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 msgr2=0x7f50a8158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.021+0000 7f50aee00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 0x7f50a8158050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f509001c720 tx=0x7f5090005790 comp rx=0 tx=0).stop 2024-09-17T03:30:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 shutdown_connections 2024-09-17T03:30:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5084074400 0x7f5084076910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50a815e4e0 0x7f50a814da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a815aa50 0x7f50a8158050 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 --2- 172.21.15.46:0/2388184219 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f50a80cd7a0 0x7f50a80aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 >> 172.21.15.46:0/2388184219 conn(0x7f50a80b2040 msgr2=0x7f50a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 shutdown_connections 2024-09-17T03:30:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.025+0000 7f50aee00640 1 -- 172.21.15.46:0/2388184219 wait complete. 2024-09-17T03:30:07.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:30:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 -- 172.21.15.46:0/1208066956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c155a00 msgr2=0x7f701c15f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 --2- 172.21.15.46:0/1208066956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c155a00 0x7f701c15f070 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f700c009900 tx=0x7f700c01c7e0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 -- 172.21.15.46:0/1208066956 shutdown_connections 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 --2- 172.21.15.46:0/1208066956 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c0b2300 0x7f701c0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 --2- 172.21.15.46:0/1208066956 >> v2:172.21.15.99:3300/0 conn(0x7f701c15f5b0 0x7f701c0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 --2- 172.21.15.46:0/1208066956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c155a00 0x7f701c15f070 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 -- 172.21.15.46:0/1208066956 >> 172.21.15.46:0/1208066956 conn(0x7f701c06cdf0 msgr2=0x7f701c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 -- 172.21.15.46:0/1208066956 shutdown_connections 2024-09-17T03:30:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.217+0000 7f701b200640 1 -- 172.21.15.46:0/1208066956 wait complete. 2024-09-17T03:30:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 Processor -- start 2024-09-17T03:30:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 -- start start 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 0x7f701c149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c155a00 0x7f701c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701c15f5b0 0x7f701c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f701c0726f0 con 0x7f701c0b2300 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f701c072570 con 0x7f701c15f5b0 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f701c072870 con 0x7f701c155a00 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 0x7f701c149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 0x7f701c149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46840/0 (socket says 172.21.15.46:46840) 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 -- 172.21.15.46:0/2025245702 learned_addr learned my addr 172.21.15.46:0/2025245702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:07.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f701a800640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701c15f5b0 0x7f701c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7013e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c155a00 0x7f701c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 -- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c155a00 msgr2=0x7f701c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c155a00 0x7f701c0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 -- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701c15f5b0 msgr2=0x7f701c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701c15f5b0 0x7f701c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7019e00640 1 -- 172.21.15.46:0/2025245702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c14a530 con 0x7f701c0b2300 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.221+0000 7f7013e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c155a00 0x7f701c0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:07.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7019e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 0x7f701c149ff0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f700c009850 tx=0x7f700c0a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0b2040 con 0x7f701c0b2300 2024-09-17T03:30:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f700c004340 con 0x7f701c0b2300 2024-09-17T03:30:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c146e20 con 0x7f701c0b2300 2024-09-17T03:30:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0044f0 con 0x7f701c0b2300 2024-09-17T03:30:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c148380 con 0x7f701c0b2300 2024-09-17T03:30:07.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c147340 con 0x7f701c0b2300 2024-09-17T03:30:07.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f700c005290 con 0x7f701c0b2300 2024-09-17T03:30:07.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7012000640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6ff8074400 0x7f6ff8076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5415+0+0 (secure 0 0 0) 0x7f700c12c920 con 0x7f701c0b2300 2024-09-17T03:30:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.225+0000 7f7013e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6ff8074400 0x7f6ff8076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:07.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.229+0000 7f7013e00640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6ff8074400 0x7f6ff8076910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f701c147df0 tx=0x7f7004005eb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.229+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f701c147340 con 0x7f701c0b2300 2024-09-17T03:30:07.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:07.409+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f701c072df0 con 0x7f701c0b2300 2024-09-17T03:30:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.285+0000 7f7012000640 1 -- 172.21.15.46:0/2025245702 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) ==== 169+0+0 (secure 0 0 0) 0x7f700c016490 con 0x7f701c0b2300 2024-09-17T03:30:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:30:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6ff8074400 msgr2=0x7f6ff8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6ff8074400 0x7f6ff8076910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f701c147df0 tx=0x7f7004005eb0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 msgr2=0x7f701c149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 0x7f701c149ff0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f700c009850 tx=0x7f700c0a5700 comp rx=0 tx=0).stop 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 shutdown_connections 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6ff8074400 0x7f6ff8076910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701c15f5b0 0x7f701c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f701c155a00 0x7f701c0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 --2- 172.21.15.46:0/2025245702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f701c0b2300 0x7f701c149ff0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 >> 172.21.15.46:0/2025245702 conn(0x7f701c06cdf0 msgr2=0x7f701c0cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.293+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 shutdown_connections 2024-09-17T03:30:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.297+0000 7f701b200640 1 -- 172.21.15.46:0/2025245702 wait complete. 2024-09-17T03:30:08.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:30:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 -- 172.21.15.46:0/544915635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 msgr2=0x7f5bc815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 --2- 172.21.15.46:0/544915635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 0x7f5bc815d480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009840 tx=0x7f5bb801c790 comp rx=0 tx=0).stop 2024-09-17T03:30:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 -- 172.21.15.46:0/544915635 shutdown_connections 2024-09-17T03:30:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 --2- 172.21.15.46:0/544915635 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc8160060 0x7f5bc8074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 --2- 172.21.15.46:0/544915635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 0x7f5bc815d480 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 -- 172.21.15.46:0/544915635 >> 172.21.15.46:0/544915635 conn(0x7f5bc80b4560 msgr2=0x7f5bc80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 -- 172.21.15.46:0/544915635 shutdown_connections 2024-09-17T03:30:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 -- 172.21.15.46:0/544915635 wait complete. 2024-09-17T03:30:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 Processor -- start 2024-09-17T03:30:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.493+0000 7f5bccc00640 1 -- start start 2024-09-17T03:30:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 0x7f5bc814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 0x7f5bc8142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc8160470 0x7f5bc80d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc816a440 con 0x7f5bc8160060 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc816a2c0 con 0x7f5bc81536f0 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc816a5c0 con 0x7f5bc8160470 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 0x7f5bc8142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 0x7f5bc8142d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46848/0 (socket says 172.21.15.46:46848) 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 -- 172.21.15.46:0/1896971606 learned_addr learned my addr 172.21.15.46:0/1896971606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bc7400640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 0x7f5bc814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc8160470 0x7f5bc80d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 -- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc8160470 msgr2=0x7f5bc80d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc8160470 0x7f5bc80d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 -- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 msgr2=0x7f5bc814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 0x7f5bc814b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 -- 172.21.15.46:0/1896971606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81fa7b0 con 0x7f5bc8160060 2024-09-17T03:30:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbea00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 0x7f5bc8142d00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e9a0 tx=0x7f5bb000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000cd80 con 0x7f5bc8160060 2024-09-17T03:30:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb0004020 con 0x7f5bc8160060 2024-09-17T03:30:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81faa90 con 0x7f5bc8160060 2024-09-17T03:30:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004450 con 0x7f5bc8160060 2024-09-17T03:30:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81fb050 con 0x7f5bc8160060 2024-09-17T03:30:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.497+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc815d480 con 0x7f5bc8160060 2024-09-17T03:30:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.501+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f5bb0003600 con 0x7f5bc8160060 2024-09-17T03:30:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.501+0000 7f5bbfe00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba4074430 0x7f5ba4076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.501+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f5bb00937d0 con 0x7f5bc8160060 2024-09-17T03:30:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.501+0000 7f5bc7400640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba4074430 0x7f5ba4076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.501+0000 7f5bc7400640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba4074430 0x7f5ba4076940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ae000 tx=0x7f5bb801afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:08.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.505+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5bb0060250 con 0x7f5bc8160060 2024-09-17T03:30:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.681+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f5bc80720a0 con 0x7f5bc8160060 2024-09-17T03:30:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.685+0000 7f5bbfe00640 1 -- 172.21.15.46:0/1896971606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7f5bb005fb50 con 0x7f5bc8160060 2024-09-17T03:30:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba4074430 msgr2=0x7f5ba4076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba4074430 0x7f5ba4076940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ae000 tx=0x7f5bb801afd0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 msgr2=0x7f5bc8142d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 0x7f5bc8142d00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000e9a0 tx=0x7f5bb000ee60 comp rx=0 tx=0).stop 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 shutdown_connections 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5ba4074430 0x7f5ba4076940 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5bc8160470 0x7f5bc80d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8160060 0x7f5bc8142d00 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1896971606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5bc81536f0 0x7f5bc814b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 >> 172.21.15.46:0/1896971606 conn(0x7f5bc80b4560 msgr2=0x7f5bc80715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 shutdown_connections 2024-09-17T03:30:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.689+0000 7f5bccc00640 1 -- 172.21.15.46:0/1896971606 wait complete. 2024-09-17T03:30:08.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-17T03:30:08.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 -- 172.21.15.46:0/170657284 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc0cd7a0 msgr2=0x7f8cdc0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/170657284 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc0cd7a0 0x7f8cdc0cdbb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009900 tx=0x7f8cc801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 -- 172.21.15.46:0/170657284 shutdown_connections 2024-09-17T03:30:08.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/170657284 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc0cd7a0 0x7f8cdc0cdbb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/170657284 >> v2:172.21.15.46:3300/0 conn(0x7f8cdc15aa50 0x7f8cdc0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 -- 172.21.15.46:0/170657284 >> 172.21.15.46:0/170657284 conn(0x7f8cdc0b2040 msgr2=0x7f8cdc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:08.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 -- 172.21.15.46:0/170657284 shutdown_connections 2024-09-17T03:30:08.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.881+0000 7f8ce3a00640 1 -- 172.21.15.46:0/170657284 wait complete. 2024-09-17T03:30:08.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 Processor -- start 2024-09-17T03:30:08.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 -- start start 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 0x7f8cdc14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cdc15aa50 0x7f8cdc159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc15e4e0 0x7f8cdc139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1683e0 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc168260 con 0x7f8cdc15aa50 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc168560 con 0x7f8cdc15e4e0 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 0x7f8cdc14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 0x7f8cdc14bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46866/0 (socket says 172.21.15.46:46866) 2024-09-17T03:30:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 -- 172.21.15.46:0/3185308336 learned_addr learned my addr 172.21.15.46:0/3185308336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc15e4e0 0x7f8cdc139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8cdbe00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cdc15aa50 0x7f8cdc159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 -- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc15e4e0 msgr2=0x7f8cdc139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc15e4e0 0x7f8cdc139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 -- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cdc15aa50 msgr2=0x7f8cdc159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cdc15aa50 0x7f8cdc159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 -- 172.21.15.46:0/3185308336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1588c0 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce1000640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 0x7f8cdc14bda0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00ee40 tx=0x7f8ccc012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc01c030 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ccc012e70 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc158ba0 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc004d90 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.885+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc159160 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.889+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc0cdbb0 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.889+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f8ccc01f070 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.889+0000 7f8cd8c00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8cbc074430 0x7f8cbc076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:08.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.889+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f8ccc097740 con 0x7f8cdc0cd7a0 2024-09-17T03:30:08.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.889+0000 7f8cdbe00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8cbc074430 0x7f8cbc076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:08.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.893+0000 7f8cdbe00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8cbc074430 0x7f8cbc076940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80af000 tx=0x7f8cc801b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:08.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:08.893+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ccc063880 con 0x7f8cdc0cd7a0 2024-09-17T03:30:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.133+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f8cdc0b16a0 con 0x7f8cdc0cd7a0 2024-09-17T03:30:09.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.285+0000 7f8cd8c00640 1 -- 172.21.15.46:0/3185308336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v419) ==== 167+0+0 (secure 0 0 0) 0x7f8ccc0682b0 con 0x7f8cdc0cd7a0 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8cbc074430 msgr2=0x7f8cbc076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8cbc074430 0x7f8cbc076940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80af000 tx=0x7f8cc801b120 comp rx=0 tx=0).stop 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 msgr2=0x7f8cdc14bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 0x7f8cdc14bda0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00ee40 tx=0x7f8ccc012490 comp rx=0 tx=0).stop 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 shutdown_connections 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8cbc074430 0x7f8cbc076940 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8cdc15e4e0 0x7f8cdc139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cdc15aa50 0x7f8cdc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 --2- 172.21.15.46:0/3185308336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cdc0cd7a0 0x7f8cdc14bda0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 >> 172.21.15.46:0/3185308336 conn(0x7f8cdc0b2040 msgr2=0x7f8cdc150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 shutdown_connections 2024-09-17T03:30:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.289+0000 7f8ce3a00640 1 -- 172.21.15.46:0/3185308336 wait complete. 2024-09-17T03:30:09.307 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-17T03:30:09.308 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-17T03:30:09.308 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:30:09.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:30:09.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 -- 172.21.15.46:0/937622084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015c5d0 msgr2=0x7f1500073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 --2- 172.21.15.46:0/937622084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015c5d0 0x7f1500073dc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009900 tx=0x7f14f401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 -- 172.21.15.46:0/937622084 shutdown_connections 2024-09-17T03:30:09.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 --2- 172.21.15.46:0/937622084 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1500074300 0x7f150015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 --2- 172.21.15.46:0/937622084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015c5d0 0x7f1500073dc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 -- 172.21.15.46:0/937622084 >> 172.21.15.46:0/937622084 conn(0x7f15000b2c40 msgr2=0x7f15000b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:09.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 -- 172.21.15.46:0/937622084 shutdown_connections 2024-09-17T03:30:09.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 -- 172.21.15.46:0/937622084 wait complete. 2024-09-17T03:30:09.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 Processor -- start 2024-09-17T03:30:09.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.469+0000 7f1508000640 1 -- start start 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1500074300 0x7f15001fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f150015c5d0 0x7f1500153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 0x7f150014d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f150006f980 con 0x7f150015f6b0 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15000b3d50 con 0x7f1500074300 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f150006fb00 con 0x7f150015c5d0 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 0x7f150014d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 0x7f150014d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46890/0 (socket says 172.21.15.46:46890) 2024-09-17T03:30:09.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 -- 172.21.15.46:0/787670943 learned_addr learned my addr 172.21.15.46:0/787670943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1504c00640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f150015c5d0 0x7f1500153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1505600640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1500074300 0x7f15001fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 -- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f150015c5d0 msgr2=0x7f1500153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f150015c5d0 0x7f1500153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 -- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1500074300 msgr2=0x7f15001fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1500074300 0x7f15001fc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 -- 172.21.15.46:0/787670943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15001fcdb0 con 0x7f150015f6b0 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1504c00640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f150015c5d0 0x7f1500153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1506000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 0x7f150014d7f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c740 tx=0x7f14f800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:09.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8003b80 con 0x7f150015f6b0 2024-09-17T03:30:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f800ce50 con 0x7f150015f6b0 2024-09-17T03:30:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f800a850 con 0x7f150015f6b0 2024-09-17T03:30:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15001fd090 con 0x7f150015f6b0 2024-09-17T03:30:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15001f8fd0 con 0x7f150015f6b0 2024-09-17T03:30:09.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.473+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15001fcb20 con 0x7f150015f6b0 2024-09-17T03:30:09.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.477+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f14f800a9f0 con 0x7f150015f6b0 2024-09-17T03:30:09.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.477+0000 7f14fd600640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14e0074430 0x7f14e0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.477+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f14f8092f00 con 0x7f150015f6b0 2024-09-17T03:30:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.477+0000 7f1505600640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14e0074430 0x7f14e0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.477+0000 7f1505600640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14e0074430 0x7f14e0076940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f14f40af000 tx=0x7f14f401b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:09.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.481+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14f805f600 con 0x7f150015f6b0 2024-09-17T03:30:09.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.657+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f150015f170 con 0x7f150015f6b0 2024-09-17T03:30:09.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.657+0000 7f14fd600640 1 -- 172.21.15.46:0/787670943 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+12997 (secure 0 0 0) 0x7f14f805ef00 con 0x7f150015f6b0 2024-09-17T03:30:09.658 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:09.658 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:30:08.280235+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6836/3671761014":"2024-09-18T03:30:04.916743+0000","172.21.15.46:6834/3671761014":"2024-09-18T03:30:04.916743+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:30:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14e0074430 msgr2=0x7f14e0076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14e0074430 0x7f14e0076940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f14f40af000 tx=0x7f14f401b0f0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 msgr2=0x7f150014d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 0x7f150014d7f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c740 tx=0x7f14f800cc00 comp rx=0 tx=0).stop 2024-09-17T03:30:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 shutdown_connections 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14e0074430 0x7f14e0076940 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f150015f6b0 0x7f150014d7f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f150015c5d0 0x7f1500153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 --2- 172.21.15.46:0/787670943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1500074300 0x7f15001fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 >> 172.21.15.46:0/787670943 conn(0x7f15000b2c40 msgr2=0x7f150014f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 shutdown_connections 2024-09-17T03:30:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.661+0000 7f1508000640 1 -- 172.21.15.46:0/787670943 wait complete. 2024-09-17T03:30:09.681 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:30:09.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:30:09.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 -- 172.21.15.46:0/1874044783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0950005360 msgr2=0x7f095000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 --2- 172.21.15.46:0/1874044783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0950005360 0x7f095000b9d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0964009b40 tx=0x7f096401d290 comp rx=0 tx=0).stop 2024-09-17T03:30:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 -- 172.21.15.46:0/1874044783 shutdown_connections 2024-09-17T03:30:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 --2- 172.21.15.46:0/1874044783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0950005360 0x7f095000b9d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 -- 172.21.15.46:0/1874044783 >> 172.21.15.46:0/1874044783 conn(0x7f096c0708c0 msgr2=0x7f096c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 -- 172.21.15.46:0/1874044783 shutdown_connections 2024-09-17T03:30:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 -- 172.21.15.46:0/1874044783 wait complete. 2024-09-17T03:30:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 Processor -- start 2024-09-17T03:30:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.845+0000 7f0971800640 1 -- start start 2024-09-17T03:30:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0950005360 0x7f096c1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 0x7f096c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c0cb470 0x7f096c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f096c160ed0 con 0x7f096c0cb060 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f096c160d50 con 0x7f096c0cb470 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f096c161050 con 0x7f0950005360 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 0x7f096c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 0x7f096c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46900/0 (socket says 172.21.15.46:46900) 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 -- 172.21.15.46:0/3201695623 learned_addr learned my addr 172.21.15.46:0/3201695623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f096aa00640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0950005360 0x7f096c1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f096b400640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c0cb470 0x7f096c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 -- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0950005360 msgr2=0x7f096c1457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0950005360 0x7f096c1457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 -- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c0cb470 msgr2=0x7f096c0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c0cb470 0x7f096c0aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 -- 172.21.15.46:0/3201695623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c147ff0 con 0x7f096c0cb060 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f096b400640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c0cb470 0x7f096c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0962000640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 0x7f096c0af9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0954003f30 tx=0x7f0954001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954011640 con 0x7f096c0cb060 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0954016020 con 0x7f096c0cb060 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c1482d0 con 0x7f096c0cb060 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954011a60 con 0x7f096c0cb060 2024-09-17T03:30:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c148890 con 0x7f096c0cb060 2024-09-17T03:30:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.849+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c0cb8e0 con 0x7f096c0cb060 2024-09-17T03:30:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.853+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f0954011c00 con 0x7f096c0cb060 2024-09-17T03:30:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.853+0000 7f0963400640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0948074430 0x7f0948076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.853+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f0954093f50 con 0x7f096c0cb060 2024-09-17T03:30:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.853+0000 7f096aa00640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0948074430 0x7f0948076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.853+0000 7f096aa00640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0948074430 0x7f0948076940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f095c001ae0 tx=0x7f095c006b30 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:09.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:09.857+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0954020020 con 0x7f096c0cb060 2024-09-17T03:30:10.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.029+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f096c074580 con 0x7f096c0cb060 2024-09-17T03:30:10.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.029+0000 7f0963400640 1 -- 172.21.15.46:0/3201695623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f0954060a50 con 0x7f096c0cb060 2024-09-17T03:30:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:30:10.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0948074430 msgr2=0x7f0948076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:10.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0948074430 0x7f0948076940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f095c001ae0 tx=0x7f095c006b30 comp rx=0 tx=0).stop 2024-09-17T03:30:10.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 msgr2=0x7f096c0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:10.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 0x7f096c0af9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0954003f30 tx=0x7f0954001700 comp rx=0 tx=0).stop 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 shutdown_connections 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0948074430 0x7f0948076940 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f096c0cb470 0x7f096c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096c0cb060 0x7f096c0af9c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 --2- 172.21.15.46:0/3201695623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0950005360 0x7f096c1457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 >> 172.21.15.46:0/3201695623 conn(0x7f096c0708c0 msgr2=0x7f096c075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 shutdown_connections 2024-09-17T03:30:10.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.033+0000 7f0971800640 1 -- 172.21.15.46:0/3201695623 wait complete. 2024-09-17T03:30:10.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-17T03:30:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 -- 172.21.15.46:0/2778519607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec075340 msgr2=0x7f09ec15c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 --2- 172.21.15.46:0/2778519607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec075340 0x7f09ec15c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009900 tx=0x7f09d401c7f0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 -- 172.21.15.46:0/2778519607 shutdown_connections 2024-09-17T03:30:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 --2- 172.21.15.46:0/2778519607 >> v2:172.21.15.99:3300/0 conn(0x7f09ec0cda80 0x7f09ec0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 --2- 172.21.15.46:0/2778519607 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec075340 0x7f09ec15c920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 -- 172.21.15.46:0/2778519607 >> 172.21.15.46:0/2778519607 conn(0x7f09ec0b41f0 msgr2=0x7f09ec0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 -- 172.21.15.46:0/2778519607 shutdown_connections 2024-09-17T03:30:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.213+0000 7f09f2400640 1 -- 172.21.15.46:0/2778519607 wait complete. 2024-09-17T03:30:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 Processor -- start 2024-09-17T03:30:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 -- start start 2024-09-17T03:30:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 0x7f09ec157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 0x7f09ec15f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec06f350 0x7f09ec15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec169ee0 con 0x7f09ec06f350 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec169d60 con 0x7f09ec0cda80 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f2400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec16a060 con 0x7f09ec0746e0 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec06f350 0x7f09ec15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 0x7f09ec157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 0x7f09ec15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 0x7f09ec157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:46502/0 (socket says 172.21.15.46:46502) 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 -- 172.21.15.46:0/1976399420 learned_addr learned my addr 172.21.15.46:0/1976399420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 0x7f09ec15f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:53566/0 (socket says 172.21.15.46:53566) 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 -- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 msgr2=0x7f09ec15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 0x7f09ec15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 -- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec06f350 msgr2=0x7f09ec15fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec06f350 0x7f09ec15fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 -- 172.21.15.46:0/1976399420 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec1587f0 con 0x7f09ec0746e0 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09e3e00640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 0x7f09ec15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1a00640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec06f350 0x7f09ec15fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.217+0000 7f09f1000640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 0x7f09ec157a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09dc001ec0 tx=0x7f09dc00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc017070 con 0x7f09ec0746e0 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09dc003600 con 0x7f09ec0746e0 2024-09-17T03:30:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc003a50 con 0x7f09ec0746e0 2024-09-17T03:30:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec158330 con 0x7f09ec0746e0 2024-09-17T03:30:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec157fa0 con 0x7f09ec0746e0 2024-09-17T03:30:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec15c920 con 0x7f09ec0746e0 2024-09-17T03:30:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f09dc01a070 con 0x7f09ec0746e0 2024-09-17T03:30:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e9600640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09c8074400 0x7f09c8076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f09dc092440 con 0x7f09ec0746e0 2024-09-17T03:30:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.221+0000 7f09e3e00640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09c8074400 0x7f09c8076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.225+0000 7f09e3e00640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09c8074400 0x7f09c8076910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f09d40af000 tx=0x7f09d401b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.229+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f09dc05e500 con 0x7f09ec0746e0 2024-09-17T03:30:10.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:10.469+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f09ec0b4f30 con 0x7f09ec0746e0 2024-09-17T03:30:11.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.297+0000 7f09e9600640 1 -- 172.21.15.46:0/1976399420 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v421) ==== 177+0+0 (secure 0 0 0) 0x7f09dc062f30 con 0x7f09ec0746e0 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09c8074400 msgr2=0x7f09c8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09c8074400 0x7f09c8076910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f09d40af000 tx=0x7f09d401b090 comp rx=0 tx=0).stop 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 msgr2=0x7f09ec157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 0x7f09ec157a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f09dc001ec0 tx=0x7f09dc00c890 comp rx=0 tx=0).stop 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 shutdown_connections 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09c8074400 0x7f09c8076910 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ec06f350 0x7f09ec15fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec0cda80 0x7f09ec15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 --2- 172.21.15.46:0/1976399420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09ec0746e0 0x7f09ec157a60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 >> 172.21.15.46:0/1976399420 conn(0x7f09ec0b41f0 msgr2=0x7f09ec0cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 shutdown_connections 2024-09-17T03:30:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.301+0000 7f09f2400640 1 -- 172.21.15.46:0/1976399420 wait complete. 2024-09-17T03:30:11.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:30:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 -- 172.21.15.46:0/1016624492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 msgr2=0x7ff6a815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 --2- 172.21.15.46:0/1016624492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a815bb60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff698009900 tx=0x7ff69801c820 comp rx=0 tx=0).stop 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 -- 172.21.15.46:0/1016624492 shutdown_connections 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 --2- 172.21.15.46:0/1016624492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6a81527b0 0x7ff6a8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 --2- 172.21.15.46:0/1016624492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a815bb60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 -- 172.21.15.46:0/1016624492 >> 172.21.15.46:0/1016624492 conn(0x7ff6a80b2040 msgr2=0x7ff6a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 -- 172.21.15.46:0/1016624492 shutdown_connections 2024-09-17T03:30:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.489+0000 7ff6ae400640 1 -- 172.21.15.46:0/1016624492 wait complete. 2024-09-17T03:30:11.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 Processor -- start 2024-09-17T03:30:11.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 -- start start 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6a80b4de0 0x7ff6a81f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a8149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a81527b0 0x7ff6a814a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8168ca0 con 0x7ff6a80b5c70 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8168b20 con 0x7ff6a81527b0 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a8168e20 con 0x7ff6a80b4de0 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a8149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a8149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46918/0 (socket says 172.21.15.46:46918) 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 -- 172.21.15.46:0/2685530246 learned_addr learned my addr 172.21.15.46:0/2685530246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6a7400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6a80b4de0 0x7ff6a81f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6a7e00640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a81527b0 0x7ff6a814a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 -- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6a80b4de0 msgr2=0x7ff6a81f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6a80b4de0 0x7ff6a81f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 -- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a81527b0 msgr2=0x7ff6a814a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a81527b0 0x7ff6a814a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 -- 172.21.15.46:0/2685530246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a81fc570 con 0x7ff6a80b5c70 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69fe00640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a8149320 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e7c0 tx=0x7ff69000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000cc60 con 0x7ff6a80b5c70 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff69000ce00 con 0x7ff6a80b5c70 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690005280 con 0x7ff6a80b5c70 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a81fc850 con 0x7ff6a80b5c70 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.493+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a81f8510 con 0x7ff6a80b5c70 2024-09-17T03:30:11.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.497+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a815bb60 con 0x7ff6a80b5c70 2024-09-17T03:30:11.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.497+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7ff690005420 con 0x7ff6a80b5c70 2024-09-17T03:30:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.497+0000 7ff69f400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff684074430 0x7ff684076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.497+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7ff690093b50 con 0x7ff6a80b5c70 2024-09-17T03:30:11.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.497+0000 7ff6a7400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff684074430 0x7ff684076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.501+0000 7ff6a7400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff684074430 0x7ff684076940 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6980af000 tx=0x7ff69801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:11.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.501+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff690060690 con 0x7ff6a80b5c70 2024-09-17T03:30:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff6a8149d30 con 0x7ff6a80b5c70 2024-09-17T03:30:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff69f400640 1 -- 172.21.15.46:0/2685530246 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+2294 (secure 0 0 0) 0x7ff6a8149d30 con 0x7ff6a80b5c70 2024-09-17T03:30:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:30:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:30:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:30:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:30:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:30:11.682 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:30:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff684074430 msgr2=0x7ff684076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff684074430 0x7ff684076940 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6980af000 tx=0x7ff69801b060 comp rx=0 tx=0).stop 2024-09-17T03:30:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 msgr2=0x7ff6a8149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a8149320 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e7c0 tx=0x7ff69000ec80 comp rx=0 tx=0).stop 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 shutdown_connections 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff684074430 0x7ff684076940 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a81527b0 0x7ff6a814a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6a80b5c70 0x7ff6a8149320 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 --2- 172.21.15.46:0/2685530246 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6a80b4de0 0x7ff6a81f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 >> 172.21.15.46:0/2685530246 conn(0x7ff6a80b2040 msgr2=0x7ff6a80af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 shutdown_connections 2024-09-17T03:30:11.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.681+0000 7ff6ae400640 1 -- 172.21.15.46:0/2685530246 wait complete. 2024-09-17T03:30:11.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:30:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 -- 172.21.15.46:0/1831342418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef415d6c0 msgr2=0x7f0ef415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 --2- 172.21.15.46:0/1831342418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef415d6c0 0x7f0ef415dad0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40719d0 tx=0x7f0eec01ccf0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 -- 172.21.15.46:0/1831342418 shutdown_connections 2024-09-17T03:30:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 --2- 172.21.15.46:0/1831342418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef415d6c0 0x7f0ef415dad0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 --2- 172.21.15.46:0/1831342418 >> v2:172.21.15.99:3300/0 conn(0x7f0ef415efd0 0x7f0ef415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 -- 172.21.15.46:0/1831342418 >> 172.21.15.46:0/1831342418 conn(0x7f0ef40b2040 msgr2=0x7f0ef40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 -- 172.21.15.46:0/1831342418 shutdown_connections 2024-09-17T03:30:11.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.865+0000 7f0efc600640 1 -- 172.21.15.46:0/1831342418 wait complete. 2024-09-17T03:30:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 Processor -- start 2024-09-17T03:30:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 -- start start 2024-09-17T03:30:11.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 0x7f0ef41fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ef415d6c0 0x7f0ef41652f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ef41f6030 0x7f0ef41f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef40b4ec0 con 0x7f0ef40cf900 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef40b4d40 con 0x7f0ef415d6c0 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef40b5040 con 0x7f0ef41f6030 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 0x7f0ef41fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 0x7f0ef41fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46932/0 (socket says 172.21.15.46:46932) 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 -- 172.21.15.46:0/480707144 learned_addr learned my addr 172.21.15.46:0/480707144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9200640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ef415d6c0 0x7f0ef41652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efa600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ef41f6030 0x7f0ef41f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 -- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ef41f6030 msgr2=0x7f0ef41f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ef41f6030 0x7f0ef41f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 -- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ef415d6c0 msgr2=0x7f0ef41652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:11.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ef415d6c0 0x7f0ef41652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:11.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 -- 172.21.15.46:0/480707144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef41fa2c0 con 0x7f0ef40cf900 2024-09-17T03:30:11.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efa600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ef41f6030 0x7f0ef41f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:11.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9200640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ef415d6c0 0x7f0ef41652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:11.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef9c00640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 0x7f0ef41fb360 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800b410 tx=0x7f0ee800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8015030 con 0x7f0ef40cf900 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ee8004020 con 0x7f0ef40cf900 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef41fa5a0 con 0x7f0ef40cf900 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8004470 con 0x7f0ef40cf900 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.869+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41fab60 con 0x7f0ef40cf900 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.873+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef415dad0 con 0x7f0ef40cf900 2024-09-17T03:30:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.873+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f0ee8018070 con 0x7f0ef40cf900 2024-09-17T03:30:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.873+0000 7f0ef1600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ed4074400 0x7f0ed4076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.873+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f0ee8092d50 con 0x7f0ef40cf900 2024-09-17T03:30:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.873+0000 7f0ef9200640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ed4074400 0x7f0ed4076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.873+0000 7f0ef9200640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ed4074400 0x7f0ed4076910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0ef413e2b0 tx=0x7f0ee0005eb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:11.877+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ee805f450 con 0x7f0ef40cf900 2024-09-17T03:30:12.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.053+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0ef40b0200 con 0x7f0ef40cf900 2024-09-17T03:30:12.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.057+0000 7f0ef1600640 1 -- 172.21.15.46:0/480707144 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f0ee805ed50 con 0x7f0ef40cf900 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ed4074400 msgr2=0x7f0ed4076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ed4074400 0x7f0ed4076910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0ef413e2b0 tx=0x7f0ee0005eb0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 msgr2=0x7f0ef41fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 0x7f0ef41fb360 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800b410 tx=0x7f0ee800b8d0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 shutdown_connections 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0ed4074400 0x7f0ed4076910 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ef41f6030 0x7f0ef41f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ef415d6c0 0x7f0ef41652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 --2- 172.21.15.46:0/480707144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ef40cf900 0x7f0ef41fb360 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 >> 172.21.15.46:0/480707144 conn(0x7f0ef40b2040 msgr2=0x7f0ef40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 shutdown_connections 2024-09-17T03:30:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.061+0000 7f0efc600640 1 -- 172.21.15.46:0/480707144 wait complete. 2024-09-17T03:30:12.080 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:30:12.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:30:12.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:30:12.272 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:30:12.272 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:12.272 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:12.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:30:12.281 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:30:12.282 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:12.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:30:12.292 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:12.292 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:12.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:30:12.331 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:30:12.331 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:12.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:30:12.342 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:30:12.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 -- 172.21.15.46:0/2103901559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 msgr2=0x7fd81815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 --2- 172.21.15.46:0/2103901559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd81815bb60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd808009840 tx=0x7fd80801c760 comp rx=0 tx=0).stop 2024-09-17T03:30:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 -- 172.21.15.46:0/2103901559 shutdown_connections 2024-09-17T03:30:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 --2- 172.21.15.46:0/2103901559 >> v2:172.21.15.99:3300/0 conn(0x7fd8181527b0 0x7fd818074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 --2- 172.21.15.46:0/2103901559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd81815bb60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 -- 172.21.15.46:0/2103901559 >> 172.21.15.46:0/2103901559 conn(0x7fd8180b2040 msgr2=0x7fd8180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 -- 172.21.15.46:0/2103901559 shutdown_connections 2024-09-17T03:30:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.533+0000 7fd81d000640 1 -- 172.21.15.46:0/2103901559 wait complete. 2024-09-17T03:30:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 Processor -- start 2024-09-17T03:30:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 -- start start 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8180b4de0 0x7fd8181491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd818151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8181527b0 0x7fd818151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd818168bb0 con 0x7fd8181527b0 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd818168a30 con 0x7fd8180b4de0 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd818168d30 con 0x7fd8180b5c70 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd816800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8181527b0 0x7fd818151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd818151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd815e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8180b4de0 0x7fd8181491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd818151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:59850/0 (socket says 172.21.15.46:59850) 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 -- 172.21.15.46:0/102893623 learned_addr learned my addr 172.21.15.46:0/102893623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 -- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8180b4de0 msgr2=0x7fd8181491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8180b4de0 0x7fd8181491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 -- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8181527b0 msgr2=0x7fd818151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8181527b0 0x7fd818151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 -- 172.21.15.46:0/102893623 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8180ccb80 con 0x7fd8180b5c70 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd815e00640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8180b4de0 0x7fd8181491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd816800640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8181527b0 0x7fd818151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80fe00640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd818151760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e9c0 tx=0x7fd80000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000cd80 con 0x7fd8180b5c70 2024-09-17T03:30:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd800004020 con 0x7fd8180b5c70 2024-09-17T03:30:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800004450 con 0x7fd8180b5c70 2024-09-17T03:30:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81814b3b0 con 0x7fd8180b5c70 2024-09-17T03:30:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.537+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8180cce20 con 0x7fd8180b5c70 2024-09-17T03:30:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.541+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81814b120 con 0x7fd8180b5c70 2024-09-17T03:30:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.541+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fd800003600 con 0x7fd8180b5c70 2024-09-17T03:30:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.541+0000 7fd80e000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7f4074400 0x7fd7f4076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.541+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7fd800093770 con 0x7fd8180b5c70 2024-09-17T03:30:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.541+0000 7fd815e00640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7f4074400 0x7fd7f4076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.545+0000 7fd815e00640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7f4074400 0x7fd7f4076910 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8080af000 tx=0x7fd80801afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.545+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd80005fc20 con 0x7fd8180b5c70 2024-09-17T03:30:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.757+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd81815bb60 con 0x7fd8180b5c70 2024-09-17T03:30:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.757+0000 7fd80e000640 1 -- 172.21.15.46:0/102893623 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+3919 (secure 0 0 0) 0x7fd800064650 con 0x7fd8180b5c70 2024-09-17T03:30:12.758 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:12.758 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"btime":"2024-09-17T03:30:05:919628+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3006140239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3006140239},{"type":"v1","addr":"172.21.15.46:6837","nonce":3006140239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/3273226584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3273226584},{"type":"v1","addr":"172.21.15.99:6833","nonce":3273226584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":246154,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2375477362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2375477362},{"type":"v1","addr":"172.21.15.99:6835","nonce":2375477362}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":246571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/1000190812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1000190812},{"type":"v1","addr":"172.21.15.46:6836","nonce":1000190812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[]} 2024-09-17T03:30:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7f4074400 msgr2=0x7fd7f4076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7f4074400 0x7fd7f4076910 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8080af000 tx=0x7fd80801afa0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 msgr2=0x7fd818151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd818151760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e9c0 tx=0x7fd80000ee80 comp rx=0 tx=0).stop 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 shutdown_connections 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7f4074400 0x7fd7f4076910 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8181527b0 0x7fd818151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd8180b5c70 0x7fd818151760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 --2- 172.21.15.46:0/102893623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8180b4de0 0x7fd8181491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 >> 172.21.15.46:0/102893623 conn(0x7fd8180b2040 msgr2=0x7fd8180b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 shutdown_connections 2024-09-17T03:30:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:12.761+0000 7fd81d000640 1 -- 172.21.15.46:0/102893623 wait complete. 2024-09-17T03:30:12.781 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:30:12.781 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:30:12.781 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:30:12.782 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:30:12.782 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:30:12.782 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:30:12.782 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:30:12.782 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:30:12.782 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:30:12.783 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:30:12.783 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:30:12.783 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:30:12.783 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:30:12.783 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:30:12.784 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:30:12.784 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:30:12.798 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:30:12.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:30:12.803 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:30:12.804 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:30:12.804 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:30:12.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:30:12.809 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:30:12.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:30:12.853 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:30:12.853 DEBUG:teuthology.parallel:result is None 2024-09-17T03:30:12.854 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:30:12.854 DEBUG:teuthology.parallel:result is None 2024-09-17T03:30:12.857 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:30:12.860 DEBUG:teuthology.parallel:result is None 2024-09-17T03:30:12.862 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:30:12.862 DEBUG:teuthology.parallel:result is None 2024-09-17T03:30:12.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:30:12.958 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:30:12.959 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:30:12.960 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:30:12.961 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:30:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 -- 172.21.15.46:0/1137798596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50074b20 msgr2=0x7f1f5015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 --2- 172.21.15.46:0/1137798596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50074b20 0x7f1f5015cf90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009c80 tx=0x7f1f4801cda0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 -- 172.21.15.46:0/1137798596 shutdown_connections 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 --2- 172.21.15.46:0/1137798596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50074b20 0x7f1f5015cf90 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 --2- 172.21.15.46:0/1137798596 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f500b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 -- 172.21.15.46:0/1137798596 >> 172.21.15.46:0/1137798596 conn(0x7f1f500708c0 msgr2=0x7f1f50070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 -- 172.21.15.46:0/1137798596 shutdown_connections 2024-09-17T03:30:13.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.029+0000 7f1f57a00640 1 -- 172.21.15.46:0/1137798596 wait complete. 2024-09-17T03:30:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 Processor -- start 2024-09-17T03:30:13.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 -- start start 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f5014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50160d50 0x7f1f501fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f5015ea10 0x7f1f5015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5016a870 con 0x7f1f50160d50 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5016a6f0 con 0x7f1f5015ea10 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5016a9f0 con 0x7f1f50157030 2024-09-17T03:30:13.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f5014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50160d50 0x7f1f501fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f5014ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:59898/0 (socket says 172.21.15.46:59898) 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 -- 172.21.15.46:0/2507365886 learned_addr learned my addr 172.21.15.46:0/2507365886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f5015ea10 0x7f1f5015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 -- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f5015ea10 msgr2=0x7f1f5015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f5015ea10 0x7f1f5015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 -- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50160d50 msgr2=0x7f1f501fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50160d50 0x7f1f501fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 -- 172.21.15.46:0/2507365886 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5015f360 con 0x7f1f50157030 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f47e00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50160d50 0x7f1f501fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:13.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f55000640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f5014ccb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000d750 tx=0x7f1f4000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40017070 con 0x7f1f50157030 2024-09-17T03:30:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f40009d20 con 0x7f1f50157030 2024-09-17T03:30:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40005280 con 0x7f1f50157030 2024-09-17T03:30:13.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5014d390 con 0x7f1f50157030 2024-09-17T03:30:13.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.033+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5006cf20 con 0x7f1f50157030 2024-09-17T03:30:13.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.037+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5014d560 con 0x7f1f50157030 2024-09-17T03:30:13.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.037+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f1f40005420 con 0x7f1f50157030 2024-09-17T03:30:13.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.037+0000 7f1f4d600640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f2c074400 0x7f1f2c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.037+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(79..79 src has 1..79) ==== 4943+0+0 (secure 0 0 0) 0x7f1f40093230 con 0x7f1f50157030 2024-09-17T03:30:13.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.037+0000 7f1f47e00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f2c074400 0x7f1f2c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.037+0000 7f1f47e00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f2c074400 0x7f1f2c076910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f500d15e0 tx=0x7f1f38009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:13.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.041+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f5015f360 con 0x7f1f50157030 2024-09-17T03:30:13.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.229+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1f501fd8e0 con 0x7f1f50157030 2024-09-17T03:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.313+0000 7f1f4d600640 1 -- 172.21.15.46:0/2507365886 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) ==== 97+0+0 (secure 0 0 0) 0x7f1f4000de60 con 0x7f1f50157030 2024-09-17T03:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f2c074400 msgr2=0x7f1f2c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f2c074400 0x7f1f2c076910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f500d15e0 tx=0x7f1f38009040 comp rx=0 tx=0).stop 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 msgr2=0x7f1f5014ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f5014ccb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000d750 tx=0x7f1f4000dc10 comp rx=0 tx=0).stop 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 shutdown_connections 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1f2c074400 0x7f1f2c076910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f5015ea10 0x7f1f5015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f50160d50 0x7f1f501fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 --2- 172.21.15.46:0/2507365886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1f50157030 0x7f1f5014ccb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 >> 172.21.15.46:0/2507365886 conn(0x7f1f500708c0 msgr2=0x7f1f500d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 shutdown_connections 2024-09-17T03:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.317+0000 7f1f57a00640 1 -- 172.21.15.46:0/2507365886 wait complete. 2024-09-17T03:30:13.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:30:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 -- 172.21.15.46:0/2595245144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 msgr2=0x7f11e40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 --2- 172.21.15.46:0/2595245144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 0x7f11e40cdbb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009840 tx=0x7f11cc01c740 comp rx=0 tx=0).stop 2024-09-17T03:30:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 -- 172.21.15.46:0/2595245144 shutdown_connections 2024-09-17T03:30:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 --2- 172.21.15.46:0/2595245144 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 0x7f11e40cdbb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 --2- 172.21.15.46:0/2595245144 >> v2:172.21.15.99:3300/0 conn(0x7f11e415aa50 0x7f11e40cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 -- 172.21.15.46:0/2595245144 >> 172.21.15.46:0/2595245144 conn(0x7f11e40b2040 msgr2=0x7f11e40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 -- 172.21.15.46:0/2595245144 shutdown_connections 2024-09-17T03:30:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.497+0000 7f11eb000640 1 -- 172.21.15.46:0/2595245144 wait complete. 2024-09-17T03:30:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 Processor -- start 2024-09-17T03:30:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 -- start start 2024-09-17T03:30:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 0x7f11e415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 0x7f11e4154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e415e4e0 0x7f11e4074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11e41663f0 con 0x7f11e415aa50 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f11e4166270 con 0x7f11e415e4e0 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11eb000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f11e4166570 con 0x7f11e40cd7a0 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 0x7f11e4154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11db400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 0x7f11e4154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45358/0 (socket says 172.21.15.46:45358) 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.501+0000 7f11db400640 1 -- 172.21.15.46:0/3110748010 learned_addr learned my addr 172.21.15.46:0/3110748010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11e3e00640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 0x7f11e415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11e9000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e415e4e0 0x7f11e4074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11db400640 1 -- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 msgr2=0x7f11e415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11db400640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 0x7f11e415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11db400640 1 -- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e415e4e0 msgr2=0x7f11e4074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11db400640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e415e4e0 0x7f11e4074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11db400640 1 -- 172.21.15.46:0/3110748010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e4153c60 con 0x7f11e415aa50 2024-09-17T03:30:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11db400640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 0x7f11e4154930 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009790 tx=0x7f11cc004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0bb070 con 0x7f11e415aa50 2024-09-17T03:30:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11cc0a5c30 con 0x7f11e415aa50 2024-09-17T03:30:13.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e41fc790 con 0x7f11e415aa50 2024-09-17T03:30:13.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0b2040 con 0x7f11e415aa50 2024-09-17T03:30:13.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e41fcd50 con 0x7f11e415aa50 2024-09-17T03:30:13.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e41fc500 con 0x7f11e415aa50 2024-09-17T03:30:13.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.505+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f11cc005030 con 0x7f11e415aa50 2024-09-17T03:30:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.509+0000 7f11e0c00640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f11c0074400 0x7f11c0076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.509+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f11cc12c2e0 con 0x7f11e415aa50 2024-09-17T03:30:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.509+0000 7f11e3e00640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f11c0074400 0x7f11c0076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.509+0000 7f11e3e00640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f11c0074400 0x7f11c0076910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009600 tx=0x7f11d40091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.513+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11cc016330 con 0x7f11e415aa50 2024-09-17T03:30:13.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.745+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f11e4166a40 con 0x7f11e415aa50 2024-09-17T03:30:13.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11e0c00640 1 -- 172.21.15.46:0/3110748010 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) ==== 80+0+5304 (secure 0 0 0) 0x7f11cc0f8710 con 0x7f11e415aa50 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:30:13.748 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.749 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.750 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:30:13.752 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.753 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.754 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.755 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.756 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.757 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:30:13.758 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:30:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:30:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.759 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:30:13.760 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.761 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:30:13.762 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:30:13.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f11c0074400 msgr2=0x7f11c0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f11c0074400 0x7f11c0076910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009600 tx=0x7f11d40091e0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 msgr2=0x7f11e4154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 0x7f11e4154930 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009790 tx=0x7f11cc004620 comp rx=0 tx=0).stop 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 shutdown_connections 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f11c0074400 0x7f11c0076910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e415e4e0 0x7f11e4074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e415aa50 0x7f11e4154930 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 --2- 172.21.15.46:0/3110748010 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11e40cd7a0 0x7f11e415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 >> 172.21.15.46:0/3110748010 conn(0x7f11e40b2040 msgr2=0x7f11e4167710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 shutdown_connections 2024-09-17T03:30:13.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.749+0000 7f11eb000640 1 -- 172.21.15.46:0/3110748010 wait complete. 2024-09-17T03:30:13.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:30:13.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 -- 172.21.15.46:0/2684052450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40cefa0 msgr2=0x7fbae40cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 --2- 172.21.15.46:0/2684052450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40cefa0 0x7fbae40cf3b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009840 tx=0x7fbacc01c740 comp rx=0 tx=0).stop 2024-09-17T03:30:13.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 -- 172.21.15.46:0/2684052450 shutdown_connections 2024-09-17T03:30:13.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 --2- 172.21.15.46:0/2684052450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40cefa0 0x7fbae40cf3b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 -- 172.21.15.46:0/2684052450 >> 172.21.15.46:0/2684052450 conn(0x7fbae40b2040 msgr2=0x7fbae40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:13.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 -- 172.21.15.46:0/2684052450 shutdown_connections 2024-09-17T03:30:13.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.929+0000 7fbaeb000640 1 -- 172.21.15.46:0/2684052450 wait complete. 2024-09-17T03:30:13.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 Processor -- start 2024-09-17T03:30:13.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 -- start start 2024-09-17T03:30:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40749a0 0x7fbae4156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbae40b3340 0x7fbae4144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 0x7fbae40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbae416b350 con 0x7fbae40cefa0 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbae416b1d0 con 0x7fbae40749a0 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbae416b4d0 con 0x7fbae40b3340 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 0x7fbae40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 0x7fbae40ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45366/0 (socket says 172.21.15.46:45366) 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 -- 172.21.15.46:0/1415752298 learned_addr learned my addr 172.21.15.46:0/1415752298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbadbe00640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbae40b3340 0x7fbae4144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae3e00640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40749a0 0x7fbae4156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 -- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbae40b3340 msgr2=0x7fbae4144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbae40b3340 0x7fbae4144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 -- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40749a0 msgr2=0x7fbae4156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40749a0 0x7fbae4156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 -- 172.21.15.46:0/1415752298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae4144540 con 0x7fbae40cefa0 2024-09-17T03:30:13.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae9000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 0x7fbae40ae8e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c730 tx=0x7fbadc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc003b80 con 0x7fbae40cefa0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbadc003d20 con 0x7fbae40cefa0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae40cd2e0 con 0x7fbae40cefa0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc00aa50 con 0x7fbae40cefa0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.933+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae40cd8a0 con 0x7fbae40cefa0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.937+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae40cf3b0 con 0x7fbae40cefa0 2024-09-17T03:30:13.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.937+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fbadc00abf0 con 0x7fbae40cefa0 2024-09-17T03:30:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.937+0000 7fbae0c00640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbac0074430 0x7fbac0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.937+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7fbadc092f80 con 0x7fbae40cefa0 2024-09-17T03:30:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.937+0000 7fbae3e00640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbac0074430 0x7fbac0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:13.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.937+0000 7fbae3e00640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbac0074430 0x7fbac0076940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbad40078b0 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:13.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:13.941+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbadc05f690 con 0x7fbae40cefa0 2024-09-17T03:30:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbae415fef0 con 0x7fbae40cefa0 2024-09-17T03:30:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbae0c00640 1 -- 172.21.15.46:0/1415752298 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7fbadc05ef90 con 0x7fbae40cefa0 2024-09-17T03:30:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:14.120 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:30:14.120 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:30:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:30:14.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:14.121 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:30:14.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbac0074430 msgr2=0x7fbac0076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:14.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbac0074430 0x7fbac0076940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbad40078b0 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 msgr2=0x7fbae40ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 0x7fbae40ae8e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c730 tx=0x7fbadc00cbf0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 shutdown_connections 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbac0074430 0x7fbac0076940 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbae40cefa0 0x7fbae40ae8e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbae40b3340 0x7fbae4144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 --2- 172.21.15.46:0/1415752298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae40749a0 0x7fbae4156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 >> 172.21.15.46:0/1415752298 conn(0x7fbae40b2040 msgr2=0x7fbae40b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 shutdown_connections 2024-09-17T03:30:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.121+0000 7fbaeb000640 1 -- 172.21.15.46:0/1415752298 wait complete. 2024-09-17T03:30:14.143 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:30:14.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 -- 172.21.15.46:0/4228012779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20074b20 msgr2=0x7f7f2015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 --2- 172.21.15.46:0/4228012779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20074b20 0x7f7f2015cf90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7f200713e0 tx=0x7f7f1801caf0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 -- 172.21.15.46:0/4228012779 shutdown_connections 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 --2- 172.21.15.46:0/4228012779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20074b20 0x7f7f2015cf90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 --2- 172.21.15.46:0/4228012779 >> v2:172.21.15.99:3300/0 conn(0x7f7f20157030 0x7f7f200b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 -- 172.21.15.46:0/4228012779 >> 172.21.15.46:0/4228012779 conn(0x7f7f200708c0 msgr2=0x7f7f20070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 -- 172.21.15.46:0/4228012779 shutdown_connections 2024-09-17T03:30:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.309+0000 7f7f25000640 1 -- 172.21.15.46:0/4228012779 wait complete. 2024-09-17T03:30:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 Processor -- start 2024-09-17T03:30:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 -- start start 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 0x7f7f2013bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20160d50 0x7f7f201fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f201fab30 0x7f7f2014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2016a790 con 0x7f7f20160d50 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2016a610 con 0x7f7f20157030 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f25000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2016a910 con 0x7f7f201fab30 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f1de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 0x7f7f2013bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20160d50 0x7f7f201fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f1de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 0x7f7f2013bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:59888/0 (socket says 172.21.15.46:59888) 2024-09-17T03:30:14.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f1de00640 1 -- 172.21.15.46:0/1738075067 learned_addr learned my addr 172.21.15.46:0/1738075067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:14.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.313+0000 7f7f1e800640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f201fab30 0x7f7f2014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1de00640 1 -- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f201fab30 msgr2=0x7f7f2014e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1de00640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f201fab30 0x7f7f2014e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1de00640 1 -- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20160d50 msgr2=0x7f7f201fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1de00640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20160d50 0x7f7f201fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1de00640 1 -- 172.21.15.46:0/1738075067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2014d8a0 con 0x7f7f20157030 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1e800640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f201fab30 0x7f7f2014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f1de00640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 0x7f7f2013bd10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e820 tx=0x7f7f1000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000ccc0 con 0x7f7f20157030 2024-09-17T03:30:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f1000ce80 con 0x7f7f20157030 2024-09-17T03:30:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f200b40a0 con 0x7f7f20157030 2024-09-17T03:30:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10005280 con 0x7f7f20157030 2024-09-17T03:30:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2014db10 con 0x7f7f20157030 2024-09-17T03:30:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef80052a0 con 0x7f7f20157030 2024-09-17T03:30:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.317+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f7f10005420 con 0x7f7f20157030 2024-09-17T03:30:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.321+0000 7f7f16000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f00074400 0x7f7f00076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.321+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f7f10093590 con 0x7f7f20157030 2024-09-17T03:30:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.321+0000 7f7f1d400640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f00074400 0x7f7f00076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.321+0000 7f7f1d400640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f00074400 0x7f7f00076910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7f20156320 tx=0x7f7f0c0013a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.325+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f1005faa0 con 0x7f7f20157030 2024-09-17T03:30:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:14.501+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7ef8005860 con 0x7f7f20157030 2024-09-17T03:30:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.337+0000 7f7f16000640 1 -- 172.21.15.46:0/1738075067 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f7f100644d0 con 0x7f7f20157030 2024-09-17T03:30:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:30:15.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f00074400 msgr2=0x7f7f00076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f00074400 0x7f7f00076910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7f20156320 tx=0x7f7f0c0013a0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 msgr2=0x7f7f2013bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 0x7f7f2013bd10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e820 tx=0x7f7f1000ece0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 shutdown_connections 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f00074400 0x7f7f00076910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f201fab30 0x7f7f2014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20160d50 0x7f7f201fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 --2- 172.21.15.46:0/1738075067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f20157030 0x7f7f2013bd10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 >> 172.21.15.46:0/1738075067 conn(0x7f7f200708c0 msgr2=0x7f7f20152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 shutdown_connections 2024-09-17T03:30:15.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.341+0000 7f7f25000640 1 -- 172.21.15.46:0/1738075067 wait complete. 2024-09-17T03:30:15.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 -- 172.21.15.46:0/4265860541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb00b5c70 msgr2=0x7f5eb015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 --2- 172.21.15.46:0/4265860541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb00b5c70 0x7f5eb015bb60 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0070710 tx=0x7f5ea001c740 comp rx=0 tx=0).stop 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 -- 172.21.15.46:0/4265860541 shutdown_connections 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 --2- 172.21.15.46:0/4265860541 >> v2:172.21.15.99:3300/0 conn(0x7f5eb01527b0 0x7f5eb0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 --2- 172.21.15.46:0/4265860541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb00b5c70 0x7f5eb015bb60 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 -- 172.21.15.46:0/4265860541 >> 172.21.15.46:0/4265860541 conn(0x7f5eb00b2040 msgr2=0x7f5eb00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 -- 172.21.15.46:0/4265860541 shutdown_connections 2024-09-17T03:30:15.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.537+0000 7f5eb6200640 1 -- 172.21.15.46:0/4265860541 wait complete. 2024-09-17T03:30:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 Processor -- start 2024-09-17T03:30:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 -- start start 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb00b4de0 0x7f5eb01fa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5eb00b5c70 0x7f5eb00cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 0x7f5eb0149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0168b20 con 0x7f5eb01527b0 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01689a0 con 0x7f5eb00b4de0 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eb6200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb0168ca0 con 0x7f5eb00b5c70 2024-09-17T03:30:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 0x7f5eb0149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 0x7f5eb0149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45392/0 (socket says 172.21.15.46:45392) 2024-09-17T03:30:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 -- 172.21.15.46:0/450381455 learned_addr learned my addr 172.21.15.46:0/450381455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eaf400640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb00b4de0 0x7f5eb01fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5ea7e00640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5eb00b5c70 0x7f5eb00cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 -- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5eb00b5c70 msgr2=0x7f5eb00cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5eb00b5c70 0x7f5eb00cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 -- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb00b4de0 msgr2=0x7f5eb01fa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb00b4de0 0x7f5eb01fa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 -- 172.21.15.46:0/450381455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb0149f80 con 0x7f5eb01527b0 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.541+0000 7f5eafe00640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 0x7f5eb0149a40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c730 tx=0x7f5ea800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003b80 con 0x7f5eb01527b0 2024-09-17T03:30:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea8003d40 con 0x7f5eb01527b0 2024-09-17T03:30:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb014fa30 con 0x7f5eb01527b0 2024-09-17T03:30:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80052d0 con 0x7f5eb01527b0 2024-09-17T03:30:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb01f9d60 con 0x7f5eb01527b0 2024-09-17T03:30:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb015bb60 con 0x7f5eb01527b0 2024-09-17T03:30:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f5ea8005680 con 0x7f5eb01527b0 2024-09-17T03:30:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5ea7400640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e8c074400 0x7f5e8c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5227+0+0 (secure 0 0 0) 0x7f5ea8092f30 con 0x7f5eb01527b0 2024-09-17T03:30:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.545+0000 7f5eaf400640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e8c074400 0x7f5e8c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.549+0000 7f5eaf400640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e8c074400 0x7f5e8c076910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00af000 tx=0x7f5ea001af80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.553+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5eb0149f80 con 0x7f5eb01527b0 2024-09-17T03:30:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:15.729+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5eb0150ef0 con 0x7f5eb01527b0 2024-09-17T03:30:16.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.333+0000 7f5ea7400640 1 -- 172.21.15.46:0/450381455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f5ea805f070 con 0x7f5eb01527b0 2024-09-17T03:30:16.335 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:30:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.337+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e8c074400 msgr2=0x7f5e8c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.337+0000 7f5eb6200640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e8c074400 0x7f5e8c076910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00af000 tx=0x7f5ea001af80 comp rx=0 tx=0).stop 2024-09-17T03:30:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.337+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 msgr2=0x7f5eb0149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.337+0000 7f5eb6200640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 0x7f5eb0149a40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c730 tx=0x7f5ea800cbf0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 shutdown_connections 2024-09-17T03:30:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e8c074400 0x7f5e8c076910 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5eb01527b0 0x7f5eb0149a40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5eb00b5c70 0x7f5eb00cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 --2- 172.21.15.46:0/450381455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5eb00b4de0 0x7f5eb01fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 >> 172.21.15.46:0/450381455 conn(0x7f5eb00b2040 msgr2=0x7f5eb0154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 shutdown_connections 2024-09-17T03:30:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.341+0000 7f5eb6200640 1 -- 172.21.15.46:0/450381455 wait complete. 2024-09-17T03:30:16.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:30:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 -- 172.21.15.46:0/3594310301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c0cd2e0 msgr2=0x7f532c070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 --2- 172.21.15.46:0/3594310301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c0cd2e0 0x7f532c070e80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f532c1464a0 tx=0x7f531c01c800 comp rx=0 tx=0).stop 2024-09-17T03:30:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 -- 172.21.15.46:0/3594310301 shutdown_connections 2024-09-17T03:30:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 --2- 172.21.15.46:0/3594310301 >> v2:172.21.15.99:3300/0 conn(0x7f532c1602a0 0x7f532c1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 --2- 172.21.15.46:0/3594310301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c0cd2e0 0x7f532c070e80 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 -- 172.21.15.46:0/3594310301 >> 172.21.15.46:0/3594310301 conn(0x7f532c0b2c40 msgr2=0x7f532c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 -- 172.21.15.46:0/3594310301 shutdown_connections 2024-09-17T03:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.545+0000 7f5331a00640 1 -- 172.21.15.46:0/3594310301 wait complete. 2024-09-17T03:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 Processor -- start 2024-09-17T03:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 -- start start 2024-09-17T03:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f532c0cd2e0 0x7f532c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c15cd20 0x7f532c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 0x7f532c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f532c16a240 con 0x7f532c1602a0 2024-09-17T03:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f532c16a0c0 con 0x7f532c15cd20 2024-09-17T03:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f5331a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f532c16a3c0 con 0x7f532c0cd2e0 2024-09-17T03:30:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 0x7f532c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 0x7f532c1597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45408/0 (socket says 172.21.15.46:45408) 2024-09-17T03:30:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 -- 172.21.15.46:0/3794162535 learned_addr learned my addr 172.21.15.46:0/3794162535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532aa00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f532c0cd2e0 0x7f532c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532a000640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c15cd20 0x7f532c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 -- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f532c0cd2e0 msgr2=0x7f532c158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f532c0cd2e0 0x7f532c158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 -- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c15cd20 msgr2=0x7f532c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c15cd20 0x7f532c155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532b400640 1 -- 172.21.15.46:0/3794162535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c13d250 con 0x7f532c1602a0 2024-09-17T03:30:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.549+0000 7f532a000640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c15cd20 0x7f532c155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f532b400640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 0x7f532c1597e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f532400cae0 tx=0x7f532400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324016020 con 0x7f532c1602a0 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f532400ab40 con 0x7f532c1602a0 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532c13d530 con 0x7f532c1602a0 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532400ce20 con 0x7f532c1602a0 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532c153f50 con 0x7f532c1602a0 2024-09-17T03:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.553+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532c1606b0 con 0x7f532c1602a0 2024-09-17T03:30:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.557+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f5324019020 con 0x7f532c1602a0 2024-09-17T03:30:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.557+0000 7f5322a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f530c074430 0x7f530c076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.557+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5609+0+0 (secure 0 0 0) 0x7f5324093920 con 0x7f532c1602a0 2024-09-17T03:30:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.557+0000 7f532aa00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f530c074430 0x7f530c076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.557+0000 7f532aa00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f530c074430 0x7f530c076940 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f531c0af000 tx=0x7f531c01b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:16.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.561+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5324020020 con 0x7f532c1602a0 2024-09-17T03:30:16.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:16.769+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f532c153d20 con 0x7f532c1602a0 2024-09-17T03:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.337+0000 7f5322a00640 1 -- 172.21.15.46:0/3794162535 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) ==== 174+0+0 (secure 0 0 0) 0x7f5324060060 con 0x7f532c1602a0 2024-09-17T03:30:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 12 and data pool 13 2024-09-17T03:30:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.345+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f530c074430 msgr2=0x7f530c076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.345+0000 7f5331a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f530c074430 0x7f530c076940 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f531c0af000 tx=0x7f531c01b040 comp rx=0 tx=0).stop 2024-09-17T03:30:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.345+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 msgr2=0x7f532c1597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.345+0000 7f5331a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 0x7f532c1597e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f532400cae0 tx=0x7f532400cbd0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.353+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 shutdown_connections 2024-09-17T03:30:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.353+0000 7f5331a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f530c074430 0x7f530c076940 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.353+0000 7f5331a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f532c1602a0 0x7f532c1597e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.353+0000 7f5331a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c15cd20 0x7f532c155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.353+0000 7f5331a00640 1 --2- 172.21.15.46:0/3794162535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f532c0cd2e0 0x7f532c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.353+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 >> 172.21.15.46:0/3794162535 conn(0x7f532c0b2c40 msgr2=0x7f532c13f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:17.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.357+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 shutdown_connections 2024-09-17T03:30:17.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.357+0000 7f5331a00640 1 -- 172.21.15.46:0/3794162535 wait complete. 2024-09-17T03:30:17.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 -- 172.21.15.46:0/1824827972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b4006370 msgr2=0x7f81b400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 --2- 172.21.15.46:0/1824827972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b4006370 0x7f81b400cae0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f81c800c660 tx=0x7f81c801d2e0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 -- 172.21.15.46:0/1824827972 shutdown_connections 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 --2- 172.21.15.46:0/1824827972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81b4006370 0x7f81b400cae0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 -- 172.21.15.46:0/1824827972 >> 172.21.15.46:0/1824827972 conn(0x7f81d00b2040 msgr2=0x7f81d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 -- 172.21.15.46:0/1824827972 shutdown_connections 2024-09-17T03:30:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.565+0000 7f81d5200640 1 -- 172.21.15.46:0/1824827972 wait complete. 2024-09-17T03:30:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 Processor -- start 2024-09-17T03:30:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 -- start start 2024-09-17T03:30:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81b4006370 0x7f81d0140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81d01e8a80 0x7f81d01e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 0x7f81d0143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81d015e160 con 0x7f81d0143300 2024-09-17T03:30:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81d015dfe0 con 0x7f81b4006370 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f81d015e2e0 con 0x7f81d01e8a80 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 0x7f81d0143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 0x7f81d0143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45422/0 (socket says 172.21.15.46:45422) 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 -- 172.21.15.46:0/1955713119 learned_addr learned my addr 172.21.15.46:0/1955713119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81c7e00640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81d01e8a80 0x7f81d01e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81cde00640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81b4006370 0x7f81d0140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 -- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81d01e8a80 msgr2=0x7f81d01e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81d01e8a80 0x7f81d01e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 -- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81b4006370 msgr2=0x7f81d0140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81b4006370 0x7f81d0140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 -- 172.21.15.46:0/1955713119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d01412b0 con 0x7f81d0143300 2024-09-17T03:30:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81ce800640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 0x7f81d0143710 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f81c801cda0 tx=0x7f81c80ada50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c800c280 con 0x7f81d0143300 2024-09-17T03:30:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81c8003c70 con 0x7f81d0143300 2024-09-17T03:30:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8002090 con 0x7f81d0143300 2024-09-17T03:30:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d0144230 con 0x7f81d0143300 2024-09-17T03:30:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.569+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d01447f0 con 0x7f81d0143300 2024-09-17T03:30:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.573+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d015ecd0 con 0x7f81d0143300 2024-09-17T03:30:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.573+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f81c80ab030 con 0x7f81d0143300 2024-09-17T03:30:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.573+0000 7f81c6000640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81ac074400 0x7f81ac076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.573+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f81c812d5c0 con 0x7f81d0143300 2024-09-17T03:30:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.573+0000 7f81cde00640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81ac074400 0x7f81ac076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.573+0000 7f81cde00640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81ac074400 0x7f81ac076910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f81c0000c10 tx=0x7f81c0006b80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.577+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f81d01412b0 con 0x7f81d0143300 2024-09-17T03:30:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.753+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81d015f070 con 0x7f81d0143300 2024-09-17T03:30:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.753+0000 7f81c6000640 1 -- 172.21.15.46:0/1955713119 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16697 (secure 0 0 0) 0x7f81c8017090 con 0x7f81d0143300 2024-09-17T03:30:17.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:17.755 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:30:16.771456+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:30:14.515457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T03:30:15.740742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:30:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81ac074400 msgr2=0x7f81ac076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81ac074400 0x7f81ac076910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f81c0000c10 tx=0x7f81c0006b80 comp rx=0 tx=0).stop 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 msgr2=0x7f81d0143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 0x7f81d0143710 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f81c801cda0 tx=0x7f81c80ada50 comp rx=0 tx=0).stop 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 shutdown_connections 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81ac074400 0x7f81ac076910 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81d0143300 0x7f81d0143710 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81d01e8a80 0x7f81d01e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 --2- 172.21.15.46:0/1955713119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81b4006370 0x7f81d0140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 >> 172.21.15.46:0/1955713119 conn(0x7f81d00b2040 msgr2=0x7f81d015a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 shutdown_connections 2024-09-17T03:30:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.757+0000 7f81d5200640 1 -- 172.21.15.46:0/1955713119 wait complete. 2024-09-17T03:30:17.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 -- 172.21.15.46:0/3001407048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b4de0 msgr2=0x7f93d00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 --2- 172.21.15.46:0/3001407048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b4de0 0x7f93d00b51f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f93d00703e0 tx=0x7f93c801ccc0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 -- 172.21.15.46:0/3001407048 shutdown_connections 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 --2- 172.21.15.46:0/3001407048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b4de0 0x7f93d00b51f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 --2- 172.21.15.46:0/3001407048 >> v2:172.21.15.99:3300/0 conn(0x7f93d01527b0 0x7f93d0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 --2- 172.21.15.46:0/3001407048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d00b5c70 0x7f93d015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 -- 172.21.15.46:0/3001407048 >> 172.21.15.46:0/3001407048 conn(0x7f93d00b2040 msgr2=0x7f93d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 -- 172.21.15.46:0/3001407048 shutdown_connections 2024-09-17T03:30:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.937+0000 7f93d6000640 1 -- 172.21.15.46:0/3001407048 wait complete. 2024-09-17T03:30:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 Processor -- start 2024-09-17T03:30:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 -- start start 2024-09-17T03:30:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d00b4de0 0x7f93d014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 0x7f93d00b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d01527b0 0x7f93d00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93d015f6b0 con 0x7f93d00b5c70 2024-09-17T03:30:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93d015f530 con 0x7f93d00b4de0 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93d6000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f93d015f830 con 0x7f93d01527b0 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 0x7f93d00b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 0x7f93d00b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45444/0 (socket says 172.21.15.46:45444) 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93c7e00640 1 -- 172.21.15.46:0/543842297 learned_addr learned my addr 172.21.15.46:0/543842297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.941+0000 7f93cf400640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d00b4de0 0x7f93d014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93cfe00640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d01527b0 0x7f93d00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7e00640 1 -- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d01527b0 msgr2=0x7f93d00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7e00640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d01527b0 0x7f93d00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7e00640 1 -- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d00b4de0 msgr2=0x7f93d014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7e00640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d00b4de0 0x7f93d014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7e00640 1 -- 172.21.15.46:0/543842297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d01565a0 con 0x7f93d00b5c70 2024-09-17T03:30:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7e00640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 0x7f93d00b07b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f93b800c970 tx=0x7f93b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8014070 con 0x7f93d00b5c70 2024-09-17T03:30:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b8007740 con 0x7f93d00b5c70 2024-09-17T03:30:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8007b90 con 0x7f93d00b5c70 2024-09-17T03:30:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d00cdca0 con 0x7f93d00b5c70 2024-09-17T03:30:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d0156d60 con 0x7f93d00b5c70 2024-09-17T03:30:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d0156b80 con 0x7f93d00b5c70 2024-09-17T03:30:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.945+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f93b8017070 con 0x7f93d00b5c70 2024-09-17T03:30:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.949+0000 7f93c7400640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93ac074430 0x7f93ac076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.949+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f93b8092fd0 con 0x7f93d00b5c70 2024-09-17T03:30:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.949+0000 7f93cf400640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93ac074430 0x7f93ac076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:17.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.949+0000 7f93cf400640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93ac074430 0x7f93ac076940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f93c00098f0 tx=0x7f93c0006f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:17.953+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93d0156b80 con 0x7f93d00b5c70 2024-09-17T03:30:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.129+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93d015fd90 con 0x7f93d00b5c70 2024-09-17T03:30:18.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.129+0000 7f93c7400640 1 -- 172.21.15.46:0/543842297 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16697 (secure 0 0 0) 0x7f93b805f3f0 con 0x7f93d00b5c70 2024-09-17T03:30:18.129 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:18.129 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:30:16.771456+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:30:14.515457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T03:30:15.740742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:30:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93ac074430 msgr2=0x7f93ac076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93ac074430 0x7f93ac076940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f93c00098f0 tx=0x7f93c0006f90 comp rx=0 tx=0).stop 2024-09-17T03:30:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 msgr2=0x7f93d00b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 0x7f93d00b07b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f93b800c970 tx=0x7f93b800ce30 comp rx=0 tx=0).stop 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 shutdown_connections 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93ac074430 0x7f93ac076940 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d01527b0 0x7f93d00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d00b5c70 0x7f93d00b07b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 --2- 172.21.15.46:0/543842297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d00b4de0 0x7f93d014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 >> 172.21.15.46:0/543842297 conn(0x7f93d00b2040 msgr2=0x7f93d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 shutdown_connections 2024-09-17T03:30:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.133+0000 7f93d6000640 1 -- 172.21.15.46:0/543842297 wait complete. 2024-09-17T03:30:18.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:30:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 -- 172.21.15.46:0/2910488504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0749a0 msgr2=0x7f647c074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 --2- 172.21.15.46:0/2910488504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0749a0 0x7f647c074db0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6470009840 tx=0x7f647001c820 comp rx=0 tx=0).stop 2024-09-17T03:30:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 -- 172.21.15.46:0/2910488504 shutdown_connections 2024-09-17T03:30:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 --2- 172.21.15.46:0/2910488504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0749a0 0x7f647c074db0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 -- 172.21.15.46:0/2910488504 >> 172.21.15.46:0/2910488504 conn(0x7f647c0b2040 msgr2=0x7f647c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 -- 172.21.15.46:0/2910488504 shutdown_connections 2024-09-17T03:30:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 -- 172.21.15.46:0/2910488504 wait complete. 2024-09-17T03:30:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 Processor -- start 2024-09-17T03:30:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.317+0000 7f6484000640 1 -- start start 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f647c0749a0 0x7f647c1fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 0x7f647c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647c0cefa0 0x7f647c1f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f647c16b350 con 0x7f647c0b3340 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f647c16b1d0 con 0x7f647c0cefa0 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f647c16b4d0 con 0x7f647c0749a0 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 0x7f647c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 0x7f647c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45460/0 (socket says 172.21.15.46:45460) 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 -- 172.21.15.46:0/1069880030 learned_addr learned my addr 172.21.15.46:0/1069880030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6481600640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f647c0749a0 0x7f647c1fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6482000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647c0cefa0 0x7f647c1f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 -- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f647c0749a0 msgr2=0x7f647c1fcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f647c0749a0 0x7f647c1fcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 -- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647c0cefa0 msgr2=0x7f647c1f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647c0cefa0 0x7f647c1f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 -- 172.21.15.46:0/1069880030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c1fc270 con 0x7f647c0b3340 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6482000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647c0cefa0 0x7f647c1f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6480c00640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 0x7f647c159870 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f646800c3f0 tx=0x7f646800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468017070 con 0x7f647c0b3340 2024-09-17T03:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64680053c0 con 0x7f647c0b3340 2024-09-17T03:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468003860 con 0x7f647c0b3340 2024-09-17T03:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c13d660 con 0x7f647c0b3340 2024-09-17T03:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c1f96b0 con 0x7f647c0b3340 2024-09-17T03:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.321+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c074db0 con 0x7f647c0b3340 2024-09-17T03:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.325+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f646801a070 con 0x7f647c0b3340 2024-09-17T03:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.325+0000 7f6479600640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f645c074400 0x7f645c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.325+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f64680924d0 con 0x7f647c0b3340 2024-09-17T03:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.325+0000 7f6481600640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f645c074400 0x7f645c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.325+0000 7f6481600640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f645c074400 0x7f645c076910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f64700af000 tx=0x7f647001b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.329+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f646805e500 con 0x7f647c0b3340 2024-09-17T03:30:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:18.541+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f647c13ea40 con 0x7f647c0b3340 2024-09-17T03:30:19.320 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-09-17T03:30:19.321+0000 7f13bf600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:30:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6479600640 1 -- 172.21.15.46:0/1069880030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7f6468062f30 con 0x7f647c0b3340 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f645c074400 msgr2=0x7f645c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f645c074400 0x7f645c076910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f64700af000 tx=0x7f647001b060 comp rx=0 tx=0).stop 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 msgr2=0x7f647c159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 0x7f647c159870 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f646800c3f0 tx=0x7f646800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 shutdown_connections 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f645c074400 0x7f645c076910 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f647c0cefa0 0x7f647c1f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0b3340 0x7f647c159870 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 --2- 172.21.15.46:0/1069880030 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f647c0749a0 0x7f647c1fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 >> 172.21.15.46:0/1069880030 conn(0x7f647c0b2040 msgr2=0x7f647c13af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.365+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 shutdown_connections 2024-09-17T03:30:19.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.369+0000 7f6484000640 1 -- 172.21.15.46:0/1069880030 wait complete. 2024-09-17T03:30:19.392 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:30:19.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:30:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 -- 172.21.15.46:0/216360217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 msgr2=0x7ffbe815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 --2- 172.21.15.46:0/216360217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe815bb60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8070710 tx=0x7ffbd801c740 comp rx=0 tx=0).stop 2024-09-17T03:30:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 -- 172.21.15.46:0/216360217 shutdown_connections 2024-09-17T03:30:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 --2- 172.21.15.46:0/216360217 >> v2:172.21.15.99:3300/0 conn(0x7ffbe81527b0 0x7ffbe8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 --2- 172.21.15.46:0/216360217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe815bb60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 -- 172.21.15.46:0/216360217 >> 172.21.15.46:0/216360217 conn(0x7ffbe80b2040 msgr2=0x7ffbe80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 -- 172.21.15.46:0/216360217 shutdown_connections 2024-09-17T03:30:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.561+0000 7ffbed600640 1 -- 172.21.15.46:0/216360217 wait complete. 2024-09-17T03:30:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 Processor -- start 2024-09-17T03:30:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 -- start start 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbe80b4de0 0x7ffbe8140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe80cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe81527b0 0x7ffbe80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8168b20 con 0x7ffbe80b5c70 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81689a0 con 0x7ffbe81527b0 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe8168ca0 con 0x7ffbe80b4de0 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe80cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe80cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45490/0 (socket says 172.21.15.46:45490) 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 -- 172.21.15.46:0/3551085659 learned_addr learned my addr 172.21.15.46:0/3551085659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbe6a00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbe80b4de0 0x7ffbe8140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbe7400640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe81527b0 0x7ffbe80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 -- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbe80b4de0 msgr2=0x7ffbe8140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbe80b4de0 0x7ffbe8140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 -- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe81527b0 msgr2=0x7ffbe80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe81527b0 0x7ffbe80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 -- 172.21.15.46:0/3551085659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe80aaa20 con 0x7ffbe80b5c70 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdfe00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe80cf210 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000d750 tx=0x7ffbd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0017070 con 0x7ffbe80b5c70 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbd0009d20 con 0x7ffbe80b5c70 2024-09-17T03:30:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe80ce730 con 0x7ffbe80b5c70 2024-09-17T03:30:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0005280 con 0x7ffbe80b5c70 2024-09-17T03:30:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.565+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe80ac1f0 con 0x7ffbe80b5c70 2024-09-17T03:30:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.569+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe80ce520 con 0x7ffbe80b5c70 2024-09-17T03:30:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.569+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7ffbd0005420 con 0x7ffbe80b5c70 2024-09-17T03:30:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.569+0000 7ffbdea00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffbc4074430 0x7ffbc4076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.569+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7ffbd0093250 con 0x7ffbe80b5c70 2024-09-17T03:30:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.569+0000 7ffbe6a00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffbc4074430 0x7ffbc4076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.569+0000 7ffbe6a00640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffbc4074430 0x7ffbc4076940 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80af000 tx=0x7ffbd801af80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.573+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffbd000de60 con 0x7ffbe80b5c70 2024-09-17T03:30:19.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:19.785+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ffbe815bb60 con 0x7ffbe80b5c70 2024-09-17T03:30:20.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.369+0000 7ffbdea00640 1 -- 172.21.15.46:0/3551085659 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7ffbd005f280 con 0x7ffbe80b5c70 2024-09-17T03:30:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffbc4074430 msgr2=0x7ffbc4076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffbc4074430 0x7ffbc4076940 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80af000 tx=0x7ffbd801af80 comp rx=0 tx=0).stop 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 msgr2=0x7ffbe80cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe80cf210 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000d750 tx=0x7ffbd000dc10 comp rx=0 tx=0).stop 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 shutdown_connections 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffbc4074430 0x7ffbc4076940 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe81527b0 0x7ffbe80cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80b5c70 0x7ffbe80cf210 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 --2- 172.21.15.46:0/3551085659 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffbe80b4de0 0x7ffbe8140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 >> 172.21.15.46:0/3551085659 conn(0x7ffbe80b2040 msgr2=0x7ffbe8151c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 shutdown_connections 2024-09-17T03:30:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.373+0000 7ffbed600640 1 -- 172.21.15.46:0/3551085659 wait complete. 2024-09-17T03:30:20.398 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:30:20.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 -- 172.21.15.46:0/2457667869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15f5c0 msgr2=0x7fa0ec15f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 --2- 172.21.15.46:0/2457667869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15f5c0 0x7fa0ec15f9d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0ced20 tx=0x7fa0e401caf0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 -- 172.21.15.46:0/2457667869 shutdown_connections 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 --2- 172.21.15.46:0/2457667869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15f5c0 0x7fa0ec15f9d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 --2- 172.21.15.46:0/2457667869 >> v2:172.21.15.99:3300/0 conn(0x7fa0ec0d1140 0x7fa0ec0b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 -- 172.21.15.46:0/2457667869 >> 172.21.15.46:0/2457667869 conn(0x7fa0ec0b41f0 msgr2=0x7fa0ec0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 -- 172.21.15.46:0/2457667869 shutdown_connections 2024-09-17T03:30:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.561+0000 7fa0f1000640 1 -- 172.21.15.46:0/2457667869 wait complete. 2024-09-17T03:30:20.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 Processor -- start 2024-09-17T03:30:20.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 -- start start 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 0x7fa0ec09aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ec0d1140 0x7fa0ec1f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0ec15f5c0 0x7fa0ec1fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec168ff0 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec168e70 con 0x7fa0ec0d1140 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0f1000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec169170 con 0x7fa0ec15f5c0 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 0x7fa0ec09aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 0x7fa0ec09aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45510/0 (socket says 172.21.15.46:45510) 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 -- 172.21.15.46:0/3682694928 learned_addr learned my addr 172.21.15.46:0/3682694928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eaa00640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ec0d1140 0x7fa0ec1f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0ebe00640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0ec15f5c0 0x7fa0ec1fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 -- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0ec15f5c0 msgr2=0x7fa0ec1fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0ec15f5c0 0x7fa0ec1fa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 -- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ec0d1140 msgr2=0x7fa0ec1f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ec0d1140 0x7fa0ec1f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eb400640 1 -- 172.21.15.46:0/3682694928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec09e8c0 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.565+0000 7fa0eaa00640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ec0d1140 0x7fa0ec1f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0eb400640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 0x7fa0ec09aa20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00d720 tx=0x7fa0dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc017070 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0dc009d20 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec09eb60 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc005280 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec09b310 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec0b0730 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fa0dc005420 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0e3400640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa0cc074430 0x7fa0cc076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fa0dc093150 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.569+0000 7fa0eaa00640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa0cc074430 0x7fa0cc076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.573+0000 7fa0eaa00640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa0cc074430 0x7fa0cc076940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0cfe80 tx=0x7fa0d8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.573+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa0dc05f2d0 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0ec070d60 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0e3400640 1 -- 172.21.15.46:0/3682694928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5241 (secure 0 0 0) 0x7fa0dc063d00 con 0x7fa0ec06dbc0 2024-09-17T03:30:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:20.784 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":55,"btime":"2024-09-17T03:30:20:367227+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:30:16.771417+0000","modified":"2024-09-17T03:30:19.787080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246888":{"gid":246888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2776262749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2776262749},{"type":"v1","addr":"172.21.15.46:6835","nonce":2776262749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246888,"qdb_cluster":[246888]},"id":6}]} 2024-09-17T03:30:20.785 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 55 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa0cc074430 msgr2=0x7fa0cc076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa0cc074430 0x7fa0cc076940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0cfe80 tx=0x7fa0d8009040 comp rx=0 tx=0).stop 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 msgr2=0x7fa0ec09aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 0x7fa0ec09aa20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00d720 tx=0x7fa0dc00dbe0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 shutdown_connections 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa0cc074430 0x7fa0cc076940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa0ec15f5c0 0x7fa0ec1fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ec0d1140 0x7fa0ec1f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 --2- 172.21.15.46:0/3682694928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbc0 0x7fa0ec09aa20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 >> 172.21.15.46:0/3682694928 conn(0x7fa0ec0b41f0 msgr2=0x7fa0ec0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 shutdown_connections 2024-09-17T03:30:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.785+0000 7fa0f1000640 1 -- 172.21.15.46:0/3682694928 wait complete. 2024-09-17T03:30:20.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:30:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.965+0000 7f88d4800640 1 -- 172.21.15.46:0/1948506902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc157030 msgr2=0x7f88cc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.965+0000 7f88d4800640 1 --2- 172.21.15.46:0/1948506902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc157030 0x7f88cc0b5a80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009840 tx=0x7f88c001c790 comp rx=0 tx=0).stop 2024-09-17T03:30:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.965+0000 7f88d4800640 1 -- 172.21.15.46:0/1948506902 shutdown_connections 2024-09-17T03:30:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.965+0000 7f88d4800640 1 --2- 172.21.15.46:0/1948506902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc157030 0x7f88cc0b5a80 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.965+0000 7f88d4800640 1 -- 172.21.15.46:0/1948506902 >> 172.21.15.46:0/1948506902 conn(0x7f88cc0708c0 msgr2=0x7f88cc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.965+0000 7f88d4800640 1 -- 172.21.15.46:0/1948506902 shutdown_connections 2024-09-17T03:30:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 -- 172.21.15.46:0/1948506902 wait complete. 2024-09-17T03:30:20.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 Processor -- start 2024-09-17T03:30:20.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 -- start start 2024-09-17T03:30:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f88cc157030 0x7f88cc150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88cc160d50 0x7f88cc149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 0x7f88cc1fc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc16b830 con 0x7f88cc14a720 2024-09-17T03:30:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc16b6b0 con 0x7f88cc160d50 2024-09-17T03:30:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.969+0000 7f88d4800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc16b9b0 con 0x7f88cc157030 2024-09-17T03:30:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 0x7f88cc1fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 0x7f88cc1fc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45540/0 (socket says 172.21.15.46:45540) 2024-09-17T03:30:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 -- 172.21.15.46:0/468452241 learned_addr learned my addr 172.21.15.46:0/468452241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d1e00640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f88cc157030 0x7f88cc150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d1400640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88cc160d50 0x7f88cc149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 -- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f88cc157030 msgr2=0x7f88cc150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f88cc157030 0x7f88cc150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 -- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88cc160d50 msgr2=0x7f88cc149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88cc160d50 0x7f88cc149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 -- 172.21.15.46:0/468452241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc14a530 con 0x7f88cc14a720 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d2800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 0x7f88cc1fc030 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f88c800cb40 tx=0x7f88c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8016020 con 0x7f88cc14a720 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c800ab60 con 0x7f88cc14a720 2024-09-17T03:30:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc156390 con 0x7f88cc14a720 2024-09-17T03:30:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8012420 con 0x7f88cc14a720 2024-09-17T03:30:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc152cd0 con 0x7f88cc14a720 2024-09-17T03:30:20.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc156600 con 0x7f88cc14a720 2024-09-17T03:30:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.973+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f88c8019020 con 0x7f88cc14a720 2024-09-17T03:30:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.977+0000 7f88c6000640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f88b0074430 0x7f88b0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.977+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f88c80933a0 con 0x7f88cc14a720 2024-09-17T03:30:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.977+0000 7f88d1e00640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f88b0074430 0x7f88b0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.977+0000 7f88d1e00640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f88b0074430 0x7f88b0076940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f88c00af000 tx=0x7f88c00b2040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:20.981+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f88c8020020 con 0x7f88cc14a720 2024-09-17T03:30:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88cc0d2260 con 0x7f88cc14a720 2024-09-17T03:30:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88c6000640 1 -- 172.21.15.46:0/468452241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16684 (secure 0 0 0) 0x7f88c805fa60 con 0x7f88cc14a720 2024-09-17T03:30:21.157 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:21.157 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:30:18.353581+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:30:14.515457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T03:30:15.740742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:30:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f88b0074430 msgr2=0x7f88b0076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f88b0074430 0x7f88b0076940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f88c00af000 tx=0x7f88c00b2040 comp rx=0 tx=0).stop 2024-09-17T03:30:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 msgr2=0x7f88cc1fc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 0x7f88cc1fc030 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f88c800cb40 tx=0x7f88c800cc30 comp rx=0 tx=0).stop 2024-09-17T03:30:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 shutdown_connections 2024-09-17T03:30:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f88b0074430 0x7f88b0076940 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88cc14a720 0x7f88cc1fc030 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88cc160d50 0x7f88cc149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 --2- 172.21.15.46:0/468452241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f88cc157030 0x7f88cc150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 >> 172.21.15.46:0/468452241 conn(0x7f88cc0708c0 msgr2=0x7f88cc14e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 shutdown_connections 2024-09-17T03:30:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.157+0000 7f88d4800640 1 -- 172.21.15.46:0/468452241 wait complete. 2024-09-17T03:30:21.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:30:21.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:30:21.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.337+0000 7fbb89600640 1 -- 172.21.15.46:0/2185104766 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84074b20 msgr2=0x7fbb8415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.337+0000 7fbb89600640 1 --2- 172.21.15.46:0/2185104766 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84074b20 0x7fbb8415cf90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009c80 tx=0x7fbb7c01cda0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.337+0000 7fbb89600640 1 -- 172.21.15.46:0/2185104766 shutdown_connections 2024-09-17T03:30:21.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.337+0000 7fbb89600640 1 --2- 172.21.15.46:0/2185104766 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84074b20 0x7fbb8415cf90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.337+0000 7fbb89600640 1 --2- 172.21.15.46:0/2185104766 >> v2:172.21.15.99:3300/0 conn(0x7fbb84157030 0x7fbb840b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.337+0000 7fbb89600640 1 -- 172.21.15.46:0/2185104766 >> 172.21.15.46:0/2185104766 conn(0x7fbb840708c0 msgr2=0x7fbb84070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 -- 172.21.15.46:0/2185104766 shutdown_connections 2024-09-17T03:30:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 -- 172.21.15.46:0/2185104766 wait complete. 2024-09-17T03:30:21.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 Processor -- start 2024-09-17T03:30:21.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 -- start start 2024-09-17T03:30:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb84157030 0x7fbb84140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84160d50 0x7fbb840b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 0x7fbb8414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8416a740 con 0x7fbb841540f0 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8416a5c0 con 0x7fbb84157030 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.341+0000 7fbb89600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8416a8c0 con 0x7fbb84160d50 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 0x7fbb8414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 0x7fbb8414e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45548/0 (socket says 172.21.15.46:45548) 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 -- 172.21.15.46:0/1023311865 learned_addr learned my addr 172.21.15.46:0/1023311865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb82a00640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb84157030 0x7fbb84140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb7be00640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84160d50 0x7fbb840b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 -- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84160d50 msgr2=0x7fbb840b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84160d50 0x7fbb840b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 -- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb84157030 msgr2=0x7fbb84140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb84157030 0x7fbb84140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 -- 172.21.15.46:0/1023311865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb840b2270 con 0x7fbb841540f0 2024-09-17T03:30:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb82a00640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb84157030 0x7fbb84140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb83400640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 0x7fbb8414e7c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c01c810 tx=0x7fbb7c005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0b2040 con 0x7fbb841540f0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb7c0a5a70 con 0x7fbb841540f0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb84153e10 con 0x7fbb841540f0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c004240 con 0x7fbb841540f0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb840b3cb0 con 0x7fbb841540f0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84153b60 con 0x7fbb841540f0 2024-09-17T03:30:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.345+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fbb7c0bd030 con 0x7fbb841540f0 2024-09-17T03:30:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.349+0000 7fbb7aa00640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb60074400 0x7fbb60076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.349+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fbb7c12cf10 con 0x7fbb841540f0 2024-09-17T03:30:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.349+0000 7fbb82a00640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb60074400 0x7fbb60076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.349+0000 7fbb82a00640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb60074400 0x7fbb60076910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009b90 tx=0x7fbb740091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.353+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb7c0c4020 con 0x7fbb841540f0 2024-09-17T03:30:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.525+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb8406b540 con 0x7fbb60074400 2024-09-17T03:30:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.529+0000 7fbb7aa00640 1 -- 172.21.15.46:0/1023311865 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248208 (secure 0 0 0) 0x7fbb74011be0 con 0x7fbb60074400 2024-09-17T03:30:21.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:30:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb60074400 msgr2=0x7fbb60076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb60074400 0x7fbb60076910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009b90 tx=0x7fbb740091e0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 msgr2=0x7fbb8414e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 0x7fbb8414e7c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c01c810 tx=0x7fbb7c005c20 comp rx=0 tx=0).stop 2024-09-17T03:30:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 shutdown_connections 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb60074400 0x7fbb60076910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb841540f0 0x7fbb8414e7c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb84160d50 0x7fbb840b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 --2- 172.21.15.46:0/1023311865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb84157030 0x7fbb84140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 >> 172.21.15.46:0/1023311865 conn(0x7fbb840708c0 msgr2=0x7fbb84151ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 shutdown_connections 2024-09-17T03:30:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.533+0000 7fbb89600640 1 -- 172.21.15.46:0/1023311865 wait complete. 2024-09-17T03:30:21.554 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T03:30:21.390774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":114,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":527768,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":518259,"kb_avail":754446952,"statfs":{"total":773094113280,"available":772553678848,"internally_reserved":0,"allocated":9543680,"data_stored":4734567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":530697776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001148"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356490+0000","last_change":"2024-09-17T03:30:16.356490+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356490+0000","last_undegraded":"2024-09-17T03:30:16.356490+0000","last_fullsized":"2024-09-17T03:30:16.356490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041228+0000","last_change":"2024-09-17T03:30:17.370486+0000","last_active":"2024-09-17T03:30:19.041228+0000","last_peered":"2024-09-17T03:30:19.041228+0000","last_clean":"2024-09-17T03:30:19.041228+0000","last_became_active":"2024-09-17T03:30:17.370107+0000","last_became_peered":"2024-09-17T03:30:17.370107+0000","last_unstale":"2024-09-17T03:30:19.041228+0000","last_undegraded":"2024-09-17T03:30:19.041228+0000","last_fullsized":"2024-09-17T03:30:19.041228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:27.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361223+0000","last_change":"2024-09-17T03:30:16.361223+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361223+0000","last_undegraded":"2024-09-17T03:30:16.361223+0000","last_fullsized":"2024-09-17T03:30:16.361223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350657+0000","last_change":"2024-09-17T03:30:16.350657+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350657+0000","last_undegraded":"2024-09-17T03:30:16.350657+0000","last_fullsized":"2024-09-17T03:30:16.350657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362688+0000","last_change":"2024-09-17T03:30:16.362688+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362688+0000","last_undegraded":"2024-09-17T03:30:16.362688+0000","last_fullsized":"2024-09-17T03:30:16.362688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047927+0000","last_change":"2024-09-17T03:30:17.388212+0000","last_active":"2024-09-17T03:30:19.047927+0000","last_peered":"2024-09-17T03:30:19.047927+0000","last_clean":"2024-09-17T03:30:19.047927+0000","last_became_active":"2024-09-17T03:30:17.387941+0000","last_became_peered":"2024-09-17T03:30:17.387941+0000","last_unstale":"2024-09-17T03:30:19.047927+0000","last_undegraded":"2024-09-17T03:30:19.047927+0000","last_fullsized":"2024-09-17T03:30:19.047927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:30.231182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356272+0000","last_change":"2024-09-17T03:30:16.356272+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356272+0000","last_undegraded":"2024-09-17T03:30:16.356272+0000","last_fullsized":"2024-09-17T03:30:16.356272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047750+0000","last_change":"2024-09-17T03:30:17.383305+0000","last_active":"2024-09-17T03:30:19.047750+0000","last_peered":"2024-09-17T03:30:19.047750+0000","last_clean":"2024-09-17T03:30:19.047750+0000","last_became_active":"2024-09-17T03:30:17.382995+0000","last_became_peered":"2024-09-17T03:30:17.382995+0000","last_unstale":"2024-09-17T03:30:19.047750+0000","last_undegraded":"2024-09-17T03:30:19.047750+0000","last_fullsized":"2024-09-17T03:30:19.047750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:03.641896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047928+0000","last_change":"2024-09-17T03:30:17.374521+0000","last_active":"2024-09-17T03:30:19.047928+0000","last_peered":"2024-09-17T03:30:19.047928+0000","last_clean":"2024-09-17T03:30:19.047928+0000","last_became_active":"2024-09-17T03:30:17.374185+0000","last_became_peered":"2024-09-17T03:30:17.374185+0000","last_unstale":"2024-09-17T03:30:19.047928+0000","last_undegraded":"2024-09-17T03:30:19.047928+0000","last_fullsized":"2024-09-17T03:30:19.047928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:57.551527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353169+0000","last_change":"2024-09-17T03:30:16.353169+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353169+0000","last_undegraded":"2024-09-17T03:30:16.353169+0000","last_fullsized":"2024-09-17T03:30:16.353169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040669+0000","last_change":"2024-09-17T03:30:17.373313+0000","last_active":"2024-09-17T03:30:19.040669+0000","last_peered":"2024-09-17T03:30:19.040669+0000","last_clean":"2024-09-17T03:30:19.040669+0000","last_became_active":"2024-09-17T03:30:17.372950+0000","last_became_peered":"2024-09-17T03:30:17.372950+0000","last_unstale":"2024-09-17T03:30:19.040669+0000","last_undegraded":"2024-09-17T03:30:19.040669+0000","last_fullsized":"2024-09-17T03:30:19.040669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356094+0000","last_change":"2024-09-17T03:30:16.356094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356094+0000","last_undegraded":"2024-09-17T03:30:16.356094+0000","last_fullsized":"2024-09-17T03:30:16.356094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363504+0000","last_change":"2024-09-17T03:30:16.363504+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363504+0000","last_undegraded":"2024-09-17T03:30:16.363504+0000","last_fullsized":"2024-09-17T03:30:16.363504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350224+0000","last_change":"2024-09-17T03:30:16.350224+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350224+0000","last_undegraded":"2024-09-17T03:30:16.350224+0000","last_fullsized":"2024-09-17T03:30:16.350224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348144+0000","last_change":"2024-09-17T03:30:16.348144+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348144+0000","last_undegraded":"2024-09-17T03:30:16.348144+0000","last_fullsized":"2024-09-17T03:30:16.348144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.358745+0000","last_change":"2024-09-17T03:30:16.358745+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.358745+0000","last_undegraded":"2024-09-17T03:30:16.358745+0000","last_fullsized":"2024-09-17T03:30:16.358745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.364506+0000","last_change":"2024-09-17T03:30:16.364506+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.364506+0000","last_undegraded":"2024-09-17T03:30:16.364506+0000","last_fullsized":"2024-09-17T03:30:16.364506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.351260+0000","last_change":"2024-09-17T03:30:16.351260+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.351260+0000","last_undegraded":"2024-09-17T03:30:16.351260+0000","last_fullsized":"2024-09-17T03:30:16.351260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346459+0000","last_change":"2024-09-17T03:30:16.346459+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346459+0000","last_undegraded":"2024-09-17T03:30:16.346459+0000","last_fullsized":"2024-09-17T03:30:16.346459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350038+0000","last_change":"2024-09-17T03:30:16.350038+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350038+0000","last_undegraded":"2024-09-17T03:30:16.350038+0000","last_fullsized":"2024-09-17T03:30:16.350038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349235+0000","last_change":"2024-09-17T03:30:16.349235+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349235+0000","last_undegraded":"2024-09-17T03:30:16.349235+0000","last_fullsized":"2024-09-17T03:30:16.349235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040270+0000","last_change":"2024-09-17T03:30:17.374044+0000","last_active":"2024-09-17T03:30:19.040270+0000","last_peered":"2024-09-17T03:30:19.040270+0000","last_clean":"2024-09-17T03:30:19.040270+0000","last_became_active":"2024-09-17T03:30:17.373700+0000","last_became_peered":"2024-09-17T03:30:17.373700+0000","last_unstale":"2024-09-17T03:30:19.040270+0000","last_undegraded":"2024-09-17T03:30:19.040270+0000","last_fullsized":"2024-09-17T03:30:19.040270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:04.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354041+0000","last_change":"2024-09-17T03:30:16.354041+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354041+0000","last_undegraded":"2024-09-17T03:30:16.354041+0000","last_fullsized":"2024-09-17T03:30:16.354041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349327+0000","last_change":"2024-09-17T03:30:16.349327+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349327+0000","last_undegraded":"2024-09-17T03:30:16.349327+0000","last_fullsized":"2024-09-17T03:30:16.349327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041180+0000","last_change":"2024-09-17T03:30:17.374952+0000","last_active":"2024-09-17T03:30:19.041180+0000","last_peered":"2024-09-17T03:30:19.041180+0000","last_clean":"2024-09-17T03:30:19.041180+0000","last_became_active":"2024-09-17T03:30:17.374579+0000","last_became_peered":"2024-09-17T03:30:17.374579+0000","last_unstale":"2024-09-17T03:30:19.041180+0000","last_undegraded":"2024-09-17T03:30:19.041180+0000","last_fullsized":"2024-09-17T03:30:19.041180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.728454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361780+0000","last_change":"2024-09-17T03:30:16.361780+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361780+0000","last_undegraded":"2024-09-17T03:30:16.361780+0000","last_fullsized":"2024-09-17T03:30:16.361780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.367168+0000","last_change":"2024-09-17T03:30:16.367168+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.367168+0000","last_undegraded":"2024-09-17T03:30:16.367168+0000","last_fullsized":"2024-09-17T03:30:16.367168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.355538+0000","last_change":"2024-09-17T03:30:16.355538+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.355538+0000","last_undegraded":"2024-09-17T03:30:16.355538+0000","last_fullsized":"2024-09-17T03:30:16.355538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040169+0000","last_change":"2024-09-17T03:30:17.378897+0000","last_active":"2024-09-17T03:30:19.040169+0000","last_peered":"2024-09-17T03:30:19.040169+0000","last_clean":"2024-09-17T03:30:19.040169+0000","last_became_active":"2024-09-17T03:30:17.378212+0000","last_became_peered":"2024-09-17T03:30:17.378212+0000","last_unstale":"2024-09-17T03:30:19.040169+0000","last_undegraded":"2024-09-17T03:30:19.040169+0000","last_fullsized":"2024-09-17T03:30:19.040169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:13.896559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040290+0000","last_change":"2024-09-17T03:30:17.382864+0000","last_active":"2024-09-17T03:30:19.040290+0000","last_peered":"2024-09-17T03:30:19.040290+0000","last_clean":"2024-09-17T03:30:19.040290+0000","last_became_active":"2024-09-17T03:30:17.381934+0000","last_became_peered":"2024-09-17T03:30:17.381934+0000","last_unstale":"2024-09-17T03:30:19.040290+0000","last_undegraded":"2024-09-17T03:30:19.040290+0000","last_fullsized":"2024-09-17T03:30:19.040290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:12.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360461+0000","last_change":"2024-09-17T03:30:16.360461+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360461+0000","last_undegraded":"2024-09-17T03:30:16.360461+0000","last_fullsized":"2024-09-17T03:30:16.360461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353448+0000","last_change":"2024-09-17T03:30:16.353448+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353448+0000","last_undegraded":"2024-09-17T03:30:16.353448+0000","last_fullsized":"2024-09-17T03:30:16.353448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363769+0000","last_change":"2024-09-17T03:30:16.363769+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363769+0000","last_undegraded":"2024-09-17T03:30:16.363769+0000","last_fullsized":"2024-09-17T03:30:16.363769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371726+0000","last_change":"2024-09-17T03:30:16.366033+0000","last_active":"2024-09-17T03:30:16.371726+0000","last_peered":"2024-09-17T03:30:16.371726+0000","last_clean":"2024-09-17T03:30:16.371726+0000","last_became_active":"2024-09-17T03:30:16.365643+0000","last_became_peered":"2024-09-17T03:30:16.365643+0000","last_unstale":"2024-09-17T03:30:16.371726+0000","last_undegraded":"2024-09-17T03:30:16.371726+0000","last_fullsized":"2024-09-17T03:30:16.371726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:05.152896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354849+0000","last_change":"2024-09-17T03:30:16.354849+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354849+0000","last_undegraded":"2024-09-17T03:30:16.354849+0000","last_fullsized":"2024-09-17T03:30:16.354849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375348+0000","last_change":"2024-09-17T03:30:16.374265+0000","last_active":"2024-09-17T03:30:16.375348+0000","last_peered":"2024-09-17T03:30:16.375348+0000","last_clean":"2024-09-17T03:30:16.375348+0000","last_became_active":"2024-09-17T03:30:16.372671+0000","last_became_peered":"2024-09-17T03:30:16.372671+0000","last_unstale":"2024-09-17T03:30:16.375348+0000","last_undegraded":"2024-09-17T03:30:16.375348+0000","last_fullsized":"2024-09-17T03:30:16.375348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:20.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354990+0000","last_change":"2024-09-17T03:30:16.354990+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354990+0000","last_undegraded":"2024-09-17T03:30:16.354990+0000","last_fullsized":"2024-09-17T03:30:16.354990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.359637+0000","last_change":"2024-09-17T03:30:16.355580+0000","last_active":"2024-09-17T03:30:16.359637+0000","last_peered":"2024-09-17T03:30:16.359637+0000","last_clean":"2024-09-17T03:30:16.359637+0000","last_became_active":"2024-09-17T03:30:16.355240+0000","last_became_peered":"2024-09-17T03:30:16.355240+0000","last_unstale":"2024-09-17T03:30:16.359637+0000","last_undegraded":"2024-09-17T03:30:16.359637+0000","last_fullsized":"2024-09-17T03:30:16.359637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:31.022404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360687+0000","last_change":"2024-09-17T03:30:16.360687+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360687+0000","last_undegraded":"2024-09-17T03:30:16.360687+0000","last_fullsized":"2024-09-17T03:30:16.360687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361157+0000","last_change":"2024-09-17T03:30:16.357957+0000","last_active":"2024-09-17T03:30:16.361157+0000","last_peered":"2024-09-17T03:30:16.361157+0000","last_clean":"2024-09-17T03:30:16.361157+0000","last_became_active":"2024-09-17T03:30:16.357597+0000","last_became_peered":"2024-09-17T03:30:16.357597+0000","last_unstale":"2024-09-17T03:30:16.361157+0000","last_undegraded":"2024-09-17T03:30:16.361157+0000","last_fullsized":"2024-09-17T03:30:16.361157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:25.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040526+0000","last_change":"2024-09-17T03:30:17.382905+0000","last_active":"2024-09-17T03:30:19.040526+0000","last_peered":"2024-09-17T03:30:19.040526+0000","last_clean":"2024-09-17T03:30:19.040526+0000","last_became_active":"2024-09-17T03:30:17.382343+0000","last_became_peered":"2024-09-17T03:30:17.382343+0000","last_unstale":"2024-09-17T03:30:19.040526+0000","last_undegraded":"2024-09-17T03:30:19.040526+0000","last_fullsized":"2024-09-17T03:30:19.040526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:08.921865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376896+0000","last_change":"2024-09-17T03:30:16.375643+0000","last_active":"2024-09-17T03:30:16.376896+0000","last_peered":"2024-09-17T03:30:16.376896+0000","last_clean":"2024-09-17T03:30:16.376896+0000","last_became_active":"2024-09-17T03:30:16.375378+0000","last_became_peered":"2024-09-17T03:30:16.375378+0000","last_unstale":"2024-09-17T03:30:16.376896+0000","last_undegraded":"2024-09-17T03:30:16.376896+0000","last_fullsized":"2024-09-17T03:30:16.376896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.037881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362120+0000","last_change":"2024-09-17T03:30:16.362120+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362120+0000","last_undegraded":"2024-09-17T03:30:16.362120+0000","last_fullsized":"2024-09-17T03:30:16.362120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375682+0000","last_change":"2024-09-17T03:30:16.374780+0000","last_active":"2024-09-17T03:30:16.375682+0000","last_peered":"2024-09-17T03:30:16.375682+0000","last_clean":"2024-09-17T03:30:16.375682+0000","last_became_active":"2024-09-17T03:30:16.374312+0000","last_became_peered":"2024-09-17T03:30:16.374312+0000","last_unstale":"2024-09-17T03:30:16.375682+0000","last_undegraded":"2024-09-17T03:30:16.375682+0000","last_fullsized":"2024-09-17T03:30:16.375682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.479057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362017+0000","last_change":"2024-09-17T03:30:16.362017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362017+0000","last_undegraded":"2024-09-17T03:30:16.362017+0000","last_fullsized":"2024-09-17T03:30:16.362017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363741+0000","last_change":"2024-09-17T03:30:16.360075+0000","last_active":"2024-09-17T03:30:16.363741+0000","last_peered":"2024-09-17T03:30:16.363741+0000","last_clean":"2024-09-17T03:30:16.363741+0000","last_became_active":"2024-09-17T03:30:16.359771+0000","last_became_peered":"2024-09-17T03:30:16.359771+0000","last_unstale":"2024-09-17T03:30:16.363741+0000","last_undegraded":"2024-09-17T03:30:16.363741+0000","last_fullsized":"2024-09-17T03:30:16.363741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:01.924364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.347497+0000","last_change":"2024-09-17T03:30:16.347497+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.347497+0000","last_undegraded":"2024-09-17T03:30:16.347497+0000","last_fullsized":"2024-09-17T03:30:16.347497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039799+0000","last_change":"2024-09-17T03:30:16.365783+0000","last_active":"2024-09-17T03:30:19.039799+0000","last_peered":"2024-09-17T03:30:19.039799+0000","last_clean":"2024-09-17T03:30:19.039799+0000","last_became_active":"2024-09-17T03:30:16.365439+0000","last_became_peered":"2024-09-17T03:30:16.365439+0000","last_unstale":"2024-09-17T03:30:19.039799+0000","last_undegraded":"2024-09-17T03:30:19.039799+0000","last_fullsized":"2024-09-17T03:30:19.039799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:48.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348017+0000","last_change":"2024-09-17T03:30:16.348017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348017+0000","last_undegraded":"2024-09-17T03:30:16.348017+0000","last_fullsized":"2024-09-17T03:30:16.348017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371783+0000","last_change":"2024-09-17T03:30:16.369999+0000","last_active":"2024-09-17T03:30:16.371783+0000","last_peered":"2024-09-17T03:30:16.371783+0000","last_clean":"2024-09-17T03:30:16.371783+0000","last_became_active":"2024-09-17T03:30:16.368392+0000","last_became_peered":"2024-09-17T03:30:16.368392+0000","last_unstale":"2024-09-17T03:30:16.371783+0000","last_undegraded":"2024-09-17T03:30:16.371783+0000","last_fullsized":"2024-09-17T03:30:16.371783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:10.071007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047957+0000","last_change":"2024-09-17T03:30:17.380149+0000","last_active":"2024-09-17T03:30:19.047957+0000","last_peered":"2024-09-17T03:30:19.047957+0000","last_clean":"2024-09-17T03:30:19.047957+0000","last_became_active":"2024-09-17T03:30:17.379829+0000","last_became_peered":"2024-09-17T03:30:17.379829+0000","last_unstale":"2024-09-17T03:30:19.047957+0000","last_undegraded":"2024-09-17T03:30:19.047957+0000","last_fullsized":"2024-09-17T03:30:19.047957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:17.605539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.374805+0000","last_change":"2024-09-17T03:30:16.373477+0000","last_active":"2024-09-17T03:30:16.374805+0000","last_peered":"2024-09-17T03:30:16.374805+0000","last_clean":"2024-09-17T03:30:16.374805+0000","last_became_active":"2024-09-17T03:30:16.373060+0000","last_became_peered":"2024-09-17T03:30:16.373060+0000","last_unstale":"2024-09-17T03:30:16.374805+0000","last_undegraded":"2024-09-17T03:30:16.374805+0000","last_fullsized":"2024-09-17T03:30:16.374805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:06.209920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346582+0000","last_change":"2024-09-17T03:30:16.346582+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346582+0000","last_undegraded":"2024-09-17T03:30:16.346582+0000","last_fullsized":"2024-09-17T03:30:16.346582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376938+0000","last_change":"2024-09-17T03:30:16.376370+0000","last_active":"2024-09-17T03:30:16.376938+0000","last_peered":"2024-09-17T03:30:16.376938+0000","last_clean":"2024-09-17T03:30:16.376938+0000","last_became_active":"2024-09-17T03:30:16.376027+0000","last_became_peered":"2024-09-17T03:30:16.376027+0000","last_unstale":"2024-09-17T03:30:16.376938+0000","last_undegraded":"2024-09-17T03:30:16.376938+0000","last_fullsized":"2024-09-17T03:30:16.376938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:51.451736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366652+0000","last_change":"2024-09-17T03:30:16.366652+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366652+0000","last_undegraded":"2024-09-17T03:30:16.366652+0000","last_fullsized":"2024-09-17T03:30:16.366652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047718+0000","last_change":"2024-09-17T03:30:16.370573+0000","last_active":"2024-09-17T03:30:19.047718+0000","last_peered":"2024-09-17T03:30:19.047718+0000","last_clean":"2024-09-17T03:30:19.047718+0000","last_became_active":"2024-09-17T03:30:16.369166+0000","last_became_peered":"2024-09-17T03:30:16.369166+0000","last_unstale":"2024-09-17T03:30:19.047718+0000","last_undegraded":"2024-09-17T03:30:19.047718+0000","last_fullsized":"2024-09-17T03:30:19.047718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:23.751200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346578+0000","last_change":"2024-09-17T03:30:16.346578+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346578+0000","last_undegraded":"2024-09-17T03:30:16.346578+0000","last_fullsized":"2024-09-17T03:30:16.346578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040259+0000","last_change":"2024-09-17T03:30:16.367290+0000","last_active":"2024-09-17T03:30:19.040259+0000","last_peered":"2024-09-17T03:30:19.040259+0000","last_clean":"2024-09-17T03:30:19.040259+0000","last_became_active":"2024-09-17T03:30:16.365363+0000","last_became_peered":"2024-09-17T03:30:16.365363+0000","last_unstale":"2024-09-17T03:30:19.040259+0000","last_undegraded":"2024-09-17T03:30:19.040259+0000","last_fullsized":"2024-09-17T03:30:19.040259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:46.817707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363020+0000","last_change":"2024-09-17T03:30:16.363020+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363020+0000","last_undegraded":"2024-09-17T03:30:16.363020+0000","last_fullsized":"2024-09-17T03:30:16.363020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371240+0000","last_change":"2024-09-17T03:30:16.369373+0000","last_active":"2024-09-17T03:30:16.371240+0000","last_peered":"2024-09-17T03:30:16.371240+0000","last_clean":"2024-09-17T03:30:16.371240+0000","last_became_active":"2024-09-17T03:30:16.369082+0000","last_became_peered":"2024-09-17T03:30:16.369082+0000","last_unstale":"2024-09-17T03:30:16.371240+0000","last_undegraded":"2024-09-17T03:30:16.371240+0000","last_fullsized":"2024-09-17T03:30:16.371240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:46.125971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039947+0000","last_change":"2024-09-17T03:30:17.382255+0000","last_active":"2024-09-17T03:30:19.039947+0000","last_peered":"2024-09-17T03:30:19.039947+0000","last_clean":"2024-09-17T03:30:19.039947+0000","last_became_active":"2024-09-17T03:30:17.381901+0000","last_became_peered":"2024-09-17T03:30:17.381901+0000","last_unstale":"2024-09-17T03:30:19.039947+0000","last_undegraded":"2024-09-17T03:30:19.039947+0000","last_fullsized":"2024-09-17T03:30:19.039947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.620927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047671+0000","last_change":"2024-09-17T03:30:16.371647+0000","last_active":"2024-09-17T03:30:19.047671+0000","last_peered":"2024-09-17T03:30:19.047671+0000","last_clean":"2024-09-17T03:30:19.047671+0000","last_became_active":"2024-09-17T03:30:16.371336+0000","last_became_peered":"2024-09-17T03:30:16.371336+0000","last_unstale":"2024-09-17T03:30:19.047671+0000","last_undegraded":"2024-09-17T03:30:19.047671+0000","last_fullsized":"2024-09-17T03:30:19.047671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:19.546103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039882+0000","last_change":"2024-09-17T03:30:17.372667+0000","last_active":"2024-09-17T03:30:19.039882+0000","last_peered":"2024-09-17T03:30:19.039882+0000","last_clean":"2024-09-17T03:30:19.039882+0000","last_became_active":"2024-09-17T03:30:17.372310+0000","last_became_peered":"2024-09-17T03:30:17.372310+0000","last_unstale":"2024-09-17T03:30:19.039882+0000","last_undegraded":"2024-09-17T03:30:19.039882+0000","last_fullsized":"2024-09-17T03:30:19.039882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:46.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363382+0000","last_change":"2024-09-17T03:30:16.359214+0000","last_active":"2024-09-17T03:30:16.363382+0000","last_peered":"2024-09-17T03:30:16.363382+0000","last_clean":"2024-09-17T03:30:16.363382+0000","last_became_active":"2024-09-17T03:30:16.358883+0000","last_became_peered":"2024-09-17T03:30:16.358883+0000","last_unstale":"2024-09-17T03:30:16.363382+0000","last_undegraded":"2024-09-17T03:30:16.363382+0000","last_fullsized":"2024-09-17T03:30:16.363382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:47.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361307+0000","last_change":"2024-09-17T03:30:16.361307+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361307+0000","last_undegraded":"2024-09-17T03:30:16.361307+0000","last_fullsized":"2024-09-17T03:30:16.361307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371756+0000","last_change":"2024-09-17T03:30:16.369933+0000","last_active":"2024-09-17T03:30:16.371756+0000","last_peered":"2024-09-17T03:30:16.371756+0000","last_clean":"2024-09-17T03:30:16.371756+0000","last_became_active":"2024-09-17T03:30:16.369307+0000","last_became_peered":"2024-09-17T03:30:16.369307+0000","last_unstale":"2024-09-17T03:30:16.371756+0000","last_undegraded":"2024-09-17T03:30:16.371756+0000","last_fullsized":"2024-09-17T03:30:16.371756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:29.392547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354255+0000","last_change":"2024-09-17T03:30:16.354255+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354255+0000","last_undegraded":"2024-09-17T03:30:16.354255+0000","last_fullsized":"2024-09-17T03:30:16.354255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367756+0000","last_change":"2024-09-17T03:30:16.364256+0000","last_active":"2024-09-17T03:30:16.367756+0000","last_peered":"2024-09-17T03:30:16.367756+0000","last_clean":"2024-09-17T03:30:16.367756+0000","last_became_active":"2024-09-17T03:30:16.363880+0000","last_became_peered":"2024-09-17T03:30:16.363880+0000","last_unstale":"2024-09-17T03:30:16.367756+0000","last_undegraded":"2024-09-17T03:30:16.367756+0000","last_fullsized":"2024-09-17T03:30:16.367756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366139+0000","last_change":"2024-09-17T03:30:16.366139+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366139+0000","last_undegraded":"2024-09-17T03:30:16.366139+0000","last_fullsized":"2024-09-17T03:30:16.366139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364390+0000","last_change":"2024-09-17T03:30:16.361309+0000","last_active":"2024-09-17T03:30:16.364390+0000","last_peered":"2024-09-17T03:30:16.364390+0000","last_clean":"2024-09-17T03:30:16.364390+0000","last_became_active":"2024-09-17T03:30:16.360955+0000","last_became_peered":"2024-09-17T03:30:16.360955+0000","last_unstale":"2024-09-17T03:30:16.364390+0000","last_undegraded":"2024-09-17T03:30:16.364390+0000","last_fullsized":"2024-09-17T03:30:16.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:33.036462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349551+0000","last_change":"2024-09-17T03:30:16.349551+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349551+0000","last_undegraded":"2024-09-17T03:30:16.349551+0000","last_fullsized":"2024-09-17T03:30:16.349551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.366789+0000","last_change":"2024-09-17T03:30:16.359453+0000","last_active":"2024-09-17T03:30:16.366789+0000","last_peered":"2024-09-17T03:30:16.366789+0000","last_clean":"2024-09-17T03:30:16.366789+0000","last_became_active":"2024-09-17T03:30:16.358930+0000","last_became_peered":"2024-09-17T03:30:16.358930+0000","last_unstale":"2024-09-17T03:30:16.366789+0000","last_undegraded":"2024-09-17T03:30:16.366789+0000","last_fullsized":"2024-09-17T03:30:16.366789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:34.925865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048154+0000","last_change":"2024-09-17T03:30:17.380496+0000","last_active":"2024-09-17T03:30:19.048154+0000","last_peered":"2024-09-17T03:30:19.048154+0000","last_clean":"2024-09-17T03:30:19.048154+0000","last_became_active":"2024-09-17T03:30:17.380195+0000","last_became_peered":"2024-09-17T03:30:17.380195+0000","last_unstale":"2024-09-17T03:30:19.048154+0000","last_undegraded":"2024-09-17T03:30:19.048154+0000","last_fullsized":"2024-09-17T03:30:19.048154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:32.240069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370053+0000","last_change":"2024-09-17T03:30:16.366802+0000","last_active":"2024-09-17T03:30:16.370053+0000","last_peered":"2024-09-17T03:30:16.370053+0000","last_clean":"2024-09-17T03:30:16.370053+0000","last_became_active":"2024-09-17T03:30:16.366497+0000","last_became_peered":"2024-09-17T03:30:16.366497+0000","last_unstale":"2024-09-17T03:30:16.370053+0000","last_undegraded":"2024-09-17T03:30:16.370053+0000","last_fullsized":"2024-09-17T03:30:16.370053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:31.288276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348588+0000","last_change":"2024-09-17T03:30:16.348588+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348588+0000","last_undegraded":"2024-09-17T03:30:16.348588+0000","last_fullsized":"2024-09-17T03:30:16.348588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367253+0000","last_change":"2024-09-17T03:30:16.362330+0000","last_active":"2024-09-17T03:30:16.367253+0000","last_peered":"2024-09-17T03:30:16.367253+0000","last_clean":"2024-09-17T03:30:16.367253+0000","last_became_active":"2024-09-17T03:30:16.361960+0000","last_became_peered":"2024-09-17T03:30:16.361960+0000","last_unstale":"2024-09-17T03:30:16.367253+0000","last_undegraded":"2024-09-17T03:30:16.367253+0000","last_fullsized":"2024-09-17T03:30:16.367253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:27.002346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047756+0000","last_change":"2024-09-17T03:30:17.373912+0000","last_active":"2024-09-17T03:30:19.047756+0000","last_peered":"2024-09-17T03:30:19.047756+0000","last_clean":"2024-09-17T03:30:19.047756+0000","last_became_active":"2024-09-17T03:30:17.373542+0000","last_became_peered":"2024-09-17T03:30:17.373542+0000","last_unstale":"2024-09-17T03:30:19.047756+0000","last_undegraded":"2024-09-17T03:30:19.047756+0000","last_fullsized":"2024-09-17T03:30:19.047756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:28.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361044+0000","last_change":"2024-09-17T03:30:16.354858+0000","last_active":"2024-09-17T03:30:16.361044+0000","last_peered":"2024-09-17T03:30:16.361044+0000","last_clean":"2024-09-17T03:30:16.361044+0000","last_became_active":"2024-09-17T03:30:16.354399+0000","last_became_peered":"2024-09-17T03:30:16.354399+0000","last_unstale":"2024-09-17T03:30:16.361044+0000","last_undegraded":"2024-09-17T03:30:16.361044+0000","last_fullsized":"2024-09-17T03:30:16.361044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:47.686970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363094+0000","last_change":"2024-09-17T03:30:16.363094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363094+0000","last_undegraded":"2024-09-17T03:30:16.363094+0000","last_fullsized":"2024-09-17T03:30:16.363094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041218+0000","last_change":"2024-09-17T03:30:16.368986+0000","last_active":"2024-09-17T03:30:19.041218+0000","last_peered":"2024-09-17T03:30:19.041218+0000","last_clean":"2024-09-17T03:30:19.041218+0000","last_became_active":"2024-09-17T03:30:16.368638+0000","last_became_peered":"2024-09-17T03:30:16.368638+0000","last_unstale":"2024-09-17T03:30:19.041218+0000","last_undegraded":"2024-09-17T03:30:19.041218+0000","last_fullsized":"2024-09-17T03:30:19.041218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:04.104352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360154+0000","last_change":"2024-09-17T03:30:16.360154+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360154+0000","last_undegraded":"2024-09-17T03:30:16.360154+0000","last_fullsized":"2024-09-17T03:30:16.360154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041203+0000","last_change":"2024-09-17T03:30:16.371245+0000","last_active":"2024-09-17T03:30:19.041203+0000","last_peered":"2024-09-17T03:30:19.041203+0000","last_clean":"2024-09-17T03:30:19.041203+0000","last_became_active":"2024-09-17T03:30:16.370661+0000","last_became_peered":"2024-09-17T03:30:16.370661+0000","last_unstale":"2024-09-17T03:30:19.041203+0000","last_undegraded":"2024-09-17T03:30:19.041203+0000","last_fullsized":"2024-09-17T03:30:19.041203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:47.753727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353740+0000","last_change":"2024-09-17T03:30:16.353740+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353740+0000","last_undegraded":"2024-09-17T03:30:16.353740+0000","last_fullsized":"2024-09-17T03:30:16.353740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370305+0000","last_change":"2024-09-17T03:30:16.366822+0000","last_active":"2024-09-17T03:30:16.370305+0000","last_peered":"2024-09-17T03:30:16.370305+0000","last_clean":"2024-09-17T03:30:16.370305+0000","last_became_active":"2024-09-17T03:30:16.364923+0000","last_became_peered":"2024-09-17T03:30:16.364923+0000","last_unstale":"2024-09-17T03:30:16.370305+0000","last_undegraded":"2024-09-17T03:30:16.370305+0000","last_fullsized":"2024-09-17T03:30:16.370305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:02.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363106+0000","last_change":"2024-09-17T03:30:16.363106+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363106+0000","last_undegraded":"2024-09-17T03:30:16.363106+0000","last_fullsized":"2024-09-17T03:30:16.363106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371866+0000","last_change":"2024-09-17T03:30:16.369886+0000","last_active":"2024-09-17T03:30:16.371866+0000","last_peered":"2024-09-17T03:30:16.371866+0000","last_clean":"2024-09-17T03:30:16.371866+0000","last_became_active":"2024-09-17T03:30:16.369399+0000","last_became_peered":"2024-09-17T03:30:16.369399+0000","last_unstale":"2024-09-17T03:30:16.371866+0000","last_undegraded":"2024-09-17T03:30:16.371866+0000","last_fullsized":"2024-09-17T03:30:16.371866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:30.365530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047759+0000","last_change":"2024-09-17T03:30:17.372555+0000","last_active":"2024-09-17T03:30:19.047759+0000","last_peered":"2024-09-17T03:30:19.047759+0000","last_clean":"2024-09-17T03:30:19.047759+0000","last_became_active":"2024-09-17T03:30:17.372225+0000","last_became_peered":"2024-09-17T03:30:17.372225+0000","last_unstale":"2024-09-17T03:30:19.047759+0000","last_undegraded":"2024-09-17T03:30:19.047759+0000","last_fullsized":"2024-09-17T03:30:19.047759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:49.139488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363251+0000","last_change":"2024-09-17T03:30:16.359549+0000","last_active":"2024-09-17T03:30:16.363251+0000","last_peered":"2024-09-17T03:30:16.363251+0000","last_clean":"2024-09-17T03:30:16.363251+0000","last_became_active":"2024-09-17T03:30:16.359149+0000","last_became_peered":"2024-09-17T03:30:16.359149+0000","last_unstale":"2024-09-17T03:30:16.363251+0000","last_undegraded":"2024-09-17T03:30:16.363251+0000","last_fullsized":"2024-09-17T03:30:16.363251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:15.534467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346444+0000","last_change":"2024-09-17T03:30:16.346444+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346444+0000","last_undegraded":"2024-09-17T03:30:16.346444+0000","last_fullsized":"2024-09-17T03:30:16.346444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375718+0000","last_change":"2024-09-17T03:30:16.374752+0000","last_active":"2024-09-17T03:30:16.375718+0000","last_peered":"2024-09-17T03:30:16.375718+0000","last_clean":"2024-09-17T03:30:16.375718+0000","last_became_active":"2024-09-17T03:30:16.374286+0000","last_became_peered":"2024-09-17T03:30:16.374286+0000","last_unstale":"2024-09-17T03:30:16.375718+0000","last_undegraded":"2024-09-17T03:30:16.375718+0000","last_fullsized":"2024-09-17T03:30:16.375718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:38.480874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361539+0000","last_change":"2024-09-17T03:30:16.361539+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361539+0000","last_undegraded":"2024-09-17T03:30:16.361539+0000","last_fullsized":"2024-09-17T03:30:16.361539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371481+0000","last_change":"2024-09-17T03:30:16.367991+0000","last_active":"2024-09-17T03:30:16.371481+0000","last_peered":"2024-09-17T03:30:16.371481+0000","last_clean":"2024-09-17T03:30:16.371481+0000","last_became_active":"2024-09-17T03:30:16.367669+0000","last_became_peered":"2024-09-17T03:30:16.367669+0000","last_unstale":"2024-09-17T03:30:16.371481+0000","last_undegraded":"2024-09-17T03:30:16.371481+0000","last_fullsized":"2024-09-17T03:30:16.371481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:30.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353079+0000","last_change":"2024-09-17T03:30:16.353079+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353079+0000","last_undegraded":"2024-09-17T03:30:16.353079+0000","last_fullsized":"2024-09-17T03:30:16.353079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.372120+0000","last_change":"2024-09-17T03:30:16.367493+0000","last_active":"2024-09-17T03:30:16.372120+0000","last_peered":"2024-09-17T03:30:16.372120+0000","last_clean":"2024-09-17T03:30:16.372120+0000","last_became_active":"2024-09-17T03:30:16.367090+0000","last_became_peered":"2024-09-17T03:30:16.367090+0000","last_unstale":"2024-09-17T03:30:16.372120+0000","last_undegraded":"2024-09-17T03:30:16.372120+0000","last_fullsized":"2024-09-17T03:30:16.372120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:47.067459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040368+0000","last_change":"2024-09-17T03:30:17.374985+0000","last_active":"2024-09-17T03:30:19.040368+0000","last_peered":"2024-09-17T03:30:19.040368+0000","last_clean":"2024-09-17T03:30:19.040368+0000","last_became_active":"2024-09-17T03:30:17.373033+0000","last_became_peered":"2024-09-17T03:30:17.373033+0000","last_unstale":"2024-09-17T03:30:19.040368+0000","last_undegraded":"2024-09-17T03:30:19.040368+0000","last_fullsized":"2024-09-17T03:30:19.040368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:56.073407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.377262+0000","last_change":"2024-09-17T03:30:16.376615+0000","last_active":"2024-09-17T03:30:16.377262+0000","last_peered":"2024-09-17T03:30:16.377262+0000","last_clean":"2024-09-17T03:30:16.377262+0000","last_became_active":"2024-09-17T03:30:16.376148+0000","last_became_peered":"2024-09-17T03:30:16.376148+0000","last_unstale":"2024-09-17T03:30:16.377262+0000","last_undegraded":"2024-09-17T03:30:16.377262+0000","last_fullsized":"2024-09-17T03:30:16.377262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364429+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:30:16.364429+0000","last_peered":"2024-09-17T03:30:16.364429+0000","last_clean":"2024-09-17T03:30:16.364429+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:30:16.364429+0000","last_undegraded":"2024-09-17T03:30:16.364429+0000","last_fullsized":"2024-09-17T03:30:16.364429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64652,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307188,"statfs":{"total":96636764160,"available":96570560512,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64072,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307768,"statfs":{"total":96636764160,"available":96571154432,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62716,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309124,"statfs":{"total":96636764160,"available":96572542976,"internally_reserved":0,"allocated":1024000,"data_stored":435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72392,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299448,"statfs":{"total":96636764160,"available":96562634752,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68240,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68816,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303024,"statfs":{"total":96636764160,"available":96566296576,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1077248,"data_stored":456225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68268,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303572,"statfs":{"total":96636764160,"available":96566857728,"internally_reserved":0,"allocated":1073152,"data_stored":454785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:30:21.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 -- 172.21.15.46:0/883985466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 msgr2=0x7fc83406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 --2- 172.21.15.46:0/883985466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc83406d200 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009bc0 tx=0x7fc82c01cb30 comp rx=0 tx=0).stop 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 -- 172.21.15.46:0/883985466 shutdown_connections 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 --2- 172.21.15.46:0/883985466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc83406d200 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 --2- 172.21.15.46:0/883985466 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc834159130 0x7fc83415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 -- 172.21.15.46:0/883985466 >> 172.21.15.46:0/883985466 conn(0x7fc8340b35f0 msgr2=0x7fc8340b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 -- 172.21.15.46:0/883985466 shutdown_connections 2024-09-17T03:30:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 -- 172.21.15.46:0/883985466 wait complete. 2024-09-17T03:30:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 Processor -- start 2024-09-17T03:30:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.753+0000 7fc839600640 1 -- start start 2024-09-17T03:30:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc83406cdf0 0x7fc8341fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc834157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc834159130 0x7fc83415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc834168df0 con 0x7fc8340703a0 2024-09-17T03:30:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc834168c70 con 0x7fc834159130 2024-09-17T03:30:21.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc834168f70 con 0x7fc83406cdf0 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc834157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc834157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45560/0 (socket says 172.21.15.46:45560) 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 -- 172.21.15.46:0/3357417458 learned_addr learned my addr 172.21.15.46:0/3357417458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc832a00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc83406cdf0 0x7fc8341fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc833400640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc834159130 0x7fc83415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 -- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc83406cdf0 msgr2=0x7fc8341fcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc83406cdf0 0x7fc8341fcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 -- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc834159130 msgr2=0x7fc83415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc834159130 0x7fc83415f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 -- 172.21.15.46:0/3357417458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8340cf1a0 con 0x7fc8340703a0 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc833400640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc834159130 0x7fc83415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82be00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc834157080 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c370 tx=0x7fc81c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c017070 con 0x7fc8340703a0 2024-09-17T03:30:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc81c003600 con 0x7fc8340703a0 2024-09-17T03:30:21.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8340cf480 con 0x7fc8340703a0 2024-09-17T03:30:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c003a30 con 0x7fc8340703a0 2024-09-17T03:30:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8340aa6b0 con 0x7fc8340703a0 2024-09-17T03:30:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.757+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8340aa4d0 con 0x7fc8340703a0 2024-09-17T03:30:21.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.761+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fc81c01a070 con 0x7fc8340703a0 2024-09-17T03:30:21.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.761+0000 7fc82aa00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc810074430 0x7fc810076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:21.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.761+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fc81c092280 con 0x7fc8340703a0 2024-09-17T03:30:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.761+0000 7fc832a00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc810074430 0x7fc810076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.761+0000 7fc832a00640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc810074430 0x7fc810076940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc824004630 tx=0x7fc82400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.765+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc8340aa4d0 con 0x7fc8340703a0 2024-09-17T03:30:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.933+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc83406b540 con 0x7fc810074430 2024-09-17T03:30:21.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.941+0000 7fc82aa00640 1 -- 172.21.15.46:0/3357417458 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248208 (secure 0 0 0) 0x7fc82400b7e0 con 0x7fc810074430 2024-09-17T03:30:21.942 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc810074430 msgr2=0x7fc810076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc810074430 0x7fc810076940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc824004630 tx=0x7fc82400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 msgr2=0x7fc834157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc834157080 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c370 tx=0x7fc81c00c830 comp rx=0 tx=0).stop 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 shutdown_connections 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc810074430 0x7fc810076940 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc834159130 0x7fc83415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8340703a0 0x7fc834157080 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 --2- 172.21.15.46:0/3357417458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc83406cdf0 0x7fc8341fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:21.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 >> 172.21.15.46:0/3357417458 conn(0x7fc8340b35f0 msgr2=0x7fc8341646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:21.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 shutdown_connections 2024-09-17T03:30:21.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:21.945+0000 7fc839600640 1 -- 172.21.15.46:0/3357417458 wait complete. 2024-09-17T03:30:21.965 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T03:30:21.390774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":114,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":527768,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":518259,"kb_avail":754446952,"statfs":{"total":773094113280,"available":772553678848,"internally_reserved":0,"allocated":9543680,"data_stored":4734567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":530697776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001148"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356490+0000","last_change":"2024-09-17T03:30:16.356490+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356490+0000","last_undegraded":"2024-09-17T03:30:16.356490+0000","last_fullsized":"2024-09-17T03:30:16.356490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041228+0000","last_change":"2024-09-17T03:30:17.370486+0000","last_active":"2024-09-17T03:30:19.041228+0000","last_peered":"2024-09-17T03:30:19.041228+0000","last_clean":"2024-09-17T03:30:19.041228+0000","last_became_active":"2024-09-17T03:30:17.370107+0000","last_became_peered":"2024-09-17T03:30:17.370107+0000","last_unstale":"2024-09-17T03:30:19.041228+0000","last_undegraded":"2024-09-17T03:30:19.041228+0000","last_fullsized":"2024-09-17T03:30:19.041228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:27.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361223+0000","last_change":"2024-09-17T03:30:16.361223+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361223+0000","last_undegraded":"2024-09-17T03:30:16.361223+0000","last_fullsized":"2024-09-17T03:30:16.361223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350657+0000","last_change":"2024-09-17T03:30:16.350657+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350657+0000","last_undegraded":"2024-09-17T03:30:16.350657+0000","last_fullsized":"2024-09-17T03:30:16.350657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362688+0000","last_change":"2024-09-17T03:30:16.362688+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362688+0000","last_undegraded":"2024-09-17T03:30:16.362688+0000","last_fullsized":"2024-09-17T03:30:16.362688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047927+0000","last_change":"2024-09-17T03:30:17.388212+0000","last_active":"2024-09-17T03:30:19.047927+0000","last_peered":"2024-09-17T03:30:19.047927+0000","last_clean":"2024-09-17T03:30:19.047927+0000","last_became_active":"2024-09-17T03:30:17.387941+0000","last_became_peered":"2024-09-17T03:30:17.387941+0000","last_unstale":"2024-09-17T03:30:19.047927+0000","last_undegraded":"2024-09-17T03:30:19.047927+0000","last_fullsized":"2024-09-17T03:30:19.047927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:30.231182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356272+0000","last_change":"2024-09-17T03:30:16.356272+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356272+0000","last_undegraded":"2024-09-17T03:30:16.356272+0000","last_fullsized":"2024-09-17T03:30:16.356272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047750+0000","last_change":"2024-09-17T03:30:17.383305+0000","last_active":"2024-09-17T03:30:19.047750+0000","last_peered":"2024-09-17T03:30:19.047750+0000","last_clean":"2024-09-17T03:30:19.047750+0000","last_became_active":"2024-09-17T03:30:17.382995+0000","last_became_peered":"2024-09-17T03:30:17.382995+0000","last_unstale":"2024-09-17T03:30:19.047750+0000","last_undegraded":"2024-09-17T03:30:19.047750+0000","last_fullsized":"2024-09-17T03:30:19.047750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:03.641896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047928+0000","last_change":"2024-09-17T03:30:17.374521+0000","last_active":"2024-09-17T03:30:19.047928+0000","last_peered":"2024-09-17T03:30:19.047928+0000","last_clean":"2024-09-17T03:30:19.047928+0000","last_became_active":"2024-09-17T03:30:17.374185+0000","last_became_peered":"2024-09-17T03:30:17.374185+0000","last_unstale":"2024-09-17T03:30:19.047928+0000","last_undegraded":"2024-09-17T03:30:19.047928+0000","last_fullsized":"2024-09-17T03:30:19.047928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:57.551527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353169+0000","last_change":"2024-09-17T03:30:16.353169+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353169+0000","last_undegraded":"2024-09-17T03:30:16.353169+0000","last_fullsized":"2024-09-17T03:30:16.353169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040669+0000","last_change":"2024-09-17T03:30:17.373313+0000","last_active":"2024-09-17T03:30:19.040669+0000","last_peered":"2024-09-17T03:30:19.040669+0000","last_clean":"2024-09-17T03:30:19.040669+0000","last_became_active":"2024-09-17T03:30:17.372950+0000","last_became_peered":"2024-09-17T03:30:17.372950+0000","last_unstale":"2024-09-17T03:30:19.040669+0000","last_undegraded":"2024-09-17T03:30:19.040669+0000","last_fullsized":"2024-09-17T03:30:19.040669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356094+0000","last_change":"2024-09-17T03:30:16.356094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356094+0000","last_undegraded":"2024-09-17T03:30:16.356094+0000","last_fullsized":"2024-09-17T03:30:16.356094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363504+0000","last_change":"2024-09-17T03:30:16.363504+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363504+0000","last_undegraded":"2024-09-17T03:30:16.363504+0000","last_fullsized":"2024-09-17T03:30:16.363504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350224+0000","last_change":"2024-09-17T03:30:16.350224+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350224+0000","last_undegraded":"2024-09-17T03:30:16.350224+0000","last_fullsized":"2024-09-17T03:30:16.350224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348144+0000","last_change":"2024-09-17T03:30:16.348144+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348144+0000","last_undegraded":"2024-09-17T03:30:16.348144+0000","last_fullsized":"2024-09-17T03:30:16.348144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.358745+0000","last_change":"2024-09-17T03:30:16.358745+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.358745+0000","last_undegraded":"2024-09-17T03:30:16.358745+0000","last_fullsized":"2024-09-17T03:30:16.358745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.364506+0000","last_change":"2024-09-17T03:30:16.364506+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.364506+0000","last_undegraded":"2024-09-17T03:30:16.364506+0000","last_fullsized":"2024-09-17T03:30:16.364506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.351260+0000","last_change":"2024-09-17T03:30:16.351260+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.351260+0000","last_undegraded":"2024-09-17T03:30:16.351260+0000","last_fullsized":"2024-09-17T03:30:16.351260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346459+0000","last_change":"2024-09-17T03:30:16.346459+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346459+0000","last_undegraded":"2024-09-17T03:30:16.346459+0000","last_fullsized":"2024-09-17T03:30:16.346459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350038+0000","last_change":"2024-09-17T03:30:16.350038+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350038+0000","last_undegraded":"2024-09-17T03:30:16.350038+0000","last_fullsized":"2024-09-17T03:30:16.350038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349235+0000","last_change":"2024-09-17T03:30:16.349235+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349235+0000","last_undegraded":"2024-09-17T03:30:16.349235+0000","last_fullsized":"2024-09-17T03:30:16.349235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040270+0000","last_change":"2024-09-17T03:30:17.374044+0000","last_active":"2024-09-17T03:30:19.040270+0000","last_peered":"2024-09-17T03:30:19.040270+0000","last_clean":"2024-09-17T03:30:19.040270+0000","last_became_active":"2024-09-17T03:30:17.373700+0000","last_became_peered":"2024-09-17T03:30:17.373700+0000","last_unstale":"2024-09-17T03:30:19.040270+0000","last_undegraded":"2024-09-17T03:30:19.040270+0000","last_fullsized":"2024-09-17T03:30:19.040270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:04.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354041+0000","last_change":"2024-09-17T03:30:16.354041+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354041+0000","last_undegraded":"2024-09-17T03:30:16.354041+0000","last_fullsized":"2024-09-17T03:30:16.354041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349327+0000","last_change":"2024-09-17T03:30:16.349327+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349327+0000","last_undegraded":"2024-09-17T03:30:16.349327+0000","last_fullsized":"2024-09-17T03:30:16.349327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041180+0000","last_change":"2024-09-17T03:30:17.374952+0000","last_active":"2024-09-17T03:30:19.041180+0000","last_peered":"2024-09-17T03:30:19.041180+0000","last_clean":"2024-09-17T03:30:19.041180+0000","last_became_active":"2024-09-17T03:30:17.374579+0000","last_became_peered":"2024-09-17T03:30:17.374579+0000","last_unstale":"2024-09-17T03:30:19.041180+0000","last_undegraded":"2024-09-17T03:30:19.041180+0000","last_fullsized":"2024-09-17T03:30:19.041180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.728454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361780+0000","last_change":"2024-09-17T03:30:16.361780+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361780+0000","last_undegraded":"2024-09-17T03:30:16.361780+0000","last_fullsized":"2024-09-17T03:30:16.361780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.367168+0000","last_change":"2024-09-17T03:30:16.367168+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.367168+0000","last_undegraded":"2024-09-17T03:30:16.367168+0000","last_fullsized":"2024-09-17T03:30:16.367168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.355538+0000","last_change":"2024-09-17T03:30:16.355538+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.355538+0000","last_undegraded":"2024-09-17T03:30:16.355538+0000","last_fullsized":"2024-09-17T03:30:16.355538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040169+0000","last_change":"2024-09-17T03:30:17.378897+0000","last_active":"2024-09-17T03:30:19.040169+0000","last_peered":"2024-09-17T03:30:19.040169+0000","last_clean":"2024-09-17T03:30:19.040169+0000","last_became_active":"2024-09-17T03:30:17.378212+0000","last_became_peered":"2024-09-17T03:30:17.378212+0000","last_unstale":"2024-09-17T03:30:19.040169+0000","last_undegraded":"2024-09-17T03:30:19.040169+0000","last_fullsized":"2024-09-17T03:30:19.040169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:13.896559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040290+0000","last_change":"2024-09-17T03:30:17.382864+0000","last_active":"2024-09-17T03:30:19.040290+0000","last_peered":"2024-09-17T03:30:19.040290+0000","last_clean":"2024-09-17T03:30:19.040290+0000","last_became_active":"2024-09-17T03:30:17.381934+0000","last_became_peered":"2024-09-17T03:30:17.381934+0000","last_unstale":"2024-09-17T03:30:19.040290+0000","last_undegraded":"2024-09-17T03:30:19.040290+0000","last_fullsized":"2024-09-17T03:30:19.040290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:12.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360461+0000","last_change":"2024-09-17T03:30:16.360461+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360461+0000","last_undegraded":"2024-09-17T03:30:16.360461+0000","last_fullsized":"2024-09-17T03:30:16.360461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353448+0000","last_change":"2024-09-17T03:30:16.353448+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353448+0000","last_undegraded":"2024-09-17T03:30:16.353448+0000","last_fullsized":"2024-09-17T03:30:16.353448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363769+0000","last_change":"2024-09-17T03:30:16.363769+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363769+0000","last_undegraded":"2024-09-17T03:30:16.363769+0000","last_fullsized":"2024-09-17T03:30:16.363769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371726+0000","last_change":"2024-09-17T03:30:16.366033+0000","last_active":"2024-09-17T03:30:16.371726+0000","last_peered":"2024-09-17T03:30:16.371726+0000","last_clean":"2024-09-17T03:30:16.371726+0000","last_became_active":"2024-09-17T03:30:16.365643+0000","last_became_peered":"2024-09-17T03:30:16.365643+0000","last_unstale":"2024-09-17T03:30:16.371726+0000","last_undegraded":"2024-09-17T03:30:16.371726+0000","last_fullsized":"2024-09-17T03:30:16.371726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:05.152896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354849+0000","last_change":"2024-09-17T03:30:16.354849+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354849+0000","last_undegraded":"2024-09-17T03:30:16.354849+0000","last_fullsized":"2024-09-17T03:30:16.354849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375348+0000","last_change":"2024-09-17T03:30:16.374265+0000","last_active":"2024-09-17T03:30:16.375348+0000","last_peered":"2024-09-17T03:30:16.375348+0000","last_clean":"2024-09-17T03:30:16.375348+0000","last_became_active":"2024-09-17T03:30:16.372671+0000","last_became_peered":"2024-09-17T03:30:16.372671+0000","last_unstale":"2024-09-17T03:30:16.375348+0000","last_undegraded":"2024-09-17T03:30:16.375348+0000","last_fullsized":"2024-09-17T03:30:16.375348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:20.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354990+0000","last_change":"2024-09-17T03:30:16.354990+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354990+0000","last_undegraded":"2024-09-17T03:30:16.354990+0000","last_fullsized":"2024-09-17T03:30:16.354990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.359637+0000","last_change":"2024-09-17T03:30:16.355580+0000","last_active":"2024-09-17T03:30:16.359637+0000","last_peered":"2024-09-17T03:30:16.359637+0000","last_clean":"2024-09-17T03:30:16.359637+0000","last_became_active":"2024-09-17T03:30:16.355240+0000","last_became_peered":"2024-09-17T03:30:16.355240+0000","last_unstale":"2024-09-17T03:30:16.359637+0000","last_undegraded":"2024-09-17T03:30:16.359637+0000","last_fullsized":"2024-09-17T03:30:16.359637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:31.022404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360687+0000","last_change":"2024-09-17T03:30:16.360687+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360687+0000","last_undegraded":"2024-09-17T03:30:16.360687+0000","last_fullsized":"2024-09-17T03:30:16.360687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361157+0000","last_change":"2024-09-17T03:30:16.357957+0000","last_active":"2024-09-17T03:30:16.361157+0000","last_peered":"2024-09-17T03:30:16.361157+0000","last_clean":"2024-09-17T03:30:16.361157+0000","last_became_active":"2024-09-17T03:30:16.357597+0000","last_became_peered":"2024-09-17T03:30:16.357597+0000","last_unstale":"2024-09-17T03:30:16.361157+0000","last_undegraded":"2024-09-17T03:30:16.361157+0000","last_fullsized":"2024-09-17T03:30:16.361157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:25.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040526+0000","last_change":"2024-09-17T03:30:17.382905+0000","last_active":"2024-09-17T03:30:19.040526+0000","last_peered":"2024-09-17T03:30:19.040526+0000","last_clean":"2024-09-17T03:30:19.040526+0000","last_became_active":"2024-09-17T03:30:17.382343+0000","last_became_peered":"2024-09-17T03:30:17.382343+0000","last_unstale":"2024-09-17T03:30:19.040526+0000","last_undegraded":"2024-09-17T03:30:19.040526+0000","last_fullsized":"2024-09-17T03:30:19.040526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:08.921865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376896+0000","last_change":"2024-09-17T03:30:16.375643+0000","last_active":"2024-09-17T03:30:16.376896+0000","last_peered":"2024-09-17T03:30:16.376896+0000","last_clean":"2024-09-17T03:30:16.376896+0000","last_became_active":"2024-09-17T03:30:16.375378+0000","last_became_peered":"2024-09-17T03:30:16.375378+0000","last_unstale":"2024-09-17T03:30:16.376896+0000","last_undegraded":"2024-09-17T03:30:16.376896+0000","last_fullsized":"2024-09-17T03:30:16.376896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.037881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362120+0000","last_change":"2024-09-17T03:30:16.362120+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362120+0000","last_undegraded":"2024-09-17T03:30:16.362120+0000","last_fullsized":"2024-09-17T03:30:16.362120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375682+0000","last_change":"2024-09-17T03:30:16.374780+0000","last_active":"2024-09-17T03:30:16.375682+0000","last_peered":"2024-09-17T03:30:16.375682+0000","last_clean":"2024-09-17T03:30:16.375682+0000","last_became_active":"2024-09-17T03:30:16.374312+0000","last_became_peered":"2024-09-17T03:30:16.374312+0000","last_unstale":"2024-09-17T03:30:16.375682+0000","last_undegraded":"2024-09-17T03:30:16.375682+0000","last_fullsized":"2024-09-17T03:30:16.375682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.479057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362017+0000","last_change":"2024-09-17T03:30:16.362017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362017+0000","last_undegraded":"2024-09-17T03:30:16.362017+0000","last_fullsized":"2024-09-17T03:30:16.362017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363741+0000","last_change":"2024-09-17T03:30:16.360075+0000","last_active":"2024-09-17T03:30:16.363741+0000","last_peered":"2024-09-17T03:30:16.363741+0000","last_clean":"2024-09-17T03:30:16.363741+0000","last_became_active":"2024-09-17T03:30:16.359771+0000","last_became_peered":"2024-09-17T03:30:16.359771+0000","last_unstale":"2024-09-17T03:30:16.363741+0000","last_undegraded":"2024-09-17T03:30:16.363741+0000","last_fullsized":"2024-09-17T03:30:16.363741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:01.924364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.347497+0000","last_change":"2024-09-17T03:30:16.347497+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.347497+0000","last_undegraded":"2024-09-17T03:30:16.347497+0000","last_fullsized":"2024-09-17T03:30:16.347497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039799+0000","last_change":"2024-09-17T03:30:16.365783+0000","last_active":"2024-09-17T03:30:19.039799+0000","last_peered":"2024-09-17T03:30:19.039799+0000","last_clean":"2024-09-17T03:30:19.039799+0000","last_became_active":"2024-09-17T03:30:16.365439+0000","last_became_peered":"2024-09-17T03:30:16.365439+0000","last_unstale":"2024-09-17T03:30:19.039799+0000","last_undegraded":"2024-09-17T03:30:19.039799+0000","last_fullsized":"2024-09-17T03:30:19.039799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:48.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348017+0000","last_change":"2024-09-17T03:30:16.348017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348017+0000","last_undegraded":"2024-09-17T03:30:16.348017+0000","last_fullsized":"2024-09-17T03:30:16.348017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371783+0000","last_change":"2024-09-17T03:30:16.369999+0000","last_active":"2024-09-17T03:30:16.371783+0000","last_peered":"2024-09-17T03:30:16.371783+0000","last_clean":"2024-09-17T03:30:16.371783+0000","last_became_active":"2024-09-17T03:30:16.368392+0000","last_became_peered":"2024-09-17T03:30:16.368392+0000","last_unstale":"2024-09-17T03:30:16.371783+0000","last_undegraded":"2024-09-17T03:30:16.371783+0000","last_fullsized":"2024-09-17T03:30:16.371783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:10.071007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047957+0000","last_change":"2024-09-17T03:30:17.380149+0000","last_active":"2024-09-17T03:30:19.047957+0000","last_peered":"2024-09-17T03:30:19.047957+0000","last_clean":"2024-09-17T03:30:19.047957+0000","last_became_active":"2024-09-17T03:30:17.379829+0000","last_became_peered":"2024-09-17T03:30:17.379829+0000","last_unstale":"2024-09-17T03:30:19.047957+0000","last_undegraded":"2024-09-17T03:30:19.047957+0000","last_fullsized":"2024-09-17T03:30:19.047957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:17.605539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.374805+0000","last_change":"2024-09-17T03:30:16.373477+0000","last_active":"2024-09-17T03:30:16.374805+0000","last_peered":"2024-09-17T03:30:16.374805+0000","last_clean":"2024-09-17T03:30:16.374805+0000","last_became_active":"2024-09-17T03:30:16.373060+0000","last_became_peered":"2024-09-17T03:30:16.373060+0000","last_unstale":"2024-09-17T03:30:16.374805+0000","last_undegraded":"2024-09-17T03:30:16.374805+0000","last_fullsized":"2024-09-17T03:30:16.374805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:06.209920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346582+0000","last_change":"2024-09-17T03:30:16.346582+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346582+0000","last_undegraded":"2024-09-17T03:30:16.346582+0000","last_fullsized":"2024-09-17T03:30:16.346582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376938+0000","last_change":"2024-09-17T03:30:16.376370+0000","last_active":"2024-09-17T03:30:16.376938+0000","last_peered":"2024-09-17T03:30:16.376938+0000","last_clean":"2024-09-17T03:30:16.376938+0000","last_became_active":"2024-09-17T03:30:16.376027+0000","last_became_peered":"2024-09-17T03:30:16.376027+0000","last_unstale":"2024-09-17T03:30:16.376938+0000","last_undegraded":"2024-09-17T03:30:16.376938+0000","last_fullsized":"2024-09-17T03:30:16.376938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:51.451736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366652+0000","last_change":"2024-09-17T03:30:16.366652+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366652+0000","last_undegraded":"2024-09-17T03:30:16.366652+0000","last_fullsized":"2024-09-17T03:30:16.366652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047718+0000","last_change":"2024-09-17T03:30:16.370573+0000","last_active":"2024-09-17T03:30:19.047718+0000","last_peered":"2024-09-17T03:30:19.047718+0000","last_clean":"2024-09-17T03:30:19.047718+0000","last_became_active":"2024-09-17T03:30:16.369166+0000","last_became_peered":"2024-09-17T03:30:16.369166+0000","last_unstale":"2024-09-17T03:30:19.047718+0000","last_undegraded":"2024-09-17T03:30:19.047718+0000","last_fullsized":"2024-09-17T03:30:19.047718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:23.751200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346578+0000","last_change":"2024-09-17T03:30:16.346578+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346578+0000","last_undegraded":"2024-09-17T03:30:16.346578+0000","last_fullsized":"2024-09-17T03:30:16.346578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040259+0000","last_change":"2024-09-17T03:30:16.367290+0000","last_active":"2024-09-17T03:30:19.040259+0000","last_peered":"2024-09-17T03:30:19.040259+0000","last_clean":"2024-09-17T03:30:19.040259+0000","last_became_active":"2024-09-17T03:30:16.365363+0000","last_became_peered":"2024-09-17T03:30:16.365363+0000","last_unstale":"2024-09-17T03:30:19.040259+0000","last_undegraded":"2024-09-17T03:30:19.040259+0000","last_fullsized":"2024-09-17T03:30:19.040259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:46.817707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363020+0000","last_change":"2024-09-17T03:30:16.363020+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363020+0000","last_undegraded":"2024-09-17T03:30:16.363020+0000","last_fullsized":"2024-09-17T03:30:16.363020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371240+0000","last_change":"2024-09-17T03:30:16.369373+0000","last_active":"2024-09-17T03:30:16.371240+0000","last_peered":"2024-09-17T03:30:16.371240+0000","last_clean":"2024-09-17T03:30:16.371240+0000","last_became_active":"2024-09-17T03:30:16.369082+0000","last_became_peered":"2024-09-17T03:30:16.369082+0000","last_unstale":"2024-09-17T03:30:16.371240+0000","last_undegraded":"2024-09-17T03:30:16.371240+0000","last_fullsized":"2024-09-17T03:30:16.371240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:46.125971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039947+0000","last_change":"2024-09-17T03:30:17.382255+0000","last_active":"2024-09-17T03:30:19.039947+0000","last_peered":"2024-09-17T03:30:19.039947+0000","last_clean":"2024-09-17T03:30:19.039947+0000","last_became_active":"2024-09-17T03:30:17.381901+0000","last_became_peered":"2024-09-17T03:30:17.381901+0000","last_unstale":"2024-09-17T03:30:19.039947+0000","last_undegraded":"2024-09-17T03:30:19.039947+0000","last_fullsized":"2024-09-17T03:30:19.039947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.620927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047671+0000","last_change":"2024-09-17T03:30:16.371647+0000","last_active":"2024-09-17T03:30:19.047671+0000","last_peered":"2024-09-17T03:30:19.047671+0000","last_clean":"2024-09-17T03:30:19.047671+0000","last_became_active":"2024-09-17T03:30:16.371336+0000","last_became_peered":"2024-09-17T03:30:16.371336+0000","last_unstale":"2024-09-17T03:30:19.047671+0000","last_undegraded":"2024-09-17T03:30:19.047671+0000","last_fullsized":"2024-09-17T03:30:19.047671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:19.546103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039882+0000","last_change":"2024-09-17T03:30:17.372667+0000","last_active":"2024-09-17T03:30:19.039882+0000","last_peered":"2024-09-17T03:30:19.039882+0000","last_clean":"2024-09-17T03:30:19.039882+0000","last_became_active":"2024-09-17T03:30:17.372310+0000","last_became_peered":"2024-09-17T03:30:17.372310+0000","last_unstale":"2024-09-17T03:30:19.039882+0000","last_undegraded":"2024-09-17T03:30:19.039882+0000","last_fullsized":"2024-09-17T03:30:19.039882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:46.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363382+0000","last_change":"2024-09-17T03:30:16.359214+0000","last_active":"2024-09-17T03:30:16.363382+0000","last_peered":"2024-09-17T03:30:16.363382+0000","last_clean":"2024-09-17T03:30:16.363382+0000","last_became_active":"2024-09-17T03:30:16.358883+0000","last_became_peered":"2024-09-17T03:30:16.358883+0000","last_unstale":"2024-09-17T03:30:16.363382+0000","last_undegraded":"2024-09-17T03:30:16.363382+0000","last_fullsized":"2024-09-17T03:30:16.363382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:47.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361307+0000","last_change":"2024-09-17T03:30:16.361307+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361307+0000","last_undegraded":"2024-09-17T03:30:16.361307+0000","last_fullsized":"2024-09-17T03:30:16.361307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371756+0000","last_change":"2024-09-17T03:30:16.369933+0000","last_active":"2024-09-17T03:30:16.371756+0000","last_peered":"2024-09-17T03:30:16.371756+0000","last_clean":"2024-09-17T03:30:16.371756+0000","last_became_active":"2024-09-17T03:30:16.369307+0000","last_became_peered":"2024-09-17T03:30:16.369307+0000","last_unstale":"2024-09-17T03:30:16.371756+0000","last_undegraded":"2024-09-17T03:30:16.371756+0000","last_fullsized":"2024-09-17T03:30:16.371756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:29.392547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354255+0000","last_change":"2024-09-17T03:30:16.354255+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354255+0000","last_undegraded":"2024-09-17T03:30:16.354255+0000","last_fullsized":"2024-09-17T03:30:16.354255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367756+0000","last_change":"2024-09-17T03:30:16.364256+0000","last_active":"2024-09-17T03:30:16.367756+0000","last_peered":"2024-09-17T03:30:16.367756+0000","last_clean":"2024-09-17T03:30:16.367756+0000","last_became_active":"2024-09-17T03:30:16.363880+0000","last_became_peered":"2024-09-17T03:30:16.363880+0000","last_unstale":"2024-09-17T03:30:16.367756+0000","last_undegraded":"2024-09-17T03:30:16.367756+0000","last_fullsized":"2024-09-17T03:30:16.367756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366139+0000","last_change":"2024-09-17T03:30:16.366139+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366139+0000","last_undegraded":"2024-09-17T03:30:16.366139+0000","last_fullsized":"2024-09-17T03:30:16.366139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364390+0000","last_change":"2024-09-17T03:30:16.361309+0000","last_active":"2024-09-17T03:30:16.364390+0000","last_peered":"2024-09-17T03:30:16.364390+0000","last_clean":"2024-09-17T03:30:16.364390+0000","last_became_active":"2024-09-17T03:30:16.360955+0000","last_became_peered":"2024-09-17T03:30:16.360955+0000","last_unstale":"2024-09-17T03:30:16.364390+0000","last_undegraded":"2024-09-17T03:30:16.364390+0000","last_fullsized":"2024-09-17T03:30:16.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:33.036462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349551+0000","last_change":"2024-09-17T03:30:16.349551+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349551+0000","last_undegraded":"2024-09-17T03:30:16.349551+0000","last_fullsized":"2024-09-17T03:30:16.349551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.366789+0000","last_change":"2024-09-17T03:30:16.359453+0000","last_active":"2024-09-17T03:30:16.366789+0000","last_peered":"2024-09-17T03:30:16.366789+0000","last_clean":"2024-09-17T03:30:16.366789+0000","last_became_active":"2024-09-17T03:30:16.358930+0000","last_became_peered":"2024-09-17T03:30:16.358930+0000","last_unstale":"2024-09-17T03:30:16.366789+0000","last_undegraded":"2024-09-17T03:30:16.366789+0000","last_fullsized":"2024-09-17T03:30:16.366789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:34.925865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048154+0000","last_change":"2024-09-17T03:30:17.380496+0000","last_active":"2024-09-17T03:30:19.048154+0000","last_peered":"2024-09-17T03:30:19.048154+0000","last_clean":"2024-09-17T03:30:19.048154+0000","last_became_active":"2024-09-17T03:30:17.380195+0000","last_became_peered":"2024-09-17T03:30:17.380195+0000","last_unstale":"2024-09-17T03:30:19.048154+0000","last_undegraded":"2024-09-17T03:30:19.048154+0000","last_fullsized":"2024-09-17T03:30:19.048154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:32.240069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370053+0000","last_change":"2024-09-17T03:30:16.366802+0000","last_active":"2024-09-17T03:30:16.370053+0000","last_peered":"2024-09-17T03:30:16.370053+0000","last_clean":"2024-09-17T03:30:16.370053+0000","last_became_active":"2024-09-17T03:30:16.366497+0000","last_became_peered":"2024-09-17T03:30:16.366497+0000","last_unstale":"2024-09-17T03:30:16.370053+0000","last_undegraded":"2024-09-17T03:30:16.370053+0000","last_fullsized":"2024-09-17T03:30:16.370053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:31.288276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348588+0000","last_change":"2024-09-17T03:30:16.348588+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348588+0000","last_undegraded":"2024-09-17T03:30:16.348588+0000","last_fullsized":"2024-09-17T03:30:16.348588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367253+0000","last_change":"2024-09-17T03:30:16.362330+0000","last_active":"2024-09-17T03:30:16.367253+0000","last_peered":"2024-09-17T03:30:16.367253+0000","last_clean":"2024-09-17T03:30:16.367253+0000","last_became_active":"2024-09-17T03:30:16.361960+0000","last_became_peered":"2024-09-17T03:30:16.361960+0000","last_unstale":"2024-09-17T03:30:16.367253+0000","last_undegraded":"2024-09-17T03:30:16.367253+0000","last_fullsized":"2024-09-17T03:30:16.367253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:27.002346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047756+0000","last_change":"2024-09-17T03:30:17.373912+0000","last_active":"2024-09-17T03:30:19.047756+0000","last_peered":"2024-09-17T03:30:19.047756+0000","last_clean":"2024-09-17T03:30:19.047756+0000","last_became_active":"2024-09-17T03:30:17.373542+0000","last_became_peered":"2024-09-17T03:30:17.373542+0000","last_unstale":"2024-09-17T03:30:19.047756+0000","last_undegraded":"2024-09-17T03:30:19.047756+0000","last_fullsized":"2024-09-17T03:30:19.047756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:28.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361044+0000","last_change":"2024-09-17T03:30:16.354858+0000","last_active":"2024-09-17T03:30:16.361044+0000","last_peered":"2024-09-17T03:30:16.361044+0000","last_clean":"2024-09-17T03:30:16.361044+0000","last_became_active":"2024-09-17T03:30:16.354399+0000","last_became_peered":"2024-09-17T03:30:16.354399+0000","last_unstale":"2024-09-17T03:30:16.361044+0000","last_undegraded":"2024-09-17T03:30:16.361044+0000","last_fullsized":"2024-09-17T03:30:16.361044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:47.686970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363094+0000","last_change":"2024-09-17T03:30:16.363094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363094+0000","last_undegraded":"2024-09-17T03:30:16.363094+0000","last_fullsized":"2024-09-17T03:30:16.363094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041218+0000","last_change":"2024-09-17T03:30:16.368986+0000","last_active":"2024-09-17T03:30:19.041218+0000","last_peered":"2024-09-17T03:30:19.041218+0000","last_clean":"2024-09-17T03:30:19.041218+0000","last_became_active":"2024-09-17T03:30:16.368638+0000","last_became_peered":"2024-09-17T03:30:16.368638+0000","last_unstale":"2024-09-17T03:30:19.041218+0000","last_undegraded":"2024-09-17T03:30:19.041218+0000","last_fullsized":"2024-09-17T03:30:19.041218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:04.104352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360154+0000","last_change":"2024-09-17T03:30:16.360154+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360154+0000","last_undegraded":"2024-09-17T03:30:16.360154+0000","last_fullsized":"2024-09-17T03:30:16.360154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041203+0000","last_change":"2024-09-17T03:30:16.371245+0000","last_active":"2024-09-17T03:30:19.041203+0000","last_peered":"2024-09-17T03:30:19.041203+0000","last_clean":"2024-09-17T03:30:19.041203+0000","last_became_active":"2024-09-17T03:30:16.370661+0000","last_became_peered":"2024-09-17T03:30:16.370661+0000","last_unstale":"2024-09-17T03:30:19.041203+0000","last_undegraded":"2024-09-17T03:30:19.041203+0000","last_fullsized":"2024-09-17T03:30:19.041203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:47.753727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353740+0000","last_change":"2024-09-17T03:30:16.353740+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353740+0000","last_undegraded":"2024-09-17T03:30:16.353740+0000","last_fullsized":"2024-09-17T03:30:16.353740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370305+0000","last_change":"2024-09-17T03:30:16.366822+0000","last_active":"2024-09-17T03:30:16.370305+0000","last_peered":"2024-09-17T03:30:16.370305+0000","last_clean":"2024-09-17T03:30:16.370305+0000","last_became_active":"2024-09-17T03:30:16.364923+0000","last_became_peered":"2024-09-17T03:30:16.364923+0000","last_unstale":"2024-09-17T03:30:16.370305+0000","last_undegraded":"2024-09-17T03:30:16.370305+0000","last_fullsized":"2024-09-17T03:30:16.370305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:02.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363106+0000","last_change":"2024-09-17T03:30:16.363106+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363106+0000","last_undegraded":"2024-09-17T03:30:16.363106+0000","last_fullsized":"2024-09-17T03:30:16.363106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371866+0000","last_change":"2024-09-17T03:30:16.369886+0000","last_active":"2024-09-17T03:30:16.371866+0000","last_peered":"2024-09-17T03:30:16.371866+0000","last_clean":"2024-09-17T03:30:16.371866+0000","last_became_active":"2024-09-17T03:30:16.369399+0000","last_became_peered":"2024-09-17T03:30:16.369399+0000","last_unstale":"2024-09-17T03:30:16.371866+0000","last_undegraded":"2024-09-17T03:30:16.371866+0000","last_fullsized":"2024-09-17T03:30:16.371866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:30.365530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047759+0000","last_change":"2024-09-17T03:30:17.372555+0000","last_active":"2024-09-17T03:30:19.047759+0000","last_peered":"2024-09-17T03:30:19.047759+0000","last_clean":"2024-09-17T03:30:19.047759+0000","last_became_active":"2024-09-17T03:30:17.372225+0000","last_became_peered":"2024-09-17T03:30:17.372225+0000","last_unstale":"2024-09-17T03:30:19.047759+0000","last_undegraded":"2024-09-17T03:30:19.047759+0000","last_fullsized":"2024-09-17T03:30:19.047759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:49.139488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363251+0000","last_change":"2024-09-17T03:30:16.359549+0000","last_active":"2024-09-17T03:30:16.363251+0000","last_peered":"2024-09-17T03:30:16.363251+0000","last_clean":"2024-09-17T03:30:16.363251+0000","last_became_active":"2024-09-17T03:30:16.359149+0000","last_became_peered":"2024-09-17T03:30:16.359149+0000","last_unstale":"2024-09-17T03:30:16.363251+0000","last_undegraded":"2024-09-17T03:30:16.363251+0000","last_fullsized":"2024-09-17T03:30:16.363251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:15.534467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346444+0000","last_change":"2024-09-17T03:30:16.346444+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346444+0000","last_undegraded":"2024-09-17T03:30:16.346444+0000","last_fullsized":"2024-09-17T03:30:16.346444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375718+0000","last_change":"2024-09-17T03:30:16.374752+0000","last_active":"2024-09-17T03:30:16.375718+0000","last_peered":"2024-09-17T03:30:16.375718+0000","last_clean":"2024-09-17T03:30:16.375718+0000","last_became_active":"2024-09-17T03:30:16.374286+0000","last_became_peered":"2024-09-17T03:30:16.374286+0000","last_unstale":"2024-09-17T03:30:16.375718+0000","last_undegraded":"2024-09-17T03:30:16.375718+0000","last_fullsized":"2024-09-17T03:30:16.375718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:38.480874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361539+0000","last_change":"2024-09-17T03:30:16.361539+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361539+0000","last_undegraded":"2024-09-17T03:30:16.361539+0000","last_fullsized":"2024-09-17T03:30:16.361539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371481+0000","last_change":"2024-09-17T03:30:16.367991+0000","last_active":"2024-09-17T03:30:16.371481+0000","last_peered":"2024-09-17T03:30:16.371481+0000","last_clean":"2024-09-17T03:30:16.371481+0000","last_became_active":"2024-09-17T03:30:16.367669+0000","last_became_peered":"2024-09-17T03:30:16.367669+0000","last_unstale":"2024-09-17T03:30:16.371481+0000","last_undegraded":"2024-09-17T03:30:16.371481+0000","last_fullsized":"2024-09-17T03:30:16.371481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:30.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353079+0000","last_change":"2024-09-17T03:30:16.353079+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353079+0000","last_undegraded":"2024-09-17T03:30:16.353079+0000","last_fullsized":"2024-09-17T03:30:16.353079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.372120+0000","last_change":"2024-09-17T03:30:16.367493+0000","last_active":"2024-09-17T03:30:16.372120+0000","last_peered":"2024-09-17T03:30:16.372120+0000","last_clean":"2024-09-17T03:30:16.372120+0000","last_became_active":"2024-09-17T03:30:16.367090+0000","last_became_peered":"2024-09-17T03:30:16.367090+0000","last_unstale":"2024-09-17T03:30:16.372120+0000","last_undegraded":"2024-09-17T03:30:16.372120+0000","last_fullsized":"2024-09-17T03:30:16.372120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:47.067459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040368+0000","last_change":"2024-09-17T03:30:17.374985+0000","last_active":"2024-09-17T03:30:19.040368+0000","last_peered":"2024-09-17T03:30:19.040368+0000","last_clean":"2024-09-17T03:30:19.040368+0000","last_became_active":"2024-09-17T03:30:17.373033+0000","last_became_peered":"2024-09-17T03:30:17.373033+0000","last_unstale":"2024-09-17T03:30:19.040368+0000","last_undegraded":"2024-09-17T03:30:19.040368+0000","last_fullsized":"2024-09-17T03:30:19.040368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:56.073407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.377262+0000","last_change":"2024-09-17T03:30:16.376615+0000","last_active":"2024-09-17T03:30:16.377262+0000","last_peered":"2024-09-17T03:30:16.377262+0000","last_clean":"2024-09-17T03:30:16.377262+0000","last_became_active":"2024-09-17T03:30:16.376148+0000","last_became_peered":"2024-09-17T03:30:16.376148+0000","last_unstale":"2024-09-17T03:30:16.377262+0000","last_undegraded":"2024-09-17T03:30:16.377262+0000","last_fullsized":"2024-09-17T03:30:16.377262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364429+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:30:16.364429+0000","last_peered":"2024-09-17T03:30:16.364429+0000","last_clean":"2024-09-17T03:30:16.364429+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:30:16.364429+0000","last_undegraded":"2024-09-17T03:30:16.364429+0000","last_fullsized":"2024-09-17T03:30:16.364429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64652,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307188,"statfs":{"total":96636764160,"available":96570560512,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64072,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307768,"statfs":{"total":96636764160,"available":96571154432,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62716,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309124,"statfs":{"total":96636764160,"available":96572542976,"internally_reserved":0,"allocated":1024000,"data_stored":435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72392,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299448,"statfs":{"total":96636764160,"available":96562634752,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68240,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68816,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303024,"statfs":{"total":96636764160,"available":96566296576,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1077248,"data_stored":456225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68268,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303572,"statfs":{"total":96636764160,"available":96566857728,"internally_reserved":0,"allocated":1073152,"data_stored":454785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:30:21.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:30:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dce200640 1 -- 172.21.15.46:0/1546447701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 msgr2=0x7f4dac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dce200640 1 --2- 172.21.15.46:0/1546447701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dac00b9d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009ec0 tx=0x7f4dc001d5f0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dc7e00640 1 -- 172.21.15.46:0/1546447701 reap_dead start 2024-09-17T03:30:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dce200640 1 -- 172.21.15.46:0/1546447701 shutdown_connections 2024-09-17T03:30:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dce200640 1 -- 172.21.15.46:0/1546447701 >> 172.21.15.46:0/1546447701 conn(0x7f4dc80b2040 msgr2=0x7f4dc80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dce200640 1 -- 172.21.15.46:0/1546447701 shutdown_connections 2024-09-17T03:30:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.169+0000 7f4dce200640 1 -- 172.21.15.46:0/1546447701 wait complete. 2024-09-17T03:30:22.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 Processor -- start 2024-09-17T03:30:22.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 -- start start 2024-09-17T03:30:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dc814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4dc81e6150 0x7f4dc81f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81e6560 0x7f4dc81f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc815cba0 con 0x7f4dac005160 2024-09-17T03:30:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc815bf00 con 0x7f4dc81e6560 2024-09-17T03:30:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dce200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc815cd20 con 0x7f4dc81e6150 2024-09-17T03:30:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dc814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dc814dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45580/0 (socket says 172.21.15.46:45580) 2024-09-17T03:30:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 -- 172.21.15.46:0/3326461241 learned_addr learned my addr 172.21.15.46:0/3326461241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dbfe00640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4dc81e6150 0x7f4dc81f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7e00640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81e6560 0x7f4dc81f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 -- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4dc81e6150 msgr2=0x7f4dc81f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4dc81e6150 0x7f4dc81f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 -- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81e6560 msgr2=0x7f4dc81f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81e6560 0x7f4dc81f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7400640 1 -- 172.21.15.46:0/3326461241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc815ace0 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dbfe00640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4dc81e6150 0x7f4dc81f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.173+0000 7f4dc7e00640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81e6560 0x7f4dc81f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dc7400640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dc814dc90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001870 tx=0x7f4db8012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8010b40 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4db8017030 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc815afc0 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8012e60 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81e3950 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc8073cf0 con 0x7f4dac005160 2024-09-17T03:30:22.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.177+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f4db8013450 con 0x7f4dac005160 2024-09-17T03:30:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.181+0000 7f4dbf400640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4da4074400 0x7f4da4076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.181+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f4db8096f90 con 0x7f4dac005160 2024-09-17T03:30:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.181+0000 7f4dbfe00640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4da4074400 0x7f4da4076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.181+0000 7f4dbfe00640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4da4074400 0x7f4da4076910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80b1250 tx=0x7f4db0006b20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.181+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4db8063a90 con 0x7f4dac005160 2024-09-17T03:30:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.357+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4dc815aa90 con 0x7f4da4074400 2024-09-17T03:30:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.361+0000 7f4dbf400640 1 -- 172.21.15.46:0/3326461241 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248208 (secure 0 0 0) 0x7f4dc815aa90 con 0x7f4da4074400 2024-09-17T03:30:22.364 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4da4074400 msgr2=0x7f4da4076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4da4074400 0x7f4da4076910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80b1250 tx=0x7f4db0006b20 comp rx=0 tx=0).stop 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 msgr2=0x7f4dc814dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dc814dc90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001870 tx=0x7f4db8012b20 comp rx=0 tx=0).stop 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 shutdown_connections 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4da4074400 0x7f4da4076910 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81e6560 0x7f4dc81f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4dc81e6150 0x7f4dc81f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 --2- 172.21.15.46:0/3326461241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac005160 0x7f4dc814dc90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 >> 172.21.15.46:0/3326461241 conn(0x7f4dc80b2040 msgr2=0x7f4dc80b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 shutdown_connections 2024-09-17T03:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.365+0000 7f4dce200640 1 -- 172.21.15.46:0/3326461241 wait complete. 2024-09-17T03:30:22.386 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T03:30:21.390774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":114,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":527768,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":518259,"kb_avail":754446952,"statfs":{"total":773094113280,"available":772553678848,"internally_reserved":0,"allocated":9543680,"data_stored":4734567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":530697776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001148"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356490+0000","last_change":"2024-09-17T03:30:16.356490+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356490+0000","last_undegraded":"2024-09-17T03:30:16.356490+0000","last_fullsized":"2024-09-17T03:30:16.356490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041228+0000","last_change":"2024-09-17T03:30:17.370486+0000","last_active":"2024-09-17T03:30:19.041228+0000","last_peered":"2024-09-17T03:30:19.041228+0000","last_clean":"2024-09-17T03:30:19.041228+0000","last_became_active":"2024-09-17T03:30:17.370107+0000","last_became_peered":"2024-09-17T03:30:17.370107+0000","last_unstale":"2024-09-17T03:30:19.041228+0000","last_undegraded":"2024-09-17T03:30:19.041228+0000","last_fullsized":"2024-09-17T03:30:19.041228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:27.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361223+0000","last_change":"2024-09-17T03:30:16.361223+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361223+0000","last_undegraded":"2024-09-17T03:30:16.361223+0000","last_fullsized":"2024-09-17T03:30:16.361223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350657+0000","last_change":"2024-09-17T03:30:16.350657+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350657+0000","last_undegraded":"2024-09-17T03:30:16.350657+0000","last_fullsized":"2024-09-17T03:30:16.350657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362688+0000","last_change":"2024-09-17T03:30:16.362688+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362688+0000","last_undegraded":"2024-09-17T03:30:16.362688+0000","last_fullsized":"2024-09-17T03:30:16.362688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047927+0000","last_change":"2024-09-17T03:30:17.388212+0000","last_active":"2024-09-17T03:30:19.047927+0000","last_peered":"2024-09-17T03:30:19.047927+0000","last_clean":"2024-09-17T03:30:19.047927+0000","last_became_active":"2024-09-17T03:30:17.387941+0000","last_became_peered":"2024-09-17T03:30:17.387941+0000","last_unstale":"2024-09-17T03:30:19.047927+0000","last_undegraded":"2024-09-17T03:30:19.047927+0000","last_fullsized":"2024-09-17T03:30:19.047927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:30.231182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356272+0000","last_change":"2024-09-17T03:30:16.356272+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356272+0000","last_undegraded":"2024-09-17T03:30:16.356272+0000","last_fullsized":"2024-09-17T03:30:16.356272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047750+0000","last_change":"2024-09-17T03:30:17.383305+0000","last_active":"2024-09-17T03:30:19.047750+0000","last_peered":"2024-09-17T03:30:19.047750+0000","last_clean":"2024-09-17T03:30:19.047750+0000","last_became_active":"2024-09-17T03:30:17.382995+0000","last_became_peered":"2024-09-17T03:30:17.382995+0000","last_unstale":"2024-09-17T03:30:19.047750+0000","last_undegraded":"2024-09-17T03:30:19.047750+0000","last_fullsized":"2024-09-17T03:30:19.047750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:03.641896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047928+0000","last_change":"2024-09-17T03:30:17.374521+0000","last_active":"2024-09-17T03:30:19.047928+0000","last_peered":"2024-09-17T03:30:19.047928+0000","last_clean":"2024-09-17T03:30:19.047928+0000","last_became_active":"2024-09-17T03:30:17.374185+0000","last_became_peered":"2024-09-17T03:30:17.374185+0000","last_unstale":"2024-09-17T03:30:19.047928+0000","last_undegraded":"2024-09-17T03:30:19.047928+0000","last_fullsized":"2024-09-17T03:30:19.047928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:57.551527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353169+0000","last_change":"2024-09-17T03:30:16.353169+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353169+0000","last_undegraded":"2024-09-17T03:30:16.353169+0000","last_fullsized":"2024-09-17T03:30:16.353169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040669+0000","last_change":"2024-09-17T03:30:17.373313+0000","last_active":"2024-09-17T03:30:19.040669+0000","last_peered":"2024-09-17T03:30:19.040669+0000","last_clean":"2024-09-17T03:30:19.040669+0000","last_became_active":"2024-09-17T03:30:17.372950+0000","last_became_peered":"2024-09-17T03:30:17.372950+0000","last_unstale":"2024-09-17T03:30:19.040669+0000","last_undegraded":"2024-09-17T03:30:19.040669+0000","last_fullsized":"2024-09-17T03:30:19.040669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356094+0000","last_change":"2024-09-17T03:30:16.356094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356094+0000","last_undegraded":"2024-09-17T03:30:16.356094+0000","last_fullsized":"2024-09-17T03:30:16.356094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363504+0000","last_change":"2024-09-17T03:30:16.363504+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363504+0000","last_undegraded":"2024-09-17T03:30:16.363504+0000","last_fullsized":"2024-09-17T03:30:16.363504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350224+0000","last_change":"2024-09-17T03:30:16.350224+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350224+0000","last_undegraded":"2024-09-17T03:30:16.350224+0000","last_fullsized":"2024-09-17T03:30:16.350224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348144+0000","last_change":"2024-09-17T03:30:16.348144+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348144+0000","last_undegraded":"2024-09-17T03:30:16.348144+0000","last_fullsized":"2024-09-17T03:30:16.348144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.358745+0000","last_change":"2024-09-17T03:30:16.358745+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.358745+0000","last_undegraded":"2024-09-17T03:30:16.358745+0000","last_fullsized":"2024-09-17T03:30:16.358745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.364506+0000","last_change":"2024-09-17T03:30:16.364506+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.364506+0000","last_undegraded":"2024-09-17T03:30:16.364506+0000","last_fullsized":"2024-09-17T03:30:16.364506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.351260+0000","last_change":"2024-09-17T03:30:16.351260+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.351260+0000","last_undegraded":"2024-09-17T03:30:16.351260+0000","last_fullsized":"2024-09-17T03:30:16.351260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346459+0000","last_change":"2024-09-17T03:30:16.346459+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346459+0000","last_undegraded":"2024-09-17T03:30:16.346459+0000","last_fullsized":"2024-09-17T03:30:16.346459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350038+0000","last_change":"2024-09-17T03:30:16.350038+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350038+0000","last_undegraded":"2024-09-17T03:30:16.350038+0000","last_fullsized":"2024-09-17T03:30:16.350038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349235+0000","last_change":"2024-09-17T03:30:16.349235+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349235+0000","last_undegraded":"2024-09-17T03:30:16.349235+0000","last_fullsized":"2024-09-17T03:30:16.349235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040270+0000","last_change":"2024-09-17T03:30:17.374044+0000","last_active":"2024-09-17T03:30:19.040270+0000","last_peered":"2024-09-17T03:30:19.040270+0000","last_clean":"2024-09-17T03:30:19.040270+0000","last_became_active":"2024-09-17T03:30:17.373700+0000","last_became_peered":"2024-09-17T03:30:17.373700+0000","last_unstale":"2024-09-17T03:30:19.040270+0000","last_undegraded":"2024-09-17T03:30:19.040270+0000","last_fullsized":"2024-09-17T03:30:19.040270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:04.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354041+0000","last_change":"2024-09-17T03:30:16.354041+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354041+0000","last_undegraded":"2024-09-17T03:30:16.354041+0000","last_fullsized":"2024-09-17T03:30:16.354041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349327+0000","last_change":"2024-09-17T03:30:16.349327+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349327+0000","last_undegraded":"2024-09-17T03:30:16.349327+0000","last_fullsized":"2024-09-17T03:30:16.349327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041180+0000","last_change":"2024-09-17T03:30:17.374952+0000","last_active":"2024-09-17T03:30:19.041180+0000","last_peered":"2024-09-17T03:30:19.041180+0000","last_clean":"2024-09-17T03:30:19.041180+0000","last_became_active":"2024-09-17T03:30:17.374579+0000","last_became_peered":"2024-09-17T03:30:17.374579+0000","last_unstale":"2024-09-17T03:30:19.041180+0000","last_undegraded":"2024-09-17T03:30:19.041180+0000","last_fullsized":"2024-09-17T03:30:19.041180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.728454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361780+0000","last_change":"2024-09-17T03:30:16.361780+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361780+0000","last_undegraded":"2024-09-17T03:30:16.361780+0000","last_fullsized":"2024-09-17T03:30:16.361780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.367168+0000","last_change":"2024-09-17T03:30:16.367168+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.367168+0000","last_undegraded":"2024-09-17T03:30:16.367168+0000","last_fullsized":"2024-09-17T03:30:16.367168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.355538+0000","last_change":"2024-09-17T03:30:16.355538+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.355538+0000","last_undegraded":"2024-09-17T03:30:16.355538+0000","last_fullsized":"2024-09-17T03:30:16.355538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040169+0000","last_change":"2024-09-17T03:30:17.378897+0000","last_active":"2024-09-17T03:30:19.040169+0000","last_peered":"2024-09-17T03:30:19.040169+0000","last_clean":"2024-09-17T03:30:19.040169+0000","last_became_active":"2024-09-17T03:30:17.378212+0000","last_became_peered":"2024-09-17T03:30:17.378212+0000","last_unstale":"2024-09-17T03:30:19.040169+0000","last_undegraded":"2024-09-17T03:30:19.040169+0000","last_fullsized":"2024-09-17T03:30:19.040169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:13.896559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040290+0000","last_change":"2024-09-17T03:30:17.382864+0000","last_active":"2024-09-17T03:30:19.040290+0000","last_peered":"2024-09-17T03:30:19.040290+0000","last_clean":"2024-09-17T03:30:19.040290+0000","last_became_active":"2024-09-17T03:30:17.381934+0000","last_became_peered":"2024-09-17T03:30:17.381934+0000","last_unstale":"2024-09-17T03:30:19.040290+0000","last_undegraded":"2024-09-17T03:30:19.040290+0000","last_fullsized":"2024-09-17T03:30:19.040290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:12.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360461+0000","last_change":"2024-09-17T03:30:16.360461+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360461+0000","last_undegraded":"2024-09-17T03:30:16.360461+0000","last_fullsized":"2024-09-17T03:30:16.360461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353448+0000","last_change":"2024-09-17T03:30:16.353448+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353448+0000","last_undegraded":"2024-09-17T03:30:16.353448+0000","last_fullsized":"2024-09-17T03:30:16.353448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363769+0000","last_change":"2024-09-17T03:30:16.363769+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363769+0000","last_undegraded":"2024-09-17T03:30:16.363769+0000","last_fullsized":"2024-09-17T03:30:16.363769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371726+0000","last_change":"2024-09-17T03:30:16.366033+0000","last_active":"2024-09-17T03:30:16.371726+0000","last_peered":"2024-09-17T03:30:16.371726+0000","last_clean":"2024-09-17T03:30:16.371726+0000","last_became_active":"2024-09-17T03:30:16.365643+0000","last_became_peered":"2024-09-17T03:30:16.365643+0000","last_unstale":"2024-09-17T03:30:16.371726+0000","last_undegraded":"2024-09-17T03:30:16.371726+0000","last_fullsized":"2024-09-17T03:30:16.371726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:05.152896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354849+0000","last_change":"2024-09-17T03:30:16.354849+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354849+0000","last_undegraded":"2024-09-17T03:30:16.354849+0000","last_fullsized":"2024-09-17T03:30:16.354849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375348+0000","last_change":"2024-09-17T03:30:16.374265+0000","last_active":"2024-09-17T03:30:16.375348+0000","last_peered":"2024-09-17T03:30:16.375348+0000","last_clean":"2024-09-17T03:30:16.375348+0000","last_became_active":"2024-09-17T03:30:16.372671+0000","last_became_peered":"2024-09-17T03:30:16.372671+0000","last_unstale":"2024-09-17T03:30:16.375348+0000","last_undegraded":"2024-09-17T03:30:16.375348+0000","last_fullsized":"2024-09-17T03:30:16.375348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:20.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354990+0000","last_change":"2024-09-17T03:30:16.354990+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354990+0000","last_undegraded":"2024-09-17T03:30:16.354990+0000","last_fullsized":"2024-09-17T03:30:16.354990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.359637+0000","last_change":"2024-09-17T03:30:16.355580+0000","last_active":"2024-09-17T03:30:16.359637+0000","last_peered":"2024-09-17T03:30:16.359637+0000","last_clean":"2024-09-17T03:30:16.359637+0000","last_became_active":"2024-09-17T03:30:16.355240+0000","last_became_peered":"2024-09-17T03:30:16.355240+0000","last_unstale":"2024-09-17T03:30:16.359637+0000","last_undegraded":"2024-09-17T03:30:16.359637+0000","last_fullsized":"2024-09-17T03:30:16.359637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:31.022404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360687+0000","last_change":"2024-09-17T03:30:16.360687+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360687+0000","last_undegraded":"2024-09-17T03:30:16.360687+0000","last_fullsized":"2024-09-17T03:30:16.360687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361157+0000","last_change":"2024-09-17T03:30:16.357957+0000","last_active":"2024-09-17T03:30:16.361157+0000","last_peered":"2024-09-17T03:30:16.361157+0000","last_clean":"2024-09-17T03:30:16.361157+0000","last_became_active":"2024-09-17T03:30:16.357597+0000","last_became_peered":"2024-09-17T03:30:16.357597+0000","last_unstale":"2024-09-17T03:30:16.361157+0000","last_undegraded":"2024-09-17T03:30:16.361157+0000","last_fullsized":"2024-09-17T03:30:16.361157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:25.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040526+0000","last_change":"2024-09-17T03:30:17.382905+0000","last_active":"2024-09-17T03:30:19.040526+0000","last_peered":"2024-09-17T03:30:19.040526+0000","last_clean":"2024-09-17T03:30:19.040526+0000","last_became_active":"2024-09-17T03:30:17.382343+0000","last_became_peered":"2024-09-17T03:30:17.382343+0000","last_unstale":"2024-09-17T03:30:19.040526+0000","last_undegraded":"2024-09-17T03:30:19.040526+0000","last_fullsized":"2024-09-17T03:30:19.040526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:08.921865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376896+0000","last_change":"2024-09-17T03:30:16.375643+0000","last_active":"2024-09-17T03:30:16.376896+0000","last_peered":"2024-09-17T03:30:16.376896+0000","last_clean":"2024-09-17T03:30:16.376896+0000","last_became_active":"2024-09-17T03:30:16.375378+0000","last_became_peered":"2024-09-17T03:30:16.375378+0000","last_unstale":"2024-09-17T03:30:16.376896+0000","last_undegraded":"2024-09-17T03:30:16.376896+0000","last_fullsized":"2024-09-17T03:30:16.376896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.037881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362120+0000","last_change":"2024-09-17T03:30:16.362120+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362120+0000","last_undegraded":"2024-09-17T03:30:16.362120+0000","last_fullsized":"2024-09-17T03:30:16.362120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375682+0000","last_change":"2024-09-17T03:30:16.374780+0000","last_active":"2024-09-17T03:30:16.375682+0000","last_peered":"2024-09-17T03:30:16.375682+0000","last_clean":"2024-09-17T03:30:16.375682+0000","last_became_active":"2024-09-17T03:30:16.374312+0000","last_became_peered":"2024-09-17T03:30:16.374312+0000","last_unstale":"2024-09-17T03:30:16.375682+0000","last_undegraded":"2024-09-17T03:30:16.375682+0000","last_fullsized":"2024-09-17T03:30:16.375682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.479057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362017+0000","last_change":"2024-09-17T03:30:16.362017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362017+0000","last_undegraded":"2024-09-17T03:30:16.362017+0000","last_fullsized":"2024-09-17T03:30:16.362017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363741+0000","last_change":"2024-09-17T03:30:16.360075+0000","last_active":"2024-09-17T03:30:16.363741+0000","last_peered":"2024-09-17T03:30:16.363741+0000","last_clean":"2024-09-17T03:30:16.363741+0000","last_became_active":"2024-09-17T03:30:16.359771+0000","last_became_peered":"2024-09-17T03:30:16.359771+0000","last_unstale":"2024-09-17T03:30:16.363741+0000","last_undegraded":"2024-09-17T03:30:16.363741+0000","last_fullsized":"2024-09-17T03:30:16.363741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:01.924364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.347497+0000","last_change":"2024-09-17T03:30:16.347497+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.347497+0000","last_undegraded":"2024-09-17T03:30:16.347497+0000","last_fullsized":"2024-09-17T03:30:16.347497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039799+0000","last_change":"2024-09-17T03:30:16.365783+0000","last_active":"2024-09-17T03:30:19.039799+0000","last_peered":"2024-09-17T03:30:19.039799+0000","last_clean":"2024-09-17T03:30:19.039799+0000","last_became_active":"2024-09-17T03:30:16.365439+0000","last_became_peered":"2024-09-17T03:30:16.365439+0000","last_unstale":"2024-09-17T03:30:19.039799+0000","last_undegraded":"2024-09-17T03:30:19.039799+0000","last_fullsized":"2024-09-17T03:30:19.039799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:48.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348017+0000","last_change":"2024-09-17T03:30:16.348017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348017+0000","last_undegraded":"2024-09-17T03:30:16.348017+0000","last_fullsized":"2024-09-17T03:30:16.348017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371783+0000","last_change":"2024-09-17T03:30:16.369999+0000","last_active":"2024-09-17T03:30:16.371783+0000","last_peered":"2024-09-17T03:30:16.371783+0000","last_clean":"2024-09-17T03:30:16.371783+0000","last_became_active":"2024-09-17T03:30:16.368392+0000","last_became_peered":"2024-09-17T03:30:16.368392+0000","last_unstale":"2024-09-17T03:30:16.371783+0000","last_undegraded":"2024-09-17T03:30:16.371783+0000","last_fullsized":"2024-09-17T03:30:16.371783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:10.071007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047957+0000","last_change":"2024-09-17T03:30:17.380149+0000","last_active":"2024-09-17T03:30:19.047957+0000","last_peered":"2024-09-17T03:30:19.047957+0000","last_clean":"2024-09-17T03:30:19.047957+0000","last_became_active":"2024-09-17T03:30:17.379829+0000","last_became_peered":"2024-09-17T03:30:17.379829+0000","last_unstale":"2024-09-17T03:30:19.047957+0000","last_undegraded":"2024-09-17T03:30:19.047957+0000","last_fullsized":"2024-09-17T03:30:19.047957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:17.605539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.374805+0000","last_change":"2024-09-17T03:30:16.373477+0000","last_active":"2024-09-17T03:30:16.374805+0000","last_peered":"2024-09-17T03:30:16.374805+0000","last_clean":"2024-09-17T03:30:16.374805+0000","last_became_active":"2024-09-17T03:30:16.373060+0000","last_became_peered":"2024-09-17T03:30:16.373060+0000","last_unstale":"2024-09-17T03:30:16.374805+0000","last_undegraded":"2024-09-17T03:30:16.374805+0000","last_fullsized":"2024-09-17T03:30:16.374805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:06.209920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346582+0000","last_change":"2024-09-17T03:30:16.346582+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346582+0000","last_undegraded":"2024-09-17T03:30:16.346582+0000","last_fullsized":"2024-09-17T03:30:16.346582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376938+0000","last_change":"2024-09-17T03:30:16.376370+0000","last_active":"2024-09-17T03:30:16.376938+0000","last_peered":"2024-09-17T03:30:16.376938+0000","last_clean":"2024-09-17T03:30:16.376938+0000","last_became_active":"2024-09-17T03:30:16.376027+0000","last_became_peered":"2024-09-17T03:30:16.376027+0000","last_unstale":"2024-09-17T03:30:16.376938+0000","last_undegraded":"2024-09-17T03:30:16.376938+0000","last_fullsized":"2024-09-17T03:30:16.376938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:51.451736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366652+0000","last_change":"2024-09-17T03:30:16.366652+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366652+0000","last_undegraded":"2024-09-17T03:30:16.366652+0000","last_fullsized":"2024-09-17T03:30:16.366652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047718+0000","last_change":"2024-09-17T03:30:16.370573+0000","last_active":"2024-09-17T03:30:19.047718+0000","last_peered":"2024-09-17T03:30:19.047718+0000","last_clean":"2024-09-17T03:30:19.047718+0000","last_became_active":"2024-09-17T03:30:16.369166+0000","last_became_peered":"2024-09-17T03:30:16.369166+0000","last_unstale":"2024-09-17T03:30:19.047718+0000","last_undegraded":"2024-09-17T03:30:19.047718+0000","last_fullsized":"2024-09-17T03:30:19.047718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:23.751200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346578+0000","last_change":"2024-09-17T03:30:16.346578+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346578+0000","last_undegraded":"2024-09-17T03:30:16.346578+0000","last_fullsized":"2024-09-17T03:30:16.346578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040259+0000","last_change":"2024-09-17T03:30:16.367290+0000","last_active":"2024-09-17T03:30:19.040259+0000","last_peered":"2024-09-17T03:30:19.040259+0000","last_clean":"2024-09-17T03:30:19.040259+0000","last_became_active":"2024-09-17T03:30:16.365363+0000","last_became_peered":"2024-09-17T03:30:16.365363+0000","last_unstale":"2024-09-17T03:30:19.040259+0000","last_undegraded":"2024-09-17T03:30:19.040259+0000","last_fullsized":"2024-09-17T03:30:19.040259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:46.817707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363020+0000","last_change":"2024-09-17T03:30:16.363020+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363020+0000","last_undegraded":"2024-09-17T03:30:16.363020+0000","last_fullsized":"2024-09-17T03:30:16.363020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371240+0000","last_change":"2024-09-17T03:30:16.369373+0000","last_active":"2024-09-17T03:30:16.371240+0000","last_peered":"2024-09-17T03:30:16.371240+0000","last_clean":"2024-09-17T03:30:16.371240+0000","last_became_active":"2024-09-17T03:30:16.369082+0000","last_became_peered":"2024-09-17T03:30:16.369082+0000","last_unstale":"2024-09-17T03:30:16.371240+0000","last_undegraded":"2024-09-17T03:30:16.371240+0000","last_fullsized":"2024-09-17T03:30:16.371240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:46.125971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039947+0000","last_change":"2024-09-17T03:30:17.382255+0000","last_active":"2024-09-17T03:30:19.039947+0000","last_peered":"2024-09-17T03:30:19.039947+0000","last_clean":"2024-09-17T03:30:19.039947+0000","last_became_active":"2024-09-17T03:30:17.381901+0000","last_became_peered":"2024-09-17T03:30:17.381901+0000","last_unstale":"2024-09-17T03:30:19.039947+0000","last_undegraded":"2024-09-17T03:30:19.039947+0000","last_fullsized":"2024-09-17T03:30:19.039947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.620927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047671+0000","last_change":"2024-09-17T03:30:16.371647+0000","last_active":"2024-09-17T03:30:19.047671+0000","last_peered":"2024-09-17T03:30:19.047671+0000","last_clean":"2024-09-17T03:30:19.047671+0000","last_became_active":"2024-09-17T03:30:16.371336+0000","last_became_peered":"2024-09-17T03:30:16.371336+0000","last_unstale":"2024-09-17T03:30:19.047671+0000","last_undegraded":"2024-09-17T03:30:19.047671+0000","last_fullsized":"2024-09-17T03:30:19.047671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:19.546103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039882+0000","last_change":"2024-09-17T03:30:17.372667+0000","last_active":"2024-09-17T03:30:19.039882+0000","last_peered":"2024-09-17T03:30:19.039882+0000","last_clean":"2024-09-17T03:30:19.039882+0000","last_became_active":"2024-09-17T03:30:17.372310+0000","last_became_peered":"2024-09-17T03:30:17.372310+0000","last_unstale":"2024-09-17T03:30:19.039882+0000","last_undegraded":"2024-09-17T03:30:19.039882+0000","last_fullsized":"2024-09-17T03:30:19.039882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:46.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363382+0000","last_change":"2024-09-17T03:30:16.359214+0000","last_active":"2024-09-17T03:30:16.363382+0000","last_peered":"2024-09-17T03:30:16.363382+0000","last_clean":"2024-09-17T03:30:16.363382+0000","last_became_active":"2024-09-17T03:30:16.358883+0000","last_became_peered":"2024-09-17T03:30:16.358883+0000","last_unstale":"2024-09-17T03:30:16.363382+0000","last_undegraded":"2024-09-17T03:30:16.363382+0000","last_fullsized":"2024-09-17T03:30:16.363382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:47.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361307+0000","last_change":"2024-09-17T03:30:16.361307+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361307+0000","last_undegraded":"2024-09-17T03:30:16.361307+0000","last_fullsized":"2024-09-17T03:30:16.361307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371756+0000","last_change":"2024-09-17T03:30:16.369933+0000","last_active":"2024-09-17T03:30:16.371756+0000","last_peered":"2024-09-17T03:30:16.371756+0000","last_clean":"2024-09-17T03:30:16.371756+0000","last_became_active":"2024-09-17T03:30:16.369307+0000","last_became_peered":"2024-09-17T03:30:16.369307+0000","last_unstale":"2024-09-17T03:30:16.371756+0000","last_undegraded":"2024-09-17T03:30:16.371756+0000","last_fullsized":"2024-09-17T03:30:16.371756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:29.392547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354255+0000","last_change":"2024-09-17T03:30:16.354255+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354255+0000","last_undegraded":"2024-09-17T03:30:16.354255+0000","last_fullsized":"2024-09-17T03:30:16.354255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367756+0000","last_change":"2024-09-17T03:30:16.364256+0000","last_active":"2024-09-17T03:30:16.367756+0000","last_peered":"2024-09-17T03:30:16.367756+0000","last_clean":"2024-09-17T03:30:16.367756+0000","last_became_active":"2024-09-17T03:30:16.363880+0000","last_became_peered":"2024-09-17T03:30:16.363880+0000","last_unstale":"2024-09-17T03:30:16.367756+0000","last_undegraded":"2024-09-17T03:30:16.367756+0000","last_fullsized":"2024-09-17T03:30:16.367756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366139+0000","last_change":"2024-09-17T03:30:16.366139+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366139+0000","last_undegraded":"2024-09-17T03:30:16.366139+0000","last_fullsized":"2024-09-17T03:30:16.366139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364390+0000","last_change":"2024-09-17T03:30:16.361309+0000","last_active":"2024-09-17T03:30:16.364390+0000","last_peered":"2024-09-17T03:30:16.364390+0000","last_clean":"2024-09-17T03:30:16.364390+0000","last_became_active":"2024-09-17T03:30:16.360955+0000","last_became_peered":"2024-09-17T03:30:16.360955+0000","last_unstale":"2024-09-17T03:30:16.364390+0000","last_undegraded":"2024-09-17T03:30:16.364390+0000","last_fullsized":"2024-09-17T03:30:16.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:33.036462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349551+0000","last_change":"2024-09-17T03:30:16.349551+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349551+0000","last_undegraded":"2024-09-17T03:30:16.349551+0000","last_fullsized":"2024-09-17T03:30:16.349551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.366789+0000","last_change":"2024-09-17T03:30:16.359453+0000","last_active":"2024-09-17T03:30:16.366789+0000","last_peered":"2024-09-17T03:30:16.366789+0000","last_clean":"2024-09-17T03:30:16.366789+0000","last_became_active":"2024-09-17T03:30:16.358930+0000","last_became_peered":"2024-09-17T03:30:16.358930+0000","last_unstale":"2024-09-17T03:30:16.366789+0000","last_undegraded":"2024-09-17T03:30:16.366789+0000","last_fullsized":"2024-09-17T03:30:16.366789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:34.925865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048154+0000","last_change":"2024-09-17T03:30:17.380496+0000","last_active":"2024-09-17T03:30:19.048154+0000","last_peered":"2024-09-17T03:30:19.048154+0000","last_clean":"2024-09-17T03:30:19.048154+0000","last_became_active":"2024-09-17T03:30:17.380195+0000","last_became_peered":"2024-09-17T03:30:17.380195+0000","last_unstale":"2024-09-17T03:30:19.048154+0000","last_undegraded":"2024-09-17T03:30:19.048154+0000","last_fullsized":"2024-09-17T03:30:19.048154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:32.240069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370053+0000","last_change":"2024-09-17T03:30:16.366802+0000","last_active":"2024-09-17T03:30:16.370053+0000","last_peered":"2024-09-17T03:30:16.370053+0000","last_clean":"2024-09-17T03:30:16.370053+0000","last_became_active":"2024-09-17T03:30:16.366497+0000","last_became_peered":"2024-09-17T03:30:16.366497+0000","last_unstale":"2024-09-17T03:30:16.370053+0000","last_undegraded":"2024-09-17T03:30:16.370053+0000","last_fullsized":"2024-09-17T03:30:16.370053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:31.288276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348588+0000","last_change":"2024-09-17T03:30:16.348588+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348588+0000","last_undegraded":"2024-09-17T03:30:16.348588+0000","last_fullsized":"2024-09-17T03:30:16.348588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367253+0000","last_change":"2024-09-17T03:30:16.362330+0000","last_active":"2024-09-17T03:30:16.367253+0000","last_peered":"2024-09-17T03:30:16.367253+0000","last_clean":"2024-09-17T03:30:16.367253+0000","last_became_active":"2024-09-17T03:30:16.361960+0000","last_became_peered":"2024-09-17T03:30:16.361960+0000","last_unstale":"2024-09-17T03:30:16.367253+0000","last_undegraded":"2024-09-17T03:30:16.367253+0000","last_fullsized":"2024-09-17T03:30:16.367253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:27.002346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047756+0000","last_change":"2024-09-17T03:30:17.373912+0000","last_active":"2024-09-17T03:30:19.047756+0000","last_peered":"2024-09-17T03:30:19.047756+0000","last_clean":"2024-09-17T03:30:19.047756+0000","last_became_active":"2024-09-17T03:30:17.373542+0000","last_became_peered":"2024-09-17T03:30:17.373542+0000","last_unstale":"2024-09-17T03:30:19.047756+0000","last_undegraded":"2024-09-17T03:30:19.047756+0000","last_fullsized":"2024-09-17T03:30:19.047756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:28.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361044+0000","last_change":"2024-09-17T03:30:16.354858+0000","last_active":"2024-09-17T03:30:16.361044+0000","last_peered":"2024-09-17T03:30:16.361044+0000","last_clean":"2024-09-17T03:30:16.361044+0000","last_became_active":"2024-09-17T03:30:16.354399+0000","last_became_peered":"2024-09-17T03:30:16.354399+0000","last_unstale":"2024-09-17T03:30:16.361044+0000","last_undegraded":"2024-09-17T03:30:16.361044+0000","last_fullsized":"2024-09-17T03:30:16.361044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:47.686970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363094+0000","last_change":"2024-09-17T03:30:16.363094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363094+0000","last_undegraded":"2024-09-17T03:30:16.363094+0000","last_fullsized":"2024-09-17T03:30:16.363094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041218+0000","last_change":"2024-09-17T03:30:16.368986+0000","last_active":"2024-09-17T03:30:19.041218+0000","last_peered":"2024-09-17T03:30:19.041218+0000","last_clean":"2024-09-17T03:30:19.041218+0000","last_became_active":"2024-09-17T03:30:16.368638+0000","last_became_peered":"2024-09-17T03:30:16.368638+0000","last_unstale":"2024-09-17T03:30:19.041218+0000","last_undegraded":"2024-09-17T03:30:19.041218+0000","last_fullsized":"2024-09-17T03:30:19.041218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:04.104352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360154+0000","last_change":"2024-09-17T03:30:16.360154+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360154+0000","last_undegraded":"2024-09-17T03:30:16.360154+0000","last_fullsized":"2024-09-17T03:30:16.360154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041203+0000","last_change":"2024-09-17T03:30:16.371245+0000","last_active":"2024-09-17T03:30:19.041203+0000","last_peered":"2024-09-17T03:30:19.041203+0000","last_clean":"2024-09-17T03:30:19.041203+0000","last_became_active":"2024-09-17T03:30:16.370661+0000","last_became_peered":"2024-09-17T03:30:16.370661+0000","last_unstale":"2024-09-17T03:30:19.041203+0000","last_undegraded":"2024-09-17T03:30:19.041203+0000","last_fullsized":"2024-09-17T03:30:19.041203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:47.753727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353740+0000","last_change":"2024-09-17T03:30:16.353740+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353740+0000","last_undegraded":"2024-09-17T03:30:16.353740+0000","last_fullsized":"2024-09-17T03:30:16.353740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370305+0000","last_change":"2024-09-17T03:30:16.366822+0000","last_active":"2024-09-17T03:30:16.370305+0000","last_peered":"2024-09-17T03:30:16.370305+0000","last_clean":"2024-09-17T03:30:16.370305+0000","last_became_active":"2024-09-17T03:30:16.364923+0000","last_became_peered":"2024-09-17T03:30:16.364923+0000","last_unstale":"2024-09-17T03:30:16.370305+0000","last_undegraded":"2024-09-17T03:30:16.370305+0000","last_fullsized":"2024-09-17T03:30:16.370305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:02.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363106+0000","last_change":"2024-09-17T03:30:16.363106+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363106+0000","last_undegraded":"2024-09-17T03:30:16.363106+0000","last_fullsized":"2024-09-17T03:30:16.363106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371866+0000","last_change":"2024-09-17T03:30:16.369886+0000","last_active":"2024-09-17T03:30:16.371866+0000","last_peered":"2024-09-17T03:30:16.371866+0000","last_clean":"2024-09-17T03:30:16.371866+0000","last_became_active":"2024-09-17T03:30:16.369399+0000","last_became_peered":"2024-09-17T03:30:16.369399+0000","last_unstale":"2024-09-17T03:30:16.371866+0000","last_undegraded":"2024-09-17T03:30:16.371866+0000","last_fullsized":"2024-09-17T03:30:16.371866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:30.365530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047759+0000","last_change":"2024-09-17T03:30:17.372555+0000","last_active":"2024-09-17T03:30:19.047759+0000","last_peered":"2024-09-17T03:30:19.047759+0000","last_clean":"2024-09-17T03:30:19.047759+0000","last_became_active":"2024-09-17T03:30:17.372225+0000","last_became_peered":"2024-09-17T03:30:17.372225+0000","last_unstale":"2024-09-17T03:30:19.047759+0000","last_undegraded":"2024-09-17T03:30:19.047759+0000","last_fullsized":"2024-09-17T03:30:19.047759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:49.139488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363251+0000","last_change":"2024-09-17T03:30:16.359549+0000","last_active":"2024-09-17T03:30:16.363251+0000","last_peered":"2024-09-17T03:30:16.363251+0000","last_clean":"2024-09-17T03:30:16.363251+0000","last_became_active":"2024-09-17T03:30:16.359149+0000","last_became_peered":"2024-09-17T03:30:16.359149+0000","last_unstale":"2024-09-17T03:30:16.363251+0000","last_undegraded":"2024-09-17T03:30:16.363251+0000","last_fullsized":"2024-09-17T03:30:16.363251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:15.534467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346444+0000","last_change":"2024-09-17T03:30:16.346444+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346444+0000","last_undegraded":"2024-09-17T03:30:16.346444+0000","last_fullsized":"2024-09-17T03:30:16.346444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375718+0000","last_change":"2024-09-17T03:30:16.374752+0000","last_active":"2024-09-17T03:30:16.375718+0000","last_peered":"2024-09-17T03:30:16.375718+0000","last_clean":"2024-09-17T03:30:16.375718+0000","last_became_active":"2024-09-17T03:30:16.374286+0000","last_became_peered":"2024-09-17T03:30:16.374286+0000","last_unstale":"2024-09-17T03:30:16.375718+0000","last_undegraded":"2024-09-17T03:30:16.375718+0000","last_fullsized":"2024-09-17T03:30:16.375718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:38.480874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361539+0000","last_change":"2024-09-17T03:30:16.361539+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361539+0000","last_undegraded":"2024-09-17T03:30:16.361539+0000","last_fullsized":"2024-09-17T03:30:16.361539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371481+0000","last_change":"2024-09-17T03:30:16.367991+0000","last_active":"2024-09-17T03:30:16.371481+0000","last_peered":"2024-09-17T03:30:16.371481+0000","last_clean":"2024-09-17T03:30:16.371481+0000","last_became_active":"2024-09-17T03:30:16.367669+0000","last_became_peered":"2024-09-17T03:30:16.367669+0000","last_unstale":"2024-09-17T03:30:16.371481+0000","last_undegraded":"2024-09-17T03:30:16.371481+0000","last_fullsized":"2024-09-17T03:30:16.371481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:30.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353079+0000","last_change":"2024-09-17T03:30:16.353079+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353079+0000","last_undegraded":"2024-09-17T03:30:16.353079+0000","last_fullsized":"2024-09-17T03:30:16.353079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.372120+0000","last_change":"2024-09-17T03:30:16.367493+0000","last_active":"2024-09-17T03:30:16.372120+0000","last_peered":"2024-09-17T03:30:16.372120+0000","last_clean":"2024-09-17T03:30:16.372120+0000","last_became_active":"2024-09-17T03:30:16.367090+0000","last_became_peered":"2024-09-17T03:30:16.367090+0000","last_unstale":"2024-09-17T03:30:16.372120+0000","last_undegraded":"2024-09-17T03:30:16.372120+0000","last_fullsized":"2024-09-17T03:30:16.372120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:47.067459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040368+0000","last_change":"2024-09-17T03:30:17.374985+0000","last_active":"2024-09-17T03:30:19.040368+0000","last_peered":"2024-09-17T03:30:19.040368+0000","last_clean":"2024-09-17T03:30:19.040368+0000","last_became_active":"2024-09-17T03:30:17.373033+0000","last_became_peered":"2024-09-17T03:30:17.373033+0000","last_unstale":"2024-09-17T03:30:19.040368+0000","last_undegraded":"2024-09-17T03:30:19.040368+0000","last_fullsized":"2024-09-17T03:30:19.040368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:56.073407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.377262+0000","last_change":"2024-09-17T03:30:16.376615+0000","last_active":"2024-09-17T03:30:16.377262+0000","last_peered":"2024-09-17T03:30:16.377262+0000","last_clean":"2024-09-17T03:30:16.377262+0000","last_became_active":"2024-09-17T03:30:16.376148+0000","last_became_peered":"2024-09-17T03:30:16.376148+0000","last_unstale":"2024-09-17T03:30:16.377262+0000","last_undegraded":"2024-09-17T03:30:16.377262+0000","last_fullsized":"2024-09-17T03:30:16.377262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364429+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:30:16.364429+0000","last_peered":"2024-09-17T03:30:16.364429+0000","last_clean":"2024-09-17T03:30:16.364429+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:30:16.364429+0000","last_undegraded":"2024-09-17T03:30:16.364429+0000","last_fullsized":"2024-09-17T03:30:16.364429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64652,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307188,"statfs":{"total":96636764160,"available":96570560512,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64072,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307768,"statfs":{"total":96636764160,"available":96571154432,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62716,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309124,"statfs":{"total":96636764160,"available":96572542976,"internally_reserved":0,"allocated":1024000,"data_stored":435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72392,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299448,"statfs":{"total":96636764160,"available":96562634752,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68240,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68816,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303024,"statfs":{"total":96636764160,"available":96566296576,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1077248,"data_stored":456225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68268,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303572,"statfs":{"total":96636764160,"available":96566857728,"internally_reserved":0,"allocated":1073152,"data_stored":454785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:30:22.390 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3f is not active+clean 2024-09-17T03:30:22.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.356490+0000', 'last_change': '2024-09-17T03:30:16.356490+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.356490+0000', 'last_undegraded': '2024-09-17T03:30:16.356490+0000', 'last_fullsized': '2024-09-17T03:30:16.356490+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.390 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3d is not active+clean 2024-09-17T03:30:22.390 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.361223+0000', 'last_change': '2024-09-17T03:30:16.361223+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.361223+0000', 'last_undegraded': '2024-09-17T03:30:16.361223+0000', 'last_fullsized': '2024-09-17T03:30:16.361223+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3c is not active+clean 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.350657+0000', 'last_change': '2024-09-17T03:30:16.350657+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.350657+0000', 'last_undegraded': '2024-09-17T03:30:16.350657+0000', 'last_fullsized': '2024-09-17T03:30:16.350657+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3b is not active+clean 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.362688+0000', 'last_change': '2024-09-17T03:30:16.362688+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.362688+0000', 'last_undegraded': '2024-09-17T03:30:16.362688+0000', 'last_fullsized': '2024-09-17T03:30:16.362688+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:PG 13.39 is not active+clean 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.356272+0000', 'last_change': '2024-09-17T03:30:16.356272+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.356272+0000', 'last_undegraded': '2024-09-17T03:30:16.356272+0000', 'last_fullsized': '2024-09-17T03:30:16.356272+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.391 INFO:tasks.ceph.ceph_manager.ceph:PG 13.36 is not active+clean 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.353169+0000', 'last_change': '2024-09-17T03:30:16.353169+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.353169+0000', 'last_undegraded': '2024-09-17T03:30:16.353169+0000', 'last_fullsized': '2024-09-17T03:30:16.353169+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:PG 13.34 is not active+clean 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.356094+0000', 'last_change': '2024-09-17T03:30:16.356094+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.356094+0000', 'last_undegraded': '2024-09-17T03:30:16.356094+0000', 'last_fullsized': '2024-09-17T03:30:16.356094+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:PG 13.33 is not active+clean 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.363504+0000', 'last_change': '2024-09-17T03:30:16.363504+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.363504+0000', 'last_undegraded': '2024-09-17T03:30:16.363504+0000', 'last_fullsized': '2024-09-17T03:30:16.363504+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:PG 13.32 is not active+clean 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.350224+0000', 'last_change': '2024-09-17T03:30:16.350224+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.350224+0000', 'last_undegraded': '2024-09-17T03:30:16.350224+0000', 'last_fullsized': '2024-09-17T03:30:16.350224+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:PG 13.31 is not active+clean 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.348144+0000', 'last_change': '2024-09-17T03:30:16.348144+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.348144+0000', 'last_undegraded': '2024-09-17T03:30:16.348144+0000', 'last_fullsized': '2024-09-17T03:30:16.348144+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:PG 13.30 is not active+clean 2024-09-17T03:30:22.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.358745+0000', 'last_change': '2024-09-17T03:30:16.358745+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.358745+0000', 'last_undegraded': '2024-09-17T03:30:16.358745+0000', 'last_fullsized': '2024-09-17T03:30:16.358745+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2f is not active+clean 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.364506+0000', 'last_change': '2024-09-17T03:30:16.364506+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.364506+0000', 'last_undegraded': '2024-09-17T03:30:16.364506+0000', 'last_fullsized': '2024-09-17T03:30:16.364506+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2e is not active+clean 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.351260+0000', 'last_change': '2024-09-17T03:30:16.351260+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.351260+0000', 'last_undegraded': '2024-09-17T03:30:16.351260+0000', 'last_fullsized': '2024-09-17T03:30:16.351260+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2d is not active+clean 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.346459+0000', 'last_change': '2024-09-17T03:30:16.346459+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.346459+0000', 'last_undegraded': '2024-09-17T03:30:16.346459+0000', 'last_fullsized': '2024-09-17T03:30:16.346459+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2c is not active+clean 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.350038+0000', 'last_change': '2024-09-17T03:30:16.350038+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.350038+0000', 'last_undegraded': '2024-09-17T03:30:16.350038+0000', 'last_fullsized': '2024-09-17T03:30:16.350038+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2b is not active+clean 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.349235+0000', 'last_change': '2024-09-17T03:30:16.349235+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.349235+0000', 'last_undegraded': '2024-09-17T03:30:16.349235+0000', 'last_fullsized': '2024-09-17T03:30:16.349235+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:PG 13.29 is not active+clean 2024-09-17T03:30:22.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.354041+0000', 'last_change': '2024-09-17T03:30:16.354041+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.354041+0000', 'last_undegraded': '2024-09-17T03:30:16.354041+0000', 'last_fullsized': '2024-09-17T03:30:16.354041+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:PG 13.28 is not active+clean 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.349327+0000', 'last_change': '2024-09-17T03:30:16.349327+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.349327+0000', 'last_undegraded': '2024-09-17T03:30:16.349327+0000', 'last_fullsized': '2024-09-17T03:30:16.349327+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:PG 13.26 is not active+clean 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.361780+0000', 'last_change': '2024-09-17T03:30:16.361780+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.361780+0000', 'last_undegraded': '2024-09-17T03:30:16.361780+0000', 'last_fullsized': '2024-09-17T03:30:16.361780+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:PG 13.25 is not active+clean 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.367168+0000', 'last_change': '2024-09-17T03:30:16.367168+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.367168+0000', 'last_undegraded': '2024-09-17T03:30:16.367168+0000', 'last_fullsized': '2024-09-17T03:30:16.367168+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:PG 13.24 is not active+clean 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.355538+0000', 'last_change': '2024-09-17T03:30:16.355538+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.355538+0000', 'last_undegraded': '2024-09-17T03:30:16.355538+0000', 'last_fullsized': '2024-09-17T03:30:16.355538+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:PG 13.21 is not active+clean 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.360461+0000', 'last_change': '2024-09-17T03:30:16.360461+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.360461+0000', 'last_undegraded': '2024-09-17T03:30:16.360461+0000', 'last_fullsized': '2024-09-17T03:30:16.360461+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.394 INFO:tasks.ceph.ceph_manager.ceph:PG 13.20 is not active+clean 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.353448+0000', 'last_change': '2024-09-17T03:30:16.353448+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.353448+0000', 'last_undegraded': '2024-09-17T03:30:16.353448+0000', 'last_fullsized': '2024-09-17T03:30:16.353448+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1e is not active+clean 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.363769+0000', 'last_change': '2024-09-17T03:30:16.363769+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.363769+0000', 'last_undegraded': '2024-09-17T03:30:16.363769+0000', 'last_fullsized': '2024-09-17T03:30:16.363769+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1f is not active+clean 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.354849+0000', 'last_change': '2024-09-17T03:30:16.354849+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.354849+0000', 'last_undegraded': '2024-09-17T03:30:16.354849+0000', 'last_fullsized': '2024-09-17T03:30:16.354849+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1c is not active+clean 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.354990+0000', 'last_change': '2024-09-17T03:30:16.354990+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.354990+0000', 'last_undegraded': '2024-09-17T03:30:16.354990+0000', 'last_fullsized': '2024-09-17T03:30:16.354990+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1d is not active+clean 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.360687+0000', 'last_change': '2024-09-17T03:30:16.360687+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.360687+0000', 'last_undegraded': '2024-09-17T03:30:16.360687+0000', 'last_fullsized': '2024-09-17T03:30:16.360687+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1b is not active+clean 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.362120+0000', 'last_change': '2024-09-17T03:30:16.362120+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.362120+0000', 'last_undegraded': '2024-09-17T03:30:16.362120+0000', 'last_fullsized': '2024-09-17T03:30:16.362120+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.395 INFO:tasks.ceph.ceph_manager.ceph:PG 13.18 is not active+clean 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.362017+0000', 'last_change': '2024-09-17T03:30:16.362017+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.362017+0000', 'last_undegraded': '2024-09-17T03:30:16.362017+0000', 'last_fullsized': '2024-09-17T03:30:16.362017+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:PG 13.19 is not active+clean 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.347497+0000', 'last_change': '2024-09-17T03:30:16.347497+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.347497+0000', 'last_undegraded': '2024-09-17T03:30:16.347497+0000', 'last_fullsized': '2024-09-17T03:30:16.347497+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:PG 13.16 is not active+clean 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.348017+0000', 'last_change': '2024-09-17T03:30:16.348017+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.348017+0000', 'last_undegraded': '2024-09-17T03:30:16.348017+0000', 'last_fullsized': '2024-09-17T03:30:16.348017+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:PG 13.14 is not active+clean 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.346582+0000', 'last_change': '2024-09-17T03:30:16.346582+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.346582+0000', 'last_undegraded': '2024-09-17T03:30:16.346582+0000', 'last_fullsized': '2024-09-17T03:30:16.346582+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:PG 13.15 is not active+clean 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.366652+0000', 'last_change': '2024-09-17T03:30:16.366652+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.366652+0000', 'last_undegraded': '2024-09-17T03:30:16.366652+0000', 'last_fullsized': '2024-09-17T03:30:16.366652+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.396 INFO:tasks.ceph.ceph_manager.ceph:PG 13.12 is not active+clean 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.346578+0000', 'last_change': '2024-09-17T03:30:16.346578+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.346578+0000', 'last_undegraded': '2024-09-17T03:30:16.346578+0000', 'last_fullsized': '2024-09-17T03:30:16.346578+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:PG 13.13 is not active+clean 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.363020+0000', 'last_change': '2024-09-17T03:30:16.363020+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.363020+0000', 'last_undegraded': '2024-09-17T03:30:16.363020+0000', 'last_fullsized': '2024-09-17T03:30:16.363020+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:PG 13.e is not active+clean 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.361307+0000', 'last_change': '2024-09-17T03:30:16.361307+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.361307+0000', 'last_undegraded': '2024-09-17T03:30:16.361307+0000', 'last_fullsized': '2024-09-17T03:30:16.361307+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:PG 13.f is not active+clean 2024-09-17T03:30:22.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.354255+0000', 'last_change': '2024-09-17T03:30:16.354255+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.354255+0000', 'last_undegraded': '2024-09-17T03:30:16.354255+0000', 'last_fullsized': '2024-09-17T03:30:16.354255+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.475 INFO:tasks.ceph.ceph_manager.ceph:PG 13.d is not active+clean 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.366139+0000', 'last_change': '2024-09-17T03:30:16.366139+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.366139+0000', 'last_undegraded': '2024-09-17T03:30:16.366139+0000', 'last_fullsized': '2024-09-17T03:30:16.366139+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:PG 13.a is not active+clean 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.349551+0000', 'last_change': '2024-09-17T03:30:16.349551+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.349551+0000', 'last_undegraded': '2024-09-17T03:30:16.349551+0000', 'last_fullsized': '2024-09-17T03:30:16.349551+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:PG 13.8 is not active+clean 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.348588+0000', 'last_change': '2024-09-17T03:30:16.348588+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.348588+0000', 'last_undegraded': '2024-09-17T03:30:16.348588+0000', 'last_fullsized': '2024-09-17T03:30:16.348588+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:PG 13.6 is not active+clean 2024-09-17T03:30:22.476 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.363094+0000', 'last_change': '2024-09-17T03:30:16.363094+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.363094+0000', 'last_undegraded': '2024-09-17T03:30:16.363094+0000', 'last_fullsized': '2024-09-17T03:30:16.363094+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:PG 13.7 is not active+clean 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.360154+0000', 'last_change': '2024-09-17T03:30:16.360154+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.360154+0000', 'last_undegraded': '2024-09-17T03:30:16.360154+0000', 'last_fullsized': '2024-09-17T03:30:16.360154+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:PG 13.4 is not active+clean 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.353740+0000', 'last_change': '2024-09-17T03:30:16.353740+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.353740+0000', 'last_undegraded': '2024-09-17T03:30:16.353740+0000', 'last_fullsized': '2024-09-17T03:30:16.353740+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:PG 13.5 is not active+clean 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.363106+0000', 'last_change': '2024-09-17T03:30:16.363106+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.363106+0000', 'last_undegraded': '2024-09-17T03:30:16.363106+0000', 'last_fullsized': '2024-09-17T03:30:16.363106+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:30:22.477 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3 is not active+clean 2024-09-17T03:30:22.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.346444+0000', 'last_change': '2024-09-17T03:30:16.346444+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.346444+0000', 'last_undegraded': '2024-09-17T03:30:16.346444+0000', 'last_fullsized': '2024-09-17T03:30:16.346444+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:30:22.478 INFO:tasks.ceph.ceph_manager.ceph:PG 13.0 is not active+clean 2024-09-17T03:30:22.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.361539+0000', 'last_change': '2024-09-17T03:30:16.361539+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.361539+0000', 'last_undegraded': '2024-09-17T03:30:16.361539+0000', 'last_fullsized': '2024-09-17T03:30:16.361539+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:30:22.478 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1 is not active+clean 2024-09-17T03:30:22.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:30:16.353079+0000', 'last_change': '2024-09-17T03:30:16.353079+0000', 'last_active': '2024-09-17T03:30:16.330767+0000', 'last_peered': '2024-09-17T03:30:16.330767+0000', 'last_clean': '2024-09-17T03:30:16.330767+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:30:16.353079+0000', 'last_undegraded': '2024-09-17T03:30:16.353079+0000', 'last_fullsized': '2024-09-17T03:30:16.353079+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'last_clean_scrub_stamp': '2024-09-17T03:30:16.330767+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:30:22.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:30:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f386ce00640 1 -- 172.21.15.46:0/1001788135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 msgr2=0x7f384c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f386ce00640 1 --2- 172.21.15.46:0/1001788135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f384c004810 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3850001990 tx=0x7f385001f030 comp rx=0 tx=0).stop 2024-09-17T03:30:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f3866800640 1 -- 172.21.15.46:0/1001788135 reap_dead start 2024-09-17T03:30:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f386ce00640 1 -- 172.21.15.46:0/1001788135 shutdown_connections 2024-09-17T03:30:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f386ce00640 1 -- 172.21.15.46:0/1001788135 >> 172.21.15.46:0/1001788135 conn(0x7f38680b35f0 msgr2=0x7f38680b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f386ce00640 1 -- 172.21.15.46:0/1001788135 shutdown_connections 2024-09-17T03:30:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.641+0000 7f386ce00640 1 -- 172.21.15.46:0/1001788135 wait complete. 2024-09-17T03:30:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 Processor -- start 2024-09-17T03:30:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 -- start start 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f3868147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f38680ab7d0 0x7f38680b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f386815ad50 0x7f386815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3868069000 con 0x7f384c003fa0 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3868071ee0 con 0x7f386815ad50 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3868069180 con 0x7f38680ab7d0 2024-09-17T03:30:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f3868147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f3868147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34520/0 (socket says 172.21.15.46:34520) 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 -- 172.21.15.46:0/2017264399 learned_addr learned my addr 172.21.15.46:0/2017264399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f385fe00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f38680ab7d0 0x7f38680b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3866800640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f386815ad50 0x7f386815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 -- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f38680ab7d0 msgr2=0x7f38680b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f38680ab7d0 0x7f38680b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 -- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f386815ad50 msgr2=0x7f386815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f386815ad50 0x7f386815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 -- 172.21.15.46:0/2017264399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386815b6a0 con 0x7f384c003fa0 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f385fe00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f38680ab7d0 0x7f38680b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f3865e00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f3868147d60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3858011ab0 tx=0x7f3858011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385801e070 con 0x7f384c003fa0 2024-09-17T03:30:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f385800fc70 con 0x7f384c003fa0 2024-09-17T03:30:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38681f6d20 con 0x7f384c003fa0 2024-09-17T03:30:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858019670 con 0x7f384c003fa0 2024-09-17T03:30:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.645+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38681f72e0 con 0x7f384c003fa0 2024-09-17T03:30:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.649+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38680d0990 con 0x7f384c003fa0 2024-09-17T03:30:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.649+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f3858021070 con 0x7f384c003fa0 2024-09-17T03:30:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.649+0000 7f385e000640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3844074400 0x7f3844076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.649+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f38580936c0 con 0x7f384c003fa0 2024-09-17T03:30:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.649+0000 7f385fe00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3844074400 0x7f3844076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.649+0000 7f385fe00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3844074400 0x7f3844076910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38680746e0 tx=0x7f38500b3040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.653+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f386815b6a0 con 0x7f384c003fa0 2024-09-17T03:30:22.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.889+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f38680abf40 con 0x7f384c003fa0 2024-09-17T03:30:22.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f385e000640 1 -- 172.21.15.46:0/2017264399 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1144 (secure 0 0 0) 0x7f385800fe60 con 0x7f384c003fa0 2024-09-17T03:30:22.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:22.891 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1726543255,"num_in_osds":8,"osd_in_since":1726543245,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":50},{"state_name":"creating+peering","count":47}],"num_pgs":97,"num_pools":3,"num_objects":10,"data_bytes":592014,"bytes_used":540434432,"bytes_avail":772553678848,"bytes_total":773094113280,"inactive_pgs_ratio":0.48453608155250549,"write_bytes_sec":1706,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":55,"btime":"2024-09-17T03:30:20:367227+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":246888}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-17T03:29:51.384262+0000","services":{}},"progress_events":{}} 2024-09-17T03:30:22.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3844074400 msgr2=0x7f3844076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3844074400 0x7f3844076910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f38680746e0 tx=0x7f38500b3040 comp rx=0 tx=0).stop 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 msgr2=0x7f3868147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f3868147d60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3858011ab0 tx=0x7f3858011f70 comp rx=0 tx=0).stop 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 shutdown_connections 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3844074400 0x7f3844076910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f386815ad50 0x7f386815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f38680ab7d0 0x7f38680b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 --2- 172.21.15.46:0/2017264399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f384c003fa0 0x7f3868147d60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 >> 172.21.15.46:0/2017264399 conn(0x7f38680b35f0 msgr2=0x7f386815f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 shutdown_connections 2024-09-17T03:30:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:22.893+0000 7f386ce00640 1 -- 172.21.15.46:0/2017264399 wait complete. 2024-09-17T03:30:22.914 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:30:22.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 -- 172.21.15.46:0/1851982786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ac003c40 msgr2=0x7f02ac0040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 --2- 172.21.15.46:0/1851982786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ac003c40 0x7f02ac0040b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009b60 tx=0x7f02b801d240 comp rx=0 tx=0).stop 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 -- 172.21.15.46:0/1851982786 shutdown_connections 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 --2- 172.21.15.46:0/1851982786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ac003c40 0x7f02ac0040b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 -- 172.21.15.46:0/1851982786 >> 172.21.15.46:0/1851982786 conn(0x7f02c40b2040 msgr2=0x7f02c40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 -- 172.21.15.46:0/1851982786 shutdown_connections 2024-09-17T03:30:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.073+0000 7f02cc600640 1 -- 172.21.15.46:0/1851982786 wait complete. 2024-09-17T03:30:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 Processor -- start 2024-09-17T03:30:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 -- start start 2024-09-17T03:30:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02ac003c40 0x7f02c40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02c40ce5c0 0x7f02c41fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 0x7f02c41fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02c40b5690 con 0x7f02c41580d0 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f02c40b5510 con 0x7f02ac003c40 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02cc600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f02c415f180 con 0x7f02c40ce5c0 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 0x7f02c41fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 0x7f02c41fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34544/0 (socket says 172.21.15.46:34544) 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 -- 172.21.15.46:0/3110349412 learned_addr learned my addr 172.21.15.46:0/3110349412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02c9200640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02c40ce5c0 0x7f02c41fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02c9c00640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02ac003c40 0x7f02c40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 -- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02c40ce5c0 msgr2=0x7f02c41fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02c40ce5c0 0x7f02c41fc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 -- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02ac003c40 msgr2=0x7f02c40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02ac003c40 0x7f02c40ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 -- 172.21.15.46:0/3110349412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c4158f10 con 0x7f02c41580d0 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02ca600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 0x7f02c41fcc00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f90 tx=0x7f02bc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc010890 con 0x7f02c41580d0 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02bc017070 con 0x7f02c41580d0 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.077+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc010de0 con 0x7f02c41580d0 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.081+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c4158670 con 0x7f02c41580d0 2024-09-17T03:30:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.081+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c4158c30 con 0x7f02c41580d0 2024-09-17T03:30:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.081+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c415f6e0 con 0x7f02c41580d0 2024-09-17T03:30:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.081+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f02bc01c460 con 0x7f02c41580d0 2024-09-17T03:30:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.081+0000 7f02c1600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f02a4074400 0x7f02a4076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.085+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f02bc0933e0 con 0x7f02c41580d0 2024-09-17T03:30:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.085+0000 7f02c9c00640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f02a4074400 0x7f02a4076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.085+0000 7f02c9c00640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f02a4074400 0x7f02a4076910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f02b80ae000 tx=0x7f02b8019fe0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.089+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f02bc05fb70 con 0x7f02c41580d0 2024-09-17T03:30:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.257+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02c406b540 con 0x7f02a4074400 2024-09-17T03:30:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.261+0000 7f02c1600640 1 -- 172.21.15.46:0/3110349412 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248208 (secure 0 0 0) 0x7f02b80042e0 con 0x7f02a4074400 2024-09-17T03:30:23.264 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f02a4074400 msgr2=0x7f02a4076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f02a4074400 0x7f02a4076910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f02b80ae000 tx=0x7f02b8019fe0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 msgr2=0x7f02c41fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 0x7f02c41fcc00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f02bc000f90 tx=0x7f02bc00ea10 comp rx=0 tx=0).stop 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 shutdown_connections 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f02a4074400 0x7f02a4076910 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41580d0 0x7f02c41fcc00 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02c40ce5c0 0x7f02c41fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 --2- 172.21.15.46:0/3110349412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02ac003c40 0x7f02c40ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 >> 172.21.15.46:0/3110349412 conn(0x7f02c40b2040 msgr2=0x7f02c4159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 shutdown_connections 2024-09-17T03:30:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:23.265+0000 7f02cc600640 1 -- 172.21.15.46:0/3110349412 wait complete. 2024-09-17T03:30:23.289 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T03:30:21.390774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":114,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":527768,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":518259,"kb_avail":754446952,"statfs":{"total":773094113280,"available":772553678848,"internally_reserved":0,"allocated":9543680,"data_stored":4734567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":530697776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001148"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356490+0000","last_change":"2024-09-17T03:30:16.356490+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356490+0000","last_undegraded":"2024-09-17T03:30:16.356490+0000","last_fullsized":"2024-09-17T03:30:16.356490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041228+0000","last_change":"2024-09-17T03:30:17.370486+0000","last_active":"2024-09-17T03:30:19.041228+0000","last_peered":"2024-09-17T03:30:19.041228+0000","last_clean":"2024-09-17T03:30:19.041228+0000","last_became_active":"2024-09-17T03:30:17.370107+0000","last_became_peered":"2024-09-17T03:30:17.370107+0000","last_unstale":"2024-09-17T03:30:19.041228+0000","last_undegraded":"2024-09-17T03:30:19.041228+0000","last_fullsized":"2024-09-17T03:30:19.041228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:27.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361223+0000","last_change":"2024-09-17T03:30:16.361223+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361223+0000","last_undegraded":"2024-09-17T03:30:16.361223+0000","last_fullsized":"2024-09-17T03:30:16.361223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350657+0000","last_change":"2024-09-17T03:30:16.350657+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350657+0000","last_undegraded":"2024-09-17T03:30:16.350657+0000","last_fullsized":"2024-09-17T03:30:16.350657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362688+0000","last_change":"2024-09-17T03:30:16.362688+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362688+0000","last_undegraded":"2024-09-17T03:30:16.362688+0000","last_fullsized":"2024-09-17T03:30:16.362688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047927+0000","last_change":"2024-09-17T03:30:17.388212+0000","last_active":"2024-09-17T03:30:19.047927+0000","last_peered":"2024-09-17T03:30:19.047927+0000","last_clean":"2024-09-17T03:30:19.047927+0000","last_became_active":"2024-09-17T03:30:17.387941+0000","last_became_peered":"2024-09-17T03:30:17.387941+0000","last_unstale":"2024-09-17T03:30:19.047927+0000","last_undegraded":"2024-09-17T03:30:19.047927+0000","last_fullsized":"2024-09-17T03:30:19.047927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:30.231182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356272+0000","last_change":"2024-09-17T03:30:16.356272+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356272+0000","last_undegraded":"2024-09-17T03:30:16.356272+0000","last_fullsized":"2024-09-17T03:30:16.356272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047750+0000","last_change":"2024-09-17T03:30:17.383305+0000","last_active":"2024-09-17T03:30:19.047750+0000","last_peered":"2024-09-17T03:30:19.047750+0000","last_clean":"2024-09-17T03:30:19.047750+0000","last_became_active":"2024-09-17T03:30:17.382995+0000","last_became_peered":"2024-09-17T03:30:17.382995+0000","last_unstale":"2024-09-17T03:30:19.047750+0000","last_undegraded":"2024-09-17T03:30:19.047750+0000","last_fullsized":"2024-09-17T03:30:19.047750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:03.641896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047928+0000","last_change":"2024-09-17T03:30:17.374521+0000","last_active":"2024-09-17T03:30:19.047928+0000","last_peered":"2024-09-17T03:30:19.047928+0000","last_clean":"2024-09-17T03:30:19.047928+0000","last_became_active":"2024-09-17T03:30:17.374185+0000","last_became_peered":"2024-09-17T03:30:17.374185+0000","last_unstale":"2024-09-17T03:30:19.047928+0000","last_undegraded":"2024-09-17T03:30:19.047928+0000","last_fullsized":"2024-09-17T03:30:19.047928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:57.551527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353169+0000","last_change":"2024-09-17T03:30:16.353169+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353169+0000","last_undegraded":"2024-09-17T03:30:16.353169+0000","last_fullsized":"2024-09-17T03:30:16.353169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040669+0000","last_change":"2024-09-17T03:30:17.373313+0000","last_active":"2024-09-17T03:30:19.040669+0000","last_peered":"2024-09-17T03:30:19.040669+0000","last_clean":"2024-09-17T03:30:19.040669+0000","last_became_active":"2024-09-17T03:30:17.372950+0000","last_became_peered":"2024-09-17T03:30:17.372950+0000","last_unstale":"2024-09-17T03:30:19.040669+0000","last_undegraded":"2024-09-17T03:30:19.040669+0000","last_fullsized":"2024-09-17T03:30:19.040669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.356094+0000","last_change":"2024-09-17T03:30:16.356094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.356094+0000","last_undegraded":"2024-09-17T03:30:16.356094+0000","last_fullsized":"2024-09-17T03:30:16.356094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363504+0000","last_change":"2024-09-17T03:30:16.363504+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363504+0000","last_undegraded":"2024-09-17T03:30:16.363504+0000","last_fullsized":"2024-09-17T03:30:16.363504+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350224+0000","last_change":"2024-09-17T03:30:16.350224+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350224+0000","last_undegraded":"2024-09-17T03:30:16.350224+0000","last_fullsized":"2024-09-17T03:30:16.350224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348144+0000","last_change":"2024-09-17T03:30:16.348144+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348144+0000","last_undegraded":"2024-09-17T03:30:16.348144+0000","last_fullsized":"2024-09-17T03:30:16.348144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.358745+0000","last_change":"2024-09-17T03:30:16.358745+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.358745+0000","last_undegraded":"2024-09-17T03:30:16.358745+0000","last_fullsized":"2024-09-17T03:30:16.358745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.364506+0000","last_change":"2024-09-17T03:30:16.364506+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.364506+0000","last_undegraded":"2024-09-17T03:30:16.364506+0000","last_fullsized":"2024-09-17T03:30:16.364506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.351260+0000","last_change":"2024-09-17T03:30:16.351260+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.351260+0000","last_undegraded":"2024-09-17T03:30:16.351260+0000","last_fullsized":"2024-09-17T03:30:16.351260+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346459+0000","last_change":"2024-09-17T03:30:16.346459+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346459+0000","last_undegraded":"2024-09-17T03:30:16.346459+0000","last_fullsized":"2024-09-17T03:30:16.346459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.350038+0000","last_change":"2024-09-17T03:30:16.350038+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.350038+0000","last_undegraded":"2024-09-17T03:30:16.350038+0000","last_fullsized":"2024-09-17T03:30:16.350038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349235+0000","last_change":"2024-09-17T03:30:16.349235+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349235+0000","last_undegraded":"2024-09-17T03:30:16.349235+0000","last_fullsized":"2024-09-17T03:30:16.349235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040270+0000","last_change":"2024-09-17T03:30:17.374044+0000","last_active":"2024-09-17T03:30:19.040270+0000","last_peered":"2024-09-17T03:30:19.040270+0000","last_clean":"2024-09-17T03:30:19.040270+0000","last_became_active":"2024-09-17T03:30:17.373700+0000","last_became_peered":"2024-09-17T03:30:17.373700+0000","last_unstale":"2024-09-17T03:30:19.040270+0000","last_undegraded":"2024-09-17T03:30:19.040270+0000","last_fullsized":"2024-09-17T03:30:19.040270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:04.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354041+0000","last_change":"2024-09-17T03:30:16.354041+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354041+0000","last_undegraded":"2024-09-17T03:30:16.354041+0000","last_fullsized":"2024-09-17T03:30:16.354041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349327+0000","last_change":"2024-09-17T03:30:16.349327+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349327+0000","last_undegraded":"2024-09-17T03:30:16.349327+0000","last_fullsized":"2024-09-17T03:30:16.349327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041180+0000","last_change":"2024-09-17T03:30:17.374952+0000","last_active":"2024-09-17T03:30:19.041180+0000","last_peered":"2024-09-17T03:30:19.041180+0000","last_clean":"2024-09-17T03:30:19.041180+0000","last_became_active":"2024-09-17T03:30:17.374579+0000","last_became_peered":"2024-09-17T03:30:17.374579+0000","last_unstale":"2024-09-17T03:30:19.041180+0000","last_undegraded":"2024-09-17T03:30:19.041180+0000","last_fullsized":"2024-09-17T03:30:19.041180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.728454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361780+0000","last_change":"2024-09-17T03:30:16.361780+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361780+0000","last_undegraded":"2024-09-17T03:30:16.361780+0000","last_fullsized":"2024-09-17T03:30:16.361780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.367168+0000","last_change":"2024-09-17T03:30:16.367168+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.367168+0000","last_undegraded":"2024-09-17T03:30:16.367168+0000","last_fullsized":"2024-09-17T03:30:16.367168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.355538+0000","last_change":"2024-09-17T03:30:16.355538+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.355538+0000","last_undegraded":"2024-09-17T03:30:16.355538+0000","last_fullsized":"2024-09-17T03:30:16.355538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040169+0000","last_change":"2024-09-17T03:30:17.378897+0000","last_active":"2024-09-17T03:30:19.040169+0000","last_peered":"2024-09-17T03:30:19.040169+0000","last_clean":"2024-09-17T03:30:19.040169+0000","last_became_active":"2024-09-17T03:30:17.378212+0000","last_became_peered":"2024-09-17T03:30:17.378212+0000","last_unstale":"2024-09-17T03:30:19.040169+0000","last_undegraded":"2024-09-17T03:30:19.040169+0000","last_fullsized":"2024-09-17T03:30:19.040169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:13.896559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040290+0000","last_change":"2024-09-17T03:30:17.382864+0000","last_active":"2024-09-17T03:30:19.040290+0000","last_peered":"2024-09-17T03:30:19.040290+0000","last_clean":"2024-09-17T03:30:19.040290+0000","last_became_active":"2024-09-17T03:30:17.381934+0000","last_became_peered":"2024-09-17T03:30:17.381934+0000","last_unstale":"2024-09-17T03:30:19.040290+0000","last_undegraded":"2024-09-17T03:30:19.040290+0000","last_fullsized":"2024-09-17T03:30:19.040290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:12.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360461+0000","last_change":"2024-09-17T03:30:16.360461+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360461+0000","last_undegraded":"2024-09-17T03:30:16.360461+0000","last_fullsized":"2024-09-17T03:30:16.360461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353448+0000","last_change":"2024-09-17T03:30:16.353448+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353448+0000","last_undegraded":"2024-09-17T03:30:16.353448+0000","last_fullsized":"2024-09-17T03:30:16.353448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363769+0000","last_change":"2024-09-17T03:30:16.363769+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363769+0000","last_undegraded":"2024-09-17T03:30:16.363769+0000","last_fullsized":"2024-09-17T03:30:16.363769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371726+0000","last_change":"2024-09-17T03:30:16.366033+0000","last_active":"2024-09-17T03:30:16.371726+0000","last_peered":"2024-09-17T03:30:16.371726+0000","last_clean":"2024-09-17T03:30:16.371726+0000","last_became_active":"2024-09-17T03:30:16.365643+0000","last_became_peered":"2024-09-17T03:30:16.365643+0000","last_unstale":"2024-09-17T03:30:16.371726+0000","last_undegraded":"2024-09-17T03:30:16.371726+0000","last_fullsized":"2024-09-17T03:30:16.371726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:05.152896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354849+0000","last_change":"2024-09-17T03:30:16.354849+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354849+0000","last_undegraded":"2024-09-17T03:30:16.354849+0000","last_fullsized":"2024-09-17T03:30:16.354849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375348+0000","last_change":"2024-09-17T03:30:16.374265+0000","last_active":"2024-09-17T03:30:16.375348+0000","last_peered":"2024-09-17T03:30:16.375348+0000","last_clean":"2024-09-17T03:30:16.375348+0000","last_became_active":"2024-09-17T03:30:16.372671+0000","last_became_peered":"2024-09-17T03:30:16.372671+0000","last_unstale":"2024-09-17T03:30:16.375348+0000","last_undegraded":"2024-09-17T03:30:16.375348+0000","last_fullsized":"2024-09-17T03:30:16.375348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:20.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354990+0000","last_change":"2024-09-17T03:30:16.354990+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354990+0000","last_undegraded":"2024-09-17T03:30:16.354990+0000","last_fullsized":"2024-09-17T03:30:16.354990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.359637+0000","last_change":"2024-09-17T03:30:16.355580+0000","last_active":"2024-09-17T03:30:16.359637+0000","last_peered":"2024-09-17T03:30:16.359637+0000","last_clean":"2024-09-17T03:30:16.359637+0000","last_became_active":"2024-09-17T03:30:16.355240+0000","last_became_peered":"2024-09-17T03:30:16.355240+0000","last_unstale":"2024-09-17T03:30:16.359637+0000","last_undegraded":"2024-09-17T03:30:16.359637+0000","last_fullsized":"2024-09-17T03:30:16.359637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:31.022404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360687+0000","last_change":"2024-09-17T03:30:16.360687+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360687+0000","last_undegraded":"2024-09-17T03:30:16.360687+0000","last_fullsized":"2024-09-17T03:30:16.360687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361157+0000","last_change":"2024-09-17T03:30:16.357957+0000","last_active":"2024-09-17T03:30:16.361157+0000","last_peered":"2024-09-17T03:30:16.361157+0000","last_clean":"2024-09-17T03:30:16.361157+0000","last_became_active":"2024-09-17T03:30:16.357597+0000","last_became_peered":"2024-09-17T03:30:16.357597+0000","last_unstale":"2024-09-17T03:30:16.361157+0000","last_undegraded":"2024-09-17T03:30:16.361157+0000","last_fullsized":"2024-09-17T03:30:16.361157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:25.727900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040526+0000","last_change":"2024-09-17T03:30:17.382905+0000","last_active":"2024-09-17T03:30:19.040526+0000","last_peered":"2024-09-17T03:30:19.040526+0000","last_clean":"2024-09-17T03:30:19.040526+0000","last_became_active":"2024-09-17T03:30:17.382343+0000","last_became_peered":"2024-09-17T03:30:17.382343+0000","last_unstale":"2024-09-17T03:30:19.040526+0000","last_undegraded":"2024-09-17T03:30:19.040526+0000","last_fullsized":"2024-09-17T03:30:19.040526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:08.921865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376896+0000","last_change":"2024-09-17T03:30:16.375643+0000","last_active":"2024-09-17T03:30:16.376896+0000","last_peered":"2024-09-17T03:30:16.376896+0000","last_clean":"2024-09-17T03:30:16.376896+0000","last_became_active":"2024-09-17T03:30:16.375378+0000","last_became_peered":"2024-09-17T03:30:16.375378+0000","last_unstale":"2024-09-17T03:30:16.376896+0000","last_undegraded":"2024-09-17T03:30:16.376896+0000","last_fullsized":"2024-09-17T03:30:16.376896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.037881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362120+0000","last_change":"2024-09-17T03:30:16.362120+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362120+0000","last_undegraded":"2024-09-17T03:30:16.362120+0000","last_fullsized":"2024-09-17T03:30:16.362120+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375682+0000","last_change":"2024-09-17T03:30:16.374780+0000","last_active":"2024-09-17T03:30:16.375682+0000","last_peered":"2024-09-17T03:30:16.375682+0000","last_clean":"2024-09-17T03:30:16.375682+0000","last_became_active":"2024-09-17T03:30:16.374312+0000","last_became_peered":"2024-09-17T03:30:16.374312+0000","last_unstale":"2024-09-17T03:30:16.375682+0000","last_undegraded":"2024-09-17T03:30:16.375682+0000","last_fullsized":"2024-09-17T03:30:16.375682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.479057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.362017+0000","last_change":"2024-09-17T03:30:16.362017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.362017+0000","last_undegraded":"2024-09-17T03:30:16.362017+0000","last_fullsized":"2024-09-17T03:30:16.362017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363741+0000","last_change":"2024-09-17T03:30:16.360075+0000","last_active":"2024-09-17T03:30:16.363741+0000","last_peered":"2024-09-17T03:30:16.363741+0000","last_clean":"2024-09-17T03:30:16.363741+0000","last_became_active":"2024-09-17T03:30:16.359771+0000","last_became_peered":"2024-09-17T03:30:16.359771+0000","last_unstale":"2024-09-17T03:30:16.363741+0000","last_undegraded":"2024-09-17T03:30:16.363741+0000","last_fullsized":"2024-09-17T03:30:16.363741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:01.924364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.347497+0000","last_change":"2024-09-17T03:30:16.347497+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.347497+0000","last_undegraded":"2024-09-17T03:30:16.347497+0000","last_fullsized":"2024-09-17T03:30:16.347497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039799+0000","last_change":"2024-09-17T03:30:16.365783+0000","last_active":"2024-09-17T03:30:19.039799+0000","last_peered":"2024-09-17T03:30:19.039799+0000","last_clean":"2024-09-17T03:30:19.039799+0000","last_became_active":"2024-09-17T03:30:16.365439+0000","last_became_peered":"2024-09-17T03:30:16.365439+0000","last_unstale":"2024-09-17T03:30:19.039799+0000","last_undegraded":"2024-09-17T03:30:19.039799+0000","last_fullsized":"2024-09-17T03:30:19.039799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:48.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348017+0000","last_change":"2024-09-17T03:30:16.348017+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348017+0000","last_undegraded":"2024-09-17T03:30:16.348017+0000","last_fullsized":"2024-09-17T03:30:16.348017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371783+0000","last_change":"2024-09-17T03:30:16.369999+0000","last_active":"2024-09-17T03:30:16.371783+0000","last_peered":"2024-09-17T03:30:16.371783+0000","last_clean":"2024-09-17T03:30:16.371783+0000","last_became_active":"2024-09-17T03:30:16.368392+0000","last_became_peered":"2024-09-17T03:30:16.368392+0000","last_unstale":"2024-09-17T03:30:16.371783+0000","last_undegraded":"2024-09-17T03:30:16.371783+0000","last_fullsized":"2024-09-17T03:30:16.371783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:10.071007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047957+0000","last_change":"2024-09-17T03:30:17.380149+0000","last_active":"2024-09-17T03:30:19.047957+0000","last_peered":"2024-09-17T03:30:19.047957+0000","last_clean":"2024-09-17T03:30:19.047957+0000","last_became_active":"2024-09-17T03:30:17.379829+0000","last_became_peered":"2024-09-17T03:30:17.379829+0000","last_unstale":"2024-09-17T03:30:19.047957+0000","last_undegraded":"2024-09-17T03:30:19.047957+0000","last_fullsized":"2024-09-17T03:30:19.047957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:17.605539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.374805+0000","last_change":"2024-09-17T03:30:16.373477+0000","last_active":"2024-09-17T03:30:16.374805+0000","last_peered":"2024-09-17T03:30:16.374805+0000","last_clean":"2024-09-17T03:30:16.374805+0000","last_became_active":"2024-09-17T03:30:16.373060+0000","last_became_peered":"2024-09-17T03:30:16.373060+0000","last_unstale":"2024-09-17T03:30:16.374805+0000","last_undegraded":"2024-09-17T03:30:16.374805+0000","last_fullsized":"2024-09-17T03:30:16.374805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:06.209920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346582+0000","last_change":"2024-09-17T03:30:16.346582+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346582+0000","last_undegraded":"2024-09-17T03:30:16.346582+0000","last_fullsized":"2024-09-17T03:30:16.346582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.376938+0000","last_change":"2024-09-17T03:30:16.376370+0000","last_active":"2024-09-17T03:30:16.376938+0000","last_peered":"2024-09-17T03:30:16.376938+0000","last_clean":"2024-09-17T03:30:16.376938+0000","last_became_active":"2024-09-17T03:30:16.376027+0000","last_became_peered":"2024-09-17T03:30:16.376027+0000","last_unstale":"2024-09-17T03:30:16.376938+0000","last_undegraded":"2024-09-17T03:30:16.376938+0000","last_fullsized":"2024-09-17T03:30:16.376938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:51.451736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366652+0000","last_change":"2024-09-17T03:30:16.366652+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366652+0000","last_undegraded":"2024-09-17T03:30:16.366652+0000","last_fullsized":"2024-09-17T03:30:16.366652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047718+0000","last_change":"2024-09-17T03:30:16.370573+0000","last_active":"2024-09-17T03:30:19.047718+0000","last_peered":"2024-09-17T03:30:19.047718+0000","last_clean":"2024-09-17T03:30:19.047718+0000","last_became_active":"2024-09-17T03:30:16.369166+0000","last_became_peered":"2024-09-17T03:30:16.369166+0000","last_unstale":"2024-09-17T03:30:19.047718+0000","last_undegraded":"2024-09-17T03:30:19.047718+0000","last_fullsized":"2024-09-17T03:30:19.047718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:23.751200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346578+0000","last_change":"2024-09-17T03:30:16.346578+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346578+0000","last_undegraded":"2024-09-17T03:30:16.346578+0000","last_fullsized":"2024-09-17T03:30:16.346578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040259+0000","last_change":"2024-09-17T03:30:16.367290+0000","last_active":"2024-09-17T03:30:19.040259+0000","last_peered":"2024-09-17T03:30:19.040259+0000","last_clean":"2024-09-17T03:30:19.040259+0000","last_became_active":"2024-09-17T03:30:16.365363+0000","last_became_peered":"2024-09-17T03:30:16.365363+0000","last_unstale":"2024-09-17T03:30:19.040259+0000","last_undegraded":"2024-09-17T03:30:19.040259+0000","last_fullsized":"2024-09-17T03:30:19.040259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:46.817707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363020+0000","last_change":"2024-09-17T03:30:16.363020+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363020+0000","last_undegraded":"2024-09-17T03:30:16.363020+0000","last_fullsized":"2024-09-17T03:30:16.363020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371240+0000","last_change":"2024-09-17T03:30:16.369373+0000","last_active":"2024-09-17T03:30:16.371240+0000","last_peered":"2024-09-17T03:30:16.371240+0000","last_clean":"2024-09-17T03:30:16.371240+0000","last_became_active":"2024-09-17T03:30:16.369082+0000","last_became_peered":"2024-09-17T03:30:16.369082+0000","last_unstale":"2024-09-17T03:30:16.371240+0000","last_undegraded":"2024-09-17T03:30:16.371240+0000","last_fullsized":"2024-09-17T03:30:16.371240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:46.125971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039947+0000","last_change":"2024-09-17T03:30:17.382255+0000","last_active":"2024-09-17T03:30:19.039947+0000","last_peered":"2024-09-17T03:30:19.039947+0000","last_clean":"2024-09-17T03:30:19.039947+0000","last_became_active":"2024-09-17T03:30:17.381901+0000","last_became_peered":"2024-09-17T03:30:17.381901+0000","last_unstale":"2024-09-17T03:30:19.039947+0000","last_undegraded":"2024-09-17T03:30:19.039947+0000","last_fullsized":"2024-09-17T03:30:19.039947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.620927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047671+0000","last_change":"2024-09-17T03:30:16.371647+0000","last_active":"2024-09-17T03:30:19.047671+0000","last_peered":"2024-09-17T03:30:19.047671+0000","last_clean":"2024-09-17T03:30:19.047671+0000","last_became_active":"2024-09-17T03:30:16.371336+0000","last_became_peered":"2024-09-17T03:30:16.371336+0000","last_unstale":"2024-09-17T03:30:19.047671+0000","last_undegraded":"2024-09-17T03:30:19.047671+0000","last_fullsized":"2024-09-17T03:30:19.047671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:19.546103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039882+0000","last_change":"2024-09-17T03:30:17.372667+0000","last_active":"2024-09-17T03:30:19.039882+0000","last_peered":"2024-09-17T03:30:19.039882+0000","last_clean":"2024-09-17T03:30:19.039882+0000","last_became_active":"2024-09-17T03:30:17.372310+0000","last_became_peered":"2024-09-17T03:30:17.372310+0000","last_unstale":"2024-09-17T03:30:19.039882+0000","last_undegraded":"2024-09-17T03:30:19.039882+0000","last_fullsized":"2024-09-17T03:30:19.039882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:46.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363382+0000","last_change":"2024-09-17T03:30:16.359214+0000","last_active":"2024-09-17T03:30:16.363382+0000","last_peered":"2024-09-17T03:30:16.363382+0000","last_clean":"2024-09-17T03:30:16.363382+0000","last_became_active":"2024-09-17T03:30:16.358883+0000","last_became_peered":"2024-09-17T03:30:16.358883+0000","last_unstale":"2024-09-17T03:30:16.363382+0000","last_undegraded":"2024-09-17T03:30:16.363382+0000","last_fullsized":"2024-09-17T03:30:16.363382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:47.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361307+0000","last_change":"2024-09-17T03:30:16.361307+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361307+0000","last_undegraded":"2024-09-17T03:30:16.361307+0000","last_fullsized":"2024-09-17T03:30:16.361307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371756+0000","last_change":"2024-09-17T03:30:16.369933+0000","last_active":"2024-09-17T03:30:16.371756+0000","last_peered":"2024-09-17T03:30:16.371756+0000","last_clean":"2024-09-17T03:30:16.371756+0000","last_became_active":"2024-09-17T03:30:16.369307+0000","last_became_peered":"2024-09-17T03:30:16.369307+0000","last_unstale":"2024-09-17T03:30:16.371756+0000","last_undegraded":"2024-09-17T03:30:16.371756+0000","last_fullsized":"2024-09-17T03:30:16.371756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:29.392547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.354255+0000","last_change":"2024-09-17T03:30:16.354255+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.354255+0000","last_undegraded":"2024-09-17T03:30:16.354255+0000","last_fullsized":"2024-09-17T03:30:16.354255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367756+0000","last_change":"2024-09-17T03:30:16.364256+0000","last_active":"2024-09-17T03:30:16.367756+0000","last_peered":"2024-09-17T03:30:16.367756+0000","last_clean":"2024-09-17T03:30:16.367756+0000","last_became_active":"2024-09-17T03:30:16.363880+0000","last_became_peered":"2024-09-17T03:30:16.363880+0000","last_unstale":"2024-09-17T03:30:16.367756+0000","last_undegraded":"2024-09-17T03:30:16.367756+0000","last_fullsized":"2024-09-17T03:30:16.367756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.366139+0000","last_change":"2024-09-17T03:30:16.366139+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.366139+0000","last_undegraded":"2024-09-17T03:30:16.366139+0000","last_fullsized":"2024-09-17T03:30:16.366139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364390+0000","last_change":"2024-09-17T03:30:16.361309+0000","last_active":"2024-09-17T03:30:16.364390+0000","last_peered":"2024-09-17T03:30:16.364390+0000","last_clean":"2024-09-17T03:30:16.364390+0000","last_became_active":"2024-09-17T03:30:16.360955+0000","last_became_peered":"2024-09-17T03:30:16.360955+0000","last_unstale":"2024-09-17T03:30:16.364390+0000","last_undegraded":"2024-09-17T03:30:16.364390+0000","last_fullsized":"2024-09-17T03:30:16.364390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:33.036462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.349551+0000","last_change":"2024-09-17T03:30:16.349551+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.349551+0000","last_undegraded":"2024-09-17T03:30:16.349551+0000","last_fullsized":"2024-09-17T03:30:16.349551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.366789+0000","last_change":"2024-09-17T03:30:16.359453+0000","last_active":"2024-09-17T03:30:16.366789+0000","last_peered":"2024-09-17T03:30:16.366789+0000","last_clean":"2024-09-17T03:30:16.366789+0000","last_became_active":"2024-09-17T03:30:16.358930+0000","last_became_peered":"2024-09-17T03:30:16.358930+0000","last_unstale":"2024-09-17T03:30:16.366789+0000","last_undegraded":"2024-09-17T03:30:16.366789+0000","last_fullsized":"2024-09-17T03:30:16.366789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:34.925865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048154+0000","last_change":"2024-09-17T03:30:17.380496+0000","last_active":"2024-09-17T03:30:19.048154+0000","last_peered":"2024-09-17T03:30:19.048154+0000","last_clean":"2024-09-17T03:30:19.048154+0000","last_became_active":"2024-09-17T03:30:17.380195+0000","last_became_peered":"2024-09-17T03:30:17.380195+0000","last_unstale":"2024-09-17T03:30:19.048154+0000","last_undegraded":"2024-09-17T03:30:19.048154+0000","last_fullsized":"2024-09-17T03:30:19.048154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:32.240069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370053+0000","last_change":"2024-09-17T03:30:16.366802+0000","last_active":"2024-09-17T03:30:16.370053+0000","last_peered":"2024-09-17T03:30:16.370053+0000","last_clean":"2024-09-17T03:30:16.370053+0000","last_became_active":"2024-09-17T03:30:16.366497+0000","last_became_peered":"2024-09-17T03:30:16.366497+0000","last_unstale":"2024-09-17T03:30:16.370053+0000","last_undegraded":"2024-09-17T03:30:16.370053+0000","last_fullsized":"2024-09-17T03:30:16.370053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:31.288276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.348588+0000","last_change":"2024-09-17T03:30:16.348588+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.348588+0000","last_undegraded":"2024-09-17T03:30:16.348588+0000","last_fullsized":"2024-09-17T03:30:16.348588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.367253+0000","last_change":"2024-09-17T03:30:16.362330+0000","last_active":"2024-09-17T03:30:16.367253+0000","last_peered":"2024-09-17T03:30:16.367253+0000","last_clean":"2024-09-17T03:30:16.367253+0000","last_became_active":"2024-09-17T03:30:16.361960+0000","last_became_peered":"2024-09-17T03:30:16.361960+0000","last_unstale":"2024-09-17T03:30:16.367253+0000","last_undegraded":"2024-09-17T03:30:16.367253+0000","last_fullsized":"2024-09-17T03:30:16.367253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:27.002346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047756+0000","last_change":"2024-09-17T03:30:17.373912+0000","last_active":"2024-09-17T03:30:19.047756+0000","last_peered":"2024-09-17T03:30:19.047756+0000","last_clean":"2024-09-17T03:30:19.047756+0000","last_became_active":"2024-09-17T03:30:17.373542+0000","last_became_peered":"2024-09-17T03:30:17.373542+0000","last_unstale":"2024-09-17T03:30:19.047756+0000","last_undegraded":"2024-09-17T03:30:19.047756+0000","last_fullsized":"2024-09-17T03:30:19.047756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:28.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.361044+0000","last_change":"2024-09-17T03:30:16.354858+0000","last_active":"2024-09-17T03:30:16.361044+0000","last_peered":"2024-09-17T03:30:16.361044+0000","last_clean":"2024-09-17T03:30:16.361044+0000","last_became_active":"2024-09-17T03:30:16.354399+0000","last_became_peered":"2024-09-17T03:30:16.354399+0000","last_unstale":"2024-09-17T03:30:16.361044+0000","last_undegraded":"2024-09-17T03:30:16.361044+0000","last_fullsized":"2024-09-17T03:30:16.361044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:47.686970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363094+0000","last_change":"2024-09-17T03:30:16.363094+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363094+0000","last_undegraded":"2024-09-17T03:30:16.363094+0000","last_fullsized":"2024-09-17T03:30:16.363094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041218+0000","last_change":"2024-09-17T03:30:16.368986+0000","last_active":"2024-09-17T03:30:19.041218+0000","last_peered":"2024-09-17T03:30:19.041218+0000","last_clean":"2024-09-17T03:30:19.041218+0000","last_became_active":"2024-09-17T03:30:16.368638+0000","last_became_peered":"2024-09-17T03:30:16.368638+0000","last_unstale":"2024-09-17T03:30:19.041218+0000","last_undegraded":"2024-09-17T03:30:19.041218+0000","last_fullsized":"2024-09-17T03:30:19.041218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:04.104352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.360154+0000","last_change":"2024-09-17T03:30:16.360154+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.360154+0000","last_undegraded":"2024-09-17T03:30:16.360154+0000","last_fullsized":"2024-09-17T03:30:16.360154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041203+0000","last_change":"2024-09-17T03:30:16.371245+0000","last_active":"2024-09-17T03:30:19.041203+0000","last_peered":"2024-09-17T03:30:19.041203+0000","last_clean":"2024-09-17T03:30:19.041203+0000","last_became_active":"2024-09-17T03:30:16.370661+0000","last_became_peered":"2024-09-17T03:30:16.370661+0000","last_unstale":"2024-09-17T03:30:19.041203+0000","last_undegraded":"2024-09-17T03:30:19.041203+0000","last_fullsized":"2024-09-17T03:30:19.041203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:47.753727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353740+0000","last_change":"2024-09-17T03:30:16.353740+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353740+0000","last_undegraded":"2024-09-17T03:30:16.353740+0000","last_fullsized":"2024-09-17T03:30:16.353740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.370305+0000","last_change":"2024-09-17T03:30:16.366822+0000","last_active":"2024-09-17T03:30:16.370305+0000","last_peered":"2024-09-17T03:30:16.370305+0000","last_clean":"2024-09-17T03:30:16.370305+0000","last_became_active":"2024-09-17T03:30:16.364923+0000","last_became_peered":"2024-09-17T03:30:16.364923+0000","last_unstale":"2024-09-17T03:30:16.370305+0000","last_undegraded":"2024-09-17T03:30:16.370305+0000","last_fullsized":"2024-09-17T03:30:16.370305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:02.162681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.363106+0000","last_change":"2024-09-17T03:30:16.363106+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.363106+0000","last_undegraded":"2024-09-17T03:30:16.363106+0000","last_fullsized":"2024-09-17T03:30:16.363106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371866+0000","last_change":"2024-09-17T03:30:16.369886+0000","last_active":"2024-09-17T03:30:16.371866+0000","last_peered":"2024-09-17T03:30:16.371866+0000","last_clean":"2024-09-17T03:30:16.371866+0000","last_became_active":"2024-09-17T03:30:16.369399+0000","last_became_peered":"2024-09-17T03:30:16.369399+0000","last_unstale":"2024-09-17T03:30:16.371866+0000","last_undegraded":"2024-09-17T03:30:16.371866+0000","last_fullsized":"2024-09-17T03:30:16.371866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:30.365530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047759+0000","last_change":"2024-09-17T03:30:17.372555+0000","last_active":"2024-09-17T03:30:19.047759+0000","last_peered":"2024-09-17T03:30:19.047759+0000","last_clean":"2024-09-17T03:30:19.047759+0000","last_became_active":"2024-09-17T03:30:17.372225+0000","last_became_peered":"2024-09-17T03:30:17.372225+0000","last_unstale":"2024-09-17T03:30:19.047759+0000","last_undegraded":"2024-09-17T03:30:19.047759+0000","last_fullsized":"2024-09-17T03:30:19.047759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:49.139488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.363251+0000","last_change":"2024-09-17T03:30:16.359549+0000","last_active":"2024-09-17T03:30:16.363251+0000","last_peered":"2024-09-17T03:30:16.363251+0000","last_clean":"2024-09-17T03:30:16.363251+0000","last_became_active":"2024-09-17T03:30:16.359149+0000","last_became_peered":"2024-09-17T03:30:16.359149+0000","last_unstale":"2024-09-17T03:30:16.363251+0000","last_undegraded":"2024-09-17T03:30:16.363251+0000","last_fullsized":"2024-09-17T03:30:16.363251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:15.534467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.346444+0000","last_change":"2024-09-17T03:30:16.346444+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.346444+0000","last_undegraded":"2024-09-17T03:30:16.346444+0000","last_fullsized":"2024-09-17T03:30:16.346444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.375718+0000","last_change":"2024-09-17T03:30:16.374752+0000","last_active":"2024-09-17T03:30:16.375718+0000","last_peered":"2024-09-17T03:30:16.375718+0000","last_clean":"2024-09-17T03:30:16.375718+0000","last_became_active":"2024-09-17T03:30:16.374286+0000","last_became_peered":"2024-09-17T03:30:16.374286+0000","last_unstale":"2024-09-17T03:30:16.375718+0000","last_undegraded":"2024-09-17T03:30:16.375718+0000","last_fullsized":"2024-09-17T03:30:16.375718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:38.480874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.361539+0000","last_change":"2024-09-17T03:30:16.361539+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.361539+0000","last_undegraded":"2024-09-17T03:30:16.361539+0000","last_fullsized":"2024-09-17T03:30:16.361539+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.371481+0000","last_change":"2024-09-17T03:30:16.367991+0000","last_active":"2024-09-17T03:30:16.371481+0000","last_peered":"2024-09-17T03:30:16.371481+0000","last_clean":"2024-09-17T03:30:16.371481+0000","last_became_active":"2024-09-17T03:30:16.367669+0000","last_became_peered":"2024-09-17T03:30:16.367669+0000","last_unstale":"2024-09-17T03:30:16.371481+0000","last_undegraded":"2024-09-17T03:30:16.371481+0000","last_fullsized":"2024-09-17T03:30:16.371481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:30.112805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T03:30:16.353079+0000","last_change":"2024-09-17T03:30:16.353079+0000","last_active":"2024-09-17T03:30:16.330767+0000","last_peered":"2024-09-17T03:30:16.330767+0000","last_clean":"2024-09-17T03:30:16.330767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:30:16.353079+0000","last_undegraded":"2024-09-17T03:30:16.353079+0000","last_fullsized":"2024-09-17T03:30:16.353079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.372120+0000","last_change":"2024-09-17T03:30:16.367493+0000","last_active":"2024-09-17T03:30:16.372120+0000","last_peered":"2024-09-17T03:30:16.372120+0000","last_clean":"2024-09-17T03:30:16.372120+0000","last_became_active":"2024-09-17T03:30:16.367090+0000","last_became_peered":"2024-09-17T03:30:16.367090+0000","last_unstale":"2024-09-17T03:30:16.372120+0000","last_undegraded":"2024-09-17T03:30:16.372120+0000","last_fullsized":"2024-09-17T03:30:16.372120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:47.067459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040368+0000","last_change":"2024-09-17T03:30:17.374985+0000","last_active":"2024-09-17T03:30:19.040368+0000","last_peered":"2024-09-17T03:30:19.040368+0000","last_clean":"2024-09-17T03:30:19.040368+0000","last_became_active":"2024-09-17T03:30:17.373033+0000","last_became_peered":"2024-09-17T03:30:17.373033+0000","last_unstale":"2024-09-17T03:30:19.040368+0000","last_undegraded":"2024-09-17T03:30:19.040368+0000","last_fullsized":"2024-09-17T03:30:19.040368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:56.073407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.377262+0000","last_change":"2024-09-17T03:30:16.376615+0000","last_active":"2024-09-17T03:30:16.377262+0000","last_peered":"2024-09-17T03:30:16.377262+0000","last_clean":"2024-09-17T03:30:16.377262+0000","last_became_active":"2024-09-17T03:30:16.376148+0000","last_became_peered":"2024-09-17T03:30:16.376148+0000","last_unstale":"2024-09-17T03:30:16.377262+0000","last_undegraded":"2024-09-17T03:30:16.377262+0000","last_fullsized":"2024-09-17T03:30:16.377262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:47.251936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":228,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T03:30:16.364429+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:30:16.364429+0000","last_peered":"2024-09-17T03:30:16.364429+0000","last_clean":"2024-09-17T03:30:16.364429+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:30:16.364429+0000","last_undegraded":"2024-09-17T03:30:16.364429+0000","last_fullsized":"2024-09-17T03:30:16.364429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64652,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307188,"statfs":{"total":96636764160,"available":96570560512,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64072,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307768,"statfs":{"total":96636764160,"available":96571154432,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62716,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309124,"statfs":{"total":96636764160,"available":96572542976,"internally_reserved":0,"allocated":1024000,"data_stored":435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72392,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299448,"statfs":{"total":96636764160,"available":96562634752,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68240,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303600,"statfs":{"total":96636764160,"available":96566886400,"internally_reserved":0,"allocated":1036288,"data_stored":441516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68816,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303024,"statfs":{"total":96636764160,"available":96566296576,"internally_reserved":0,"allocated":1630208,"data_stored":1031884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1077248,"data_stored":456225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68268,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303572,"statfs":{"total":96636764160,"available":96566857728,"internally_reserved":0,"allocated":1073152,"data_stored":454785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:30:26.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:30:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d1a200640 1 -- 172.21.15.46:0/2423317266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf8004d50 msgr2=0x7f4cf8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d1a200640 1 --2- 172.21.15.46:0/2423317266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf8004d50 0x7f4cf8005220 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c001dc0 tx=0x7f4d0c01cb30 comp rx=0 tx=0).stop 2024-09-17T03:30:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d13e00640 1 -- 172.21.15.46:0/2423317266 reap_dead start 2024-09-17T03:30:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d1a200640 1 -- 172.21.15.46:0/2423317266 shutdown_connections 2024-09-17T03:30:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d1a200640 1 -- 172.21.15.46:0/2423317266 >> 172.21.15.46:0/2423317266 conn(0x7f4d140b2040 msgr2=0x7f4d140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d1a200640 1 -- 172.21.15.46:0/2423317266 shutdown_connections 2024-09-17T03:30:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.469+0000 7f4d1a200640 1 -- 172.21.15.46:0/2423317266 wait complete. 2024-09-17T03:30:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 Processor -- start 2024-09-17T03:30:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 -- start start 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf8004d50 0x7f4d1414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d14154d20 0x7f4d14155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 0x7f4d1414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1415cfe0 con 0x7f4d1414f480 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1415ce60 con 0x7f4cf8004d50 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1415d160 con 0x7f4d14154d20 2024-09-17T03:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 0x7f4d1414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf8004d50 0x7f4d1414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 0x7f4d1414f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34564/0 (socket says 172.21.15.46:34564) 2024-09-17T03:30:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 -- 172.21.15.46:0/526259112 learned_addr learned my addr 172.21.15.46:0/526259112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d0be00640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d14154d20 0x7f4d14155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 -- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d14154d20 msgr2=0x7f4d14155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d14154d20 0x7f4d14155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 -- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf8004d50 msgr2=0x7f4d1414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf8004d50 0x7f4d1414ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 -- 172.21.15.46:0/526259112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1414fdd0 con 0x7f4d1414f480 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d13e00640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 0x7f4d1414f890 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0ab040 tx=0x7f4d0c003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0013f0 con 0x7f4d1414f480 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d0c0a5d90 con 0x7f4d1414f480 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c006cb0 con 0x7f4d1414f480 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d141f9380 con 0x7f4d1414f480 2024-09-17T03:30:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.473+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d141f9940 con 0x7f4d1414f480 2024-09-17T03:30:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.477+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d140b51f0 con 0x7f4d1414f480 2024-09-17T03:30:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.477+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f4d0c006330 con 0x7f4d1414f480 2024-09-17T03:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.477+0000 7f4d0b400640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cf0074430 0x7f4cf0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.477+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f4d0c12cf40 con 0x7f4d1414f480 2024-09-17T03:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.477+0000 7f4d13400640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cf0074430 0x7f4cf0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.477+0000 7f4d13400640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cf0074430 0x7f4cf0076940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d04001250 tx=0x7f4d04001500 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.481+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4d1414fdd0 con 0x7f4d1414f480 2024-09-17T03:30:26.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.653+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d1406b540 con 0x7f4cf0074430 2024-09-17T03:30:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.657+0000 7f4d0b400640 1 -- 172.21.15.46:0/526259112 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7f4d04011de0 con 0x7f4cf0074430 2024-09-17T03:30:26.661 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:30:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cf0074430 msgr2=0x7f4cf0076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cf0074430 0x7f4cf0076940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d04001250 tx=0x7f4d04001500 comp rx=0 tx=0).stop 2024-09-17T03:30:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 msgr2=0x7f4d1414f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 0x7f4d1414f890 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0ab040 tx=0x7f4d0c003f20 comp rx=0 tx=0).stop 2024-09-17T03:30:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 shutdown_connections 2024-09-17T03:30:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cf0074430 0x7f4cf0076940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d1414f480 0x7f4d1414f890 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d14154d20 0x7f4d14155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 --2- 172.21.15.46:0/526259112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cf8004d50 0x7f4d1414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 >> 172.21.15.46:0/526259112 conn(0x7f4d140b2040 msgr2=0x7f4d14075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 shutdown_connections 2024-09-17T03:30:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.661+0000 7f4d1a200640 1 -- 172.21.15.46:0/526259112 wait complete. 2024-09-17T03:30:26.682 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-17T03:30:25.391785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":527980,"kb_used_data":9512,"kb_used_omap":12,"kb_used_meta":518259,"kb_avail":754446740,"statfs":{"total":773094113280,"available":772553461760,"internally_reserved":0,"allocated":9740288,"data_stored":4822284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":530697776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.050454"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370591+0000","last_change":"2024-09-17T03:30:17.382295+0000","last_active":"2024-09-17T03:30:18.370591+0000","last_peered":"2024-09-17T03:30:18.370591+0000","last_clean":"2024-09-17T03:30:18.370591+0000","last_became_active":"2024-09-17T03:30:17.380843+0000","last_became_peered":"2024-09-17T03:30:17.380843+0000","last_unstale":"2024-09-17T03:30:18.370591+0000","last_undegraded":"2024-09-17T03:30:18.370591+0000","last_fullsized":"2024-09-17T03:30:18.370591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:44.806360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041228+0000","last_change":"2024-09-17T03:30:17.370486+0000","last_active":"2024-09-17T03:30:19.041228+0000","last_peered":"2024-09-17T03:30:19.041228+0000","last_clean":"2024-09-17T03:30:19.041228+0000","last_became_active":"2024-09-17T03:30:17.370107+0000","last_became_peered":"2024-09-17T03:30:17.370107+0000","last_unstale":"2024-09-17T03:30:19.041228+0000","last_undegraded":"2024-09-17T03:30:19.041228+0000","last_fullsized":"2024-09-17T03:30:19.041228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:27.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049135+0000","last_change":"2024-09-17T03:30:17.372137+0000","last_active":"2024-09-17T03:30:19.049135+0000","last_peered":"2024-09-17T03:30:19.049135+0000","last_clean":"2024-09-17T03:30:19.049135+0000","last_became_active":"2024-09-17T03:30:17.371770+0000","last_became_peered":"2024-09-17T03:30:17.371770+0000","last_unstale":"2024-09-17T03:30:19.049135+0000","last_undegraded":"2024-09-17T03:30:19.049135+0000","last_fullsized":"2024-09-17T03:30:19.049135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:49.485591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362262+0000","last_change":"2024-09-17T03:30:17.382425+0000","last_active":"2024-09-17T03:30:18.362262+0000","last_peered":"2024-09-17T03:30:18.362262+0000","last_clean":"2024-09-17T03:30:18.362262+0000","last_became_active":"2024-09-17T03:30:17.379622+0000","last_became_peered":"2024-09-17T03:30:17.379622+0000","last_unstale":"2024-09-17T03:30:18.362262+0000","last_undegraded":"2024-09-17T03:30:18.362262+0000","last_fullsized":"2024-09-17T03:30:18.362262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:44.333220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048997+0000","last_change":"2024-09-17T03:30:17.380291+0000","last_active":"2024-09-17T03:30:19.048997+0000","last_peered":"2024-09-17T03:30:19.048997+0000","last_clean":"2024-09-17T03:30:19.048997+0000","last_became_active":"2024-09-17T03:30:17.379654+0000","last_became_peered":"2024-09-17T03:30:17.379654+0000","last_unstale":"2024-09-17T03:30:19.048997+0000","last_undegraded":"2024-09-17T03:30:19.048997+0000","last_fullsized":"2024-09-17T03:30:19.048997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:03.678902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047927+0000","last_change":"2024-09-17T03:30:17.388212+0000","last_active":"2024-09-17T03:30:19.047927+0000","last_peered":"2024-09-17T03:30:19.047927+0000","last_clean":"2024-09-17T03:30:19.047927+0000","last_became_active":"2024-09-17T03:30:17.387941+0000","last_became_peered":"2024-09-17T03:30:17.387941+0000","last_unstale":"2024-09-17T03:30:19.047927+0000","last_undegraded":"2024-09-17T03:30:19.047927+0000","last_fullsized":"2024-09-17T03:30:19.047927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:30.231182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.361716+0000","last_change":"2024-09-17T03:30:17.364602+0000","last_active":"2024-09-17T03:30:18.361716+0000","last_peered":"2024-09-17T03:30:18.361716+0000","last_clean":"2024-09-17T03:30:18.361716+0000","last_became_active":"2024-09-17T03:30:17.364299+0000","last_became_peered":"2024-09-17T03:30:17.364299+0000","last_unstale":"2024-09-17T03:30:18.361716+0000","last_undegraded":"2024-09-17T03:30:18.361716+0000","last_fullsized":"2024-09-17T03:30:18.361716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:07.242421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047750+0000","last_change":"2024-09-17T03:30:17.383305+0000","last_active":"2024-09-17T03:30:19.047750+0000","last_peered":"2024-09-17T03:30:19.047750+0000","last_clean":"2024-09-17T03:30:19.047750+0000","last_became_active":"2024-09-17T03:30:17.382995+0000","last_became_peered":"2024-09-17T03:30:17.382995+0000","last_unstale":"2024-09-17T03:30:19.047750+0000","last_undegraded":"2024-09-17T03:30:19.047750+0000","last_fullsized":"2024-09-17T03:30:19.047750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:03.641896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047928+0000","last_change":"2024-09-17T03:30:17.374521+0000","last_active":"2024-09-17T03:30:19.047928+0000","last_peered":"2024-09-17T03:30:19.047928+0000","last_clean":"2024-09-17T03:30:19.047928+0000","last_became_active":"2024-09-17T03:30:17.374185+0000","last_became_peered":"2024-09-17T03:30:17.374185+0000","last_unstale":"2024-09-17T03:30:19.047928+0000","last_undegraded":"2024-09-17T03:30:19.047928+0000","last_fullsized":"2024-09-17T03:30:19.047928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:06:57.551527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370149+0000","last_change":"2024-09-17T03:30:17.366986+0000","last_active":"2024-09-17T03:30:18.370149+0000","last_peered":"2024-09-17T03:30:18.370149+0000","last_clean":"2024-09-17T03:30:18.370149+0000","last_became_active":"2024-09-17T03:30:17.366624+0000","last_became_peered":"2024-09-17T03:30:17.366624+0000","last_unstale":"2024-09-17T03:30:18.370149+0000","last_undegraded":"2024-09-17T03:30:18.370149+0000","last_fullsized":"2024-09-17T03:30:18.370149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:25.363784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040669+0000","last_change":"2024-09-17T03:30:17.373313+0000","last_active":"2024-09-17T03:30:19.040669+0000","last_peered":"2024-09-17T03:30:19.040669+0000","last_clean":"2024-09-17T03:30:19.040669+0000","last_became_active":"2024-09-17T03:30:17.372950+0000","last_became_peered":"2024-09-17T03:30:17.372950+0000","last_unstale":"2024-09-17T03:30:19.040669+0000","last_undegraded":"2024-09-17T03:30:19.040669+0000","last_fullsized":"2024-09-17T03:30:19.040669+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.369662+0000","last_change":"2024-09-17T03:30:17.367183+0000","last_active":"2024-09-17T03:30:18.369662+0000","last_peered":"2024-09-17T03:30:18.369662+0000","last_clean":"2024-09-17T03:30:18.369662+0000","last_became_active":"2024-09-17T03:30:17.366823+0000","last_became_peered":"2024-09-17T03:30:17.366823+0000","last_unstale":"2024-09-17T03:30:18.369662+0000","last_undegraded":"2024-09-17T03:30:18.369662+0000","last_fullsized":"2024-09-17T03:30:18.369662+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:13:30.863931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048757+0000","last_change":"2024-09-17T03:30:17.376311+0000","last_active":"2024-09-17T03:30:19.048757+0000","last_peered":"2024-09-17T03:30:19.048757+0000","last_clean":"2024-09-17T03:30:19.048757+0000","last_became_active":"2024-09-17T03:30:17.375718+0000","last_became_peered":"2024-09-17T03:30:17.375718+0000","last_unstale":"2024-09-17T03:30:19.048757+0000","last_undegraded":"2024-09-17T03:30:19.048757+0000","last_fullsized":"2024-09-17T03:30:19.048757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:56.161852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.361928+0000","last_change":"2024-09-17T03:30:17.381256+0000","last_active":"2024-09-17T03:30:18.361928+0000","last_peered":"2024-09-17T03:30:18.361928+0000","last_clean":"2024-09-17T03:30:18.361928+0000","last_became_active":"2024-09-17T03:30:17.379854+0000","last_became_peered":"2024-09-17T03:30:17.379854+0000","last_unstale":"2024-09-17T03:30:18.361928+0000","last_undegraded":"2024-09-17T03:30:18.361928+0000","last_fullsized":"2024-09-17T03:30:18.361928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:26.502826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362102+0000","last_change":"2024-09-17T03:30:17.361388+0000","last_active":"2024-09-17T03:30:18.362102+0000","last_peered":"2024-09-17T03:30:18.362102+0000","last_clean":"2024-09-17T03:30:18.362102+0000","last_became_active":"2024-09-17T03:30:17.359674+0000","last_became_peered":"2024-09-17T03:30:17.359674+0000","last_unstale":"2024-09-17T03:30:18.362102+0000","last_undegraded":"2024-09-17T03:30:18.362102+0000","last_fullsized":"2024-09-17T03:30:18.362102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:59.637626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048276+0000","last_change":"2024-09-17T03:30:17.362153+0000","last_active":"2024-09-17T03:30:19.048276+0000","last_peered":"2024-09-17T03:30:19.048276+0000","last_clean":"2024-09-17T03:30:19.048276+0000","last_became_active":"2024-09-17T03:30:17.361765+0000","last_became_peered":"2024-09-17T03:30:17.361765+0000","last_unstale":"2024-09-17T03:30:19.048276+0000","last_undegraded":"2024-09-17T03:30:19.048276+0000","last_fullsized":"2024-09-17T03:30:19.048276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:41.930234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049921+0000","last_change":"2024-09-17T03:30:17.385197+0000","last_active":"2024-09-17T03:30:19.049921+0000","last_peered":"2024-09-17T03:30:19.049921+0000","last_clean":"2024-09-17T03:30:19.049921+0000","last_became_active":"2024-09-17T03:30:17.383216+0000","last_became_peered":"2024-09-17T03:30:17.383216+0000","last_unstale":"2024-09-17T03:30:19.049921+0000","last_undegraded":"2024-09-17T03:30:19.049921+0000","last_fullsized":"2024-09-17T03:30:19.049921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:03.982352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362361+0000","last_change":"2024-09-17T03:30:17.366142+0000","last_active":"2024-09-17T03:30:18.362361+0000","last_peered":"2024-09-17T03:30:18.362361+0000","last_clean":"2024-09-17T03:30:18.362361+0000","last_became_active":"2024-09-17T03:30:17.365833+0000","last_became_peered":"2024-09-17T03:30:17.365833+0000","last_unstale":"2024-09-17T03:30:18.362361+0000","last_undegraded":"2024-09-17T03:30:18.362361+0000","last_fullsized":"2024-09-17T03:30:18.362361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:32.726769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041602+0000","last_change":"2024-09-17T03:30:17.357258+0000","last_active":"2024-09-17T03:30:19.041602+0000","last_peered":"2024-09-17T03:30:19.041602+0000","last_clean":"2024-09-17T03:30:19.041602+0000","last_became_active":"2024-09-17T03:30:17.356936+0000","last_became_peered":"2024-09-17T03:30:17.356936+0000","last_unstale":"2024-09-17T03:30:19.041602+0000","last_undegraded":"2024-09-17T03:30:19.041602+0000","last_fullsized":"2024-09-17T03:30:19.041602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:09.764486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362538+0000","last_change":"2024-09-17T03:30:17.373588+0000","last_active":"2024-09-17T03:30:18.362538+0000","last_peered":"2024-09-17T03:30:18.362538+0000","last_clean":"2024-09-17T03:30:18.362538+0000","last_became_active":"2024-09-17T03:30:17.368639+0000","last_became_peered":"2024-09-17T03:30:17.368639+0000","last_unstale":"2024-09-17T03:30:18.362538+0000","last_undegraded":"2024-09-17T03:30:18.362538+0000","last_fullsized":"2024-09-17T03:30:18.362538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:20.676548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362002+0000","last_change":"2024-09-17T03:30:17.375178+0000","last_active":"2024-09-17T03:30:18.362002+0000","last_peered":"2024-09-17T03:30:18.362002+0000","last_clean":"2024-09-17T03:30:18.362002+0000","last_became_active":"2024-09-17T03:30:17.374917+0000","last_became_peered":"2024-09-17T03:30:17.374917+0000","last_unstale":"2024-09-17T03:30:18.362002+0000","last_undegraded":"2024-09-17T03:30:18.362002+0000","last_fullsized":"2024-09-17T03:30:18.362002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:25.870085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040270+0000","last_change":"2024-09-17T03:30:17.374044+0000","last_active":"2024-09-17T03:30:19.040270+0000","last_peered":"2024-09-17T03:30:19.040270+0000","last_clean":"2024-09-17T03:30:19.040270+0000","last_became_active":"2024-09-17T03:30:17.373700+0000","last_became_peered":"2024-09-17T03:30:17.373700+0000","last_unstale":"2024-09-17T03:30:19.040270+0000","last_undegraded":"2024-09-17T03:30:19.040270+0000","last_fullsized":"2024-09-17T03:30:19.040270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:04.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.369612+0000","last_change":"2024-09-17T03:30:17.364084+0000","last_active":"2024-09-17T03:30:18.369612+0000","last_peered":"2024-09-17T03:30:18.369612+0000","last_clean":"2024-09-17T03:30:18.369612+0000","last_became_active":"2024-09-17T03:30:17.363703+0000","last_became_peered":"2024-09-17T03:30:17.363703+0000","last_unstale":"2024-09-17T03:30:18.369612+0000","last_undegraded":"2024-09-17T03:30:18.369612+0000","last_fullsized":"2024-09-17T03:30:18.369612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:32.025289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.361628+0000","last_change":"2024-09-17T03:30:17.369179+0000","last_active":"2024-09-17T03:30:18.361628+0000","last_peered":"2024-09-17T03:30:18.361628+0000","last_clean":"2024-09-17T03:30:18.361628+0000","last_became_active":"2024-09-17T03:30:17.368861+0000","last_became_peered":"2024-09-17T03:30:17.368861+0000","last_unstale":"2024-09-17T03:30:18.361628+0000","last_undegraded":"2024-09-17T03:30:18.361628+0000","last_fullsized":"2024-09-17T03:30:18.361628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:21.922737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041180+0000","last_change":"2024-09-17T03:30:17.374952+0000","last_active":"2024-09-17T03:30:19.041180+0000","last_peered":"2024-09-17T03:30:19.041180+0000","last_clean":"2024-09-17T03:30:19.041180+0000","last_became_active":"2024-09-17T03:30:17.374579+0000","last_became_peered":"2024-09-17T03:30:17.374579+0000","last_unstale":"2024-09-17T03:30:19.041180+0000","last_undegraded":"2024-09-17T03:30:19.041180+0000","last_fullsized":"2024-09-17T03:30:19.041180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.728454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048758+0000","last_change":"2024-09-17T03:30:17.378448+0000","last_active":"2024-09-17T03:30:19.048758+0000","last_peered":"2024-09-17T03:30:19.048758+0000","last_clean":"2024-09-17T03:30:19.048758+0000","last_became_active":"2024-09-17T03:30:17.377419+0000","last_became_peered":"2024-09-17T03:30:17.377419+0000","last_unstale":"2024-09-17T03:30:19.048758+0000","last_undegraded":"2024-09-17T03:30:19.048758+0000","last_fullsized":"2024-09-17T03:30:19.048758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:23.580314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049413+0000","last_change":"2024-09-17T03:30:17.362579+0000","last_active":"2024-09-17T03:30:19.049413+0000","last_peered":"2024-09-17T03:30:19.049413+0000","last_clean":"2024-09-17T03:30:19.049413+0000","last_became_active":"2024-09-17T03:30:17.362212+0000","last_became_peered":"2024-09-17T03:30:17.362212+0000","last_unstale":"2024-09-17T03:30:19.049413+0000","last_undegraded":"2024-09-17T03:30:19.049413+0000","last_fullsized":"2024-09-17T03:30:19.049413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:01.816041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.369976+0000","last_change":"2024-09-17T03:30:17.380381+0000","last_active":"2024-09-17T03:30:18.369976+0000","last_peered":"2024-09-17T03:30:18.369976+0000","last_clean":"2024-09-17T03:30:18.369976+0000","last_became_active":"2024-09-17T03:30:17.379985+0000","last_became_peered":"2024-09-17T03:30:17.379985+0000","last_unstale":"2024-09-17T03:30:18.369976+0000","last_undegraded":"2024-09-17T03:30:18.369976+0000","last_fullsized":"2024-09-17T03:30:18.369976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:37.436545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040169+0000","last_change":"2024-09-17T03:30:17.378897+0000","last_active":"2024-09-17T03:30:19.040169+0000","last_peered":"2024-09-17T03:30:19.040169+0000","last_clean":"2024-09-17T03:30:19.040169+0000","last_became_active":"2024-09-17T03:30:17.378212+0000","last_became_peered":"2024-09-17T03:30:17.378212+0000","last_unstale":"2024-09-17T03:30:19.040169+0000","last_undegraded":"2024-09-17T03:30:19.040169+0000","last_fullsized":"2024-09-17T03:30:19.040169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:13.896559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040290+0000","last_change":"2024-09-17T03:30:17.382864+0000","last_active":"2024-09-17T03:30:19.040290+0000","last_peered":"2024-09-17T03:30:19.040290+0000","last_clean":"2024-09-17T03:30:19.040290+0000","last_became_active":"2024-09-17T03:30:17.381934+0000","last_became_peered":"2024-09-17T03:30:17.381934+0000","last_unstale":"2024-09-17T03:30:19.040290+0000","last_undegraded":"2024-09-17T03:30:19.040290+0000","last_fullsized":"2024-09-17T03:30:19.040290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:12.086166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049087+0000","last_change":"2024-09-17T03:30:17.363570+0000","last_active":"2024-09-17T03:30:19.049087+0000","last_peered":"2024-09-17T03:30:19.049087+0000","last_clean":"2024-09-17T03:30:19.049087+0000","last_became_active":"2024-09-17T03:30:17.363224+0000","last_became_peered":"2024-09-17T03:30:17.363224+0000","last_unstale":"2024-09-17T03:30:19.049087+0000","last_undegraded":"2024-09-17T03:30:19.049087+0000","last_fullsized":"2024-09-17T03:30:19.049087+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:20.011210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.369611+0000","last_change":"2024-09-17T03:30:17.378107+0000","last_active":"2024-09-17T03:30:18.369611+0000","last_peered":"2024-09-17T03:30:18.369611+0000","last_clean":"2024-09-17T03:30:18.369611+0000","last_became_active":"2024-09-17T03:30:17.377745+0000","last_became_peered":"2024-09-17T03:30:17.377745+0000","last_unstale":"2024-09-17T03:30:18.369611+0000","last_undegraded":"2024-09-17T03:30:18.369611+0000","last_fullsized":"2024-09-17T03:30:18.369611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:47.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049358+0000","last_change":"2024-09-17T03:30:17.363814+0000","last_active":"2024-09-17T03:30:19.049358+0000","last_peered":"2024-09-17T03:30:19.049358+0000","last_clean":"2024-09-17T03:30:19.049358+0000","last_became_active":"2024-09-17T03:30:17.363475+0000","last_became_peered":"2024-09-17T03:30:17.363475+0000","last_unstale":"2024-09-17T03:30:19.049358+0000","last_undegraded":"2024-09-17T03:30:19.049358+0000","last_fullsized":"2024-09-17T03:30:19.049358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:44.166315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370789+0000","last_change":"2024-09-17T03:30:16.366033+0000","last_active":"2024-09-17T03:30:18.370789+0000","last_peered":"2024-09-17T03:30:18.370789+0000","last_clean":"2024-09-17T03:30:18.370789+0000","last_became_active":"2024-09-17T03:30:16.365643+0000","last_became_peered":"2024-09-17T03:30:16.365643+0000","last_unstale":"2024-09-17T03:30:18.370789+0000","last_undegraded":"2024-09-17T03:30:18.370789+0000","last_fullsized":"2024-09-17T03:30:18.370789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:33.246007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370635+0000","last_change":"2024-09-17T03:30:17.367572+0000","last_active":"2024-09-17T03:30:18.370635+0000","last_peered":"2024-09-17T03:30:18.370635+0000","last_clean":"2024-09-17T03:30:18.370635+0000","last_became_active":"2024-09-17T03:30:17.367059+0000","last_became_peered":"2024-09-17T03:30:17.367059+0000","last_unstale":"2024-09-17T03:30:18.370635+0000","last_undegraded":"2024-09-17T03:30:18.370635+0000","last_fullsized":"2024-09-17T03:30:18.370635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:37.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049850+0000","last_change":"2024-09-17T03:30:16.374265+0000","last_active":"2024-09-17T03:30:19.049850+0000","last_peered":"2024-09-17T03:30:19.049850+0000","last_clean":"2024-09-17T03:30:19.049850+0000","last_became_active":"2024-09-17T03:30:16.372671+0000","last_became_peered":"2024-09-17T03:30:16.372671+0000","last_unstale":"2024-09-17T03:30:19.049850+0000","last_undegraded":"2024-09-17T03:30:19.049850+0000","last_fullsized":"2024-09-17T03:30:19.049850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:45.862365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370099+0000","last_change":"2024-09-17T03:30:17.381338+0000","last_active":"2024-09-17T03:30:18.370099+0000","last_peered":"2024-09-17T03:30:18.370099+0000","last_clean":"2024-09-17T03:30:18.370099+0000","last_became_active":"2024-09-17T03:30:17.380455+0000","last_became_peered":"2024-09-17T03:30:17.380455+0000","last_unstale":"2024-09-17T03:30:18.370099+0000","last_undegraded":"2024-09-17T03:30:18.370099+0000","last_fullsized":"2024-09-17T03:30:18.370099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:03.982352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041542+0000","last_change":"2024-09-17T03:30:16.355580+0000","last_active":"2024-09-17T03:30:19.041542+0000","last_peered":"2024-09-17T03:30:19.041542+0000","last_clean":"2024-09-17T03:30:19.041542+0000","last_became_active":"2024-09-17T03:30:16.355240+0000","last_became_peered":"2024-09-17T03:30:16.355240+0000","last_unstale":"2024-09-17T03:30:19.041542+0000","last_undegraded":"2024-09-17T03:30:19.041542+0000","last_fullsized":"2024-09-17T03:30:19.041542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:05.853119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049263+0000","last_change":"2024-09-17T03:30:17.379570+0000","last_active":"2024-09-17T03:30:19.049263+0000","last_peered":"2024-09-17T03:30:19.049263+0000","last_clean":"2024-09-17T03:30:19.049263+0000","last_became_active":"2024-09-17T03:30:17.379000+0000","last_became_peered":"2024-09-17T03:30:17.379000+0000","last_unstale":"2024-09-17T03:30:19.049263+0000","last_undegraded":"2024-09-17T03:30:19.049263+0000","last_fullsized":"2024-09-17T03:30:19.049263+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:40.271632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041044+0000","last_change":"2024-09-17T03:30:16.357957+0000","last_active":"2024-09-17T03:30:19.041044+0000","last_peered":"2024-09-17T03:30:19.041044+0000","last_clean":"2024-09-17T03:30:19.041044+0000","last_became_active":"2024-09-17T03:30:16.357597+0000","last_became_peered":"2024-09-17T03:30:16.357597+0000","last_unstale":"2024-09-17T03:30:19.041044+0000","last_undegraded":"2024-09-17T03:30:19.041044+0000","last_fullsized":"2024-09-17T03:30:19.041044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:42.064982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040526+0000","last_change":"2024-09-17T03:30:17.382905+0000","last_active":"2024-09-17T03:30:19.040526+0000","last_peered":"2024-09-17T03:30:19.040526+0000","last_clean":"2024-09-17T03:30:19.040526+0000","last_became_active":"2024-09-17T03:30:17.382343+0000","last_became_peered":"2024-09-17T03:30:17.382343+0000","last_unstale":"2024-09-17T03:30:19.040526+0000","last_undegraded":"2024-09-17T03:30:19.040526+0000","last_fullsized":"2024-09-17T03:30:19.040526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:08.921865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049317+0000","last_change":"2024-09-17T03:30:16.375643+0000","last_active":"2024-09-17T03:30:19.049317+0000","last_peered":"2024-09-17T03:30:19.049317+0000","last_clean":"2024-09-17T03:30:19.049317+0000","last_became_active":"2024-09-17T03:30:16.375378+0000","last_became_peered":"2024-09-17T03:30:16.375378+0000","last_unstale":"2024-09-17T03:30:19.049317+0000","last_undegraded":"2024-09-17T03:30:19.049317+0000","last_fullsized":"2024-09-17T03:30:19.049317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:00.344825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049341+0000","last_change":"2024-09-17T03:30:17.369564+0000","last_active":"2024-09-17T03:30:19.049341+0000","last_peered":"2024-09-17T03:30:19.049341+0000","last_clean":"2024-09-17T03:30:19.049341+0000","last_became_active":"2024-09-17T03:30:17.369235+0000","last_became_peered":"2024-09-17T03:30:17.369235+0000","last_unstale":"2024-09-17T03:30:19.049341+0000","last_undegraded":"2024-09-17T03:30:19.049341+0000","last_fullsized":"2024-09-17T03:30:19.049341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:26.603216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048513+0000","last_change":"2024-09-17T03:30:16.374780+0000","last_active":"2024-09-17T03:30:19.048513+0000","last_peered":"2024-09-17T03:30:19.048513+0000","last_clean":"2024-09-17T03:30:19.048513+0000","last_became_active":"2024-09-17T03:30:16.374312+0000","last_became_peered":"2024-09-17T03:30:16.374312+0000","last_unstale":"2024-09-17T03:30:19.048513+0000","last_undegraded":"2024-09-17T03:30:19.048513+0000","last_fullsized":"2024-09-17T03:30:19.048513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:06.209920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047478+0000","last_change":"2024-09-17T03:30:17.372860+0000","last_active":"2024-09-17T03:30:19.047478+0000","last_peered":"2024-09-17T03:30:19.047478+0000","last_clean":"2024-09-17T03:30:19.047478+0000","last_became_active":"2024-09-17T03:30:17.372506+0000","last_became_peered":"2024-09-17T03:30:17.372506+0000","last_unstale":"2024-09-17T03:30:19.047478+0000","last_undegraded":"2024-09-17T03:30:19.047478+0000","last_fullsized":"2024-09-17T03:30:19.047478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:46.922271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040424+0000","last_change":"2024-09-17T03:30:16.360075+0000","last_active":"2024-09-17T03:30:19.040424+0000","last_peered":"2024-09-17T03:30:19.040424+0000","last_clean":"2024-09-17T03:30:19.040424+0000","last_became_active":"2024-09-17T03:30:16.359771+0000","last_became_peered":"2024-09-17T03:30:16.359771+0000","last_unstale":"2024-09-17T03:30:19.040424+0000","last_undegraded":"2024-09-17T03:30:19.040424+0000","last_fullsized":"2024-09-17T03:30:19.040424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:47.345743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362385+0000","last_change":"2024-09-17T03:30:17.361908+0000","last_active":"2024-09-17T03:30:18.362385+0000","last_peered":"2024-09-17T03:30:18.362385+0000","last_clean":"2024-09-17T03:30:18.362385+0000","last_became_active":"2024-09-17T03:30:17.361086+0000","last_became_peered":"2024-09-17T03:30:17.361086+0000","last_unstale":"2024-09-17T03:30:18.362385+0000","last_undegraded":"2024-09-17T03:30:18.362385+0000","last_fullsized":"2024-09-17T03:30:18.362385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:21.145820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039799+0000","last_change":"2024-09-17T03:30:16.365783+0000","last_active":"2024-09-17T03:30:19.039799+0000","last_peered":"2024-09-17T03:30:19.039799+0000","last_clean":"2024-09-17T03:30:19.039799+0000","last_became_active":"2024-09-17T03:30:16.365439+0000","last_became_peered":"2024-09-17T03:30:16.365439+0000","last_unstale":"2024-09-17T03:30:19.039799+0000","last_undegraded":"2024-09-17T03:30:19.039799+0000","last_fullsized":"2024-09-17T03:30:19.039799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:48.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.363616+0000","last_change":"2024-09-17T03:30:17.380405+0000","last_active":"2024-09-17T03:30:18.363616+0000","last_peered":"2024-09-17T03:30:18.363616+0000","last_clean":"2024-09-17T03:30:18.363616+0000","last_became_active":"2024-09-17T03:30:17.380031+0000","last_became_peered":"2024-09-17T03:30:17.380031+0000","last_unstale":"2024-09-17T03:30:18.363616+0000","last_undegraded":"2024-09-17T03:30:18.363616+0000","last_fullsized":"2024-09-17T03:30:18.363616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:40.231350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370932+0000","last_change":"2024-09-17T03:30:16.369999+0000","last_active":"2024-09-17T03:30:18.370932+0000","last_peered":"2024-09-17T03:30:18.370932+0000","last_clean":"2024-09-17T03:30:18.370932+0000","last_became_active":"2024-09-17T03:30:16.368392+0000","last_became_peered":"2024-09-17T03:30:16.368392+0000","last_unstale":"2024-09-17T03:30:18.370932+0000","last_undegraded":"2024-09-17T03:30:18.370932+0000","last_fullsized":"2024-09-17T03:30:18.370932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:35.408685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047957+0000","last_change":"2024-09-17T03:30:17.380149+0000","last_active":"2024-09-17T03:30:19.047957+0000","last_peered":"2024-09-17T03:30:19.047957+0000","last_clean":"2024-09-17T03:30:19.047957+0000","last_became_active":"2024-09-17T03:30:17.379829+0000","last_became_peered":"2024-09-17T03:30:17.379829+0000","last_unstale":"2024-09-17T03:30:19.047957+0000","last_undegraded":"2024-09-17T03:30:19.047957+0000","last_fullsized":"2024-09-17T03:30:19.047957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:17.605539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370474+0000","last_change":"2024-09-17T03:30:16.373477+0000","last_active":"2024-09-17T03:30:18.370474+0000","last_peered":"2024-09-17T03:30:18.370474+0000","last_clean":"2024-09-17T03:30:18.370474+0000","last_became_active":"2024-09-17T03:30:16.373060+0000","last_became_peered":"2024-09-17T03:30:16.373060+0000","last_unstale":"2024-09-17T03:30:18.370474+0000","last_undegraded":"2024-09-17T03:30:18.370474+0000","last_fullsized":"2024-09-17T03:30:18.370474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:38.403199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041007+0000","last_change":"2024-09-17T03:30:17.363197+0000","last_active":"2024-09-17T03:30:19.041007+0000","last_peered":"2024-09-17T03:30:19.041007+0000","last_clean":"2024-09-17T03:30:19.041007+0000","last_became_active":"2024-09-17T03:30:17.362878+0000","last_became_peered":"2024-09-17T03:30:17.362878+0000","last_unstale":"2024-09-17T03:30:19.041007+0000","last_undegraded":"2024-09-17T03:30:19.041007+0000","last_fullsized":"2024-09-17T03:30:19.041007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:56.161852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370092+0000","last_change":"2024-09-17T03:30:16.376370+0000","last_active":"2024-09-17T03:30:18.370092+0000","last_peered":"2024-09-17T03:30:18.370092+0000","last_clean":"2024-09-17T03:30:18.370092+0000","last_became_active":"2024-09-17T03:30:16.376027+0000","last_became_peered":"2024-09-17T03:30:16.376027+0000","last_unstale":"2024-09-17T03:30:18.370092+0000","last_undegraded":"2024-09-17T03:30:18.370092+0000","last_fullsized":"2024-09-17T03:30:18.370092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:45.862365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049591+0000","last_change":"2024-09-17T03:30:17.370462+0000","last_active":"2024-09-17T03:30:19.049591+0000","last_peered":"2024-09-17T03:30:19.049591+0000","last_clean":"2024-09-17T03:30:19.049591+0000","last_became_active":"2024-09-17T03:30:17.364557+0000","last_became_peered":"2024-09-17T03:30:17.364557+0000","last_unstale":"2024-09-17T03:30:19.049591+0000","last_undegraded":"2024-09-17T03:30:19.049591+0000","last_fullsized":"2024-09-17T03:30:19.049591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:47.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047718+0000","last_change":"2024-09-17T03:30:16.370573+0000","last_active":"2024-09-17T03:30:19.047718+0000","last_peered":"2024-09-17T03:30:19.047718+0000","last_clean":"2024-09-17T03:30:19.047718+0000","last_became_active":"2024-09-17T03:30:16.369166+0000","last_became_peered":"2024-09-17T03:30:16.369166+0000","last_unstale":"2024-09-17T03:30:19.047718+0000","last_undegraded":"2024-09-17T03:30:19.047718+0000","last_fullsized":"2024-09-17T03:30:19.047718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:23.751200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040666+0000","last_change":"2024-09-17T03:30:17.349507+0000","last_active":"2024-09-17T03:30:19.040666+0000","last_peered":"2024-09-17T03:30:19.040666+0000","last_clean":"2024-09-17T03:30:19.040666+0000","last_became_active":"2024-09-17T03:30:17.349030+0000","last_became_peered":"2024-09-17T03:30:17.349030+0000","last_unstale":"2024-09-17T03:30:19.040666+0000","last_undegraded":"2024-09-17T03:30:19.040666+0000","last_fullsized":"2024-09-17T03:30:19.040666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:23.580314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040259+0000","last_change":"2024-09-17T03:30:16.367290+0000","last_active":"2024-09-17T03:30:19.040259+0000","last_peered":"2024-09-17T03:30:19.040259+0000","last_clean":"2024-09-17T03:30:19.040259+0000","last_became_active":"2024-09-17T03:30:16.365363+0000","last_became_peered":"2024-09-17T03:30:16.365363+0000","last_unstale":"2024-09-17T03:30:19.040259+0000","last_undegraded":"2024-09-17T03:30:19.040259+0000","last_fullsized":"2024-09-17T03:30:19.040259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:46.817707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048975+0000","last_change":"2024-09-17T03:30:17.364236+0000","last_active":"2024-09-17T03:30:19.048975+0000","last_peered":"2024-09-17T03:30:19.048975+0000","last_clean":"2024-09-17T03:30:19.048975+0000","last_became_active":"2024-09-17T03:30:17.363117+0000","last_became_peered":"2024-09-17T03:30:17.363117+0000","last_unstale":"2024-09-17T03:30:19.048975+0000","last_undegraded":"2024-09-17T03:30:19.048975+0000","last_fullsized":"2024-09-17T03:30:19.048975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:57.630432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.363055+0000","last_change":"2024-09-17T03:30:16.369373+0000","last_active":"2024-09-17T03:30:18.363055+0000","last_peered":"2024-09-17T03:30:18.363055+0000","last_clean":"2024-09-17T03:30:18.363055+0000","last_became_active":"2024-09-17T03:30:16.369082+0000","last_became_peered":"2024-09-17T03:30:16.369082+0000","last_unstale":"2024-09-17T03:30:18.363055+0000","last_undegraded":"2024-09-17T03:30:18.363055+0000","last_fullsized":"2024-09-17T03:30:18.363055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:27.083021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039947+0000","last_change":"2024-09-17T03:30:17.382255+0000","last_active":"2024-09-17T03:30:19.039947+0000","last_peered":"2024-09-17T03:30:19.039947+0000","last_clean":"2024-09-17T03:30:19.039947+0000","last_became_active":"2024-09-17T03:30:17.381901+0000","last_became_peered":"2024-09-17T03:30:17.381901+0000","last_unstale":"2024-09-17T03:30:19.039947+0000","last_undegraded":"2024-09-17T03:30:19.039947+0000","last_fullsized":"2024-09-17T03:30:19.039947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:14.620927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047671+0000","last_change":"2024-09-17T03:30:16.371647+0000","last_active":"2024-09-17T03:30:19.047671+0000","last_peered":"2024-09-17T03:30:19.047671+0000","last_clean":"2024-09-17T03:30:19.047671+0000","last_became_active":"2024-09-17T03:30:16.371336+0000","last_became_peered":"2024-09-17T03:30:16.371336+0000","last_unstale":"2024-09-17T03:30:19.047671+0000","last_undegraded":"2024-09-17T03:30:19.047671+0000","last_fullsized":"2024-09-17T03:30:19.047671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:19.546103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039882+0000","last_change":"2024-09-17T03:30:17.372667+0000","last_active":"2024-09-17T03:30:19.039882+0000","last_peered":"2024-09-17T03:30:19.039882+0000","last_clean":"2024-09-17T03:30:19.039882+0000","last_became_active":"2024-09-17T03:30:17.372310+0000","last_became_peered":"2024-09-17T03:30:17.372310+0000","last_unstale":"2024-09-17T03:30:19.039882+0000","last_undegraded":"2024-09-17T03:30:19.039882+0000","last_fullsized":"2024-09-17T03:30:19.039882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:46.070204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.039682+0000","last_change":"2024-09-17T03:30:16.359214+0000","last_active":"2024-09-17T03:30:19.039682+0000","last_peered":"2024-09-17T03:30:19.039682+0000","last_clean":"2024-09-17T03:30:19.039682+0000","last_became_active":"2024-09-17T03:30:16.358883+0000","last_became_peered":"2024-09-17T03:30:16.358883+0000","last_unstale":"2024-09-17T03:30:19.039682+0000","last_undegraded":"2024-09-17T03:30:19.039682+0000","last_fullsized":"2024-09-17T03:30:19.039682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:36.727569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049041+0000","last_change":"2024-09-17T03:30:17.375035+0000","last_active":"2024-09-17T03:30:19.049041+0000","last_peered":"2024-09-17T03:30:19.049041+0000","last_clean":"2024-09-17T03:30:19.049041+0000","last_became_active":"2024-09-17T03:30:17.374682+0000","last_became_peered":"2024-09-17T03:30:17.374682+0000","last_unstale":"2024-09-17T03:30:19.049041+0000","last_undegraded":"2024-09-17T03:30:19.049041+0000","last_fullsized":"2024-09-17T03:30:19.049041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:09.764486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370980+0000","last_change":"2024-09-17T03:30:16.369933+0000","last_active":"2024-09-17T03:30:18.370980+0000","last_peered":"2024-09-17T03:30:18.370980+0000","last_clean":"2024-09-17T03:30:18.370980+0000","last_became_active":"2024-09-17T03:30:16.369307+0000","last_became_peered":"2024-09-17T03:30:16.369307+0000","last_unstale":"2024-09-17T03:30:18.370980+0000","last_undegraded":"2024-09-17T03:30:18.370980+0000","last_fullsized":"2024-09-17T03:30:18.370980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:14.166525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370807+0000","last_change":"2024-09-17T03:30:17.369648+0000","last_active":"2024-09-17T03:30:18.370807+0000","last_peered":"2024-09-17T03:30:18.370807+0000","last_clean":"2024-09-17T03:30:18.370807+0000","last_became_active":"2024-09-17T03:30:17.368764+0000","last_became_peered":"2024-09-17T03:30:17.368764+0000","last_unstale":"2024-09-17T03:30:18.370807+0000","last_undegraded":"2024-09-17T03:30:18.370807+0000","last_fullsized":"2024-09-17T03:30:18.370807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:48.348627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041298+0000","last_change":"2024-09-17T03:30:16.364256+0000","last_active":"2024-09-17T03:30:19.041298+0000","last_peered":"2024-09-17T03:30:19.041298+0000","last_clean":"2024-09-17T03:30:19.041298+0000","last_became_active":"2024-09-17T03:30:16.363880+0000","last_became_peered":"2024-09-17T03:30:16.363880+0000","last_unstale":"2024-09-17T03:30:19.041298+0000","last_undegraded":"2024-09-17T03:30:19.041298+0000","last_fullsized":"2024-09-17T03:30:19.041298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:30.206427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049721+0000","last_change":"2024-09-17T03:30:17.383061+0000","last_active":"2024-09-17T03:30:19.049721+0000","last_peered":"2024-09-17T03:30:19.049721+0000","last_clean":"2024-09-17T03:30:19.049721+0000","last_became_active":"2024-09-17T03:30:17.378838+0000","last_became_peered":"2024-09-17T03:30:17.378838+0000","last_unstale":"2024-09-17T03:30:19.049721+0000","last_undegraded":"2024-09-17T03:30:19.049721+0000","last_fullsized":"2024-09-17T03:30:19.049721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:13:30.863931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041302+0000","last_change":"2024-09-17T03:30:16.361309+0000","last_active":"2024-09-17T03:30:19.041302+0000","last_peered":"2024-09-17T03:30:19.041302+0000","last_clean":"2024-09-17T03:30:19.041302+0000","last_became_active":"2024-09-17T03:30:16.360955+0000","last_became_peered":"2024-09-17T03:30:16.360955+0000","last_unstale":"2024-09-17T03:30:19.041302+0000","last_undegraded":"2024-09-17T03:30:19.041302+0000","last_fullsized":"2024-09-17T03:30:19.041302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:09.086082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362109+0000","last_change":"2024-09-17T03:30:17.380732+0000","last_active":"2024-09-17T03:30:18.362109+0000","last_peered":"2024-09-17T03:30:18.362109+0000","last_clean":"2024-09-17T03:30:18.362109+0000","last_became_active":"2024-09-17T03:30:17.380405+0000","last_became_peered":"2024-09-17T03:30:17.380405+0000","last_unstale":"2024-09-17T03:30:18.362109+0000","last_undegraded":"2024-09-17T03:30:18.362109+0000","last_fullsized":"2024-09-17T03:30:18.362109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:30.274845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.363206+0000","last_change":"2024-09-17T03:30:16.359453+0000","last_active":"2024-09-17T03:30:18.363206+0000","last_peered":"2024-09-17T03:30:18.363206+0000","last_clean":"2024-09-17T03:30:18.363206+0000","last_became_active":"2024-09-17T03:30:16.358930+0000","last_became_peered":"2024-09-17T03:30:16.358930+0000","last_unstale":"2024-09-17T03:30:18.363206+0000","last_undegraded":"2024-09-17T03:30:18.363206+0000","last_fullsized":"2024-09-17T03:30:18.363206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:40.286008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048154+0000","last_change":"2024-09-17T03:30:17.380496+0000","last_active":"2024-09-17T03:30:19.048154+0000","last_peered":"2024-09-17T03:30:19.048154+0000","last_clean":"2024-09-17T03:30:19.048154+0000","last_became_active":"2024-09-17T03:30:17.380195+0000","last_became_peered":"2024-09-17T03:30:17.380195+0000","last_unstale":"2024-09-17T03:30:19.048154+0000","last_undegraded":"2024-09-17T03:30:19.048154+0000","last_fullsized":"2024-09-17T03:30:19.048154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:32.240069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362274+0000","last_change":"2024-09-17T03:30:16.366802+0000","last_active":"2024-09-17T03:30:18.362274+0000","last_peered":"2024-09-17T03:30:18.362274+0000","last_clean":"2024-09-17T03:30:18.362274+0000","last_became_active":"2024-09-17T03:30:16.366497+0000","last_became_peered":"2024-09-17T03:30:16.366497+0000","last_unstale":"2024-09-17T03:30:18.362274+0000","last_undegraded":"2024-09-17T03:30:18.362274+0000","last_fullsized":"2024-09-17T03:30:18.362274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:22.474260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.362422+0000","last_change":"2024-09-17T03:30:17.372874+0000","last_active":"2024-09-17T03:30:18.362422+0000","last_peered":"2024-09-17T03:30:18.362422+0000","last_clean":"2024-09-17T03:30:18.362422+0000","last_became_active":"2024-09-17T03:30:17.369461+0000","last_became_peered":"2024-09-17T03:30:17.369461+0000","last_unstale":"2024-09-17T03:30:18.362422+0000","last_undegraded":"2024-09-17T03:30:18.362422+0000","last_fullsized":"2024-09-17T03:30:18.362422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:22.730080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.361684+0000","last_change":"2024-09-17T03:30:16.362330+0000","last_active":"2024-09-17T03:30:18.361684+0000","last_peered":"2024-09-17T03:30:18.361684+0000","last_clean":"2024-09-17T03:30:18.361684+0000","last_became_active":"2024-09-17T03:30:16.361960+0000","last_became_peered":"2024-09-17T03:30:16.361960+0000","last_unstale":"2024-09-17T03:30:18.361684+0000","last_undegraded":"2024-09-17T03:30:18.361684+0000","last_fullsized":"2024-09-17T03:30:18.361684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:43.803212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047756+0000","last_change":"2024-09-17T03:30:17.373912+0000","last_active":"2024-09-17T03:30:19.047756+0000","last_peered":"2024-09-17T03:30:19.047756+0000","last_clean":"2024-09-17T03:30:19.047756+0000","last_became_active":"2024-09-17T03:30:17.373542+0000","last_became_peered":"2024-09-17T03:30:17.373542+0000","last_unstale":"2024-09-17T03:30:19.047756+0000","last_undegraded":"2024-09-17T03:30:19.047756+0000","last_fullsized":"2024-09-17T03:30:19.047756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:28.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.361897+0000","last_change":"2024-09-17T03:30:16.354858+0000","last_active":"2024-09-17T03:30:18.361897+0000","last_peered":"2024-09-17T03:30:18.361897+0000","last_clean":"2024-09-17T03:30:18.361897+0000","last_became_active":"2024-09-17T03:30:16.354399+0000","last_became_peered":"2024-09-17T03:30:16.354399+0000","last_unstale":"2024-09-17T03:30:18.361897+0000","last_undegraded":"2024-09-17T03:30:18.361897+0000","last_fullsized":"2024-09-17T03:30:18.361897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:55.389208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049697+0000","last_change":"2024-09-17T03:30:17.374804+0000","last_active":"2024-09-17T03:30:19.049697+0000","last_peered":"2024-09-17T03:30:19.049697+0000","last_clean":"2024-09-17T03:30:19.049697+0000","last_became_active":"2024-09-17T03:30:17.374456+0000","last_became_peered":"2024-09-17T03:30:17.374456+0000","last_unstale":"2024-09-17T03:30:19.049697+0000","last_undegraded":"2024-09-17T03:30:19.049697+0000","last_fullsized":"2024-09-17T03:30:19.049697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:39.057188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041218+0000","last_change":"2024-09-17T03:30:16.368986+0000","last_active":"2024-09-17T03:30:19.041218+0000","last_peered":"2024-09-17T03:30:19.041218+0000","last_clean":"2024-09-17T03:30:19.041218+0000","last_became_active":"2024-09-17T03:30:16.368638+0000","last_became_peered":"2024-09-17T03:30:16.368638+0000","last_unstale":"2024-09-17T03:30:19.041218+0000","last_undegraded":"2024-09-17T03:30:19.041218+0000","last_fullsized":"2024-09-17T03:30:19.041218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:04.104352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048951+0000","last_change":"2024-09-17T03:30:17.370668+0000","last_active":"2024-09-17T03:30:19.048951+0000","last_peered":"2024-09-17T03:30:19.048951+0000","last_clean":"2024-09-17T03:30:19.048951+0000","last_became_active":"2024-09-17T03:30:17.370330+0000","last_became_peered":"2024-09-17T03:30:17.370330+0000","last_unstale":"2024-09-17T03:30:19.048951+0000","last_undegraded":"2024-09-17T03:30:19.048951+0000","last_fullsized":"2024-09-17T03:30:19.048951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:28.677253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041203+0000","last_change":"2024-09-17T03:30:16.371245+0000","last_active":"2024-09-17T03:30:19.041203+0000","last_peered":"2024-09-17T03:30:19.041203+0000","last_clean":"2024-09-17T03:30:19.041203+0000","last_became_active":"2024-09-17T03:30:16.370661+0000","last_became_peered":"2024-09-17T03:30:16.370661+0000","last_unstale":"2024-09-17T03:30:19.041203+0000","last_undegraded":"2024-09-17T03:30:19.041203+0000","last_fullsized":"2024-09-17T03:30:19.041203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:47.753727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370264+0000","last_change":"2024-09-17T03:30:17.376420+0000","last_active":"2024-09-17T03:30:18.370264+0000","last_peered":"2024-09-17T03:30:18.370264+0000","last_clean":"2024-09-17T03:30:18.370264+0000","last_became_active":"2024-09-17T03:30:17.376004+0000","last_became_peered":"2024-09-17T03:30:17.376004+0000","last_unstale":"2024-09-17T03:30:18.370264+0000","last_undegraded":"2024-09-17T03:30:18.370264+0000","last_fullsized":"2024-09-17T03:30:18.370264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:51:17.569106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370003+0000","last_change":"2024-09-17T03:30:16.366822+0000","last_active":"2024-09-17T03:30:18.370003+0000","last_peered":"2024-09-17T03:30:18.370003+0000","last_clean":"2024-09-17T03:30:18.370003+0000","last_became_active":"2024-09-17T03:30:16.364923+0000","last_became_peered":"2024-09-17T03:30:16.364923+0000","last_unstale":"2024-09-17T03:30:18.370003+0000","last_undegraded":"2024-09-17T03:30:18.370003+0000","last_fullsized":"2024-09-17T03:30:18.370003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:00.344825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049875+0000","last_change":"2024-09-17T03:30:17.370111+0000","last_active":"2024-09-17T03:30:19.049875+0000","last_peered":"2024-09-17T03:30:19.049875+0000","last_clean":"2024-09-17T03:30:19.049875+0000","last_became_active":"2024-09-17T03:30:17.364099+0000","last_became_peered":"2024-09-17T03:30:17.364099+0000","last_unstale":"2024-09-17T03:30:19.049875+0000","last_undegraded":"2024-09-17T03:30:19.049875+0000","last_fullsized":"2024-09-17T03:30:19.049875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:37.436545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.370305+0000","last_change":"2024-09-17T03:30:16.369886+0000","last_active":"2024-09-17T03:30:18.370305+0000","last_peered":"2024-09-17T03:30:18.370305+0000","last_clean":"2024-09-17T03:30:18.370305+0000","last_became_active":"2024-09-17T03:30:16.369399+0000","last_became_peered":"2024-09-17T03:30:16.369399+0000","last_unstale":"2024-09-17T03:30:18.370305+0000","last_undegraded":"2024-09-17T03:30:18.370305+0000","last_fullsized":"2024-09-17T03:30:18.370305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:08.771245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047759+0000","last_change":"2024-09-17T03:30:17.372555+0000","last_active":"2024-09-17T03:30:19.047759+0000","last_peered":"2024-09-17T03:30:19.047759+0000","last_clean":"2024-09-17T03:30:19.047759+0000","last_became_active":"2024-09-17T03:30:17.372225+0000","last_became_peered":"2024-09-17T03:30:17.372225+0000","last_unstale":"2024-09-17T03:30:19.047759+0000","last_undegraded":"2024-09-17T03:30:19.047759+0000","last_fullsized":"2024-09-17T03:30:19.047759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:49.139488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041028+0000","last_change":"2024-09-17T03:30:16.359549+0000","last_active":"2024-09-17T03:30:19.041028+0000","last_peered":"2024-09-17T03:30:19.041028+0000","last_clean":"2024-09-17T03:30:19.041028+0000","last_became_active":"2024-09-17T03:30:16.359149+0000","last_became_peered":"2024-09-17T03:30:16.359149+0000","last_unstale":"2024-09-17T03:30:19.041028+0000","last_undegraded":"2024-09-17T03:30:19.041028+0000","last_fullsized":"2024-09-17T03:30:19.041028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:04.116027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.041431+0000","last_change":"2024-09-17T03:30:17.349624+0000","last_active":"2024-09-17T03:30:19.041431+0000","last_peered":"2024-09-17T03:30:19.041431+0000","last_clean":"2024-09-17T03:30:19.041431+0000","last_became_active":"2024-09-17T03:30:17.349258+0000","last_became_peered":"2024-09-17T03:30:17.349258+0000","last_unstale":"2024-09-17T03:30:19.041431+0000","last_undegraded":"2024-09-17T03:30:19.041431+0000","last_fullsized":"2024-09-17T03:30:19.041431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:57.630432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048890+0000","last_change":"2024-09-17T03:30:16.374752+0000","last_active":"2024-09-17T03:30:19.048890+0000","last_peered":"2024-09-17T03:30:19.048890+0000","last_clean":"2024-09-17T03:30:19.048890+0000","last_became_active":"2024-09-17T03:30:16.374286+0000","last_became_peered":"2024-09-17T03:30:16.374286+0000","last_unstale":"2024-09-17T03:30:19.048890+0000","last_undegraded":"2024-09-17T03:30:19.048890+0000","last_fullsized":"2024-09-17T03:30:19.048890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:41.281498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.047755+0000","last_change":"2024-09-17T03:30:17.383716+0000","last_active":"2024-09-17T03:30:19.047755+0000","last_peered":"2024-09-17T03:30:19.047755+0000","last_clean":"2024-09-17T03:30:19.047755+0000","last_became_active":"2024-09-17T03:30:17.382743+0000","last_became_peered":"2024-09-17T03:30:17.382743+0000","last_unstale":"2024-09-17T03:30:19.047755+0000","last_undegraded":"2024-09-17T03:30:19.047755+0000","last_fullsized":"2024-09-17T03:30:19.047755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:09.795612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.369845+0000","last_change":"2024-09-17T03:30:16.367991+0000","last_active":"2024-09-17T03:30:18.369845+0000","last_peered":"2024-09-17T03:30:18.369845+0000","last_clean":"2024-09-17T03:30:18.369845+0000","last_became_active":"2024-09-17T03:30:16.367669+0000","last_became_peered":"2024-09-17T03:30:16.367669+0000","last_unstale":"2024-09-17T03:30:18.369845+0000","last_undegraded":"2024-09-17T03:30:18.369845+0000","last_fullsized":"2024-09-17T03:30:18.369845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:41.281498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:18.369624+0000","last_change":"2024-09-17T03:30:17.383995+0000","last_active":"2024-09-17T03:30:18.369624+0000","last_peered":"2024-09-17T03:30:18.369624+0000","last_clean":"2024-09-17T03:30:18.369624+0000","last_became_active":"2024-09-17T03:30:17.383617+0000","last_became_peered":"2024-09-17T03:30:17.383617+0000","last_unstale":"2024-09-17T03:30:18.369624+0000","last_undegraded":"2024-09-17T03:30:18.369624+0000","last_fullsized":"2024-09-17T03:30:18.369624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:01.816041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048248+0000","last_change":"2024-09-17T03:30:16.367493+0000","last_active":"2024-09-17T03:30:19.048248+0000","last_peered":"2024-09-17T03:30:19.048248+0000","last_clean":"2024-09-17T03:30:19.048248+0000","last_became_active":"2024-09-17T03:30:16.367090+0000","last_became_peered":"2024-09-17T03:30:16.367090+0000","last_unstale":"2024-09-17T03:30:19.048248+0000","last_undegraded":"2024-09-17T03:30:19.048248+0000","last_fullsized":"2024-09-17T03:30:19.048248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:51.451736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.040368+0000","last_change":"2024-09-17T03:30:17.374985+0000","last_active":"2024-09-17T03:30:19.040368+0000","last_peered":"2024-09-17T03:30:19.040368+0000","last_clean":"2024-09-17T03:30:19.040368+0000","last_became_active":"2024-09-17T03:30:17.373033+0000","last_became_peered":"2024-09-17T03:30:17.373033+0000","last_unstale":"2024-09-17T03:30:19.040368+0000","last_undegraded":"2024-09-17T03:30:19.040368+0000","last_fullsized":"2024-09-17T03:30:19.040368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:16.330767+0000","last_clean_scrub_stamp":"2024-09-17T03:30:16.330767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:56.073407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.049324+0000","last_change":"2024-09-17T03:30:16.376615+0000","last_active":"2024-09-17T03:30:19.049324+0000","last_peered":"2024-09-17T03:30:19.049324+0000","last_clean":"2024-09-17T03:30:19.049324+0000","last_became_active":"2024-09-17T03:30:16.376148+0000","last_became_peered":"2024-09-17T03:30:16.376148+0000","last_unstale":"2024-09-17T03:30:19.049324+0000","last_undegraded":"2024-09-17T03:30:19.049324+0000","last_fullsized":"2024-09-17T03:30:19.049324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:30:15.327883+0000","last_clean_scrub_stamp":"2024-09-17T03:30:15.327883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:25.951223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":232,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T03:30:19.048713+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:30:19.048713+0000","last_peered":"2024-09-17T03:30:19.048713+0000","last_clean":"2024-09-17T03:30:19.048713+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:30:19.048713+0000","last_undegraded":"2024-09-17T03:30:19.048713+0000","last_fullsized":"2024-09-17T03:30:19.048713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64688,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307152,"statfs":{"total":96636764160,"available":96570523648,"internally_reserved":0,"allocated":1658880,"data_stored":1045029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64116,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307724,"statfs":{"total":96636764160,"available":96571109376,"internally_reserved":0,"allocated":1073152,"data_stored":455285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62752,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309088,"statfs":{"total":96636764160,"available":96572506112,"internally_reserved":0,"allocated":1060864,"data_stored":454591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72436,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299404,"statfs":{"total":96636764160,"available":96562589696,"internally_reserved":0,"allocated":1073152,"data_stored":456203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68264,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303576,"statfs":{"total":96636764160,"available":96566861824,"internally_reserved":0,"allocated":1064960,"data_stored":455173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509556,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68844,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302996,"statfs":{"total":96636764160,"available":96566267904,"internally_reserved":0,"allocated":1658880,"data_stored":1044993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1077248,"data_stored":456225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68268,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303572,"statfs":{"total":96636764160,"available":96566857728,"internally_reserved":0,"allocated":1073152,"data_stored":454785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:30:26.686 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:30:26.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:30:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 -- 172.21.15.46:0/3768063270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c0b4de0 msgr2=0x7fda4c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 --2- 172.21.15.46:0/3768063270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c0b4de0 0x7fda4c0b51f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fda4c070710 tx=0x7fda4401ccc0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 -- 172.21.15.46:0/3768063270 shutdown_connections 2024-09-17T03:30:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 --2- 172.21.15.46:0/3768063270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c0b4de0 0x7fda4c0b51f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 --2- 172.21.15.46:0/3768063270 >> v2:172.21.15.99:3300/0 conn(0x7fda4c1527b0 0x7fda4c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 -- 172.21.15.46:0/3768063270 >> 172.21.15.46:0/3768063270 conn(0x7fda4c0b2040 msgr2=0x7fda4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 -- 172.21.15.46:0/3768063270 shutdown_connections 2024-09-17T03:30:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 -- 172.21.15.46:0/3768063270 wait complete. 2024-09-17T03:30:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 Processor -- start 2024-09-17T03:30:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.889+0000 7fda53000640 1 -- start start 2024-09-17T03:30:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4c0b5c70 0x7fda4c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 0x7fda4c1fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1688b0 con 0x7fda4c1527b0 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c168730 con 0x7fda4c0b5c70 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fda4c168a30 con 0x7fda4c0b4de0 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 0x7fda4c1fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda4be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 0x7fda4c1fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34582/0 (socket says 172.21.15.46:34582) 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda4b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4c0b5c70 0x7fda4c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda4be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:55442/0 (socket says 172.21.15.46:55442) 2024-09-17T03:30:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 -- 172.21.15.46:0/3657985792 learned_addr learned my addr 172.21.15.46:0/3657985792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 -- 172.21.15.46:0/3657985792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda4c0b4de0 msgr2=0x7fda4c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 -- 172.21.15.46:0/3657985792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4c0b5c70 msgr2=0x7fda4c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4c0b5c70 0x7fda4c153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 -- 172.21.15.46:0/3657985792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c153b50 con 0x7fda4c1527b0 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda51000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 0x7fda4c1fbd70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fda44009bd0 tx=0x7fda44005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440bc070 con 0x7fda4c1527b0 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda440ad6e0 con 0x7fda4c1527b0 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c153e30 con 0x7fda4c1527b0 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440b2040 con 0x7fda4c1527b0 2024-09-17T03:30:26.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1543f0 con 0x7fda4c1527b0 2024-09-17T03:30:26.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.893+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4c0b51f0 con 0x7fda4c1527b0 2024-09-17T03:30:26.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.897+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fda440adaf0 con 0x7fda4c1527b0 2024-09-17T03:30:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.897+0000 7fda43e00640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda2c074400 0x7fda2c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.897+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fda4412cdc0 con 0x7fda4c1527b0 2024-09-17T03:30:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.897+0000 7fda4be00640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda2c074400 0x7fda2c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.897+0000 7fda4be00640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda2c074400 0x7fda2c076910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fda3c007fd0 tx=0x7fda3c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:26.901+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda44016970 con 0x7fda4c1527b0 2024-09-17T03:30:27.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.137+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fda4c154890 con 0x7fda4c1527b0 2024-09-17T03:30:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.141+0000 7fda43e00640 1 -- 172.21.15.46:0/3657985792 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v39) ==== 190+0+147 (secure 0 0 0) 0x7fda440f91a0 con 0x7fda4c1527b0 2024-09-17T03:30:27.141 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:30:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:30:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:30:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:30:27.141 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:30:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda2c074400 msgr2=0x7fda2c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda2c074400 0x7fda2c076910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fda3c007fd0 tx=0x7fda3c009040 comp rx=0 tx=0).stop 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 msgr2=0x7fda4c1fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 0x7fda4c1fbd70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fda44009bd0 tx=0x7fda44005ae0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 shutdown_connections 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda2c074400 0x7fda2c076910 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda4c1527b0 0x7fda4c1fbd70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda4c0b5c70 0x7fda4c153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 --2- 172.21.15.46:0/3657985792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda4c0b4de0 0x7fda4c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 >> 172.21.15.46:0/3657985792 conn(0x7fda4c0b2040 msgr2=0x7fda4c151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 shutdown_connections 2024-09-17T03:30:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.145+0000 7fda53000640 1 -- 172.21.15.46:0/3657985792 wait complete. 2024-09-17T03:30:27.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:30:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 -- 172.21.15.46:0/583293054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 msgr2=0x7f65481657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 --2- 172.21.15.46:0/583293054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f65481657a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f65480cce50 tx=0x7f654001cb80 comp rx=0 tx=0).stop 2024-09-17T03:30:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 -- 172.21.15.46:0/583293054 shutdown_connections 2024-09-17T03:30:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 --2- 172.21.15.46:0/583293054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f65481657a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 --2- 172.21.15.46:0/583293054 >> v2:172.21.15.99:3300/0 conn(0x7f6548160a20 0x7f6548162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 -- 172.21.15.46:0/583293054 >> 172.21.15.46:0/583293054 conn(0x7f65480b2040 msgr2=0x7f65480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 -- 172.21.15.46:0/583293054 shutdown_connections 2024-09-17T03:30:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.325+0000 7f654f200640 1 -- 172.21.15.46:0/583293054 wait complete. 2024-09-17T03:30:27.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 Processor -- start 2024-09-17T03:30:27.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 -- start start 2024-09-17T03:30:27.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f654815e8c0 0x7f65481fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6548160a20 0x7f654813c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f6548074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6548168c20 con 0x7f6548163380 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6548168aa0 con 0x7f654815e8c0 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6548168da0 con 0x7f6548160a20 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f6548074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:27.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f6548074ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34604/0 (socket says 172.21.15.46:34604) 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 -- 172.21.15.46:0/2628022431 learned_addr learned my addr 172.21.15.46:0/2628022431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f653f400640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6548160a20 0x7f654813c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f6547e00640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f654815e8c0 0x7f65481fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 -- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6548160a20 msgr2=0x7f654813c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6548160a20 0x7f654813c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 -- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f654815e8c0 msgr2=0x7f65481fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f654815e8c0 0x7f65481fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 -- 172.21.15.46:0/2628022431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6548076500 con 0x7f6548163380 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654d200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f6548074ce0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f654001c5f0 tx=0x7f6540009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400b2040 con 0x7f6548163380 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65400bb070 con 0x7f6548163380 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6548165320 con 0x7f6548163380 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.329+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540004040 con 0x7f6548163380 2024-09-17T03:30:27.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.333+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6548164d60 con 0x7f6548163380 2024-09-17T03:30:27.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.333+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654815ecd0 con 0x7f6548163380 2024-09-17T03:30:27.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.333+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f65400041e0 con 0x7f6548163380 2024-09-17T03:30:27.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.333+0000 7f6544c00640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524074400 0x7f6524076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:27.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.333+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f654012d4a0 con 0x7f6548163380 2024-09-17T03:30:27.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.333+0000 7f6547e00640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524074400 0x7f6524076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:27.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.337+0000 7f6547e00640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524074400 0x7f6524076910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6538001870 tx=0x7f6538001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:27.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.337+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6540016770 con 0x7f6548163380 2024-09-17T03:30:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6548074620 con 0x7f6548163380 2024-09-17T03:30:27.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f6544c00640 1 -- 172.21.15.46:0/2628022431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5241 (secure 0 0 0) 0x7f65400f9950 con 0x7f6548163380 2024-09-17T03:30:27.548 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:27.548 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":55,"btime":"2024-09-17T03:30:20:367227+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:30:16.771417+0000","modified":"2024-09-17T03:30:19.787080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246888":{"gid":246888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2776262749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2776262749},{"type":"v1","addr":"172.21.15.46:6835","nonce":2776262749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246888,"qdb_cluster":[246888]},"id":6}]} 2024-09-17T03:30:27.548 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 55 2024-09-17T03:30:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524074400 msgr2=0x7f6524076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524074400 0x7f6524076910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6538001870 tx=0x7f6538001380 comp rx=0 tx=0).stop 2024-09-17T03:30:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 msgr2=0x7f6548074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f6548074ce0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f654001c5f0 tx=0x7f6540009310 comp rx=0 tx=0).stop 2024-09-17T03:30:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 shutdown_connections 2024-09-17T03:30:27.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6524074400 0x7f6524076910 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6548163380 0x7f6548074ce0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6548160a20 0x7f654813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 --2- 172.21.15.46:0/2628022431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f654815e8c0 0x7f65481fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 >> 172.21.15.46:0/2628022431 conn(0x7f65480b2040 msgr2=0x7f654815d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 shutdown_connections 2024-09-17T03:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:27.549+0000 7f654f200640 1 -- 172.21.15.46:0/2628022431 wait complete. 2024-09-17T03:30:27.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:30:16.771417+0000', 'modified': '2024-09-17T03:30:19.787080+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 246888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_246888': {'gid': 246888, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/2776262749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2776262749}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2776262749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 246888, 'qdb_cluster': [246888]} 2024-09-17T03:30:27.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:30:27.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 -- 172.21.15.46:0/763857497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 msgr2=0x7f30800b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 --2- 172.21.15.46:0/763857497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30800b51f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f30800703e0 tx=0x7f307801cce0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 -- 172.21.15.46:0/763857497 shutdown_connections 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 --2- 172.21.15.46:0/763857497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30800b51f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 --2- 172.21.15.46:0/763857497 >> v2:172.21.15.99:3300/0 conn(0x7f30801527b0 0x7f3080074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 --2- 172.21.15.46:0/763857497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30800b5c70 0x7f308015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 -- 172.21.15.46:0/763857497 >> 172.21.15.46:0/763857497 conn(0x7f30800b2040 msgr2=0x7f30800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 -- 172.21.15.46:0/763857497 shutdown_connections 2024-09-17T03:30:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 -- 172.21.15.46:0/763857497 wait complete. 2024-09-17T03:30:28.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 Processor -- start 2024-09-17T03:30:28.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.141+0000 7f3088200640 1 -- start start 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30801657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30800b5c70 0x7f3080157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30801527b0 0x7f3080158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f308015f6b0 con 0x7f30800b4de0 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f308015f530 con 0x7f30800b5c70 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f308015f830 con 0x7f30801527b0 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30801657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30801657d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34622/0 (socket says 172.21.15.46:34622) 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 -- 172.21.15.46:0/1284505961 learned_addr learned my addr 172.21.15.46:0/1284505961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3084e00640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30800b5c70 0x7f3080157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3086200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30801527b0 0x7f3080158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 -- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30801527b0 msgr2=0x7f3080158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30801527b0 0x7f3080158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 -- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30800b5c70 msgr2=0x7f3080157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30800b5c70 0x7f3080157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 -- 172.21.15.46:0/1284505961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3080165d10 con 0x7f30800b4de0 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3085800640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30801657d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f307400b440 tx=0x7f307400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f307d600640 1 -- 172.21.15.46:0/1284505961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074015020 con 0x7f30800b4de0 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f307d600640 1 -- 172.21.15.46:0/1284505961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30740119a0 con 0x7f30800b4de0 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f307d600640 1 -- 172.21.15.46:0/1284505961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074011df0 con 0x7f30800b4de0 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3080158a00 con 0x7f30800b4de0 2024-09-17T03:30:28.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.145+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30801fa4c0 con 0x7f30800b4de0 2024-09-17T03:30:28.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f307d600640 1 -- 172.21.15.46:0/1284505961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f3074018020 con 0x7f30800b4de0 2024-09-17T03:30:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f307d600640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3060074430 0x7f3060076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f307d600640 1 -- 172.21.15.46:0/1284505961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f3074092d40 con 0x7f30800b4de0 2024-09-17T03:30:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f3084e00640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3060074430 0x7f3060076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f308f935000 1 Processor -- start 2024-09-17T03:30:28.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f308f935000 1 -- start start 2024-09-17T03:30:28.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.149+0000 7f3084e00640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3060074430 0x7f3060076940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f308015eb80 tx=0x7f306c006f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dba9b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dba9b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557dba88a410 con 0x557dba864950 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dba7c7b40 con 0x557dba474460 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x557dba7b6f70 con 0x557dba84dca0 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34638/0 (socket says 172.21.15.46:34638) 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 -- 172.21.15.46:0/214980922 learned_addr learned my addr 172.21.15.46:0/214980922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3085800640 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dba9b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3086200640 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dba9b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 -- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 msgr2=0x557dba9b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dba9b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 -- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 msgr2=0x557dba9b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dba9b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 -- 172.21.15.46:0/214980922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dba77e5b0 con 0x557dba864950 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3084e00640 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9b5be0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f306c006b00 tx=0x7f306c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41cbbb4c45faddb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3072000640 1 -- 172.21.15.46:0/214980922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c01b040 con 0x557dba864950 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3072000640 1 -- 172.21.15.46:0/214980922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f306c024070 con 0x557dba864950 2024-09-17T03:30:28.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f3072000640 1 -- 172.21.15.46:0/214980922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c015d70 con 0x557dba864950 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- 172.21.15.46:0/214980922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 msgr2=0x557dba9b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9b5be0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f306c006b00 tx=0x7f306c011000 comp rx=0 tx=0).stop 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- 172.21.15.46:0/214980922 shutdown_connections 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dba9b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9b5be0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 --2- 172.21.15.46:0/214980922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dba9b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- 172.21.15.46:0/214980922 >> 172.21.15.46:0/214980922 conn(0x557dba93ad90 msgr2=0x557dba96a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- 172.21.15.46:0/214980922 shutdown_connections 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- 172.21.15.46:0/214980922 wait complete. 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 -- start start 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.153+0000 7f308f935000 1 Processor -- start 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba474460 0x557dba9bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba84dca0 0x557dba9bce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557dba7515f0 con 0x557dba864950 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dba7bad80 con 0x557dba84dca0 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x557dba75d520 con 0x557dba474460 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 10 client.0 _ll_register_callbacks cb 0x557dba761830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:30:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9bd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34652/0 (socket says 172.21.15.46:34652) 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 -- 172.21.15.46:0/2314965866 learned_addr learned my addr 172.21.15.46:0/2314965866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3085800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba474460 0x557dba9bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3084e00640 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba84dca0 0x557dba9bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 -- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba474460 msgr2=0x557dba9bc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba474460 0x557dba9bc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 -- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba84dca0 msgr2=0x557dba9bce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba84dca0 0x557dba9bce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 -- 172.21.15.46:0/2314965866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dba755190 con 0x557dba864950 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3086200640 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9bd3a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f30780b0040 tx=0x7f3078003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c3b9f640bfa743 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 1 -- 172.21.15.46:0/2314965866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780bf060 con 0x557dba864950 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 1 -- 172.21.15.46:0/2314965866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f3078004040 con 0x557dba864950 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dba74f110 con 0x557dba864950 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 1 -- 172.21.15.46:0/2314965866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f30780c5050 con 0x557dba864950 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 10 client.247047 fetch_fsmap learned FSMap version 55 2024-09-17T03:30:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x557dba83bf10 con 0x557dba864950 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 1 -- 172.21.15.46:0/2314965866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557dba74f110 con 0x557dba864950 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 1 -- 172.21.15.46:0/2314965866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f30780c8020 con 0x557dba864950 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 10 client.247047 fetch_fsmap finished waiting for FSMap version 55 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 10 client.247047 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 20 client.247047 populate_metadata read hostname 'smithi046' 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba4597d0 0x557dba9c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 4 client.247047 mds_command: new command op to 246888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:30:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 --> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dba7c9a90 con 0x557dba4597d0 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3085800640 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba4597d0 0x557dba9c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3085800640 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba4597d0 0x557dba9c6d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e275ab6aacbec6b6 server_cookie=7548aacb966941df in_seq=0 out_seq=0 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.157+0000 7f3066a00640 10 client.247047 ms_handle_connect on v2:172.21.15.46:6834/2776262749 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f3066a00640 1 -- 172.21.15.46:0/2314965866 <== mds.0 v2:172.21.15.46:6834/2776262749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557dba7c9a90 con 0x557dba4597d0 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f3066a00640 10 client.247047 handle_command_reply: tid=0 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 client.247047 shutdown 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 msgr2=0x557dba9bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9bd3a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f30780b0040 tx=0x7f3078003f90 comp rx=0 tx=0).stop 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 shutdown_connections 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba4597d0 0x557dba9c6d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dba9bd3a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba84dca0 0x557dba9bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 --2- 172.21.15.46:0/2314965866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba474460 0x557dba9bc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f3066a00640 0 client.247047 ms_handle_reset on v2:172.21.15.46:6834/2776262749 2024-09-17T03:30:28.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 >> 172.21.15.46:0/2314965866 conn(0x557dba93ad90 msgr2=0x557dba971440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 shutdown_connections 2024-09-17T03:30:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 1 -- 172.21.15.46:0/2314965866 wait complete. 2024-09-17T03:30:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.161+0000 7f308f935000 20 client.247047 trim_cache size 0 max 16384 2024-09-17T03:30:28.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 Processor -- start 2024-09-17T03:30:28.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- start start 2024-09-17T03:30:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 0x557dbaa27620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba864950 0x557dbaa280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3080205220 con 0x557dba474460 2024-09-17T03:30:28.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3080204220 con 0x557dba864950 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f30802035e0 con 0x557dba84dca0 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 0x557dbaa27620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 0x557dbaa27620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34660/0 (socket says 172.21.15.46:34660) 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 -- 172.21.15.46:0/1155826117 learned_addr learned my addr 172.21.15.46:0/1155826117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 -- 172.21.15.46:0/1155826117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 msgr2=0x557dbaa27b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 -- 172.21.15.46:0/1155826117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba864950 msgr2=0x557dbaa280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba864950 0x557dbaa280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 -- 172.21.15.46:0/1155826117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dba812db0 con 0x557dba474460 2024-09-17T03:30:28.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3085800640 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 0x557dbaa27620 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f307405f1f0 tx=0x7f3074062040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96212d925feb8963 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3071600640 1 -- 172.21.15.46:0/1155826117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074068ad0 con 0x557dba474460 2024-09-17T03:30:28.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3071600640 1 -- 172.21.15.46:0/1155826117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3074092fc0 con 0x557dba474460 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f3071600640 1 -- 172.21.15.46:0/1155826117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074068640 con 0x557dba474460 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- 172.21.15.46:0/1155826117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 msgr2=0x557dbaa27620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 0x557dbaa27620 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f307405f1f0 tx=0x7f3074062040 comp rx=0 tx=0).stop 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- 172.21.15.46:0/1155826117 shutdown_connections 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba864950 0x557dbaa280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa27b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 --2- 172.21.15.46:0/1155826117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba474460 0x557dbaa27620 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.181+0000 7f308f935000 1 -- 172.21.15.46:0/1155826117 >> 172.21.15.46:0/1155826117 conn(0x557dba93ad90 msgr2=0x557dba9b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- 172.21.15.46:0/1155826117 shutdown_connections 2024-09-17T03:30:28.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- 172.21.15.46:0/1155826117 wait complete. 2024-09-17T03:30:28.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- start start 2024-09-17T03:30:28.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 Processor -- start 2024-09-17T03:30:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dbaa2b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa2bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dbaa2c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557dba7515f0 con 0x557dba864950 2024-09-17T03:30:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3080203bd0 con 0x557dba474460 2024-09-17T03:30:28.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x557dba7ba090 con 0x557dba84dca0 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 10 client.0 _ll_register_callbacks cb 0x7f3078009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dbaa2c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dbaa2c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34670/0 (socket says 172.21.15.46:34670) 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 -- 172.21.15.46:0/4206372143 learned_addr learned my addr 172.21.15.46:0/4206372143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3085800640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dbaa2b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3084e00640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa2bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 -- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 msgr2=0x557dbaa2bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa2bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 -- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 msgr2=0x557dbaa2b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dbaa2b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 -- 172.21.15.46:0/4206372143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dba77dab0 con 0x557dba864950 2024-09-17T03:30:28.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3086200640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dbaa2c020 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3078009a40 tx=0x7f3078004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f951f1e5034cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3066a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3066a00640 1 -- 172.21.15.46:0/4206372143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780bb020 con 0x557dba864950 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f3066a00640 1 -- 172.21.15.46:0/4206372143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f3078003f30 con 0x557dba864950 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.185+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3074092fc0 con 0x557dba864950 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 1 -- 172.21.15.46:0/4206372143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f30780d0050 con 0x557dba864950 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 10 client.247053 fetch_fsmap learned FSMap version 55 2024-09-17T03:30:28.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x557dba75c070 con 0x557dba864950 2024-09-17T03:30:28.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 1 -- 172.21.15.46:0/4206372143 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3074092fc0 con 0x557dba864950 2024-09-17T03:30:28.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 1 -- 172.21.15.46:0/4206372143 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f30780d3020 con 0x557dba864950 2024-09-17T03:30:28.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 10 client.247053 fetch_fsmap finished waiting for FSMap version 55 2024-09-17T03:30:28.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 10 client.247053 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:30:28.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 20 client.247053 populate_metadata read hostname 'smithi046' 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba447830 0x557dba9e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 4 client.247053 mds_command: new command op to 246888 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 --> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] -- command(tid 0: {"prefix": "status"}) -- 0x557dba7c5540 con 0x557dba447830 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3085800640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba447830 0x557dba9e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3085800640 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba447830 0x557dba9e7730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6520a54ef008e5d0 server_cookie=6f3bfcfa59cd0a41 in_seq=0 out_seq=0 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 10 client.247053 ms_handle_connect on v2:172.21.15.46:6834/2776262749 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 1 -- 172.21.15.46:0/4206372143 <== mds.0 v2:172.21.15.46:6834/2776262749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x557dba7c5540 con 0x557dba447830 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 10 client.247053 handle_command_reply: tid=0 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 client.247053 shutdown 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 msgr2=0x557dbaa2c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dbaa2c020 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3078009a40 tx=0x7f3078004450 comp rx=0 tx=0).stop 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 shutdown_connections 2024-09-17T03:30:28.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:6834/2776262749,v1:172.21.15.46:6835/2776262749] conn(0x557dba447830 0x557dba9e7730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557dba864950 0x557dbaa2c020 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x557dba84dca0 0x557dbaa2bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 --2- 172.21.15.46:0/4206372143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dba474460 0x557dbaa2b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3066a00640 0 client.247053 ms_handle_reset on v2:172.21.15.46:6834/2776262749 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 >> 172.21.15.46:0/4206372143 conn(0x557dba93ad90 msgr2=0x557dba9c66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 shutdown_connections 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 1 -- 172.21.15.46:0/4206372143 wait complete. 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f308f935000 20 client.247053 trim_cache size 0 max 16384 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:30:28.193 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 246888, 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.850296971000001, 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 55, 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 87, 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 87, 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.224991124000001 2024-09-17T03:30:28.194 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:30:28.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3060074430 msgr2=0x7f3060076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3060074430 0x7f3060076940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f308015eb80 tx=0x7f306c006f90 comp rx=0 tx=0).stop 2024-09-17T03:30:28.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 msgr2=0x7f30801657d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30801657d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f307400b440 tx=0x7f307400b900 comp rx=0 tx=0).stop 2024-09-17T03:30:28.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 shutdown_connections 2024-09-17T03:30:28.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3060074430 0x7f3060076940 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f30801527b0 0x7f3080158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30800b5c70 0x7f3080157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 --2- 172.21.15.46:0/1284505961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800b4de0 0x7f30801657d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 >> 172.21.15.46:0/1284505961 conn(0x7f30800b2040 msgr2=0x7f30801599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 shutdown_connections 2024-09-17T03:30:28.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.189+0000 7f3088200640 1 -- 172.21.15.46:0/1284505961 wait complete. 2024-09-17T03:30:28.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:30:28.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 -- 172.21.15.46:0/1496360351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec157030 msgr2=0x7f18ec0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 --2- 172.21.15.46:0/1496360351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec157030 0x7f18ec0b5a80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009840 tx=0x7f18e001c820 comp rx=0 tx=0).stop 2024-09-17T03:30:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 -- 172.21.15.46:0/1496360351 shutdown_connections 2024-09-17T03:30:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 --2- 172.21.15.46:0/1496360351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18ec074b20 0x7f18ec15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 --2- 172.21.15.46:0/1496360351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec157030 0x7f18ec0b5a80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 -- 172.21.15.46:0/1496360351 >> 172.21.15.46:0/1496360351 conn(0x7f18ec0708c0 msgr2=0x7f18ec070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 -- 172.21.15.46:0/1496360351 shutdown_connections 2024-09-17T03:30:28.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.369+0000 7f18f3600640 1 -- 172.21.15.46:0/1496360351 wait complete. 2024-09-17T03:30:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 Processor -- start 2024-09-17T03:30:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 -- start start 2024-09-17T03:30:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18ec157030 0x7f18ec1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 0x7f18ec15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18ec15c860 0x7f18ec1f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec16a6f0 con 0x7f18ec160d50 2024-09-17T03:30:28.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec16a570 con 0x7f18ec157030 2024-09-17T03:30:28.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f3600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec16a870 con 0x7f18ec15c860 2024-09-17T03:30:28.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 0x7f18ec15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 0x7f18ec15c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34688/0 (socket says 172.21.15.46:34688) 2024-09-17T03:30:28.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 -- 172.21.15.46:0/3394030760 learned_addr learned my addr 172.21.15.46:0/3394030760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:28.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f0c00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18ec157030 0x7f18ec1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18f1600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18ec15c860 0x7f18ec1f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 -- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18ec15c860 msgr2=0x7f18ec1f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18ec15c860 0x7f18ec1f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 -- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18ec157030 msgr2=0x7f18ec1576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18ec157030 0x7f18ec1576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 -- 172.21.15.46:0/3394030760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec14ce20 con 0x7f18ec160d50 2024-09-17T03:30:28.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18ebe00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 0x7f18ec15c320 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f18d8001ec0 tx=0x7f18d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.373+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8017070 con 0x7f18ec160d50 2024-09-17T03:30:28.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d8003600 con 0x7f18ec160d50 2024-09-17T03:30:28.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8003a50 con 0x7f18ec160d50 2024-09-17T03:30:28.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec14d100 con 0x7f18ec160d50 2024-09-17T03:30:28.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec157bf0 con 0x7f18ec160d50 2024-09-17T03:30:28.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec15cf90 con 0x7f18ec160d50 2024-09-17T03:30:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f18d801a070 con 0x7f18ec160d50 2024-09-17T03:30:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18e8c00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f18cc074400 0x7f18cc076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f18d8092350 con 0x7f18ec160d50 2024-09-17T03:30:28.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.377+0000 7f18f0c00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f18cc074400 0x7f18cc076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:28.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.381+0000 7f18f0c00640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f18cc074400 0x7f18cc076910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f18e00af000 tx=0x7f18e001b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:28.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.381+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f18d809a070 con 0x7f18ec160d50 2024-09-17T03:30:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.593+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f18ec0d2260 con 0x7f18ec160d50 2024-09-17T03:30:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.593+0000 7f18e8c00640 1 -- 172.21.15.46:0/3394030760 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7f18d805e4f0 con 0x7f18ec160d50 2024-09-17T03:30:28.594 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:30:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.593+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f18cc074400 msgr2=0x7f18cc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f18cc074400 0x7f18cc076910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f18e00af000 tx=0x7f18e001b060 comp rx=0 tx=0).stop 2024-09-17T03:30:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 msgr2=0x7f18ec15c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 0x7f18ec15c320 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f18d8001ec0 tx=0x7f18d800c890 comp rx=0 tx=0).stop 2024-09-17T03:30:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 shutdown_connections 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f18cc074400 0x7f18cc076910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f18ec15c860 0x7f18ec1f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec160d50 0x7f18ec15c320 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 --2- 172.21.15.46:0/3394030760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18ec157030 0x7f18ec1576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 >> 172.21.15.46:0/3394030760 conn(0x7f18ec0708c0 msgr2=0x7f18ec0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 shutdown_connections 2024-09-17T03:30:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28.597+0000 7f18f3600640 1 -- 172.21.15.46:0/3394030760 wait complete. 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:30:28.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:30:28.617 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:30:28.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:28.617 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:30:28.626 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:30:28.626 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:30:28.626 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:30:28.626 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.626 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41860sec preferred_lft 41860sec 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:30:28.627 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.628 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:30:28.629 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.629 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:30:28.629 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:30:28.629 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:30:28.629 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.629 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:30:28.632 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:30:28.632 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:30:28.632 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:30:28.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:28.633 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:30:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:30:28.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:28.694 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:30:28.744 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:30:28.744 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:30:28.744 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:30:28.744 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:30:28.745 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:30:28.745 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:30:28.745 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:30:28.745 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:30:28.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:30:28.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:30:28.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:30:28.745 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:30:28.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:30:28.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:30:28.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:28.884 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:30:28.894 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:30:28.894 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:30:28.894 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:30:28.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:28.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:30:28.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:30:28.955 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:30:29.007 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:30:29.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:30:29.008 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:30:29.058 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:30:29.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:29.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:30:29.159 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:30:29.159 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:30:29.174 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:30:29.174 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:30:29.174 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:30:29.175 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:30:29.175 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:30:29.178 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:30:29.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:29.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:30:29.199 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:30:29.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:30:29.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:30:29.308 INFO:teuthology.orchestra.run.smithi046.stdout:client247065 2024-09-17T03:30:29.308 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:29.308 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:29.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 -- 172.21.15.46:0/3453996556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380b24d0 msgr2=0x7f62380ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 --2- 172.21.15.46:0/3453996556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380b24d0 0x7f62380ce360 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6224009900 tx=0x7f622401c780 comp rx=0 tx=0).stop 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 -- 172.21.15.46:0/3453996556 shutdown_connections 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 --2- 172.21.15.46:0/3453996556 >> v2:172.21.15.99:3300/0 conn(0x7f62380ce8a0 0x7f62380cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 --2- 172.21.15.46:0/3453996556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380b24d0 0x7f62380ce360 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 -- 172.21.15.46:0/3453996556 >> 172.21.15.46:0/3453996556 conn(0x7f623806cdf0 msgr2=0x7f623806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 -- 172.21.15.46:0/3453996556 shutdown_connections 2024-09-17T03:30:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.525+0000 7f6237200640 1 -- 172.21.15.46:0/3453996556 wait complete. 2024-09-17T03:30:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 Processor -- start 2024-09-17T03:30:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 -- start start 2024-09-17T03:30:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6238074d20 0x7f623813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 0x7f6238155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623813b8f0 0x7f6238139380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6238169f20 con 0x7f62380ce8a0 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6238169da0 con 0x7f623813b8f0 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6237200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f623816a0a0 con 0x7f6238074d20 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6235400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 0x7f6238155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6235400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 0x7f6238155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34698/0 (socket says 172.21.15.46:34698) 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.529+0000 7f6235400640 1 -- 172.21.15.46:0/478535159 learned_addr learned my addr 172.21.15.46:0/478535159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235e00640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6238074d20 0x7f623813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6236800640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623813b8f0 0x7f6238139380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235400640 1 -- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6238074d20 msgr2=0x7f623813da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235400640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6238074d20 0x7f623813da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235400640 1 -- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623813b8f0 msgr2=0x7f6238139380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235400640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623813b8f0 0x7f6238139380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235400640 1 -- 172.21.15.46:0/478535159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623813c970 con 0x7f62380ce8a0 2024-09-17T03:30:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6235400640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 0x7f6238155fe0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6224009850 tx=0x7f6224005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:29.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240b2040 con 0x7f62380ce8a0 2024-09-17T03:30:29.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6224004220 con 0x7f62380ce8a0 2024-09-17T03:30:29.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224005030 con 0x7f62380ce8a0 2024-09-17T03:30:29.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623813cc50 con 0x7f62380ce8a0 2024-09-17T03:30:29.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623813d210 con 0x7f62380ce8a0 2024-09-17T03:30:29.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62380ce360 con 0x7f62380ce8a0 2024-09-17T03:30:29.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.533+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f62240051d0 con 0x7f62380ce8a0 2024-09-17T03:30:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.537+0000 7f622e000640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6218074400 0x7f6218076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:29.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.537+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f622412c7c0 con 0x7f62380ce8a0 2024-09-17T03:30:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.537+0000 7f6235e00640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6218074400 0x7f6218076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:29.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.537+0000 7f6235e00640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6218074400 0x7f6218076910 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6228001650 tx=0x7f6228009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:29.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.541+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6224016490 con 0x7f62380ce8a0 2024-09-17T03:30:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:29.749+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f623813c700 con 0x7f62380ce8a0 2024-09-17T03:30:30.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:30:30.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:30:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.409+0000 7f622e000640 1 -- 172.21.15.46:0/478535159 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f62240f9230 con 0x7f62380ce8a0 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6218074400 msgr2=0x7f6218076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6218074400 0x7f6218076910 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6228001650 tx=0x7f6228009040 comp rx=0 tx=0).stop 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 msgr2=0x7f6238155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 0x7f6238155fe0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6224009850 tx=0x7f6224005de0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 shutdown_connections 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6218074400 0x7f6218076910 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f623813b8f0 0x7f6238139380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62380ce8a0 0x7f6238155fe0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 --2- 172.21.15.46:0/478535159 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6238074d20 0x7f623813da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 >> 172.21.15.46:0/478535159 conn(0x7f623806cdf0 msgr2=0x7f62380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 shutdown_connections 2024-09-17T03:30:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.413+0000 7f6237200640 1 -- 172.21.15.46:0/478535159 wait complete. 2024-09-17T03:30:30.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:30:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 -- 172.21.15.46:0/3863655728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b5c70 msgr2=0x7fc52415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 --2- 172.21.15.46:0/3863655728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b5c70 0x7fc52415bb60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc524070710 tx=0x7fc51401c740 comp rx=0 tx=0).stop 2024-09-17T03:30:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 -- 172.21.15.46:0/3863655728 shutdown_connections 2024-09-17T03:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 --2- 172.21.15.46:0/3863655728 >> v2:172.21.15.99:3300/0 conn(0x7fc5241527b0 0x7fc524074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 --2- 172.21.15.46:0/3863655728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b5c70 0x7fc52415bb60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 -- 172.21.15.46:0/3863655728 >> 172.21.15.46:0/3863655728 conn(0x7fc5240b2040 msgr2=0x7fc5240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 -- 172.21.15.46:0/3863655728 shutdown_connections 2024-09-17T03:30:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.597+0000 7fc529a00640 1 -- 172.21.15.46:0/3863655728 wait complete. 2024-09-17T03:30:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 Processor -- start 2024-09-17T03:30:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 -- start start 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 0x7fc5241fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5240b5c70 0x7fc52413e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5241527b0 0x7fc52413e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc524168a80 con 0x7fc5240b4de0 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc524168900 con 0x7fc5241527b0 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc529a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc524168c00 con 0x7fc5240b5c70 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 0x7fc5241fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 0x7fc5241fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34708/0 (socket says 172.21.15.46:34708) 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 -- 172.21.15.46:0/550025565 learned_addr learned my addr 172.21.15.46:0/550025565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc523400640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5241527b0 0x7fc52413e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc51a000640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5240b5c70 0x7fc52413e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 -- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5240b5c70 msgr2=0x7fc52413e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5240b5c70 0x7fc52413e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 -- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5241527b0 msgr2=0x7fc52413e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5241527b0 0x7fc52413e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 -- 172.21.15.46:0/550025565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc524157340 con 0x7fc5240b4de0 2024-09-17T03:30:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.601+0000 7fc522a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 0x7fc5241fbd40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc51401cc20 tx=0x7fc5140a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5140b2040 con 0x7fc5240b4de0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5140041b0 con 0x7fc5240b4de0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc524157620 con 0x7fc5240b4de0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514003600 con 0x7fc5240b4de0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc524157be0 con 0x7fc5240b4de0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52415bb60 con 0x7fc5240b4de0 2024-09-17T03:30:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fc5140bd030 con 0x7fc5240b4de0 2024-09-17T03:30:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51b400640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc500074430 0x7fc500076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fc51412cc80 con 0x7fc5240b4de0 2024-09-17T03:30:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.605+0000 7fc51a000640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc500074430 0x7fc500076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.609+0000 7fc51a000640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc500074430 0x7fc500076940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0058d0 tx=0x7fc50c005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.613+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5140c4020 con 0x7fc5240b4de0 2024-09-17T03:30:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.809+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc524141ef0 con 0x7fc500074430 2024-09-17T03:30:30.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.809+0000 7fc51b400640 1 -- 172.21.15.46:0/550025565 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc524141ef0 con 0x7fc500074430 2024-09-17T03:30:30.810 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:30:30.810 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:30:30.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:30:30.810 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:30:30.810 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:30:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc500074430 msgr2=0x7fc500076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc500074430 0x7fc500076940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0058d0 tx=0x7fc50c005860 comp rx=0 tx=0).stop 2024-09-17T03:30:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 msgr2=0x7fc5241fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 0x7fc5241fbd40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc51401cc20 tx=0x7fc5140a5620 comp rx=0 tx=0).stop 2024-09-17T03:30:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 shutdown_connections 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc500074430 0x7fc500076940 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5241527b0 0x7fc52413e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5240b5c70 0x7fc52413e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 --2- 172.21.15.46:0/550025565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5240b4de0 0x7fc5241fbd40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 >> 172.21.15.46:0/550025565 conn(0x7fc5240b2040 msgr2=0x7fc5240b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 shutdown_connections 2024-09-17T03:30:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.813+0000 7fc529a00640 1 -- 172.21.15.46:0/550025565 wait complete. 2024-09-17T03:30:30.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:30:30.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 -- 172.21.15.46:0/2977576681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 msgr2=0x7fbaa81603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 --2- 172.21.15.46:0/2977576681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa81603e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fba90009900 tx=0x7fba9001c8b0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 -- 172.21.15.46:0/2977576681 shutdown_connections 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 --2- 172.21.15.46:0/2977576681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa81603e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 --2- 172.21.15.46:0/2977576681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbaa815c7b0 0x7fbaa8068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 -- 172.21.15.46:0/2977576681 >> 172.21.15.46:0/2977576681 conn(0x7fbaa80b35f0 msgr2=0x7fbaa80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 -- 172.21.15.46:0/2977576681 shutdown_connections 2024-09-17T03:30:30.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 -- 172.21.15.46:0/2977576681 wait complete. 2024-09-17T03:30:30.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 Processor -- start 2024-09-17T03:30:30.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:30.997+0000 7fbaaf000640 1 -- start start 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbaa815c7b0 0x7fbaa8139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa8160920 0x7fbaa814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8169bc0 con 0x7fbaa8069330 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa8169a40 con 0x7fbaa8160920 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa8169d40 con 0x7fbaa815c7b0 2024-09-17T03:30:31.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34712/0 (socket says 172.21.15.46:34712) 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 -- 172.21.15.46:0/205651911 learned_addr learned my addr 172.21.15.46:0/205651911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fba9fe00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbaa815c7b0 0x7fbaa8139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaad000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa8160920 0x7fbaa814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 -- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbaa815c7b0 msgr2=0x7fbaa8139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbaa815c7b0 0x7fbaa8139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 -- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa8160920 msgr2=0x7fbaa814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa8160920 0x7fbaa814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 -- 172.21.15.46:0/205651911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa813a510 con 0x7fbaa8069330 2024-09-17T03:30:31.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa7e00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa815e4a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fba980058d0 tx=0x7fba9800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98017070 con 0x7fbaa8069330 2024-09-17T03:30:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba98009cc0 con 0x7fbaa8069330 2024-09-17T03:30:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba980106c0 con 0x7fbaa8069330 2024-09-17T03:30:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa81fa970 con 0x7fbaa8069330 2024-09-17T03:30:31.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa815a210 con 0x7fbaa8069330 2024-09-17T03:30:31.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.001+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa81603e0 con 0x7fbaa8069330 2024-09-17T03:30:31.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.005+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fba9801a070 con 0x7fbaa8069330 2024-09-17T03:30:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.005+0000 7fbaa4c00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba84074430 0x7fba84076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.005+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fba980932e0 con 0x7fbaa8069330 2024-09-17T03:30:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.005+0000 7fba9fe00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba84074430 0x7fba84076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.005+0000 7fba9fe00640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba84074430 0x7fba84076940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fba900af000 tx=0x7fba9001b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.009+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba9805fa70 con 0x7fbaa8069330 2024-09-17T03:30:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.185+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbaa8155950 con 0x7fbaa8069330 2024-09-17T03:30:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.185+0000 7fbaa4c00640 1 -- 172.21.15.46:0/205651911 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fba9805f370 con 0x7fbaa8069330 2024-09-17T03:30:31.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba84074430 msgr2=0x7fba84076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba84074430 0x7fba84076940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fba900af000 tx=0x7fba9001b0f0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 msgr2=0x7fbaa815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa815e4a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fba980058d0 tx=0x7fba9800dde0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 shutdown_connections 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba84074430 0x7fba84076940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbaa8160920 0x7fbaa814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbaa815c7b0 0x7fbaa8139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 --2- 172.21.15.46:0/205651911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbaa8069330 0x7fbaa815e4a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 >> 172.21.15.46:0/205651911 conn(0x7fbaa80b35f0 msgr2=0x7fbaa80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 shutdown_connections 2024-09-17T03:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.189+0000 7fbaaf000640 1 -- 172.21.15.46:0/205651911 wait complete. 2024-09-17T03:30:31.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:30:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 -- 172.21.15.46:0/902036533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 msgr2=0x7fed78004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 --2- 172.21.15.46:0/902036533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed78004070 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fed84009b60 tx=0x7fed8401d290 comp rx=0 tx=0).stop 2024-09-17T03:30:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed99400640 1 -- 172.21.15.46:0/902036533 reap_dead start 2024-09-17T03:30:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 -- 172.21.15.46:0/902036533 shutdown_connections 2024-09-17T03:30:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 -- 172.21.15.46:0/902036533 >> 172.21.15.46:0/902036533 conn(0x7fed940708c0 msgr2=0x7fed94070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:31.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 -- 172.21.15.46:0/902036533 shutdown_connections 2024-09-17T03:30:31.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 -- 172.21.15.46:0/902036533 wait complete. 2024-09-17T03:30:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 Processor -- start 2024-09-17T03:30:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.373+0000 7fed9b400640 1 -- start start 2024-09-17T03:30:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed94141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed94163ce0 0x7fed94157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed941581b0 0x7fed941fa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed941693d0 con 0x7fed78003c20 2024-09-17T03:30:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed94169250 con 0x7fed94163ce0 2024-09-17T03:30:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fed94169550 con 0x7fed941581b0 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed94141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed94141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34726/0 (socket says 172.21.15.46:34726) 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 -- 172.21.15.46:0/3373012228 learned_addr learned my addr 172.21.15.46:0/3373012228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed8be00640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed94163ce0 0x7fed94157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 -- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed941581b0 msgr2=0x7fed941fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed941581b0 0x7fed941fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 -- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed94163ce0 msgr2=0x7fed94157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed94163ce0 0x7fed94157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 -- 172.21.15.46:0/3373012228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed94167ac0 con 0x7fed78003c20 2024-09-17T03:30:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed98a00640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed94141de0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fed84009b60 tx=0x7fed84005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed840b1040 con 0x7fed78003c20 2024-09-17T03:30:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed840bb070 con 0x7fed78003c20 2024-09-17T03:30:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84004290 con 0x7fed78003c20 2024-09-17T03:30:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed94167220 con 0x7fed78003c20 2024-09-17T03:30:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed941677e0 con 0x7fed78003c20 2024-09-17T03:30:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.377+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9415cf90 con 0x7fed78003c20 2024-09-17T03:30:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.381+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fed840046d0 con 0x7fed78003c20 2024-09-17T03:30:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.381+0000 7fed91600640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed70074400 0x7fed70076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.381+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fed8412e2c0 con 0x7fed78003c20 2024-09-17T03:30:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.381+0000 7fed8be00640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed70074400 0x7fed70076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.381+0000 7fed8be00640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed70074400 0x7fed70076910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fed9415a3a0 tx=0x7fed7c006b20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.385+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fed84016fa0 con 0x7fed78003c20 2024-09-17T03:30:31.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.561+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fed940d2260 con 0x7fed78003c20 2024-09-17T03:30:31.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.561+0000 7fed91600640 1 -- 172.21.15.46:0/3373012228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7fed840faa50 con 0x7fed78003c20 2024-09-17T03:30:31.564 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:30:31.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed70074400 msgr2=0x7fed70076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed70074400 0x7fed70076910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fed9415a3a0 tx=0x7fed7c006b20 comp rx=0 tx=0).stop 2024-09-17T03:30:31.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 msgr2=0x7fed94141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed94141de0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fed84009b60 tx=0x7fed84005590 comp rx=0 tx=0).stop 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 shutdown_connections 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed70074400 0x7fed70076910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed941581b0 0x7fed941fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed94163ce0 0x7fed94157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 --2- 172.21.15.46:0/3373012228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed78003c20 0x7fed94141de0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 >> 172.21.15.46:0/3373012228 conn(0x7fed940708c0 msgr2=0x7fed9415e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 shutdown_connections 2024-09-17T03:30:31.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.565+0000 7fed9b400640 1 -- 172.21.15.46:0/3373012228 wait complete. 2024-09-17T03:30:31.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:30:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 -- 172.21.15.46:0/1020339008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc0cf900 msgr2=0x7efbfc15e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 --2- 172.21.15.46:0/1020339008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc0cf900 0x7efbfc15e8c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efbec00aa10 tx=0x7efbec01dba0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 -- 172.21.15.46:0/1020339008 shutdown_connections 2024-09-17T03:30:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 --2- 172.21.15.46:0/1020339008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efbfc15e290 0x7efbfc15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 --2- 172.21.15.46:0/1020339008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc0cf900 0x7efbfc15e8c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 -- 172.21.15.46:0/1020339008 >> 172.21.15.46:0/1020339008 conn(0x7efbfc0b2040 msgr2=0x7efbfc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 -- 172.21.15.46:0/1020339008 shutdown_connections 2024-09-17T03:30:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.745+0000 7efc01600640 1 -- 172.21.15.46:0/1020339008 wait complete. 2024-09-17T03:30:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 Processor -- start 2024-09-17T03:30:31.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 -- start start 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efbfc0cf900 0x7efbfc1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efbfc15e290 0x7efbfc1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 0x7efbfc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc168270 con 0x7efbfc15ee00 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1680f0 con 0x7efbfc0cf900 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efc01600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7efbfc1683f0 con 0x7efbfc15e290 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efbfb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 0x7efbfc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efbfaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efbfc0cf900 0x7efbfc1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efbfb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 0x7efbfc06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34738/0 (socket says 172.21.15.46:34738) 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efbfb400640 1 -- 172.21.15.46:0/188017970 learned_addr learned my addr 172.21.15.46:0/188017970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.749+0000 7efbf3e00640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efbfc15e290 0x7efbfc1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbfb400640 1 -- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efbfc15e290 msgr2=0x7efbfc1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbfb400640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efbfc15e290 0x7efbfc1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbfb400640 1 -- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efbfc0cf900 msgr2=0x7efbfc1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbfb400640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efbfc0cf900 0x7efbfc1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbfb400640 1 -- 172.21.15.46:0/188017970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc156180 con 0x7efbfc15ee00 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbfb400640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 0x7efbfc06e0d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e940 tx=0x7efbf400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4004d30 con 0x7efbfc15ee00 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbf400cb20 con 0x7efbfc15ee00 2024-09-17T03:30:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc1fc670 con 0x7efbfc15ee00 2024-09-17T03:30:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4005020 con 0x7efbfc15ee00 2024-09-17T03:30:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1fcc30 con 0x7efbfc15ee00 2024-09-17T03:30:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc15e8c0 con 0x7efbfc15ee00 2024-09-17T03:30:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.753+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7efbf40051c0 con 0x7efbfc15ee00 2024-09-17T03:30:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.757+0000 7efbf2a00640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efbd8074430 0x7efbd8076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.757+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7efbf4093a60 con 0x7efbfc15ee00 2024-09-17T03:30:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.757+0000 7efbfaa00640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efbd8074430 0x7efbd8076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.757+0000 7efbfaa00640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efbd8074430 0x7efbd8076940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efbec0b0000 tx=0x7efbec0a8040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.761+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efbf4016030 con 0x7efbfc15ee00 2024-09-17T03:30:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.933+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7efbfc0b26a0 con 0x7efbfc15ee00 2024-09-17T03:30:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efbf2a00640 1 -- 172.21.15.46:0/188017970 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7efbf405fb10 con 0x7efbfc15ee00 2024-09-17T03:30:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efbd8074430 msgr2=0x7efbd8076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efbd8074430 0x7efbd8076940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efbec0b0000 tx=0x7efbec0a8040 comp rx=0 tx=0).stop 2024-09-17T03:30:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 msgr2=0x7efbfc06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:31.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 0x7efbfc06e0d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e940 tx=0x7efbf400ee00 comp rx=0 tx=0).stop 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 shutdown_connections 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7efbd8074430 0x7efbd8076940 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efbfc15ee00 0x7efbfc06e0d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7efbfc15e290 0x7efbfc1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 --2- 172.21.15.46:0/188017970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efbfc0cf900 0x7efbfc1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 >> 172.21.15.46:0/188017970 conn(0x7efbfc0b2040 msgr2=0x7efbfc13dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 shutdown_connections 2024-09-17T03:30:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:31.941+0000 7efc01600640 1 -- 172.21.15.46:0/188017970 wait complete. 2024-09-17T03:30:31.960 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:31.961+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:31.960 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:31.961+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:31.960 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:31.961+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:31.960 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:31.961+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:31.960 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:31.961+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:31.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 -- 172.21.15.46:0/3232219119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 msgr2=0x7f8e3015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 --2- 172.21.15.46:0/3232219119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 0x7f8e3015bb60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009840 tx=0x7f8e2001c760 comp rx=0 tx=0).stop 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 -- 172.21.15.46:0/3232219119 shutdown_connections 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 --2- 172.21.15.46:0/3232219119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e301527b0 0x7f8e30074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 --2- 172.21.15.46:0/3232219119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 0x7f8e3015bb60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 -- 172.21.15.46:0/3232219119 >> 172.21.15.46:0/3232219119 conn(0x7f8e300b2040 msgr2=0x7f8e300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 -- 172.21.15.46:0/3232219119 shutdown_connections 2024-09-17T03:30:32.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 -- 172.21.15.46:0/3232219119 wait complete. 2024-09-17T03:30:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 Processor -- start 2024-09-17T03:30:32.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.129+0000 7f8e34c00640 1 -- start start 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 0x7f8e301f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 0x7f8e301f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e301527b0 0x7f8e301fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30168c50 con 0x7f8e300b4de0 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30168ad0 con 0x7f8e300b5c70 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8e30168dd0 con 0x7f8e301527b0 2024-09-17T03:30:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 0x7f8e301f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 0x7f8e301f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34764/0 (socket says 172.21.15.46:34764) 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 -- 172.21.15.46:0/2724749201 learned_addr learned my addr 172.21.15.46:0/2724749201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e27e00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 0x7f8e301f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2e800640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e301527b0 0x7f8e301fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 -- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e301527b0 msgr2=0x7f8e301fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e301527b0 0x7f8e301fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 -- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 msgr2=0x7f8e301f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 0x7f8e301f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 -- 172.21.15.46:0/2724749201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3014c5b0 con 0x7f8e300b4de0 2024-09-17T03:30:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e2de00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 0x7f8e301f73f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009790 tx=0x7f8e200a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200bb070 con 0x7f8e300b4de0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e20004470 con 0x7f8e300b4de0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200b2040 con 0x7f8e300b4de0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3014c890 con 0x7f8e300b4de0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301f6ed0 con 0x7f8e300b4de0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.133+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3015bb60 con 0x7f8e300b4de0 2024-09-17T03:30:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.137+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f8e20005290 con 0x7f8e300b4de0 2024-09-17T03:30:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.137+0000 7f8e26000640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8e0c074400 0x7f8e0c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.137+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f8e2012c6f0 con 0x7f8e300b4de0 2024-09-17T03:30:32.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.137+0000 7f8e27e00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8e0c074400 0x7f8e0c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.141+0000 7f8e27e00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8e0c074400 0x7f8e0c076910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e300cfd60 tx=0x7f8e18009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.141+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8e30075580 con 0x7f8e300b4de0 2024-09-17T03:30:32.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.317+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f8e30150d00 con 0x7f8e300b4de0 2024-09-17T03:30:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.321+0000 7f8e26000640 1 -- 172.21.15.46:0/2724749201 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v47)=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f8e20016410 con 0x7f8e300b4de0 2024-09-17T03:30:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8e0c074400 msgr2=0x7f8e0c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8e0c074400 0x7f8e0c076910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e300cfd60 tx=0x7f8e18009260 comp rx=0 tx=0).stop 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 msgr2=0x7f8e301f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 0x7f8e301f73f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009790 tx=0x7f8e200a56c0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 shutdown_connections 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8e0c074400 0x7f8e0c076910 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8e301527b0 0x7f8e301fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e300b5c70 0x7f8e301f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 --2- 172.21.15.46:0/2724749201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e300b4de0 0x7f8e301f73f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 >> 172.21.15.46:0/2724749201 conn(0x7f8e300b2040 msgr2=0x7f8e3014d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:32.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 shutdown_connections 2024-09-17T03:30:32.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.325+0000 7f8e34c00640 1 -- 172.21.15.46:0/2724749201 wait complete. 2024-09-17T03:30:32.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 -- 172.21.15.46:0/1209937852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c15f5b0 msgr2=0x7fb58c0b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 --2- 172.21.15.46:0/1209937852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c15f5b0 0x7fb58c0b1dc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb574009900 tx=0x7fb57401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 -- 172.21.15.46:0/1209937852 shutdown_connections 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 --2- 172.21.15.46:0/1209937852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c15f5b0 0x7fb58c0b1dc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 --2- 172.21.15.46:0/1209937852 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb58c155a00 0x7fb58c15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 -- 172.21.15.46:0/1209937852 >> 172.21.15.46:0/1209937852 conn(0x7fb58c06cdf0 msgr2=0x7fb58c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 -- 172.21.15.46:0/1209937852 shutdown_connections 2024-09-17T03:30:32.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 -- 172.21.15.46:0/1209937852 wait complete. 2024-09-17T03:30:32.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 Processor -- start 2024-09-17T03:30:32.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.513+0000 7fb591400640 1 -- start start 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 0x7fb58c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb58c155a00 0x7fb58c1f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c15f5b0 0x7fb58c13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c168810 con 0x7fb58c0b2300 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c168690 con 0x7fb58c15f5b0 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb58c168990 con 0x7fb58c155a00 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 0x7fb58c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 0x7fb58c0ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46108/0 (socket says 172.21.15.46:46108) 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 -- 172.21.15.46:0/2407508291 learned_addr learned my addr 172.21.15.46:0/2407508291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb583e00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb58c155a00 0x7fb58c1f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb590a00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c15f5b0 0x7fb58c13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 -- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb58c155a00 msgr2=0x7fb58c1f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb58c155a00 0x7fb58c1f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 -- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c15f5b0 msgr2=0x7fb58c13cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c15f5b0 0x7fb58c13cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 -- 172.21.15.46:0/2407508291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c0cd460 con 0x7fb58c0b2300 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb590a00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c15f5b0 0x7fb58c13cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb58be00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 0x7fb58c0ccf20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00d720 tx=0x7fb57c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:32.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c017070 con 0x7fb58c0b2300 2024-09-17T03:30:32.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb57c009d20 con 0x7fb58c0b2300 2024-09-17T03:30:32.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c005280 con 0x7fb58c0b2300 2024-09-17T03:30:32.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58c13bcd0 con 0x7fb58c0b2300 2024-09-17T03:30:32.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58c13c290 con 0x7fb58c0b2300 2024-09-17T03:30:32.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.517+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58c15f070 con 0x7fb58c0b2300 2024-09-17T03:30:32.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.521+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7fb57c005420 con 0x7fb58c0b2300 2024-09-17T03:30:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.521+0000 7fb588c00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb568074430 0x7fb568076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.521+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fb57c0931d0 con 0x7fb58c0b2300 2024-09-17T03:30:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.521+0000 7fb583e00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb568074430 0x7fb568076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.521+0000 7fb583e00640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb568074430 0x7fb568076940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb5740af000 tx=0x7fb57401b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.525+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb57c05f280 con 0x7fb58c0b2300 2024-09-17T03:30:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.729+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fb58c0cc380 con 0x7fb568074430 2024-09-17T03:30:32.736 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:32.737+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:32.736 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:32.737+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:32.736 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:32.737+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:32.736 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:32.737+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:32.736 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:30:32.737+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:30:32.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.769+0000 7fb588c00640 1 -- 172.21.15.46:0/2407508291 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5740b3040 con 0x7fb568074430 2024-09-17T03:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb568074430 msgr2=0x7fb568076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb568074430 0x7fb568076940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb5740af000 tx=0x7fb57401b0f0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 msgr2=0x7fb58c0ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 0x7fb58c0ccf20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00d720 tx=0x7fb57c00dbe0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 shutdown_connections 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb568074430 0x7fb568076940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c15f5b0 0x7fb58c13cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb58c155a00 0x7fb58c1f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 --2- 172.21.15.46:0/2407508291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58c0b2300 0x7fb58c0ccf20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 >> 172.21.15.46:0/2407508291 conn(0x7fb58c06cdf0 msgr2=0x7fb58c0d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 shutdown_connections 2024-09-17T03:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.773+0000 7fb591400640 1 -- 172.21.15.46:0/2407508291 wait complete. 2024-09-17T03:30:32.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 -- 172.21.15.46:0/3232037537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 msgr2=0x7f066c15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 --2- 172.21.15.46:0/3232037537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c15bc20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f066c071130 tx=0x7f066401dc70 comp rx=0 tx=0).stop 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 -- 172.21.15.46:0/3232037537 shutdown_connections 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 --2- 172.21.15.46:0/3232037537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c15bc20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 --2- 172.21.15.46:0/3232037537 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f066c0cf620 0x7f066c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 --2- 172.21.15.46:0/3232037537 >> v2:172.21.15.99:3300/0 conn(0x7f066c15eeb0 0x7f066c0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 -- 172.21.15.46:0/3232037537 >> 172.21.15.46:0/3232037537 conn(0x7f066c0b2040 msgr2=0x7f066c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 -- 172.21.15.46:0/3232037537 shutdown_connections 2024-09-17T03:30:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.953+0000 7f0672000640 1 -- 172.21.15.46:0/3232037537 wait complete. 2024-09-17T03:30:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 Processor -- start 2024-09-17T03:30:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 -- start start 2024-09-17T03:30:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f066c0cf620 0x7f066c1314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c131a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f066c15eeb0 0x7f066c1f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f066c15e3e0 con 0x7f066c15b810 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f066c15e260 con 0x7f066c0cf620 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0672000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f066c166a50 con 0x7f066c15eeb0 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c131a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c131a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46144/0 (socket says 172.21.15.46:46144) 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 -- 172.21.15.46:0/2629616630 learned_addr learned my addr 172.21.15.46:0/2629616630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f066b400640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f066c0cf620 0x7f066c1314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f066be00640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f066c15eeb0 0x7f066c1f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 -- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f066c15eeb0 msgr2=0x7f066c1f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f066c15eeb0 0x7f066c1f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 -- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f066c0cf620 msgr2=0x7f066c1314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f066c0cf620 0x7f066c1314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.957+0000 7f0662a00640 1 -- 172.21.15.46:0/2629616630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c0ad980 con 0x7f066c15b810 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0662a00640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c131a10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f06540058d0 tx=0x7f065400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654017070 con 0x7f066c15b810 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0654009d20 con 0x7f066c15b810 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06540106c0 con 0x7f066c15b810 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c131f50 con 0x7f066c15b810 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c0ad0e0 con 0x7f066c15b810 2024-09-17T03:30:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c15bc20 con 0x7f066c15b810 2024-09-17T03:30:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.961+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) ==== 98866+0+0 (secure 0 0 0) 0x7f065401a070 con 0x7f066c15b810 2024-09-17T03:30:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.965+0000 7f0663e00640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0648074400 0x7f0648076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.965+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f0654093740 con 0x7f066c15b810 2024-09-17T03:30:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.965+0000 7f066b400640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0648074400 0x7f0648076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.965+0000 7f066b400640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0648074400 0x7f0648076910 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f065c00bad0 tx=0x7f065c00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:32.969+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f065405f8c0 con 0x7f066c15b810 2024-09-17T03:30:33.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.169+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f066c06b540 con 0x7f0648074400 2024-09-17T03:30:33.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0663e00640 1 -- 172.21.15.46:0/2629616630 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f065c010b10 con 0x7f0648074400 2024-09-17T03:30:33.172 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0648074400 msgr2=0x7f0648076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0648074400 0x7f0648076910 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f065c00bad0 tx=0x7f065c00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 msgr2=0x7f066c131a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c131a10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f06540058d0 tx=0x7f065400dde0 comp rx=0 tx=0).stop 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 shutdown_connections 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0648074400 0x7f0648076910 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f066c15eeb0 0x7f066c1f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f066c15b810 0x7f066c131a10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:33.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 --2- 172.21.15.46:0/2629616630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f066c0cf620 0x7f066c1314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:33.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 >> 172.21.15.46:0/2629616630 conn(0x7f066c0b2040 msgr2=0x7f066c0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:33.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 shutdown_connections 2024-09-17T03:30:33.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:33.173+0000 7f0672000640 1 -- 172.21.15.46:0/2629616630 wait complete. 2024-09-17T03:30:33.196 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:30:33.196 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:33.196 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:33.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:33.206 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:30:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:30:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:30:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.41894 s, 128 MB/s 2024-09-17T03:30:41.682 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:41.682 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:41.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:41.691 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:30:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:30:50.302 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:30:50.302 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.55318 s, 126 MB/s 2024-09-17T03:30:50.303 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:30:50.303 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:30:50.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:50.312 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:30:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:30:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:30:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.10352 s, 133 MB/s 2024-09-17T03:30:58.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:30:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.633+0000 7f97a6c00640 1 -- 172.21.15.46:0/1131237333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 msgr2=0x7f97a015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.633+0000 7f97a6c00640 1 --2- 172.21.15.46:0/1131237333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a015ecd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9790009900 tx=0x7f979001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.633+0000 7f97a6c00640 1 -- 172.21.15.46:0/1131237333 shutdown_connections 2024-09-17T03:30:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.633+0000 7f97a6c00640 1 --2- 172.21.15.46:0/1131237333 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a015b2d0 0x7f97a015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.633+0000 7f97a6c00640 1 --2- 172.21.15.46:0/1131237333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a015ecd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.633+0000 7f97a6c00640 1 -- 172.21.15.46:0/1131237333 >> 172.21.15.46:0/1131237333 conn(0x7f97a00b2040 msgr2=0x7f97a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 -- 172.21.15.46:0/1131237333 shutdown_connections 2024-09-17T03:30:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 -- 172.21.15.46:0/1131237333 wait complete. 2024-09-17T03:30:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 Processor -- start 2024-09-17T03:30:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 -- start start 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a015b2d0 0x7f97a0142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a015bc20 0x7f97a00b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0168d10 con 0x7f97a015e8c0 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0168b90 con 0x7f97a015b2d0 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.637+0000 7f97a6c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f97a0168e90 con 0x7f97a015bc20 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a00b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37856/0 (socket says 172.21.15.46:37856) 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 -- 172.21.15.46:0/2033416117 learned_addr learned my addr 172.21.15.46:0/2033416117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f9797e00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a015bc20 0x7f97a00b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f979fe00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a015b2d0 0x7f97a0142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 -- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a015bc20 msgr2=0x7f97a00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a015bc20 0x7f97a00b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 -- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a015b2d0 msgr2=0x7f97a0142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a015b2d0 0x7f97a0142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 -- 172.21.15.46:0/2033416117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a00ccb50 con 0x7f97a015e8c0 2024-09-17T03:30:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a4c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a00b0e40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f979800c730 tx=0x7f979800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798003b80 con 0x7f97a015e8c0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9798003d20 con 0x7f97a015e8c0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a0142e30 con 0x7f97a015e8c0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798005330 con 0x7f97a015e8c0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a00aa7b0 con 0x7f97a015e8c0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a015b6e0 con 0x7f97a015e8c0 2024-09-17T03:30:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.641+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 34) ==== 98968+0+0 (secure 0 0 0) 0x7f97980056e0 con 0x7f97a015e8c0 2024-09-17T03:30:58.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.645+0000 7f979cc00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f977c074550 0x7f977c076a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:58.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.645+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f9798093160 con 0x7f97a015e8c0 2024-09-17T03:30:58.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.645+0000 7f979fe00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f977c074550 0x7f977c076a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.645+0000 7f979fe00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f977c074550 0x7f977c076a60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97900af000 tx=0x7f979001b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.649+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f979805f8c0 con 0x7f97a015e8c0 2024-09-17T03:30:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.853+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f97a0142be0 con 0x7f977c074550 2024-09-17T03:30:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f979cc00640 1 -- 172.21.15.46:0/2033416117 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97a0142be0 con 0x7f977c074550 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f977c074550 msgr2=0x7f977c076a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f977c074550 0x7f977c076a60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97900af000 tx=0x7f979001b120 comp rx=0 tx=0).stop 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 msgr2=0x7f97a00b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a00b0e40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f979800c730 tx=0x7f979800cbf0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 shutdown_connections 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f977c074550 0x7f977c076a60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a015e8c0 0x7f97a00b0e40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a015bc20 0x7f97a00b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.865+0000 7f97a6c00640 1 --2- 172.21.15.46:0/2033416117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a015b2d0 0x7f97a0142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.869+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 >> 172.21.15.46:0/2033416117 conn(0x7f97a00b2040 msgr2=0x7f97a00b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.869+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 shutdown_connections 2024-09-17T03:30:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:58.869+0000 7f97a6c00640 1 -- 172.21.15.46:0/2033416117 wait complete. 2024-09-17T03:30:58.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:30:59.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 -- 172.21.15.46:0/4177472590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14540b24d0 msgr2=0x7f14540b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 --2- 172.21.15.46:0/4177472590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14540b24d0 0x7f14540b3130 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f144c009c80 tx=0x7f144c01cca0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 -- 172.21.15.46:0/4177472590 shutdown_connections 2024-09-17T03:30:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 --2- 172.21.15.46:0/4177472590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14540b24d0 0x7f14540b3130 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 --2- 172.21.15.46:0/4177472590 >> v2:172.21.15.99:3300/0 conn(0x7f1454155a00 0x7f145415f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 -- 172.21.15.46:0/4177472590 >> 172.21.15.46:0/4177472590 conn(0x7f145406cdf0 msgr2=0x7f145406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 -- 172.21.15.46:0/4177472590 shutdown_connections 2024-09-17T03:30:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.049+0000 7f145a600640 1 -- 172.21.15.46:0/4177472590 wait complete. 2024-09-17T03:30:59.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 Processor -- start 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 -- start start 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1454155a00 0x7f14540b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 0x7f14540d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14540cea10 0x7f1454141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1454168a90 con 0x7f145415f5b0 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1454168910 con 0x7f14540cea10 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f145a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1454168c10 con 0x7f1454155a00 2024-09-17T03:30:59.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f1453e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 0x7f14540d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f1453e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 0x7f14540d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37870/0 (socket says 172.21.15.46:37870) 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.053+0000 7f1453e00640 1 -- 172.21.15.46:0/2278596072 learned_addr learned my addr 172.21.15.46:0/2278596072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1459200640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1454155a00 0x7f14540b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1459c00640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14540cea10 0x7f1454141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1453e00640 1 -- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1454155a00 msgr2=0x7f14540b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1453e00640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1454155a00 0x7f14540b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1453e00640 1 -- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14540cea10 msgr2=0x7f1454141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:59.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1453e00640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14540cea10 0x7f1454141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1453e00640 1 -- 172.21.15.46:0/2278596072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14540cf180 con 0x7f145415f5b0 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1459c00640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14540cea10 0x7f1454141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1459200640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1454155a00 0x7f14540b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1453e00640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 0x7f14540d0b40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f144000c3f0 tx=0x7f144000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400052d0 con 0x7f145415f5b0 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1440007020 con 0x7f145415f5b0 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440007470 con 0x7f145415f5b0 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14540afd40 con 0x7f145415f5b0 2024-09-17T03:30:59.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1454141fe0 con 0x7f145415f5b0 2024-09-17T03:30:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14540ceef0 con 0x7f145415f5b0 2024-09-17T03:30:59.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.057+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 34) ==== 98968+0+0 (secure 0 0 0) 0x7f14400067e0 con 0x7f145415f5b0 2024-09-17T03:30:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.061+0000 7f1450c00640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1434074520 0x7f1434076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:30:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.061+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f1440092710 con 0x7f145415f5b0 2024-09-17T03:30:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.061+0000 7f1459200640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1434074520 0x7f1434076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:30:59.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.061+0000 7f1459200640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1434074520 0x7f1434076a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1448005fd0 tx=0x7f1448005f00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:30:59.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.065+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f144005ede0 con 0x7f145415f5b0 2024-09-17T03:30:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.265+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f145406b540 con 0x7f1434074520 2024-09-17T03:30:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f1450c00640 1 -- 172.21.15.46:0/2278596072 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f144800fb30 con 0x7f1434074520 2024-09-17T03:30:59.268 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:30:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1434074520 msgr2=0x7f1434076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1434074520 0x7f1434076a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1448005fd0 tx=0x7f1448005f00 comp rx=0 tx=0).stop 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 msgr2=0x7f14540d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 0x7f14540d0b40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f144000c3f0 tx=0x7f144000c8b0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 shutdown_connections 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1434074520 0x7f1434076a30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14540cea10 0x7f1454141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f145415f5b0 0x7f14540d0b40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 --2- 172.21.15.46:0/2278596072 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1454155a00 0x7f14540b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 >> 172.21.15.46:0/2278596072 conn(0x7f145406cdf0 msgr2=0x7f14540728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 shutdown_connections 2024-09-17T03:30:59.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:30:59.269+0000 7f145a600640 1 -- 172.21.15.46:0/2278596072 wait complete. 2024-09-17T03:30:59.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:30:59.291 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:30:59.291 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:30:59.300 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:30:59.301 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:30:59.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:30:59.302 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:31:00.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:31:00.096 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:31:01.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:31:01.303 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:31:01.303 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:31:01.314 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:31:01.314 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:31:01.315 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:01.315 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:31:03.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:31:03.317 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:31:03.317 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:31:03.329 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/99384915-3e39-4ca1-946d-8221d23ca816 2024-09-17T03:31:03.329 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T03:31:03.329 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:03.331 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T03:31:03.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone52350 2024-09-17T03:31:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 -- 172.21.15.46:0/2213751486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b5c70 msgr2=0x7f822415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 --2- 172.21.15.46:0/2213751486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b5c70 0x7f822415bb60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8224074b20 tx=0x7f821801c820 comp rx=0 tx=0).stop 2024-09-17T03:31:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 -- 172.21.15.46:0/2213751486 shutdown_connections 2024-09-17T03:31:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 --2- 172.21.15.46:0/2213751486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b5c70 0x7f822415bb60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 -- 172.21.15.46:0/2213751486 >> 172.21.15.46:0/2213751486 conn(0x7f82240b2040 msgr2=0x7f82240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 -- 172.21.15.46:0/2213751486 shutdown_connections 2024-09-17T03:31:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 -- 172.21.15.46:0/2213751486 wait complete. 2024-09-17T03:31:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 Processor -- start 2024-09-17T03:31:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.553+0000 7f822b800640 1 -- start start 2024-09-17T03:31:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 0x7f8224150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82240b5c70 0x7f822415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82241527b0 0x7f822415fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8224169bd0 con 0x7f82240b4de0 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8224169a50 con 0x7f82240b5c70 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8224169d50 con 0x7f82241527b0 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 0x7f8224150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 0x7f8224150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60242/0 (socket says 172.21.15.46:60242) 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 -- 172.21.15.46:0/3297735123 learned_addr learned my addr 172.21.15.46:0/3297735123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8223e00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82240b5c70 0x7f822415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8229800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82241527b0 0x7f822415fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 -- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82241527b0 msgr2=0x7f822415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82241527b0 0x7f822415fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 -- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82240b5c70 msgr2=0x7f822415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82240b5c70 0x7f822415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 -- 172.21.15.46:0/3297735123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82241fbdd0 con 0x7f82240b4de0 2024-09-17T03:31:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8229800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82241527b0 0x7f822415fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8228e00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 0x7f8224150790 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f82180b0040 tx=0x7f82180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180b2040 con 0x7f82240b4de0 2024-09-17T03:31:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8218003600 con 0x7f82240b4de0 2024-09-17T03:31:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82241fc0b0 con 0x7f82240b4de0 2024-09-17T03:31:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218003a50 con 0x7f82240b4de0 2024-09-17T03:31:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241fc670 con 0x7f82240b4de0 2024-09-17T03:31:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.557+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822415bb60 con 0x7f82240b4de0 2024-09-17T03:31:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.561+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 34) ==== 98968+0+0 (secure 0 0 0) 0x7f8218003bf0 con 0x7f82240b4de0 2024-09-17T03:31:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.561+0000 7f8220c00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8204074520 0x7f8204076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.561+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f821812d560 con 0x7f82240b4de0 2024-09-17T03:31:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.561+0000 7f8223e00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8204074520 0x7f8204076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.561+0000 7f8223e00640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8204074520 0x7f8204076a30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8224158a70 tx=0x7f8210009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.565+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82180bd060 con 0x7f82240b4de0 2024-09-17T03:31:03.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.769+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone52350", "target": ["mon-mgr", ""]}) -- 0x7f82241fbb80 con 0x7f8204074520 2024-09-17T03:31:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f8220c00640 1 -- 172.21.15.46:0/3297735123 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82241fbb80 con 0x7f8204074520 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8204074520 msgr2=0x7f8204076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8204074520 0x7f8204076a30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8224158a70 tx=0x7f8210009040 comp rx=0 tx=0).stop 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 msgr2=0x7f8224150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 0x7f8224150790 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f82180b0040 tx=0x7f82180040c0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 shutdown_connections 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8204074520 0x7f8204076a30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82241527b0 0x7f822415fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82240b5c70 0x7f822415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 --2- 172.21.15.46:0/3297735123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82240b4de0 0x7f8224150790 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 >> 172.21.15.46:0/3297735123 conn(0x7f82240b2040 msgr2=0x7f8224151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 shutdown_connections 2024-09-17T03:31:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:03.833+0000 7f822b800640 1 -- 172.21.15.46:0/3297735123 wait complete. 2024-09-17T03:31:03.838 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.837+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.839 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.837+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.839 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.837+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.839 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.837+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.839 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.837+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone79881 2024-09-17T03:31:03.859 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.857+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.859 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.857+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.859 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.857+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.859 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.857+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:03.859 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:03.857+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:04.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 -- 172.21.15.46:0/420953724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 msgr2=0x7f78bc074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 --2- 172.21.15.46:0/420953724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc074db0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009840 tx=0x7f78b001c820 comp rx=0 tx=0).stop 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 -- 172.21.15.46:0/420953724 shutdown_connections 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 --2- 172.21.15.46:0/420953724 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78bc0cefa0 0x7f78bc0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 --2- 172.21.15.46:0/420953724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc074db0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 -- 172.21.15.46:0/420953724 >> 172.21.15.46:0/420953724 conn(0x7f78bc0b2040 msgr2=0x7f78bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 -- 172.21.15.46:0/420953724 shutdown_connections 2024-09-17T03:31:04.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.033+0000 7f78c3a00640 1 -- 172.21.15.46:0/420953724 wait complete. 2024-09-17T03:31:04.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 Processor -- start 2024-09-17T03:31:04.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 -- start start 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc1441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78bc0cefa0 0x7f78bc144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78bc160330 0x7f78bc1414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc16a0b0 con 0x7f78bc0749a0 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc169f30 con 0x7f78bc0cefa0 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f78bc16a230 con 0x7f78bc160330 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc1441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc1441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60264/0 (socket says 172.21.15.46:60264) 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 -- 172.21.15.46:0/486593070 learned_addr learned my addr 172.21.15.46:0/486593070 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 -- 172.21.15.46:0/486593070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78bc160330 msgr2=0x7f78bc1414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78bc160330 0x7f78bc1414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 -- 172.21.15.46:0/486593070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78bc0cefa0 msgr2=0x7f78bc144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78bc0cefa0 0x7f78bc144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 -- 172.21.15.46:0/486593070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc1433f0 con 0x7f78bc0749a0 2024-09-17T03:31:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c1000640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc1441b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009790 tx=0x7f78b00a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00bb070 con 0x7f78bc0749a0 2024-09-17T03:31:04.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b0005030 con 0x7f78bc0749a0 2024-09-17T03:31:04.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00b2040 con 0x7f78bc0749a0 2024-09-17T03:31:04.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc1436d0 con 0x7f78bc0749a0 2024-09-17T03:31:04.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.037+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc143c90 con 0x7f78bc0749a0 2024-09-17T03:31:04.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.041+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc074db0 con 0x7f78bc0749a0 2024-09-17T03:31:04.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.041+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 34) ==== 98968+0+0 (secure 0 0 0) 0x7f78b00be070 con 0x7f78bc0749a0 2024-09-17T03:31:04.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.041+0000 7f78b8c00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f789c074550 0x7f789c076a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:04.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.041+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f78b012c8d0 con 0x7f78bc0749a0 2024-09-17T03:31:04.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.041+0000 7f78bbe00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f789c074550 0x7f789c076a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:04.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.041+0000 7f78bbe00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f789c074550 0x7f789c076a60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78a80058d0 tx=0x7f78a8005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.045+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f78b0016410 con 0x7f78bc0749a0 2024-09-17T03:31:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.249+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone79881", "target": ["mon-mgr", ""]}) -- 0x7f78bc0cdd60 con 0x7f789c074550 2024-09-17T03:31:04.837 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:04.837+0000 7fbe09a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:04.837 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:04.837+0000 7fbe09a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:04.837 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:04.837+0000 7fbe09a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:04.837 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:04.837+0000 7fbe09a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:04.837 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:04.837+0000 7fbe09a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.941+0000 7f78b8c00640 1 -- 172.21.15.46:0/486593070 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78bc0cdd60 con 0x7f789c074550 2024-09-17T03:31:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f789c074550 msgr2=0x7f789c076a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f789c074550 0x7f789c076a60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78a80058d0 tx=0x7f78a8005860 comp rx=0 tx=0).stop 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 msgr2=0x7f78bc1441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc1441b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009790 tx=0x7f78b00a57d0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 shutdown_connections 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f789c074550 0x7f789c076a60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78bc160330 0x7f78bc1414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78bc0cefa0 0x7f78bc144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 --2- 172.21.15.46:0/486593070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78bc0749a0 0x7f78bc1441b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 >> 172.21.15.46:0/486593070 conn(0x7f78bc0b2040 msgr2=0x7f78bc0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 shutdown_connections 2024-09-17T03:31:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:04.945+0000 7f78c3a00640 1 -- 172.21.15.46:0/486593070 wait complete. 2024-09-17T03:31:04.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone50292 2024-09-17T03:31:05.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 -- 172.21.15.46:0/3348758090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f400b1dc0 msgr2=0x7f3f400b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:05.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 --2- 172.21.15.46:0/3348758090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f400b1dc0 0x7f3f400b21d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009870 tx=0x7f3f2801c780 comp rx=0 tx=0).stop 2024-09-17T03:31:05.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 -- 172.21.15.46:0/3348758090 shutdown_connections 2024-09-17T03:31:05.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 --2- 172.21.15.46:0/3348758090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f400b1dc0 0x7f3f400b21d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:05.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 -- 172.21.15.46:0/3348758090 >> 172.21.15.46:0/3348758090 conn(0x7f3f4006cdf0 msgr2=0x7f3f4006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:05.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 -- 172.21.15.46:0/3348758090 shutdown_connections 2024-09-17T03:31:05.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.349+0000 7f3f45a00640 1 -- 172.21.15.46:0/3348758090 wait complete. 2024-09-17T03:31:05.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.353+0000 7f3f45a00640 1 Processor -- start 2024-09-17T03:31:05.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 -- start start 2024-09-17T03:31:05.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 0x7f3f4016a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:05.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f400b3130 0x7f3f40148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:05.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f40155a00 0x7f3f4014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:05.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f400d0280 con 0x7f3f400b1dc0 2024-09-17T03:31:05.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f400d0100 con 0x7f3f40155a00 2024-09-17T03:31:05.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f45a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3f400d0400 con 0x7f3f400b3130 2024-09-17T03:31:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.357+0000 7f3f3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 0x7f3f4016a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:05.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 0x7f3f4016a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60294/0 (socket says 172.21.15.46:60294) 2024-09-17T03:31:05.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 -- 172.21.15.46:0/1079822437 learned_addr learned my addr 172.21.15.46:0/1079822437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f37400640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f400b3130 0x7f3f40148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 -- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f400b3130 msgr2=0x7f3f40148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f400b3130 0x7f3f40148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 -- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f40155a00 msgr2=0x7f3f4014b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f40155a00 0x7f3f4014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 -- 172.21.15.46:0/1079822437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f40200470 con 0x7f3f400b1dc0 2024-09-17T03:31:05.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f37400640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f400b3130 0x7f3f40148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:05.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3fe00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 0x7f3f4016a2e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c410 tx=0x7f3f3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:05.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30011790 con 0x7f3f400b1dc0 2024-09-17T03:31:05.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f40200750 con 0x7f3f400b1dc0 2024-09-17T03:31:05.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f30017070 con 0x7f3f400b1dc0 2024-09-17T03:31:05.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30011d00 con 0x7f3f400b1dc0 2024-09-17T03:31:05.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.361+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f40200d10 con 0x7f3f400b1dc0 2024-09-17T03:31:05.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.365+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f400b21d0 con 0x7f3f400b1dc0 2024-09-17T03:31:05.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.365+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 34) ==== 98968+0+0 (secure 0 0 0) 0x7f3f3001c460 con 0x7f3f400b1dc0 2024-09-17T03:31:05.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.369+0000 7f3f3cc00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f1c074520 0x7f3f1c076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:05.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.369+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f3f30093050 con 0x7f3f400b1dc0 2024-09-17T03:31:05.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.369+0000 7f3f37400640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f1c074520 0x7f3f1c076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:05.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.373+0000 7f3f37400640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f1c074520 0x7f3f1c076a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3f280af000 tx=0x7f3f2801afc0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:05.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.377+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f3005f730 con 0x7f3f400b1dc0 2024-09-17T03:31:05.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.561+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 35) ==== 99121+0+0 (secure 0 0 0) 0x7f3f3005b2a0 con 0x7f3f400b1dc0 2024-09-17T03:31:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:05.857+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone50292", "target": ["mon-mgr", ""]}) -- 0x7f3f4006b540 con 0x7f3f1c074520 2024-09-17T03:31:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.313+0000 7f3f3cc00640 1 -- 172.21.15.46:0/1079822437 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f28004020 con 0x7f3f1c074520 2024-09-17T03:31:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f1c074520 msgr2=0x7f3f1c076a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f1c074520 0x7f3f1c076a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3f280af000 tx=0x7f3f2801afc0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 msgr2=0x7f3f4016a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 0x7f3f4016a2e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c410 tx=0x7f3f3000c8d0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 shutdown_connections 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f1c074520 0x7f3f1c076a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f40155a00 0x7f3f4014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f400b3130 0x7f3f40148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 --2- 172.21.15.46:0/1079822437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f400b1dc0 0x7f3f4016a2e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 >> 172.21.15.46:0/1079822437 conn(0x7f3f4006cdf0 msgr2=0x7f3f400b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 shutdown_connections 2024-09-17T03:31:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.321+0000 7f3f45a00640 1 -- 172.21.15.46:0/1079822437 wait complete. 2024-09-17T03:31:06.336 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:06.337+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:06.336 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:06.337+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:06.336 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:06.337+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:06.336 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:06.337+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:06.336 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:06.337+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:06.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone13923 2024-09-17T03:31:06.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.745+0000 7f5611c00640 1 -- 172.21.15.46:0/308277402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c0cd970 msgr2=0x7f560c0b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:06.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.745+0000 7f5611c00640 1 --2- 172.21.15.46:0/308277402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c0cd970 0x7f560c0b3340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00afd0 tx=0x7f55fc01e5a0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.753+0000 7f5611c00640 1 -- 172.21.15.46:0/308277402 shutdown_connections 2024-09-17T03:31:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.753+0000 7f5611c00640 1 --2- 172.21.15.46:0/308277402 >> v2:172.21.15.46:3300/0 conn(0x7f560c0cdd80 0x7f560c0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.753+0000 7f5611c00640 1 --2- 172.21.15.46:0/308277402 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c0cd970 0x7f560c0b3340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.753+0000 7f5611c00640 1 -- 172.21.15.46:0/308277402 >> 172.21.15.46:0/308277402 conn(0x7f560c0b2040 msgr2=0x7f560c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.757+0000 7f5611c00640 1 -- 172.21.15.46:0/308277402 shutdown_connections 2024-09-17T03:31:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.757+0000 7f5611c00640 1 -- 172.21.15.46:0/308277402 wait complete. 2024-09-17T03:31:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 Processor -- start 2024-09-17T03:31:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 -- start start 2024-09-17T03:31:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f560c0cd970 0x7f560c1f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 0x7f560c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c15aa50 0x7f560c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f560c168140 con 0x7f560c0cd970 2024-09-17T03:31:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f560c167fc0 con 0x7f560c0cdd80 2024-09-17T03:31:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f5611c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f560c1682c0 con 0x7f560c15aa50 2024-09-17T03:31:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f560aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f560c0cd970 0x7f560c1f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.765+0000 7f560a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 0x7f560c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 0x7f560c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:60210/0 (socket says 172.21.15.46:60210) 2024-09-17T03:31:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 -- 172.21.15.46:0/231079036 learned_addr learned my addr 172.21.15.46:0/231079036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560b400640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c15aa50 0x7f560c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 -- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c15aa50 msgr2=0x7f560c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c15aa50 0x7f560c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 -- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f560c0cd970 msgr2=0x7f560c1f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f560c0cd970 0x7f560c1f8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.769+0000 7f560a000640 1 -- 172.21.15.46:0/231079036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c0b1180 con 0x7f560c0cdd80 2024-09-17T03:31:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.773+0000 7f560a000640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 0x7f560c1558c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00c210 tx=0x7f55fc005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.773+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0ae050 con 0x7f560c0cdd80 2024-09-17T03:31:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.773+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55fc004320 con 0x7f560c0cdd80 2024-09-17T03:31:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.773+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0044d0 con 0x7f560c0cdd80 2024-09-17T03:31:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.777+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c0b1460 con 0x7f560c0cdd80 2024-09-17T03:31:06.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.777+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c0c5770 con 0x7f560c0cdd80 2024-09-17T03:31:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.781+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c0c54e0 con 0x7f560c0cdd80 2024-09-17T03:31:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.785+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 35) ==== 99121+0+0 (secure 0 0 0) 0x7f55fc0b1020 con 0x7f560c0cdd80 2024-09-17T03:31:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.785+0000 7f5602a00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f55ec074700 0x7f55ec076c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.785+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f55fc12ba40 con 0x7f560c0cdd80 2024-09-17T03:31:06.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.789+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f55fc018190 con 0x7f560c0cdd80 2024-09-17T03:31:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.813+0000 7f560aa00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f55ec074700 0x7f55ec076c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:06.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:06.817+0000 7f560aa00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f55ec074700 0x7f55ec076c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f55f8001870 tx=0x7f55f8001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.313+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone13923", "target": ["mon-mgr", ""]}) -- 0x7f560c0c52b0 con 0x7f55ec074700 2024-09-17T03:31:07.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.613+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 36) ==== 99172+0+0 (secure 0 0 0) 0x7f55fc0f4160 con 0x7f560c0cdd80 2024-09-17T03:31:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.961+0000 7f5602a00640 1 -- 172.21.15.46:0/231079036 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55f8001510 con 0x7f55ec074700 2024-09-17T03:31:07.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f55ec074700 msgr2=0x7f55ec076c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:07.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f55ec074700 0x7f55ec076c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f55f8001870 tx=0x7f55f8001380 comp rx=0 tx=0).stop 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 msgr2=0x7f560c1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 0x7f560c1558c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00c210 tx=0x7f55fc005780 comp rx=0 tx=0).stop 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 shutdown_connections 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f55ec074700 0x7f55ec076c10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f560c15aa50 0x7f560c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c0cdd80 0x7f560c1558c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 --2- 172.21.15.46:0/231079036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f560c0cd970 0x7f560c1f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 >> 172.21.15.46:0/231079036 conn(0x7f560c0b2040 msgr2=0x7f560c15d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 shutdown_connections 2024-09-17T03:31:07.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:07.969+0000 7f5611c00640 1 -- 172.21.15.46:0/231079036 wait complete. 2024-09-17T03:31:07.980 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:07.981+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:07.980 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:07.981+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:07.980 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:07.981+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:07.980 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:07.981+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:07.980 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:07.981+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:08.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone77144 2024-09-17T03:31:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 -- 172.21.15.46:0/100234080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 msgr2=0x7fbd08074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/100234080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd08074410 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009900 tx=0x7fbcf801c850 comp rx=0 tx=0).stop 2024-09-17T03:31:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 -- 172.21.15.46:0/100234080 shutdown_connections 2024-09-17T03:31:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/100234080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd08074410 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 -- 172.21.15.46:0/100234080 >> 172.21.15.46:0/100234080 conn(0x7fbd080708c0 msgr2=0x7fbd08070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 -- 172.21.15.46:0/100234080 shutdown_connections 2024-09-17T03:31:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.573+0000 7fbd0cc00640 1 -- 172.21.15.46:0/100234080 wait complete. 2024-09-17T03:31:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.581+0000 7fbd0cc00640 1 Processor -- start 2024-09-17T03:31:08.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.585+0000 7fbd0cc00640 1 -- start start 2024-09-17T03:31:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.589+0000 7fbd0cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd08074950 0x7fbd081439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.589+0000 7fbd0cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd081fad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.589+0000 7fbd0cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08157030 0x7fbd0814c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.589+0000 7fbd0cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0816b650 con 0x7fbd08157030 2024-09-17T03:31:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.589+0000 7fbd0cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0816b4d0 con 0x7fbd080b5a80 2024-09-17T03:31:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbd0cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0816b7d0 con 0x7fbd08074950 2024-09-17T03:31:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbd05e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd08074950 0x7fbd081439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd081fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd081fad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:60256/0 (socket says 172.21.15.46:60256) 2024-09-17T03:31:08.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 -- 172.21.15.46:0/1101390338 learned_addr learned my addr 172.21.15.46:0/1101390338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 -- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd08074950 msgr2=0x7fbd081439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd08074950 0x7fbd081439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 -- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08157030 msgr2=0x7fbd0814c7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:31:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08157030 0x7fbd0814c7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.593+0000 7fbcffe00640 1 -- 172.21.15.46:0/1101390338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd08150ae0 con 0x7fbd080b5a80 2024-09-17T03:31:08.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.601+0000 7fbcffe00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd081fad70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8019460 tx=0x7fbcf80a5680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.605+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80b2040 con 0x7fbd080b5a80 2024-09-17T03:31:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.605+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcf8004470 con 0x7fbd080b5a80 2024-09-17T03:31:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.605+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8003820 con 0x7fbd080b5a80 2024-09-17T03:31:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.605+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd08150da0 con 0x7fbd080b5a80 2024-09-17T03:31:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.605+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd08151360 con 0x7fbd080b5a80 2024-09-17T03:31:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.609+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce00052a0 con 0x7fbd080b5a80 2024-09-17T03:31:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.621+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 36) ==== 99172+0+0 (secure 0 0 0) 0x7fbcf80039c0 con 0x7fbd080b5a80 2024-09-17T03:31:08.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.621+0000 7fbcfe000640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbce4074760 0x7fbce4076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.625+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fbcf812cf00 con 0x7fbd080b5a80 2024-09-17T03:31:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.625+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbcf80164d0 con 0x7fbd080b5a80 2024-09-17T03:31:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.653+0000 7fbd05e00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbce4074760 0x7fbce4076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:08.677+0000 7fbd05e00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbce4074760 0x7fbce4076c70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b810 tx=0x7fbcf0008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:09.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:09.229+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone77144", "target": ["mon-mgr", ""]}) -- 0x7fbce0000c90 con 0x7fbce4074760 2024-09-17T03:31:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:09.661+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7fbcf80f5170 con 0x7fbd080b5a80 2024-09-17T03:31:10.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.193+0000 7fbcfe000640 1 -- 172.21.15.46:0/1101390338 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbce0000c90 con 0x7fbce4074760 2024-09-17T03:31:10.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbce4074760 msgr2=0x7fbce4076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:10.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbce4074760 0x7fbce4076c70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b810 tx=0x7fbcf0008040 comp rx=0 tx=0).stop 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 msgr2=0x7fbd081fad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd081fad70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8019460 tx=0x7fbcf80a5680 comp rx=0 tx=0).stop 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 shutdown_connections 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbce4074760 0x7fbce4076c70 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b810 tx=0x7fbcf0008040 comp rx=0 tx=0).stop 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd08157030 0x7fbd0814c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd080b5a80 0x7fbd081fad70 secure :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8019460 tx=0x7fbcf80a5680 comp rx=0 tx=0).stop 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 --2- 172.21.15.46:0/1101390338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd08074950 0x7fbd081439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.197+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 >> 172.21.15.46:0/1101390338 conn(0x7fbd080708c0 msgr2=0x7fbd080d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:10.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.201+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 shutdown_connections 2024-09-17T03:31:10.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.201+0000 7fbd0cc00640 1 -- 172.21.15.46:0/1101390338 wait complete. 2024-09-17T03:31:10.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 -- 172.21.15.46:0/1269293390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f762815b850 msgr2=0x7f76280b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 --2- 172.21.15.46:0/1269293390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f762815b850 0x7f76280b1dc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7614009230 tx=0x7f761401d880 comp rx=0 tx=0).stop 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 -- 172.21.15.46:0/1269293390 shutdown_connections 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 --2- 172.21.15.46:0/1269293390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f762815b850 0x7f76280b1dc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 -- 172.21.15.46:0/1269293390 >> 172.21.15.46:0/1269293390 conn(0x7f762806cdf0 msgr2=0x7f762806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 -- 172.21.15.46:0/1269293390 shutdown_connections 2024-09-17T03:31:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.805+0000 7f7627200640 1 -- 172.21.15.46:0/1269293390 wait complete. 2024-09-17T03:31:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.809+0000 7f7627200640 1 Processor -- start 2024-09-17T03:31:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.809+0000 7f7627200640 1 -- start start 2024-09-17T03:31:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7627200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 0x7f76281553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7627200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280b5230 0x7f76281fc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7627200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f762815b850 0x7f76281fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7627200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f762816b190 con 0x7f76280b5230 2024-09-17T03:31:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7627200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f762816b310 con 0x7f76280b2300 2024-09-17T03:31:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7627200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f762816b490 con 0x7f762815b850 2024-09-17T03:31:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7625e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 0x7f76281553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7625e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 0x7f76281553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:60276/0 (socket says 172.21.15.46:60276) 2024-09-17T03:31:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7625e00640 1 -- 172.21.15.46:0/2957823798 learned_addr learned my addr 172.21.15.46:0/2957823798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.813+0000 7f7625400640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280b5230 0x7f76281fc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:10.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.817+0000 7f7626800640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f762815b850 0x7f76281fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:10.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.817+0000 7f7625e00640 1 -- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f762815b850 msgr2=0x7f76281fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:10.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.817+0000 7f7625e00640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f762815b850 0x7f76281fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:10.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.817+0000 7f7625e00640 1 -- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280b5230 msgr2=0x7f76281fc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:10.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.817+0000 7f7625e00640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280b5230 0x7f76281fc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:10.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.817+0000 7f7625e00640 1 -- 172.21.15.46:0/2957823798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76281514e0 con 0x7f76280b2300 2024-09-17T03:31:10.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.821+0000 7f7625e00640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 0x7f76281553c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f761401a490 tx=0x7f76140056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.821+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140b3070 con 0x7f76280b2300 2024-09-17T03:31:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.821+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76140a6c70 con 0x7f76280b2300 2024-09-17T03:31:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.821+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7628157040 con 0x7f76280b2300 2024-09-17T03:31:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.821+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140a6e60 con 0x7f76280b2300 2024-09-17T03:31:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.821+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7628157600 con 0x7f76280b2300 2024-09-17T03:31:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.825+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76280b1dc0 con 0x7f76280b2300 2024-09-17T03:31:10.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.829+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f76140c0060 con 0x7f76280b2300 2024-09-17T03:31:10.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.833+0000 7f761e000640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76080747f0 0x7f7608076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:10.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.833+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f761412d500 con 0x7f76280b2300 2024-09-17T03:31:10.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.837+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76140c8020 con 0x7f76280b2300 2024-09-17T03:31:10.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.837+0000 7f7625400640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76080747f0 0x7f7608076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:10.841+0000 7f7625400640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76080747f0 0x7f7608076d00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f76200012f0 tx=0x7f7620009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.721+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f762815a450 con 0x7f76280b2300 2024-09-17T03:31:11.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.725+0000 7f761e000640 1 -- 172.21.15.46:0/2957823798 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2173 (secure 0 0 0) 0x7f7614017500 con 0x7f76280b2300 2024-09-17T03:31:11.730 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:31:11.731 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 629, 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.732 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 89, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:31:11.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 1255, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 5122467428, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 10118983680, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 762975129600, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:31:11.734 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 159603858, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 209937211, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 38, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 61 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 56, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:30:30:404049+0000", 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:11.735 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 6, 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246888 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:31:11.736 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:31:11.737 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:31:11.737 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:31:11.737 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:11.737 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:11.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.737 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 13, 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:29:51.384262+0000", 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 28.125% (5s)\n [=======.....................] (remaining: 14s)", 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.28125, 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:11.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:11.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:11.739 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:31:11.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76080747f0 msgr2=0x7f7608076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:11.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76080747f0 0x7f7608076d00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f76200012f0 tx=0x7f7620009040 comp rx=0 tx=0).stop 2024-09-17T03:31:11.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 msgr2=0x7f76281553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:11.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 0x7f76281553c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f761401a490 tx=0x7f76140056e0 comp rx=0 tx=0).stop 2024-09-17T03:31:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 shutdown_connections 2024-09-17T03:31:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f76080747f0 0x7f7608076d00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f762815b850 0x7f76281fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280b5230 0x7f76281fc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 --2- 172.21.15.46:0/2957823798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76280b2300 0x7f76281553c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.741+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 >> 172.21.15.46:0/2957823798 conn(0x7f762806cdf0 msgr2=0x7f762814f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.745+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 shutdown_connections 2024-09-17T03:31:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:11.749+0000 7f7627200640 1 -- 172.21.15.46:0/2957823798 wait complete. 2024-09-17T03:31:12.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:31:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 -- 172.21.15.46:0/3332649802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180d29e0 msgr2=0x7f73180d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 --2- 172.21.15.46:0/3332649802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180d29e0 0x7f73180d14e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73180cf390 tx=0x7f730401c8c0 comp rx=0 tx=0).stop 2024-09-17T03:31:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 -- 172.21.15.46:0/3332649802 shutdown_connections 2024-09-17T03:31:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 --2- 172.21.15.46:0/3332649802 >> v2:172.21.15.46:3300/0 conn(0x7f73180b60d0 0x7f73180b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 --2- 172.21.15.46:0/3332649802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180d29e0 0x7f73180d14e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 --2- 172.21.15.46:0/3332649802 >> v2:172.21.15.99:3300/0 conn(0x7f73180d25d0 0x7f73180d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 -- 172.21.15.46:0/3332649802 >> 172.21.15.46:0/3332649802 conn(0x7f7318073e00 msgr2=0x7f731806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 -- 172.21.15.46:0/3332649802 shutdown_connections 2024-09-17T03:31:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.333+0000 7f7320000640 1 -- 172.21.15.46:0/3332649802 wait complete. 2024-09-17T03:31:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.341+0000 7f7320000640 1 Processor -- start 2024-09-17T03:31:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.341+0000 7f7320000640 1 -- start start 2024-09-17T03:31:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.345+0000 7f7320000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180b60d0 0x7f7318160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.345+0000 7f7320000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73180d25d0 0x7f731815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.345+0000 7f7320000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73180d29e0 0x7f73180cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.345+0000 7f7320000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f731816b650 con 0x7f73180d25d0 2024-09-17T03:31:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.345+0000 7f7320000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73180d40f0 con 0x7f73180d29e0 2024-09-17T03:31:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.345+0000 7f7320000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f731816b7d0 con 0x7f73180b60d0 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73180d29e0 0x7f73180cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180b60d0 0x7f7318160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73180d29e0 0x7f73180cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:48206/0 (socket says 172.21.15.46:48206) 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731e000640 1 -- 172.21.15.46:0/1890957919 learned_addr learned my addr 172.21.15.46:0/1890957919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731cc00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73180d25d0 0x7f731815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 -- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73180d29e0 msgr2=0x7f73180cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73180d29e0 0x7f73180cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 -- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73180d25d0 msgr2=0x7f731815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73180d25d0 0x7f731815bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 -- 172.21.15.46:0/1890957919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7318161970 con 0x7f73180b60d0 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731d600640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180b60d0 0x7f7318160080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f731000be20 tx=0x7f7310009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310019070 con 0x7f73180b60d0 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7310014440 con 0x7f73180b60d0 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310014870 con 0x7f73180b60d0 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f7320000640 1 -- 172.21.15.46:0/1890957919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73181608d0 con 0x7f73180b60d0 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f731cc00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73180d25d0 0x7f731815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.349+0000 7f7320000640 1 -- 172.21.15.46:0/1890957919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7318160e90 con 0x7f73180b60d0 2024-09-17T03:31:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.353+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f731001c070 con 0x7f73180b60d0 2024-09-17T03:31:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.357+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73180b64e0 con 0x7f73180b60d0 2024-09-17T03:31:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.357+0000 7f7315600640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72f8074820 0x7f72f8076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.357+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f7310093340 con 0x7f73180b60d0 2024-09-17T03:31:13.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.369+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f731005fac0 con 0x7f73180b60d0 2024-09-17T03:31:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.373+0000 7f731cc00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72f8074820 0x7f72f8076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:13.373+0000 7f731cc00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72f8074820 0x7f72f8076d30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73040af000 tx=0x7f730401b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:14.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.009+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f73180cf490 con 0x7f73180b60d0 2024-09-17T03:31:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.013+0000 7f7315600640 1 -- 172.21.15.46:0/1890957919 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+3283 (secure 0 0 0) 0x7f731005f3c0 con 0x7f73180b60d0 2024-09-17T03:31:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:31:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:31:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:31:14.019 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:31:14.020 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 631, 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 89, 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:31:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:14.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 1823, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 7504832968, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 15097851904, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 757996261376, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 371141848, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 390387388, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 90, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 106 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:31:14.023 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 56, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:30:30:404049+0000", 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 6, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246888 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.024 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:31:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 14, 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:31:13.413788+0000", 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "daemons": { 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "summary": "", 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "a": { 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.026 INFO:teuthology.orchestra.run.smithi046.stdout: "b": { 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "c": { 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:31:14.027 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 28.125% (5s)\n [=======.....................] (remaining: 14s)", 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.28125, 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:14.028 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:31:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.025+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72f8074820 msgr2=0x7f72f8076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.025+0000 7f730aa00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72f8074820 0x7f72f8076d30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73040af000 tx=0x7f730401b100 comp rx=0 tx=0).stop 2024-09-17T03:31:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180b60d0 msgr2=0x7f7318160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180b60d0 0x7f7318160080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f731000be20 tx=0x7f7310009f70 comp rx=0 tx=0).stop 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 shutdown_connections 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72f8074820 0x7f72f8076d30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73180d29e0 0x7f73180cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73180d25d0 0x7f731815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 --2- 172.21.15.46:0/1890957919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f73180b60d0 0x7f7318160080 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 >> 172.21.15.46:0/1890957919 conn(0x7f7318073e00 msgr2=0x7f731815ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 shutdown_connections 2024-09-17T03:31:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:14.033+0000 7f730aa00640 1 -- 172.21.15.46:0/1890957919 wait complete. 2024-09-17T03:31:15.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:31:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.721+0000 7fa013c00640 1 -- 172.21.15.46:0/221131108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c0738b0 msgr2=0x7fa00c152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.721+0000 7fa013c00640 1 --2- 172.21.15.46:0/221131108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c0738b0 0x7fa00c152580 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80097f0 tx=0x7f9ff801c5c0 comp rx=0 tx=0).stop 2024-09-17T03:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.725+0000 7fa013c00640 1 -- 172.21.15.46:0/221131108 shutdown_connections 2024-09-17T03:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.725+0000 7fa013c00640 1 --2- 172.21.15.46:0/221131108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c0738b0 0x7fa00c152580 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.725+0000 7fa013c00640 1 -- 172.21.15.46:0/221131108 >> 172.21.15.46:0/221131108 conn(0x7fa00c06e1d0 msgr2=0x7fa00c06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.725+0000 7fa013c00640 1 -- 172.21.15.46:0/221131108 shutdown_connections 2024-09-17T03:31:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.725+0000 7fa013c00640 1 -- 172.21.15.46:0/221131108 wait complete. 2024-09-17T03:31:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.733+0000 7fa013c00640 1 Processor -- start 2024-09-17T03:31:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.733+0000 7fa013c00640 1 -- start start 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa013c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 0x7fa00c14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa013c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa00c075010 0x7fa00c169b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa013c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c14ef30 0x7fa00c16a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa013c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c0b4250 con 0x7fa00c0738b0 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa013c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c0b40d0 con 0x7fa00c14ef30 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa013c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c0b43d0 con 0x7fa00c075010 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa011200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 0x7fa00c14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa011200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 0x7fa00c14dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39610/0 (socket says 172.21.15.46:39610) 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa011200640 1 -- 172.21.15.46:0/2948319046 learned_addr learned my addr 172.21.15.46:0/2948319046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa011c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c14ef30 0x7fa00c16a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.737+0000 7fa00be00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa00c075010 0x7fa00c169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa011200640 1 -- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa00c075010 msgr2=0x7fa00c169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa011200640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa00c075010 0x7fa00c169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa011200640 1 -- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c14ef30 msgr2=0x7fa00c16a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa011200640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c14ef30 0x7fa00c16a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa011200640 1 -- 172.21.15.46:0/2948319046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c14e530 con 0x7fa00c0738b0 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa011200640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 0x7fa00c14dcc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80191d0 tx=0x7f9ff80aa5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80aabc0 con 0x7fa00c0738b0 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ff80aad60 con 0x7fa00c0738b0 2024-09-17T03:31:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8003860 con 0x7fa00c0738b0 2024-09-17T03:31:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00c15b1a0 con 0x7fa00c0738b0 2024-09-17T03:31:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.741+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00c0cdcd0 con 0x7fa00c0738b0 2024-09-17T03:31:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.745+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00c0cdf30 con 0x7fa00c0738b0 2024-09-17T03:31:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.745+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f9ff8003a00 con 0x7fa00c0738b0 2024-09-17T03:31:15.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.745+0000 7fa008c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9fec074840 0x7f9fec076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:15.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.745+0000 7fa00be00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9fec074840 0x7f9fec076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:15.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.749+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f9ff80f60a0 con 0x7fa00c0738b0 2024-09-17T03:31:15.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.749+0000 7fa00be00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9fec074840 0x7f9fec076d50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0afc90 tx=0x7f9ffc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:15.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:15.757+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ff8016310 con 0x7fa00c0738b0 2024-09-17T03:31:16.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.621+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa00c152580 con 0x7fa00c0738b0 2024-09-17T03:31:16.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.625+0000 7fa008c00640 1 -- 172.21.15.46:0/2948319046 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2876 (secure 0 0 0) 0x7f9ff80fa530 con 0x7fa00c0738b0 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:31:16.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 634, 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:31:16.626 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 89, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:31:16.627 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 1823, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 7504832968, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 15097851904, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 757996261376, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 371133044, 2024-09-17T03:31:16.628 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 356511963, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 90, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 98 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 56, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:30:30:404049+0000", 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 6, 2024-09-17T03:31:16.629 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246888 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:31:16.630 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 15, 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:31:15.414311+0000", 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "daemons": { 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "summary": "", 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "c": { 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "start_epoch": 0, 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "start_stamp": "0.000000", 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 0, 2024-09-17T03:31:16.631 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": {}, 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "task_status": {} 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 31.185% (8s)\n [========....................] (remaining: 18s)", 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.31185001134872437, 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:16.632 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-total-clones": { 2024-09-17T03:31:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Total 5 clones - average progress is 31.146% (0s)\n [========....................] ", 2024-09-17T03:31:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.3114599883556366, 2024-09-17T03:31:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:16.633 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:31:16.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.633+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9fec074840 msgr2=0x7f9fec076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:16.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.633+0000 7fa013c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9fec074840 0x7f9fec076d50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0afc90 tx=0x7f9ffc009040 comp rx=0 tx=0).stop 2024-09-17T03:31:16.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.633+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 msgr2=0x7fa00c14dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:16.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.633+0000 7fa013c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 0x7fa00c14dcc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80191d0 tx=0x7f9ff80aa5e0 comp rx=0 tx=0).stop 2024-09-17T03:31:16.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.637+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 shutdown_connections 2024-09-17T03:31:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.637+0000 7fa013c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9fec074840 0x7f9fec076d50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.637+0000 7fa013c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa00c14ef30 0x7fa00c16a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.637+0000 7fa013c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa00c075010 0x7fa00c169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.637+0000 7fa013c00640 1 --2- 172.21.15.46:0/2948319046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa00c0738b0 0x7fa00c14dcc0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:16.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.637+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 >> 172.21.15.46:0/2948319046 conn(0x7fa00c06e1d0 msgr2=0x7fa00c151bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.649+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 shutdown_connections 2024-09-17T03:31:16.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:16.649+0000 7fa013c00640 1 -- 172.21.15.46:0/2948319046 wait complete. 2024-09-17T03:31:16.769 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 31.185% (8s)\n [========....................] (remaining: 18s)', 'progress': 0.31185001134872437, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 31.146% (0s)\n [========....................] ', 'progress': 0.3114599883556366, 'add_to_ceph_s': True}} 2024-09-17T03:31:16.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone59755 2024-09-17T03:31:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.289+0000 7f56d1c00640 1 -- 172.21.15.46:0/728519364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c4006240 msgr2=0x7f56c4054e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.289+0000 7f56d1c00640 1 --2- 172.21.15.46:0/728519364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c4006240 0x7f56c4054e40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f56cc07b050 tx=0x7f56cc098a20 comp rx=0 tx=0).stop 2024-09-17T03:31:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.289+0000 7f56d1c00640 1 -- 172.21.15.46:0/728519364 shutdown_connections 2024-09-17T03:31:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.289+0000 7f56d1c00640 1 --2- 172.21.15.46:0/728519364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c4006240 0x7f56c4054e40 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.289+0000 7f56d1c00640 1 --2- 172.21.15.46:0/728519364 >> v2:172.21.15.99:3300/0 conn(0x7f56c40337a0 0x7f56c4051a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.289+0000 7f56d1c00640 1 -- 172.21.15.46:0/728519364 >> 172.21.15.46:0/728519364 conn(0x7f56c401d9b0 msgr2=0x7f56c4039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.293+0000 7f56d1c00640 1 -- 172.21.15.46:0/728519364 shutdown_connections 2024-09-17T03:31:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.293+0000 7f56d1c00640 1 -- 172.21.15.46:0/728519364 wait complete. 2024-09-17T03:31:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.297+0000 7f56d1c00640 1 Processor -- start 2024-09-17T03:31:17.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.301+0000 7f56d1c00640 1 -- start start 2024-09-17T03:31:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56c4005e30 0x7f56c417b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:17.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56c4006240 0x7f56c40d73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c40337a0 0x7f56c40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40ea550 con 0x7f56c40337a0 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40ea3d0 con 0x7f56c4005e30 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f56c40ea6d0 con 0x7f56c4006240 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c40337a0 0x7f56c40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c40337a0 0x7f56c40c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39632/0 (socket says 172.21.15.46:39632) 2024-09-17T03:31:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.309+0000 7f56d1200640 1 -- 172.21.15.46:0/2158082285 learned_addr learned my addr 172.21.15.46:0/2158082285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cbe00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56c4005e30 0x7f56c417b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:17.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cb400640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56c4006240 0x7f56c40d73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cbe00640 1 -- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56c4006240 msgr2=0x7f56c40d73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cbe00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56c4006240 0x7f56c40d73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cbe00640 1 -- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c40337a0 msgr2=0x7f56c40c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cbe00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c40337a0 0x7f56c40c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.313+0000 7f56cbe00640 1 -- 172.21.15.46:0/2158082285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c417edb0 con 0x7f56c4005e30 2024-09-17T03:31:17.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.321+0000 7f56cbe00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56c4005e30 0x7f56c417b200 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d630 tx=0x7f56b800daf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.321+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8017070 con 0x7f56c4005e30 2024-09-17T03:31:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.325+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b8009a70 con 0x7f56c4005e30 2024-09-17T03:31:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.325+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8010460 con 0x7f56c4005e30 2024-09-17T03:31:17.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.329+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c417f090 con 0x7f56c4005e30 2024-09-17T03:31:17.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.329+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c40dcda0 con 0x7f56c4005e30 2024-09-17T03:31:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.333+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c4051a30 con 0x7f56c4005e30 2024-09-17T03:31:17.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.333+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f56b801a070 con 0x7f56c4005e30 2024-09-17T03:31:17.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.337+0000 7f56bfe00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56ac0748c0 0x7f56ac076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.337+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f56b8093380 con 0x7f56c4005e30 2024-09-17T03:31:17.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.337+0000 7f56cb400640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56ac0748c0 0x7f56ac076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.345+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f56b805f420 con 0x7f56c4005e30 2024-09-17T03:31:17.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:17.349+0000 7f56cb400640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56ac0748c0 0x7f56ac076dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f56c417e750 tx=0x7f56c0009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.021+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone59755", "target": ["mon-mgr", ""]}) -- 0x7f56c4007360 con 0x7f56ac0748c0 2024-09-17T03:31:18.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.937+0000 7f56bfe00640 1 -- 172.21.15.46:0/2158082285 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f56c4007360 con 0x7f56ac0748c0 2024-09-17T03:31:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.941+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56ac0748c0 msgr2=0x7f56ac076dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.941+0000 7f56d1c00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56ac0748c0 0x7f56ac076dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f56c417e750 tx=0x7f56c0009060 comp rx=0 tx=0).stop 2024-09-17T03:31:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.941+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56c4005e30 msgr2=0x7f56c417b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.941+0000 7f56d1c00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56c4005e30 0x7f56c417b200 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d630 tx=0x7f56b800daf0 comp rx=0 tx=0).stop 2024-09-17T03:31:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 shutdown_connections 2024-09-17T03:31:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56ac0748c0 0x7f56ac076dd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56c40337a0 0x7f56c40c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f56c4006240 0x7f56c40d73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 --2- 172.21.15.46:0/2158082285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56c4005e30 0x7f56c417b200 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 >> 172.21.15.46:0/2158082285 conn(0x7f56c401d9b0 msgr2=0x7f56c40dbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 shutdown_connections 2024-09-17T03:31:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:18.949+0000 7f56d1c00640 1 -- 172.21.15.46:0/2158082285 wait complete. 2024-09-17T03:31:19.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone18656 2024-09-17T03:31:19.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.645+0000 7f82bd000640 1 -- 172.21.15.46:0/3554914101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b81625f0 msgr2=0x7f82b8162a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:19.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.645+0000 7f82bd000640 1 --2- 172.21.15.46:0/3554914101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b81625f0 0x7f82b8162a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82b815fa90 tx=0x7f82a401c770 comp rx=0 tx=0).stop 2024-09-17T03:31:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.649+0000 7f82bd000640 1 -- 172.21.15.46:0/3554914101 shutdown_connections 2024-09-17T03:31:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.649+0000 7f82bd000640 1 --2- 172.21.15.46:0/3554914101 >> v2:172.21.15.99:3300/0 conn(0x7f82b80cf1c0 0x7f82b80cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.649+0000 7f82bd000640 1 --2- 172.21.15.46:0/3554914101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b81625f0 0x7f82b8162a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:19.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.649+0000 7f82bd000640 1 -- 172.21.15.46:0/3554914101 >> 172.21.15.46:0/3554914101 conn(0x7f82b8073a10 msgr2=0x7f82b806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:19.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.649+0000 7f82bd000640 1 -- 172.21.15.46:0/3554914101 shutdown_connections 2024-09-17T03:31:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.653+0000 7f82bd000640 1 -- 172.21.15.46:0/3554914101 wait complete. 2024-09-17T03:31:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.661+0000 7f82bd000640 1 Processor -- start 2024-09-17T03:31:19.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.665+0000 7f82bd000640 1 -- start start 2024-09-17T03:31:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.669+0000 7f82bd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b80cf1c0 0x7f82b8150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.669+0000 7f82b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b80cf1c0 0x7f82b8150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.669+0000 7f82b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b80cf1c0 0x7f82b8150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39634/0 (socket says 172.21.15.46:39634) 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.669+0000 7f82b7400640 1 -- 172.21.15.46:0/800012053 learned_addr learned my addr 172.21.15.46:0/800012053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.669+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b80d0ce0 0x7f82b8154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.669+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b81625f0 0x7f82b8155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.673+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80d3330 con 0x7f82b80cf1c0 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.673+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80d31b0 con 0x7f82b81625f0 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.673+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f82b816c790 con 0x7f82b80d0ce0 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.673+0000 7f82b6a00640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b80d0ce0 0x7f82b8154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.677+0000 7f82b7e00640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b81625f0 0x7f82b8155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.677+0000 7f82b6a00640 1 -- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b81625f0 msgr2=0x7f82b8155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.677+0000 7f82b6a00640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b81625f0 0x7f82b8155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:19.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.677+0000 7f82b6a00640 1 -- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b80cf1c0 msgr2=0x7f82b8150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.677+0000 7f82b6a00640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b80cf1c0 0x7f82b8150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.677+0000 7f82b6a00640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b80b9d60 con 0x7f82b80d0ce0 2024-09-17T03:31:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.681+0000 7f82b6a00640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b80d0ce0 0x7f82b8154bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82b80d1f90 tx=0x7f82a40a86e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.685+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a40be070 con 0x7f82b80d0ce0 2024-09-17T03:31:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.685+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a40053c0 con 0x7f82b80d0ce0 2024-09-17T03:31:19.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.685+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a40046d0 con 0x7f82b80d0ce0 2024-09-17T03:31:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.689+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b80caba0 con 0x7f82b80d0ce0 2024-09-17T03:31:19.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.689+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b80ca0c0 con 0x7f82b80d0ce0 2024-09-17T03:31:19.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.693+0000 7f82acc00640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b80cf5d0 con 0x7f82b80d0ce0 2024-09-17T03:31:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.697+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f82a4004890 con 0x7f82b80d0ce0 2024-09-17T03:31:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.701+0000 7f82af400640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f82980748c0 0x7f8298076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:19.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.701+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f82a412d7a0 con 0x7f82b80d0ce0 2024-09-17T03:31:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.713+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82a40ac030 con 0x7f82b80d0ce0 2024-09-17T03:31:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.725+0000 7f82b7400640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f82980748c0 0x7f8298076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:19.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:19.793+0000 7f82b7400640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f82980748c0 0x7f8298076dd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f82a80099b0 tx=0x7f82a8009110 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:20.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.357+0000 7f82acc00640 1 -- 172.21.15.46:0/800012053 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone18656", "target": ["mon-mgr", ""]}) -- 0x7f82b806b540 con 0x7f82980748c0 2024-09-17T03:31:20.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.968+0000 7f82af400640 1 -- 172.21.15.46:0/800012053 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82a8009600 con 0x7f82980748c0 2024-09-17T03:31:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f82980748c0 msgr2=0x7f8298076dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f82980748c0 0x7f8298076dd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f82a80099b0 tx=0x7f82a8009110 comp rx=0 tx=0).stop 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b80d0ce0 msgr2=0x7f82b8154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b80d0ce0 0x7f82b8154bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82b80d1f90 tx=0x7f82a40a86e0 comp rx=0 tx=0).stop 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 shutdown_connections 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f82980748c0 0x7f8298076dd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82b81625f0 0x7f82b8155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f82b80d0ce0 0x7f82b8154bd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 --2- 172.21.15.46:0/800012053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b80cf1c0 0x7f82b8150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:20.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 >> 172.21.15.46:0/800012053 conn(0x7f82b8073a10 msgr2=0x7f82b8161600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 shutdown_connections 2024-09-17T03:31:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:20.976+0000 7f82bd000640 1 -- 172.21.15.46:0/800012053 wait complete. 2024-09-17T03:31:21.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone73317 2024-09-17T03:31:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.656+0000 7f81c9800640 1 -- 172.21.15.46:0/2347106406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b5520 msgr2=0x7f81c415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.656+0000 7f81c9800640 1 --2- 172.21.15.46:0/2347106406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b5520 0x7f81c415b730 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81c40afe20 tx=0x7f81bc01caa0 comp rx=0 tx=0).stop 2024-09-17T03:31:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.656+0000 7f81c9800640 1 -- 172.21.15.46:0/2347106406 shutdown_connections 2024-09-17T03:31:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.656+0000 7f81c9800640 1 --2- 172.21.15.46:0/2347106406 >> v2:172.21.15.46:3300/0 conn(0x7f81c40b2070 0x7f81c40b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.656+0000 7f81c9800640 1 --2- 172.21.15.46:0/2347106406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b5520 0x7f81c415b730 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.656+0000 7f81c9800640 1 -- 172.21.15.46:0/2347106406 >> 172.21.15.46:0/2347106406 conn(0x7f81c406e2d0 msgr2=0x7f81c406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.664+0000 7f81c9800640 1 -- 172.21.15.46:0/2347106406 shutdown_connections 2024-09-17T03:31:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.664+0000 7f81c9800640 1 -- 172.21.15.46:0/2347106406 wait complete. 2024-09-17T03:31:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.672+0000 7f81c9800640 1 Processor -- start 2024-09-17T03:31:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.672+0000 7f81c9800640 1 -- start start 2024-09-17T03:31:21.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.676+0000 7f81c9800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 0x7f81c4137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:21.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.676+0000 7f81c9800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b2070 0x7f81c4140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:21.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.676+0000 7f81c9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c415a550 0x7f81c4072c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:21.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.676+0000 7f81c2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 0x7f81c4137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:21.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 0x7f81c4137b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:53836/0 (socket says 172.21.15.46:53836) 2024-09-17T03:31:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b2070 0x7f81c4140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:21.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b2070 0x7f81c4140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:48272/0 (socket says 172.21.15.46:48272) 2024-09-17T03:31:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c415a550 0x7f81c4072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c415a550 0x7f81c4072c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39658/0 (socket says 172.21.15.46:39658) 2024-09-17T03:31:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c2a00640 1 -- 172.21.15.46:0/3479880919 learned_addr learned my addr 172.21.15.46:0/3479880919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415fa30 con 0x7f81c415a550 2024-09-17T03:31:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81c415f8b0 con 0x7f81c40b2070 2024-09-17T03:31:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.680+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f81c415fbb0 con 0x7f81c4070290 2024-09-17T03:31:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.684+0000 7f81c2a00640 1 -- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b2070 msgr2=0x7f81c4140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.684+0000 7f81c2a00640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b2070 0x7f81c4140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.684+0000 7f81c2a00640 1 -- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c415a550 msgr2=0x7f81c4072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.684+0000 7f81c2a00640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c415a550 0x7f81c4072c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.688+0000 7f81c2a00640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c4138080 con 0x7f81c4070290 2024-09-17T03:31:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.692+0000 7f81c2a00640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 0x7f81c4137b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b410 tx=0x7f81b400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.696+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4015030 con 0x7f81c4070290 2024-09-17T03:31:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.696+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b4004020 con 0x7f81c4070290 2024-09-17T03:31:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.696+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4004470 con 0x7f81c4070290 2024-09-17T03:31:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.696+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c4140d60 con 0x7f81c4070290 2024-09-17T03:31:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.696+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41438f0 con 0x7f81c4070290 2024-09-17T03:31:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.700+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f81b4018070 con 0x7f81c4070290 2024-09-17T03:31:21.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.700+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c0052a0 con 0x7f81c4070290 2024-09-17T03:31:21.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.704+0000 7f81baa00640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81a40747f0 0x7f81a4076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:21.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.708+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f81b4092d70 con 0x7f81c4070290 2024-09-17T03:31:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.708+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f81b405f600 con 0x7f81c4070290 2024-09-17T03:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.712+0000 7f81c2000640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81a40747f0 0x7f81a4076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:21.744+0000 7f81c2000640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81a40747f0 0x7f81a4076d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0af000 tx=0x7f81bc01b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:22.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.252+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone73317", "target": ["mon-mgr", ""]}) -- 0x7f819c000c90 con 0x7f81a40747f0 2024-09-17T03:31:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.948+0000 7f81baa00640 1 -- 172.21.15.46:0/3479880919 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f819c000c90 con 0x7f81a40747f0 2024-09-17T03:31:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.956+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81a40747f0 msgr2=0x7f81a4076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.956+0000 7f81c9800640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81a40747f0 0x7f81a4076d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0af000 tx=0x7f81bc01b2e0 comp rx=0 tx=0).stop 2024-09-17T03:31:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 msgr2=0x7f81c4137b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 0x7f81c4137b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b410 tx=0x7f81b400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:31:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 shutdown_connections 2024-09-17T03:31:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81a40747f0 0x7f81a4076d00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c415a550 0x7f81c4072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:22.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c40b2070 0x7f81c4140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 --2- 172.21.15.46:0/3479880919 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81c4070290 0x7f81c4137b40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 >> 172.21.15.46:0/3479880919 conn(0x7f81c406e2d0 msgr2=0x7f81c40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 shutdown_connections 2024-09-17T03:31:22.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:22.960+0000 7f81c9800640 1 -- 172.21.15.46:0/3479880919 wait complete. 2024-09-17T03:31:23.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone60228 2024-09-17T03:31:23.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.488+0000 7f0db2e00640 1 -- 172.21.15.46:0/3498240739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 msgr2=0x7f0d90003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:23.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.488+0000 7f0db2e00640 1 --2- 172.21.15.46:0/3498240739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 0x7f0d90003eb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c710 tx=0x7f0da401c6e0 comp rx=0 tx=0).stop 2024-09-17T03:31:23.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.496+0000 7f0db2e00640 1 -- 172.21.15.46:0/3498240739 shutdown_connections 2024-09-17T03:31:23.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.496+0000 7f0db2e00640 1 --2- 172.21.15.46:0/3498240739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 0x7f0d90003eb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:23.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.496+0000 7f0db2e00640 1 -- 172.21.15.46:0/3498240739 >> 172.21.15.46:0/3498240739 conn(0x7f0dac0708c0 msgr2=0x7f0dac070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.500+0000 7f0db2e00640 1 -- 172.21.15.46:0/3498240739 shutdown_connections 2024-09-17T03:31:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.500+0000 7f0db2e00640 1 -- 172.21.15.46:0/3498240739 wait complete. 2024-09-17T03:31:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.504+0000 7f0db2e00640 1 Processor -- start 2024-09-17T03:31:23.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.504+0000 7f0db2e00640 1 -- start start 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.508+0000 7f0db2e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 0x7f0dac152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.508+0000 7f0db2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac153480 0x7f0dac155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.508+0000 7f0db2e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 0x7f0dac1f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.508+0000 7f0db2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1610e0 con 0x7f0dac153480 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.508+0000 7f0db2e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac160f60 con 0x7f0d90003a60 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.508+0000 7f0db2e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac161260 con 0x7f0dac1f14e0 2024-09-17T03:31:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 0x7f0dac1f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 0x7f0dac1f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:50828/0 (socket says 172.21.15.46:50828) 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 -- 172.21.15.46:0/1708236282 learned_addr learned my addr 172.21.15.46:0/1708236282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 -- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 msgr2=0x7f0dac152ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 0x7f0dac152ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 -- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac153480 msgr2=0x7f0dac155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac153480 0x7f0dac155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 -- 172.21.15.46:0/1708236282 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac155350 con 0x7f0dac1f14e0 2024-09-17T03:31:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db0e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 0x7f0dac1f7620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0cfb10 tx=0x7f0d9c00bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:23.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c019070 con 0x7f0dac1f14e0 2024-09-17T03:31:23.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac154d90 con 0x7f0dac1f14e0 2024-09-17T03:31:23.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.512+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1f0ce0 con 0x7f0dac1f14e0 2024-09-17T03:31:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.516+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d9c014440 con 0x7f0dac1f14e0 2024-09-17T03:31:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.516+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c014890 con 0x7f0dac1f14e0 2024-09-17T03:31:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.516+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac074700 con 0x7f0dac1f14e0 2024-09-17T03:31:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.520+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f0d9c01c070 con 0x7f0dac1f14e0 2024-09-17T03:31:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.524+0000 7f0da3e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c0747f0 0x7f0d8c076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.524+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f0d9c092f90 con 0x7f0dac1f14e0 2024-09-17T03:31:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.524+0000 7f0dabe00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c0747f0 0x7f0d8c076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:23.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.532+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0d9c05fa90 con 0x7f0dac1f14e0 2024-09-17T03:31:23.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:23.532+0000 7f0dabe00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c0747f0 0x7f0d8c076d00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c6e0 tx=0x7f0da4019480 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:24.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:24.096+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone60228", "target": ["mon-mgr", ""]}) -- 0x7f0dac06b540 con 0x7f0d8c0747f0 2024-09-17T03:31:25.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.176+0000 7f0da3e00640 1 -- 172.21.15.46:0/1708236282 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0da40ac040 con 0x7f0d8c0747f0 2024-09-17T03:31:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c0747f0 msgr2=0x7f0d8c076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c0747f0 0x7f0d8c076d00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c6e0 tx=0x7f0da4019480 comp rx=0 tx=0).stop 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 msgr2=0x7f0dac1f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 0x7f0dac1f7620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0cfb10 tx=0x7f0d9c00bf00 comp rx=0 tx=0).stop 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 shutdown_connections 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0d8c0747f0 0x7f0d8c076d00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0dac1f14e0 0x7f0dac1f7620 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dac153480 0x7f0dac155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 --2- 172.21.15.46:0/1708236282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d90003a60 0x7f0dac152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 >> 172.21.15.46:0/1708236282 conn(0x7f0dac0708c0 msgr2=0x7f0dac06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 shutdown_connections 2024-09-17T03:31:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.188+0000 7f0db2e00640 1 -- 172.21.15.46:0/1708236282 wait complete. 2024-09-17T03:31:25.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28249 2024-09-17T03:31:25.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.740+0000 7fa8a3e00640 1 -- 172.21.15.46:0/2267569535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 msgr2=0x7fa89c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:25.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/2267569535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0cdbb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0cc4f0 tx=0x7fa88801e8f0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 -- 172.21.15.46:0/2267569535 shutdown_connections 2024-09-17T03:31:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/2267569535 >> v2:172.21.15.99:3300/0 conn(0x7fa89c15e4e0 0x7fa89c15e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/2267569535 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0cdbb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/2267569535 >> v2:172.21.15.46:3300/0 conn(0x7fa89c15aa50 0x7fa89c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 -- 172.21.15.46:0/2267569535 >> 172.21.15.46:0/2267569535 conn(0x7fa89c0b2040 msgr2=0x7fa89c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:25.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 -- 172.21.15.46:0/2267569535 shutdown_connections 2024-09-17T03:31:25.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.744+0000 7fa8a3e00640 1 -- 172.21.15.46:0/2267569535 wait complete. 2024-09-17T03:31:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.752+0000 7fa8a3e00640 1 Processor -- start 2024-09-17T03:31:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.752+0000 7fa8a3e00640 1 -- start start 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89c15aa50 0x7fa89c205be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89c15e4e0 0x7fa89c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c167670 con 0x7fa89c15aa50 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c0b41a0 con 0x7fa89c15e4e0 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a3e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c1677f0 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a1400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:25.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a1400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:50842/0 (socket says 172.21.15.46:50842) 2024-09-17T03:31:25.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.756+0000 7fa8a1400640 1 -- 172.21.15.46:0/3777932351 learned_addr learned my addr 172.21.15.46:0/3777932351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.760+0000 7fa8a1e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89c15e4e0 0x7fa89c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.760+0000 7fa8a1400640 1 -- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89c15e4e0 msgr2=0x7fa89c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.760+0000 7fa8a1400640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89c15e4e0 0x7fa89c158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.760+0000 7fa8a1400640 1 -- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89c15aa50 msgr2=0x7fa89c205be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.760+0000 7fa8a1400640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89c15aa50 0x7fa89c205be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.760+0000 7fa8a1400640 1 -- 172.21.15.46:0/3777932351 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c0aaa10 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.764+0000 7fa8a1400640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0aa4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa89000b3f0 tx=0x7fa89000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.764+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890015020 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.764+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa890011920 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.764+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890011d50 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.764+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c153780 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.764+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c153d40 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.768+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c06f320 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.776+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7fa890018020 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.776+0000 7fa899600640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa87c074820 0x7fa87c076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.780+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fa890017d70 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.780+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa89005fad0 con 0x7fa89c0cd7a0 2024-09-17T03:31:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.836+0000 7fa8a0a00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa87c074820 0x7fa87c076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:25.840+0000 7fa8a0a00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa87c074820 0x7fa87c076d30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa8880b0000 tx=0x7fa888000bd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:26.340+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28249", "target": ["mon-mgr", ""]}) -- 0x7fa89c06b540 con 0x7fa87c074820 2024-09-17T03:31:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.640+0000 7fa899600640 1 -- 172.21.15.46:0/3777932351 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa888008b60 con 0x7fa87c074820 2024-09-17T03:31:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa87c074820 msgr2=0x7fa87c076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa87c074820 0x7fa87c076d30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa8880b0000 tx=0x7fa888000bd0 comp rx=0 tx=0).stop 2024-09-17T03:31:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 msgr2=0x7fa89c0aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0aa4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa89000b3f0 tx=0x7fa89000b8b0 comp rx=0 tx=0).stop 2024-09-17T03:31:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 shutdown_connections 2024-09-17T03:31:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa87c074820 0x7fa87c076d30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89c15e4e0 0x7fa89c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa89c15aa50 0x7fa89c205be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 --2- 172.21.15.46:0/3777932351 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa89c0cd7a0 0x7fa89c0aa4d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 >> 172.21.15.46:0/3777932351 conn(0x7fa89c0b2040 msgr2=0x7fa89c158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 shutdown_connections 2024-09-17T03:31:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:27.648+0000 7fa8a3e00640 1 -- 172.21.15.46:0/3777932351 wait complete. 2024-09-17T03:31:27.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone16310 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 -- 172.21.15.46:0/3557505158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c163340 msgr2=0x7f606c163750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 --2- 172.21.15.46:0/3557505158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c163340 0x7f606c163750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f606c074e30 tx=0x7f606401cb30 comp rx=0 tx=0).stop 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 -- 172.21.15.46:0/3557505158 shutdown_connections 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 --2- 172.21.15.46:0/3557505158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c163340 0x7f606c163750 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 --2- 172.21.15.46:0/3557505158 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f606c166930 0x7f606c166d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 --2- 172.21.15.46:0/3557505158 >> v2:172.21.15.46:3300/0 conn(0x7f606c0af520 0x7f606c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 -- 172.21.15.46:0/3557505158 >> 172.21.15.46:0/3557505158 conn(0x7f606c0b2040 msgr2=0x7f606c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 -- 172.21.15.46:0/3557505158 shutdown_connections 2024-09-17T03:31:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.132+0000 7f6074600640 1 -- 172.21.15.46:0/3557505158 wait complete. 2024-09-17T03:31:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.144+0000 7f6074600640 1 Processor -- start 2024-09-17T03:31:28.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.144+0000 7f6074600640 1 -- start start 2024-09-17T03:31:28.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.148+0000 7f6074600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f606c0af520 0x7f606c13f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.148+0000 7f6074600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f606c163340 0x7f606c141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:28.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.148+0000 7f6074600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c166930 0x7f606c1423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.148+0000 7f6074600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f606c164c30 con 0x7f606c0af520 2024-09-17T03:31:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.148+0000 7f6074600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f606c0b42a0 con 0x7f606c166930 2024-09-17T03:31:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.148+0000 7f6074600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f606c164db0 con 0x7f606c163340 2024-09-17T03:31:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6071200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f606c163340 0x7f606c141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6072600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c166930 0x7f606c1423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:28.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6071c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f606c0af520 0x7f606c13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6071c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f606c0af520 0x7f606c13f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59690/0 (socket says 172.21.15.46:59690) 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6071c00640 1 -- 172.21.15.46:0/3457173675 learned_addr learned my addr 172.21.15.46:0/3457173675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6072600640 1 -- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f606c163340 msgr2=0x7f606c141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6072600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f606c163340 0x7f606c141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6072600640 1 -- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f606c0af520 msgr2=0x7f606c13f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6072600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f606c0af520 0x7f606c13f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:28.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.152+0000 7f6072600640 1 -- 172.21.15.46:0/3457173675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c157eb0 con 0x7f606c166930 2024-09-17T03:31:28.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.160+0000 7f6072600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c166930 0x7f606c1423a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f606c0b47b0 tx=0x7f60640a58e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.168+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60640b2040 con 0x7f606c166930 2024-09-17T03:31:28.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.168+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6064003b30 con 0x7f606c166930 2024-09-17T03:31:28.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.168+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064004030 con 0x7f606c166930 2024-09-17T03:31:28.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.168+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c158120 con 0x7f606c166930 2024-09-17T03:31:28.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.168+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c142ce0 con 0x7f606c166930 2024-09-17T03:31:28.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.172+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c163750 con 0x7f606c166930 2024-09-17T03:31:28.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.176+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f60640bd030 con 0x7f606c166930 2024-09-17T03:31:28.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.176+0000 7f6069600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f604c074840 0x7f604c076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:28.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.176+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f606412d290 con 0x7f606c166930 2024-09-17T03:31:28.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.180+0000 7f6071c00640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f604c074840 0x7f604c076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:28.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.188+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f60640c4020 con 0x7f606c166930 2024-09-17T03:31:28.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.224+0000 7f6071c00640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f604c074840 0x7f604c076d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f60600099b0 tx=0x7f6060009190 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:28.736+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone16310", "target": ["mon-mgr", ""]}) -- 0x7f606c06b540 con 0x7f604c074840 2024-09-17T03:31:29.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.492+0000 7f6069600640 1 -- 172.21.15.46:0/3457173675 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6060009600 con 0x7f604c074840 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f604c074840 msgr2=0x7f604c076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f604c074840 0x7f604c076d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f60600099b0 tx=0x7f6060009190 comp rx=0 tx=0).stop 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c166930 msgr2=0x7f606c1423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c166930 0x7f606c1423a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f606c0b47b0 tx=0x7f60640a58e0 comp rx=0 tx=0).stop 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 shutdown_connections 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f604c074840 0x7f604c076d50 secure :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f60600099b0 tx=0x7f6060009190 comp rx=0 tx=0).stop 2024-09-17T03:31:29.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606c166930 0x7f606c1423a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f606c163340 0x7f606c141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 --2- 172.21.15.46:0/3457173675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f606c0af520 0x7f606c13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.504+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 >> 172.21.15.46:0/3457173675 conn(0x7f606c0b2040 msgr2=0x7f606c158480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.508+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 shutdown_connections 2024-09-17T03:31:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:29.508+0000 7f6074600640 1 -- 172.21.15.46:0/3457173675 wait complete. 2024-09-17T03:31:29.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone560911 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 -- 172.21.15.46:0/120470797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27640b2300 msgr2=0x7f276415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 --2- 172.21.15.46:0/120470797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27640b2300 0x7f276415e3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2764155390 tx=0x7f275001d970 comp rx=0 tx=0).stop 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 -- 172.21.15.46:0/120470797 shutdown_connections 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 --2- 172.21.15.46:0/120470797 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f276415e900 0x7f276415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 --2- 172.21.15.46:0/120470797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27640b2300 0x7f276415e3c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 --2- 172.21.15.46:0/120470797 >> v2:172.21.15.46:3300/0 conn(0x7f276415b850 0x7f27640b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 -- 172.21.15.46:0/120470797 >> 172.21.15.46:0/120470797 conn(0x7f276406cdf0 msgr2=0x7f276406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.100+0000 7f276ac00640 1 -- 172.21.15.46:0/120470797 shutdown_connections 2024-09-17T03:31:30.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.104+0000 7f276ac00640 1 -- 172.21.15.46:0/120470797 wait complete. 2024-09-17T03:31:30.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.108+0000 7f276ac00640 1 Processor -- start 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.112+0000 7f276ac00640 1 -- start start 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276ac00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27640b2300 0x7f27641fc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f276415b850 0x7f27642092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 0x7f2764206690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27640b4dc0 con 0x7f276415b850 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27640b4c40 con 0x7f276415e900 2024-09-17T03:31:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276ac00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f27640b4f40 con 0x7f27640b2300 2024-09-17T03:31:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 0x7f2764206690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 0x7f2764206690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:56486/0 (socket says 172.21.15.46:56486) 2024-09-17T03:31:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.116+0000 7f276a200640 1 -- 172.21.15.46:0/91689085 learned_addr learned my addr 172.21.15.46:0/91689085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f2769800640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27640b2300 0x7f27641fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:30.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f2768e00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f276415b850 0x7f27642092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f276a200640 1 -- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27640b2300 msgr2=0x7f27641fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f276a200640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27640b2300 0x7f27641fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f276a200640 1 -- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f276415b850 msgr2=0x7f27642092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:30.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f276a200640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f276415b850 0x7f27642092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:30.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f276a200640 1 -- 172.21.15.46:0/91689085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276406d4a0 con 0x7f276415e900 2024-09-17T03:31:30.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.120+0000 7f2768e00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f276415b850 0x7f27642092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:30.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.124+0000 7f276a200640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 0x7f2764206690 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f275c00da70 tx=0x7f275c00df30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:30.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.124+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c017070 con 0x7f276415e900 2024-09-17T03:31:30.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.124+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2764206bd0 con 0x7f276415e900 2024-09-17T03:31:30.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.128+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f275c010460 con 0x7f276415e900 2024-09-17T03:31:30.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.128+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c017070 con 0x7f276415e900 2024-09-17T03:31:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.128+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2764209ef0 con 0x7f276415e900 2024-09-17T03:31:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.144+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27640b5460 con 0x7f276415e900 2024-09-17T03:31:30.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:31:30.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:31:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.152+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f275c01a070 con 0x7f276415e900 2024-09-17T03:31:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.152+0000 7f2761600640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2744074bf0 0x7f2744077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.156+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f275c00bbd0 con 0x7f276415e900 2024-09-17T03:31:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.172+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f275c05ef30 con 0x7f276415e900 2024-09-17T03:31:30.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.192+0000 7f2769800640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2744074bf0 0x7f2744077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.200+0000 7f2769800640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2744074bf0 0x7f2744077100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2764075440 tx=0x7f27580091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:30.748+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone560911", "target": ["mon-mgr", ""]}) -- 0x7f2764160720 con 0x7f2744074bf0 2024-09-17T03:31:32.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.140+0000 7f2761600640 1 -- 172.21.15.46:0/91689085 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2764160720 con 0x7f2744074bf0 2024-09-17T03:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.148+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2744074bf0 msgr2=0x7f2744077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.148+0000 7f276ac00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2744074bf0 0x7f2744077100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2764075440 tx=0x7f27580091e0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.148+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 msgr2=0x7f2764206690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.148+0000 7f276ac00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 0x7f2764206690 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f275c00da70 tx=0x7f275c00df30 comp rx=0 tx=0).stop 2024-09-17T03:31:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.156+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 shutdown_connections 2024-09-17T03:31:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.156+0000 7f276ac00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2744074bf0 0x7f2744077100 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.156+0000 7f276ac00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276415e900 0x7f2764206690 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.156+0000 7f276ac00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f276415b850 0x7f27642092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.156+0000 7f276ac00640 1 --2- 172.21.15.46:0/91689085 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27640b2300 0x7f27641fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.156+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 >> 172.21.15.46:0/91689085 conn(0x7f276406cdf0 msgr2=0x7f2764159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:32.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.168+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 shutdown_connections 2024-09-17T03:31:32.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.168+0000 7f276ac00640 1 -- 172.21.15.46:0/91689085 wait complete. 2024-09-17T03:31:32.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone955412 2024-09-17T03:31:32.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 -- 172.21.15.46:0/1301317043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b015f8f0 msgr2=0x7fc7b01557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:32.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/1301317043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b015f8f0 0x7fc7b01557e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009900 tx=0x7fc7a001c7c0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 -- 172.21.15.46:0/1301317043 shutdown_connections 2024-09-17T03:31:32.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/1301317043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b015f8f0 0x7fc7b01557e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 -- 172.21.15.46:0/1301317043 >> 172.21.15.46:0/1301317043 conn(0x7fc7b006e1d0 msgr2=0x7fc7b006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:32.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 -- 172.21.15.46:0/1301317043 shutdown_connections 2024-09-17T03:31:32.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.716+0000 7fc7b4c00640 1 -- 172.21.15.46:0/1301317043 wait complete. 2024-09-17T03:31:32.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.732+0000 7fc7b4c00640 1 Processor -- start 2024-09-17T03:31:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.732+0000 7fc7b4c00640 1 -- start start 2024-09-17T03:31:32.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7b4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 0x7fc7b014a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 0x7fc7b014a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 0x7fc7b014a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45124/0 (socket says 172.21.15.46:45124) 2024-09-17T03:31:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7ade00640 1 -- 172.21.15.46:0/3765515965 learned_addr learned my addr 172.21.15.46:0/3765515965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7b015f8f0 0x7fc7b0151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b0073d20 0x7fc7b0074130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7a7e00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7b015f8f0 0x7fc7b0151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.736+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b016d960 con 0x7fc7b00cf210 2024-09-17T03:31:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ae800640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b0073d20 0x7fc7b0074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:32.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b016d7e0 con 0x7fc7b0073d20 2024-09-17T03:31:32.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b016dae0 con 0x7fc7b015f8f0 2024-09-17T03:31:32.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ade00640 1 -- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7b015f8f0 msgr2=0x7fc7b0151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ade00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7b015f8f0 0x7fc7b0151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ade00640 1 -- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b0073d20 msgr2=0x7fc7b0074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ade00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b0073d20 0x7fc7b0074130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ade00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b0074670 con 0x7fc7b00cf210 2024-09-17T03:31:32.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7a7e00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7b015f8f0 0x7fc7b0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:32.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.740+0000 7fc7ae800640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b0073d20 0x7fc7b0074130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:32.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.744+0000 7fc7ade00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 0x7fc7b014a130 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001ae70 tx=0x7fc7a00a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:32.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.744+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00b2040 con 0x7fc7b00cf210 2024-09-17T03:31:32.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.748+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7a0004470 con 0x7fc7b00cf210 2024-09-17T03:31:32.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.748+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0003860 con 0x7fc7b00cf210 2024-09-17T03:31:32.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.748+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b0152780 con 0x7fc7b00cf210 2024-09-17T03:31:32.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.748+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b0153950 con 0x7fc7b00cf210 2024-09-17T03:31:32.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.756+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b014a670 con 0x7fc7b00cf210 2024-09-17T03:31:32.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.760+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7fc7a0003a00 con 0x7fc7b00cf210 2024-09-17T03:31:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.764+0000 7fc7a6000640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc78c074840 0x7fc78c076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:32.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.768+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fc7a012c980 con 0x7fc7b00cf210 2024-09-17T03:31:32.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.784+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc7b0074670 con 0x7fc7b00cf210 2024-09-17T03:31:32.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.796+0000 7fc7a7e00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc78c074840 0x7fc78c076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:32.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:32.804+0000 7fc7a7e00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc78c074840 0x7fc78c076d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00c6970 tx=0x7fc798007b50 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:33.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:33.360+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone955412", "target": ["mon-mgr", ""]}) -- 0x7fc7b0152ca0 con 0x7fc78c074840 2024-09-17T03:31:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.944+0000 7fc7a6000640 1 -- 172.21.15.46:0/3765515965 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc7b0152ca0 con 0x7fc78c074840 2024-09-17T03:31:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.956+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc78c074840 msgr2=0x7fc78c076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.956+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc78c074840 0x7fc78c076d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00c6970 tx=0x7fc798007b50 comp rx=0 tx=0).stop 2024-09-17T03:31:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.956+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 msgr2=0x7fc7b014a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:34.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.956+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 0x7fc7b014a130 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001ae70 tx=0x7fc7a00a56e0 comp rx=0 tx=0).stop 2024-09-17T03:31:34.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 shutdown_connections 2024-09-17T03:31:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc78c074840 0x7fc78c076d50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b0073d20 0x7fc7b0074130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc7b015f8f0 0x7fc7b0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 --2- 172.21.15.46:0/3765515965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b00cf210 0x7fc7b014a130 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 >> 172.21.15.46:0/3765515965 conn(0x7fc7b006e1d0 msgr2=0x7fc7b006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 shutdown_connections 2024-09-17T03:31:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:34.964+0000 7fc7b4c00640 1 -- 172.21.15.46:0/3765515965 wait complete. 2024-09-17T03:31:35.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone579713 2024-09-17T03:31:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.552+0000 7f1281e00640 1 -- 172.21.15.46:0/3575073955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cd260 msgr2=0x7f127c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.552+0000 7f1281e00640 1 --2- 172.21.15.46:0/3575073955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cd260 0x7f127c0cd670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f127c075760 tx=0x7f126801c620 comp rx=0 tx=0).stop 2024-09-17T03:31:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.556+0000 7f1281e00640 1 -- 172.21.15.46:0/3575073955 shutdown_connections 2024-09-17T03:31:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.556+0000 7f1281e00640 1 --2- 172.21.15.46:0/3575073955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cdbb0 0x7f127c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.556+0000 7f1281e00640 1 --2- 172.21.15.46:0/3575073955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cd260 0x7f127c0cd670 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.556+0000 7f1281e00640 1 --2- 172.21.15.46:0/3575073955 >> v2:172.21.15.46:3300/0 conn(0x7f127c0ce4c0 0x7f127c15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.556+0000 7f1281e00640 1 -- 172.21.15.46:0/3575073955 >> 172.21.15.46:0/3575073955 conn(0x7f127c0b2040 msgr2=0x7f127c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:35.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.560+0000 7f1281e00640 1 -- 172.21.15.46:0/3575073955 shutdown_connections 2024-09-17T03:31:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.560+0000 7f1281e00640 1 -- 172.21.15.46:0/3575073955 wait complete. 2024-09-17T03:31:35.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.580+0000 7f1281e00640 1 Processor -- start 2024-09-17T03:31:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.588+0000 7f1281e00640 1 -- start start 2024-09-17T03:31:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f1281e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cd260 0x7f127c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f1281e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cdbb0 0x7f127c166940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f127aa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cd260 0x7f127c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f127aa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cd260 0x7f127c1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:60282/0 (socket says 172.21.15.46:60282) 2024-09-17T03:31:35.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f127aa00640 1 -- 172.21.15.46:0/3104640307 learned_addr learned my addr 172.21.15.46:0/3104640307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c0ce4c0 0x7f127c1fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f127a000640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cdbb0 0x7f127c166940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f127b400640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c0ce4c0 0x7f127c1fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f127c0b3a80 con 0x7f127c0ce4c0 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f127c0b3900 con 0x7f127c0cdbb0 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.596+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f127c0b3c00 con 0x7f127c0cd260 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f127a000640 1 -- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cd260 msgr2=0x7f127c1644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f127a000640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cd260 0x7f127c1644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f127a000640 1 -- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c0ce4c0 msgr2=0x7f127c1fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f127a000640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c0ce4c0 0x7f127c1fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:35.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f127a000640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c1fe010 con 0x7f127c0cdbb0 2024-09-17T03:31:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f127a000640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cdbb0 0x7f127c166940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f12680ac000 tx=0x7f12680b0610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12680b0cc0 con 0x7f127c0cdbb0 2024-09-17T03:31:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1268004040 con 0x7f127c0cdbb0 2024-09-17T03:31:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268004490 con 0x7f127c0cdbb0 2024-09-17T03:31:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c1639d0 con 0x7f127c0cdbb0 2024-09-17T03:31:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.600+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c163f90 con 0x7f127c0cdbb0 2024-09-17T03:31:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.608+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f1268003600 con 0x7f127c0cdbb0 2024-09-17T03:31:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.616+0000 7f1273400640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f125c074840 0x7f125c076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.616+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f126812cd90 con 0x7f127c0cdbb0 2024-09-17T03:31:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.616+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c15e4e0 con 0x7f127c0cdbb0 2024-09-17T03:31:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.620+0000 7f127aa00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f125c074840 0x7f125c076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:35.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.624+0000 7f127aa00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f125c074840 0x7f125c076d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f126c009c00 tx=0x7f126c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:35.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:35.632+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12680be030 con 0x7f127c0cdbb0 2024-09-17T03:31:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:36.384+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone579713", "target": ["mon-mgr", ""]}) -- 0x7f127c06b540 con 0x7f125c074840 2024-09-17T03:31:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.344+0000 7f1273400640 1 -- 172.21.15.46:0/3104640307 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f126c009850 con 0x7f125c074840 2024-09-17T03:31:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f125c074840 msgr2=0x7f125c076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f125c074840 0x7f125c076d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f126c009c00 tx=0x7f126c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cdbb0 msgr2=0x7f127c166940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cdbb0 0x7f127c166940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f12680ac000 tx=0x7f12680b0610 comp rx=0 tx=0).stop 2024-09-17T03:31:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 shutdown_connections 2024-09-17T03:31:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f125c074840 0x7f125c076d50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c0ce4c0 0x7f127c1fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f127c0cdbb0 0x7f127c166940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 --2- 172.21.15.46:0/3104640307 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c0cd260 0x7f127c1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.376+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 >> 172.21.15.46:0/3104640307 conn(0x7f127c0b2040 msgr2=0x7f127c159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.380+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 shutdown_connections 2024-09-17T03:31:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.380+0000 7f1281e00640 1 -- 172.21.15.46:0/3104640307 wait complete. 2024-09-17T03:31:37.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone341414 2024-09-17T03:31:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.940+0000 7f2523e00640 1 -- 172.21.15.46:0/2328326143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0b2070 con 0x7f25240cf210 2024-09-17T03:31:37.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.940+0000 7f252be00640 1 -- 172.21.15.46:0/2328326143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 msgr2=0x7f25240cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.940+0000 7f252be00640 1 --2- 172.21.15.46:0/2328326143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240cf620 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f251c009c80 tx=0x7f251c01cd00 comp rx=0 tx=0).stop 2024-09-17T03:31:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.944+0000 7f252be00640 1 -- 172.21.15.46:0/2328326143 shutdown_connections 2024-09-17T03:31:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.944+0000 7f252be00640 1 --2- 172.21.15.46:0/2328326143 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240b3890 0x7f25240b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.944+0000 7f252be00640 1 --2- 172.21.15.46:0/2328326143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240cf620 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.944+0000 7f252be00640 1 -- 172.21.15.46:0/2328326143 >> 172.21.15.46:0/2328326143 conn(0x7f252406e1d0 msgr2=0x7f252406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.960+0000 7f252be00640 1 -- 172.21.15.46:0/2328326143 shutdown_connections 2024-09-17T03:31:37.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.968+0000 7f252be00640 1 -- 172.21.15.46:0/2328326143 wait complete. 2024-09-17T03:31:37.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.984+0000 7f252be00640 1 Processor -- start 2024-09-17T03:31:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.992+0000 7f252be00640 1 -- start start 2024-09-17T03:31:37.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.992+0000 7f252be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25240b3890 0x7f2524202db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.992+0000 7f252be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 0x7f2524208e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f252be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2529400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25240b3890 0x7f2524202db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2529400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25240b3890 0x7f2524202db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43770/0 (socket says 172.21.15.46:43770) 2024-09-17T03:31:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2528a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 0x7f2524208e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2528a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 0x7f2524208e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:60298/0 (socket says 172.21.15.46:60298) 2024-09-17T03:31:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2529e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2529e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45152/0 (socket says 172.21.15.46:45152) 2024-09-17T03:31:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f2529400640 1 -- 172.21.15.46:0/359481801 learned_addr learned my addr 172.21.15.46:0/359481801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25240ccec0 con 0x7f25240cf210 2024-09-17T03:31:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25240ccd40 con 0x7f25240b3890 2024-09-17T03:31:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:37.996+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f25240cd040 con 0x7f25240cd7a0 2024-09-17T03:31:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2528a00640 1 -- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25240b3890 msgr2=0x7f2524202db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2528a00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25240b3890 0x7f2524202db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2528a00640 1 -- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 msgr2=0x7f25240aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2528a00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2528a00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2524209b40 con 0x7f25240cd7a0 2024-09-17T03:31:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2529e00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:31:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.000+0000 7f2528a00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 0x7f2524208e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f251c0ac000 tx=0x7f251c009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.004+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0b2040 con 0x7f25240cd7a0 2024-09-17T03:31:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.004+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2524205a70 con 0x7f25240cd7a0 2024-09-17T03:31:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.004+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2524206550 con 0x7f25240cd7a0 2024-09-17T03:31:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.004+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f251c0a5e40 con 0x7f25240cd7a0 2024-09-17T03:31:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.004+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0b2040 con 0x7f25240cd7a0 2024-09-17T03:31:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.012+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000052a0 con 0x7f25240cd7a0 2024-09-17T03:31:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.012+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f251c0bd030 con 0x7f25240cd7a0 2024-09-17T03:31:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.016+0000 7f2521600640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2504074a20 0x7f2504076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.016+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f251c12cbd0 con 0x7f25240cd7a0 2024-09-17T03:31:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.024+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f251c0c4020 con 0x7f25240cd7a0 2024-09-17T03:31:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.056+0000 7f2529400640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2504074a20 0x7f2504076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.060+0000 7f2529400640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2504074a20 0x7f2504076f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2510009a00 tx=0x7f2510009110 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:38.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:38.696+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone341414", "target": ["mon-mgr", ""]}) -- 0x7f2500000c90 con 0x7f2504074a20 2024-09-17T03:31:39.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f2521600640 1 -- 172.21.15.46:0/359481801 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2500000c90 con 0x7f2504074a20 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2504074a20 msgr2=0x7f2504076f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2504074a20 0x7f2504076f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2510009a00 tx=0x7f2510009110 comp rx=0 tx=0).stop 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 msgr2=0x7f2524208e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 0x7f2524208e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f251c0ac000 tx=0x7f251c009340 comp rx=0 tx=0).stop 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 shutdown_connections 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2504074a20 0x7f2504076f30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25240cf210 0x7f25240aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25240cd7a0 0x7f2524208e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 --2- 172.21.15.46:0/359481801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25240b3890 0x7f2524202db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.492+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 >> 172.21.15.46:0/359481801 conn(0x7f252406e1d0 msgr2=0x7f25240b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.496+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 shutdown_connections 2024-09-17T03:31:39.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.496+0000 7f252be00640 1 -- 172.21.15.46:0/359481801 wait complete. 2024-09-17T03:31:39.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone616815 2024-09-17T03:31:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.992+0000 7f098e800640 1 -- 172.21.15.46:0/3779130022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980052c50 msgr2=0x7f0980038b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.992+0000 7f098e800640 1 --2- 172.21.15.46:0/3779130022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980052c50 0x7f0980038b20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f098004b9d0 tx=0x7f097401c690 comp rx=0 tx=0).stop 2024-09-17T03:31:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.996+0000 7f098e800640 1 -- 172.21.15.46:0/3779130022 shutdown_connections 2024-09-17T03:31:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.996+0000 7f098e800640 1 --2- 172.21.15.46:0/3779130022 >> v2:172.21.15.99:3300/0 conn(0x7f0980039060 0x7f09800eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.996+0000 7f098e800640 1 --2- 172.21.15.46:0/3779130022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980052c50 0x7f0980038b20 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.996+0000 7f098e800640 1 -- 172.21.15.46:0/3779130022 >> 172.21.15.46:0/3779130022 conn(0x7f0980054970 msgr2=0x7f0980054d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.996+0000 7f098e800640 1 -- 172.21.15.46:0/3779130022 shutdown_connections 2024-09-17T03:31:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:39.996+0000 7f098e800640 1 -- 172.21.15.46:0/3779130022 wait complete. 2024-09-17T03:31:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.000+0000 7f098e800640 1 Processor -- start 2024-09-17T03:31:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.000+0000 7f098e800640 1 -- start start 2024-09-17T03:31:40.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 0x7f09800d8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f098004efc0 0x7f09800df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09800494b0 0x7f09800dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f09800ed9c0 con 0x7f0980039060 2024-09-17T03:31:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09800ed840 con 0x7f098004efc0 2024-09-17T03:31:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f09800edb40 con 0x7f09800494b0 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 0x7f09800d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 0x7f09800d8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45166/0 (socket says 172.21.15.46:45166) 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098d400640 1 -- 172.21.15.46:0/3962945775 learned_addr learned my addr 172.21.15.46:0/3962945775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098fe00640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09800494b0 0x7f09800dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098ca00640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f098004efc0 0x7f09800df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098fe00640 1 -- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f098004efc0 msgr2=0x7f09800df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098fe00640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f098004efc0 0x7f09800df5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098fe00640 1 -- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 msgr2=0x7f09800d8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098fe00640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 0x7f09800d8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098fe00640 1 -- 172.21.15.46:0/3962945775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09800d8d50 con 0x7f09800494b0 2024-09-17T03:31:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.008+0000 7f098d400640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 0x7f09800d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:31:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.012+0000 7f098fe00640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09800494b0 0x7f09800dfb30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0988051dc0 tx=0x7f098806a850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.012+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098806b540 con 0x7f09800494b0 2024-09-17T03:31:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.012+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f098806a980 con 0x7f09800494b0 2024-09-17T03:31:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.012+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09880924c0 con 0x7f09800494b0 2024-09-17T03:31:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.012+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09800486f0 con 0x7f09800494b0 2024-09-17T03:31:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.012+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0980048cb0 con 0x7f09800494b0 2024-09-17T03:31:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.016+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0980038b20 con 0x7f09800494b0 2024-09-17T03:31:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.020+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f09880638b0 con 0x7f09800494b0 2024-09-17T03:31:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.020+0000 7f0985600640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09680748f0 0x7f0968076e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.020+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f098810e450 con 0x7f09800494b0 2024-09-17T03:31:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.020+0000 7f098d400640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09680748f0 0x7f0968076e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.024+0000 7f098d400640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09680748f0 0x7f0968076e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f097c001e40 tx=0x7f097c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:40.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.028+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f09880dabc0 con 0x7f09800494b0 2024-09-17T03:31:40.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:40.536+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone616815", "target": ["mon-mgr", ""]}) -- 0x7f09800dd410 con 0x7f09680748f0 2024-09-17T03:31:41.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.356+0000 7f0985600640 1 -- 172.21.15.46:0/3962945775 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09800dd410 con 0x7f09680748f0 2024-09-17T03:31:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.372+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09680748f0 msgr2=0x7f0968076e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.372+0000 7f098e800640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09680748f0 0x7f0968076e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f097c001e40 tx=0x7f097c009040 comp rx=0 tx=0).stop 2024-09-17T03:31:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.372+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09800494b0 msgr2=0x7f09800dfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.372+0000 7f098e800640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09800494b0 0x7f09800dfb30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0988051dc0 tx=0x7f098806a850 comp rx=0 tx=0).stop 2024-09-17T03:31:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 shutdown_connections 2024-09-17T03:31:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f09680748f0 0x7f0968076e00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09800494b0 0x7f09800dfb30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f098004efc0 0x7f09800df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 --2- 172.21.15.46:0/3962945775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0980039060 0x7f09800d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 >> 172.21.15.46:0/3962945775 conn(0x7f0980054970 msgr2=0x7f09800504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 shutdown_connections 2024-09-17T03:31:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.384+0000 7f098e800640 1 -- 172.21.15.46:0/3962945775 wait complete. 2024-09-17T03:31:41.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone838216 2024-09-17T03:31:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 -- 172.21.15.46:0/3628758726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cd970 msgr2=0x7f89580b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 --2- 172.21.15.46:0/3628758726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cd970 0x7f89580b3340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f894800afd0 tx=0x7f894801d340 comp rx=0 tx=0).stop 2024-09-17T03:31:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 -- 172.21.15.46:0/3628758726 shutdown_connections 2024-09-17T03:31:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 --2- 172.21.15.46:0/3628758726 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89580cdd80 0x7f89580b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 --2- 172.21.15.46:0/3628758726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cd970 0x7f89580b3340 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 -- 172.21.15.46:0/3628758726 >> 172.21.15.46:0/3628758726 conn(0x7f89580b2040 msgr2=0x7f89580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 -- 172.21.15.46:0/3628758726 shutdown_connections 2024-09-17T03:31:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.896+0000 7f895dc00640 1 -- 172.21.15.46:0/3628758726 wait complete. 2024-09-17T03:31:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.904+0000 7f895dc00640 1 Processor -- start 2024-09-17T03:31:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.904+0000 7f895dc00640 1 -- start start 2024-09-17T03:31:41.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f895dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89580cd970 0x7f8958201790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f895dc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 0x7f89580aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f895dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f895815aa50 0x7f8958157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f895dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8958168140 con 0x7f895815aa50 2024-09-17T03:31:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f895dc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8958167fc0 con 0x7f89580cdd80 2024-09-17T03:31:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f895dc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f89581682c0 con 0x7f89580cd970 2024-09-17T03:31:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8957400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f895815aa50 0x7f8958157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8956000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 0x7f89580aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8956000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 0x7f89580aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43828/0 (socket says 172.21.15.46:43828) 2024-09-17T03:31:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8956000640 1 -- 172.21.15.46:0/558355264 learned_addr learned my addr 172.21.15.46:0/558355264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8956a00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89580cd970 0x7f8958201790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8956000640 1 -- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89580cd970 msgr2=0x7f8958201790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.908+0000 7f8956000640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89580cd970 0x7f8958201790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f8956000640 1 -- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f895815aa50 msgr2=0x7f8958157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f8956000640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f895815aa50 0x7f8958157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f8956000640 1 -- 172.21.15.46:0/558355264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89581fdbc0 con 0x7f89580cdd80 2024-09-17T03:31:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f8957400640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f895815aa50 0x7f8958157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:31:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f8956000640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 0x7f89580aee80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8948000c10 tx=0x7f8948007a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:41.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480b2040 con 0x7f89580cdd80 2024-09-17T03:31:41.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8948004040 con 0x7f89580cdd80 2024-09-17T03:31:41.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.912+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948004430 con 0x7f89580cdd80 2024-09-17T03:31:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.916+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89581fdea0 con 0x7f89580cdd80 2024-09-17T03:31:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.916+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581fe460 con 0x7f89580cdd80 2024-09-17T03:31:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.916+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89581fe6a0 con 0x7f89580cdd80 2024-09-17T03:31:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.920+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f89480c0060 con 0x7f89580cdd80 2024-09-17T03:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.920+0000 7f894ea00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8938074840 0x7f8938076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.920+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f894812c1b0 con 0x7f89580cdd80 2024-09-17T03:31:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.924+0000 7f8956a00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8938074840 0x7f8938076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.928+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8948016fc0 con 0x7f89580cdd80 2024-09-17T03:31:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:41.928+0000 7f8956a00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8938074840 0x7f8938076d50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8950001e30 tx=0x7f8950009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:42.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:42.508+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone838216", "target": ["mon-mgr", ""]}) -- 0x7f895806b540 con 0x7f8938074840 2024-09-17T03:31:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.804+0000 7f894ea00640 1 -- 172.21.15.46:0/558355264 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8950001a70 con 0x7f8938074840 2024-09-17T03:31:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8938074840 msgr2=0x7f8938076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8938074840 0x7f8938076d50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8950001e30 tx=0x7f8950009040 comp rx=0 tx=0).stop 2024-09-17T03:31:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 msgr2=0x7f89580aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:43.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 0x7f89580aee80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8948000c10 tx=0x7f8948007a50 comp rx=0 tx=0).stop 2024-09-17T03:31:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 shutdown_connections 2024-09-17T03:31:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8938074840 0x7f8938076d50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f895815aa50 0x7f8958157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89580cdd80 0x7f89580aee80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 --2- 172.21.15.46:0/558355264 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f89580cd970 0x7f8958201790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 >> 172.21.15.46:0/558355264 conn(0x7f89580b2040 msgr2=0x7f89580b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 shutdown_connections 2024-09-17T03:31:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:43.812+0000 7f895dc00640 1 -- 172.21.15.46:0/558355264 wait complete. 2024-09-17T03:31:43.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone551017 2024-09-17T03:31:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.416+0000 7fbaa1e00640 1 -- 172.21.15.46:0/3481055957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba940e3a20 msgr2=0x7fba940df2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.416+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/3481055957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba940e3a20 0x7fba940df2a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fba9c077870 tx=0x7fba9c0989e0 comp rx=0 tx=0).stop 2024-09-17T03:31:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.428+0000 7fbaa1e00640 1 -- 172.21.15.46:0/3481055957 shutdown_connections 2024-09-17T03:31:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.428+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/3481055957 >> v2:172.21.15.46:3300/0 conn(0x7fba94036780 0x7fba9400add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.428+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/3481055957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba940e3a20 0x7fba940df2a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.428+0000 7fbaa1e00640 1 -- 172.21.15.46:0/3481055957 >> 172.21.15.46:0/3481055957 conn(0x7fba940513f0 msgr2=0x7fba94051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.428+0000 7fbaa1e00640 1 -- 172.21.15.46:0/3481055957 shutdown_connections 2024-09-17T03:31:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.428+0000 7fbaa1e00640 1 -- 172.21.15.46:0/3481055957 wait complete. 2024-09-17T03:31:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.432+0000 7fbaa1e00640 1 Processor -- start 2024-09-17T03:31:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.432+0000 7fbaa1e00640 1 -- start start 2024-09-17T03:31:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 0x7fba940db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba94036780 0x7fba940dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fbaa1e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba940e3a20 0x7fba940dc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fbaa1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fba940e46a0 con 0x7fba94036780 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fbaa1e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba940e4520 con 0x7fba94036370 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fbaa1e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fba940e4820 con 0x7fba940e3a20 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fba9aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 0x7fba940db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.444+0000 7fba9aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 0x7fba940db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49366/0 (socket says 172.21.15.46:49366) 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9aa00640 1 -- 172.21.15.46:0/4081638134 learned_addr learned my addr 172.21.15.46:0/4081638134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9a000640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba94036780 0x7fba940dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9aa00640 1 -- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba940e3a20 msgr2=0x7fba940dc150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:31:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9aa00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba940e3a20 0x7fba940dc150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9aa00640 1 -- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba94036780 msgr2=0x7fba940dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9aa00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba94036780 0x7fba940dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9aa00640 1 -- 172.21.15.46:0/4081638134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba940e21b0 con 0x7fba94036370 2024-09-17T03:31:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.448+0000 7fba9a000640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba94036780 0x7fba940dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:31:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.452+0000 7fba9aa00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 0x7fba940db2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fba9c077870 tx=0x7fba9c123ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:44.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.472+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c137070 con 0x7fba94036370 2024-09-17T03:31:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.472+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba9c121c90 con 0x7fba94036370 2024-09-17T03:31:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.472+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c12e040 con 0x7fba94036370 2024-09-17T03:31:44.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.476+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba940477c0 con 0x7fba94036370 2024-09-17T03:31:44.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.476+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba940553c0 con 0x7fba94036370 2024-09-17T03:31:44.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.484+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba94006430 con 0x7fba94036370 2024-09-17T03:31:44.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.492+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7fba9c1232d0 con 0x7fba94036370 2024-09-17T03:31:44.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.492+0000 7fba93400640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba7c0748f0 0x7fba7c076e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.496+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fba9c1a8960 con 0x7fba94036370 2024-09-17T03:31:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.496+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba9c092840 con 0x7fba94036370 2024-09-17T03:31:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.528+0000 7fba9a000640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba7c0748f0 0x7fba7c076e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:44.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:44.552+0000 7fba9a000640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba7c0748f0 0x7fba7c076e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fba94052540 tx=0x7fba88009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:45.176+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone551017", "target": ["mon-mgr", ""]}) -- 0x7fba940ed110 con 0x7fba7c0748f0 2024-09-17T03:31:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.108+0000 7fba93400640 1 -- 172.21.15.46:0/4081638134 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba940ed110 con 0x7fba7c0748f0 2024-09-17T03:31:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba7c0748f0 msgr2=0x7fba7c076e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba7c0748f0 0x7fba7c076e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fba94052540 tx=0x7fba88009040 comp rx=0 tx=0).stop 2024-09-17T03:31:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 msgr2=0x7fba940db2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 0x7fba940db2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fba9c077870 tx=0x7fba9c123ef0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 shutdown_connections 2024-09-17T03:31:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fba7c0748f0 0x7fba7c076e00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fba940e3a20 0x7fba940dc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba94036780 0x7fba940dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 --2- 172.21.15.46:0/4081638134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba94036370 0x7fba940db2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.112+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 >> 172.21.15.46:0/4081638134 conn(0x7fba940513f0 msgr2=0x7fba9404a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.116+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 shutdown_connections 2024-09-17T03:31:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.116+0000 7fbaa1e00640 1 -- 172.21.15.46:0/4081638134 wait complete. 2024-09-17T03:31:46.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone522018 2024-09-17T03:31:46.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.732+0000 7fbb70a00640 1 -- 172.21.15.46:0/4020810466 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00d0d0 con 0x7fbb740b4220 2024-09-17T03:31:46.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.736+0000 7fbb73200640 1 -- 172.21.15.46:0/4020810466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 msgr2=0x7fbb740b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:46.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.736+0000 7fbb73200640 1 --2- 172.21.15.46:0/4020810466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 0x7fbb740b4630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbb740734d0 tx=0x7fbb6c020aa0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.740+0000 7fbb73200640 1 -- 172.21.15.46:0/4020810466 shutdown_connections 2024-09-17T03:31:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.744+0000 7fbb73200640 1 --2- 172.21.15.46:0/4020810466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 0x7fbb740b4630 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.744+0000 7fbb73200640 1 --2- 172.21.15.46:0/4020810466 >> v2:172.21.15.46:3300/0 conn(0x7fbb740b24d0 0x7fbb7415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.744+0000 7fbb73200640 1 -- 172.21.15.46:0/4020810466 >> 172.21.15.46:0/4020810466 conn(0x7fbb7406cdf0 msgr2=0x7fbb7406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:46.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.744+0000 7fbb73200640 1 -- 172.21.15.46:0/4020810466 shutdown_connections 2024-09-17T03:31:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.744+0000 7fbb73200640 1 -- 172.21.15.46:0/4020810466 wait complete. 2024-09-17T03:31:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.752+0000 7fbb73200640 1 Processor -- start 2024-09-17T03:31:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.756+0000 7fbb73200640 1 -- start start 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb73200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 0x7fbb74136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb73200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 0x7fbb741648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb73200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb74148390 0x7fbb7414d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb73200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb74167e60 con 0x7fbb74151900 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb73200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb74167ce0 con 0x7fbb740b4220 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb73200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbb74167fe0 con 0x7fbb74148390 2024-09-17T03:31:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb71400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 0x7fbb741648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb71400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 0x7fbb741648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50914/0 (socket says 172.21.15.46:50914) 2024-09-17T03:31:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb71400640 1 -- 172.21.15.46:0/1863591019 learned_addr learned my addr 172.21.15.46:0/1863591019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.760+0000 7fbb71e00640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 0x7fbb74136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb72800640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb74148390 0x7fbb7414d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb71400640 1 -- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb74148390 msgr2=0x7fbb7414d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb71400640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb74148390 0x7fbb7414d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb71400640 1 -- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 msgr2=0x7fbb74136d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb71400640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 0x7fbb74136d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb71400640 1 -- 172.21.15.46:0/1863591019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb74137260 con 0x7fbb74151900 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb71400640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 0x7fbb741648e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbb600058d0 tx=0x7fbb6000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60017070 con 0x7fbb74151900 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb60009d20 con 0x7fbb74151900 2024-09-17T03:31:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.764+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb600106c0 con 0x7fbb74151900 2024-09-17T03:31:46.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.768+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb74166a90 con 0x7fbb74151900 2024-09-17T03:31:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.772+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb74167050 con 0x7fbb74151900 2024-09-17T03:31:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.772+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7415b850 con 0x7fbb74151900 2024-09-17T03:31:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.776+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7fbb6001a070 con 0x7fbb74151900 2024-09-17T03:31:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.776+0000 7fbb6a000640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb54074840 0x7fbb54076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.784+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fbb6005ab80 con 0x7fbb74151900 2024-09-17T03:31:46.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.788+0000 7fbb71e00640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb54074840 0x7fbb54076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.796+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb74137260 con 0x7fbb74151900 2024-09-17T03:31:46.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:46.816+0000 7fbb71e00640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb54074840 0x7fbb54076d50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb74071800 tx=0x7fbb64009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.380+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone522018", "target": ["mon-mgr", ""]}) -- 0x7fbb7406b540 con 0x7fbb54074840 2024-09-17T03:31:47.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.968+0000 7fbb6a000640 1 -- 172.21.15.46:0/1863591019 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbb64005020 con 0x7fbb54074840 2024-09-17T03:31:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb54074840 msgr2=0x7fbb54076d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb54074840 0x7fbb54076d50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbb74071800 tx=0x7fbb64009040 comp rx=0 tx=0).stop 2024-09-17T03:31:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 msgr2=0x7fbb741648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 0x7fbb741648e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbb600058d0 tx=0x7fbb6000dde0 comp rx=0 tx=0).stop 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 shutdown_connections 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb54074840 0x7fbb54076d50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb74148390 0x7fbb7414d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb74151900 0x7fbb741648e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 --2- 172.21.15.46:0/1863591019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb740b4220 0x7fbb74136d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 >> 172.21.15.46:0/1863591019 conn(0x7fbb7406cdf0 msgr2=0x7fbb7415fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 shutdown_connections 2024-09-17T03:31:47.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:47.972+0000 7fbb73200640 1 -- 172.21.15.46:0/1863591019 wait complete. 2024-09-17T03:31:48.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone582719 2024-09-17T03:31:48.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.428+0000 7f1337200640 1 -- 172.21.15.46:0/3159312817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 msgr2=0x7f13380cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:48.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.428+0000 7f1337200640 1 --2- 172.21.15.46:0/3159312817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 0x7f13380cdc70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1324009840 tx=0x7f132401c7c0 comp rx=0 tx=0).stop 2024-09-17T03:31:48.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.428+0000 7f1337200640 1 -- 172.21.15.46:0/3159312817 shutdown_connections 2024-09-17T03:31:48.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.428+0000 7f1337200640 1 --2- 172.21.15.46:0/3159312817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 0x7f1338074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:48.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.428+0000 7f1337200640 1 --2- 172.21.15.46:0/3159312817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 0x7f13380cdc70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:48.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.432+0000 7f1337200640 1 -- 172.21.15.46:0/3159312817 >> 172.21.15.46:0/3159312817 conn(0x7f13380b4290 msgr2=0x7f13380b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:48.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.432+0000 7f1337200640 1 -- 172.21.15.46:0/3159312817 shutdown_connections 2024-09-17T03:31:48.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.432+0000 7f1337200640 1 -- 172.21.15.46:0/3159312817 wait complete. 2024-09-17T03:31:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.436+0000 7f1337200640 1 Processor -- start 2024-09-17T03:31:48.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 -- start start 2024-09-17T03:31:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 0x7f13380af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 0x7f1338099230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 0x7f1338099770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1338069370 con 0x7f133815fde0 2024-09-17T03:31:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13380691f0 con 0x7f1338074a70 2024-09-17T03:31:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.440+0000 7f1337200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f13380694f0 con 0x7f13380ce1b0 2024-09-17T03:31:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 0x7f13380af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 0x7f13380af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:49386/0 (socket says 172.21.15.46:49386) 2024-09-17T03:31:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335e00640 1 -- 172.21.15.46:0/1583483760 learned_addr learned my addr 172.21.15.46:0/1583483760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335400640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 0x7f1338099230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335400640 1 -- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 msgr2=0x7f13380af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335400640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 0x7f13380af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335400640 1 -- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 msgr2=0x7f1338099770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335400640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 0x7f1338099770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335400640 1 -- 172.21.15.46:0/1583483760 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133814a3e0 con 0x7f13380ce1b0 2024-09-17T03:31:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.444+0000 7f1335e00640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 0x7f13380af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:31:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.448+0000 7f1335400640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 0x7f1338099230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f132800c9a0 tx=0x7f132800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.448+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328007bb0 con 0x7f13380ce1b0 2024-09-17T03:31:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.448+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1328007e40 con 0x7f13380ce1b0 2024-09-17T03:31:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.448+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132800f6b0 con 0x7f13380ce1b0 2024-09-17T03:31:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.448+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133809abf0 con 0x7f13380ce1b0 2024-09-17T03:31:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.448+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f133815e930 con 0x7f13380ce1b0 2024-09-17T03:31:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.452+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13100052a0 con 0x7f13380ce1b0 2024-09-17T03:31:48.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.456+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 37) ==== 99223+0+0 (secure 0 0 0) 0x7f132800f850 con 0x7f13380ce1b0 2024-09-17T03:31:48.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.456+0000 7f132e000640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1318078c00 0x7f131807b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:48.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.456+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f1328093a40 con 0x7f13380ce1b0 2024-09-17T03:31:48.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.460+0000 7f1335e00640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1318078c00 0x7f131807b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:48.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.460+0000 7f1335e00640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1318078c00 0x7f131807b110 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f132401a450 tx=0x7f13240b2040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.464+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f133814a3e0 con 0x7f13380ce1b0 2024-09-17T03:31:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.548+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f132805bca0 con 0x7f13380ce1b0 2024-09-17T03:31:48.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:48.968+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone582719", "target": ["mon-mgr", ""]}) -- 0x7f1310000c90 con 0x7f1318078c00 2024-09-17T03:31:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:49.992+0000 7f132e000640 1 -- 172.21.15.46:0/1583483760 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1310000c90 con 0x7f1318078c00 2024-09-17T03:31:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.004+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1318078c00 msgr2=0x7f131807b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.004+0000 7f1337200640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1318078c00 0x7f131807b110 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f132401a450 tx=0x7f13240b2040 comp rx=0 tx=0).stop 2024-09-17T03:31:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.004+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 msgr2=0x7f1338099230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.004+0000 7f1337200640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 0x7f1338099230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f132800c9a0 tx=0x7f132800ce60 comp rx=0 tx=0).stop 2024-09-17T03:31:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 shutdown_connections 2024-09-17T03:31:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1318078c00 0x7f131807b110 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f133815fde0 0x7f1338099770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f13380ce1b0 0x7f1338099230 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f132800c9a0 tx=0x7f132800ce60 comp rx=0 tx=0).stop 2024-09-17T03:31:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 --2- 172.21.15.46:0/1583483760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1338074a70 0x7f13380af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 >> 172.21.15.46:0/1583483760 conn(0x7f13380b4290 msgr2=0x7f13380b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 shutdown_connections 2024-09-17T03:31:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:50.008+0000 7f1337200640 1 -- 172.21.15.46:0/1583483760 wait complete. 2024-09-17T03:31:52.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:31:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90e000640 1 -- 172.21.15.46:0/2160634437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8ec0051b0 msgr2=0x7fd8ec007a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90e000640 1 --2- 172.21.15.46:0/2160634437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8ec0051b0 0x7fd8ec007a30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd900001e00 tx=0x7fd90001ca60 comp rx=0 tx=0).stop 2024-09-17T03:31:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90d600640 1 -- 172.21.15.46:0/2160634437 reap_dead start 2024-09-17T03:31:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90e000640 1 -- 172.21.15.46:0/2160634437 shutdown_connections 2024-09-17T03:31:52.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90e000640 1 -- 172.21.15.46:0/2160634437 >> 172.21.15.46:0/2160634437 conn(0x7fd90806e4a0 msgr2=0x7fd90806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:52.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90e000640 1 -- 172.21.15.46:0/2160634437 shutdown_connections 2024-09-17T03:31:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.584+0000 7fd90e000640 1 -- 172.21.15.46:0/2160634437 wait complete. 2024-09-17T03:31:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.588+0000 7fd90e000640 1 Processor -- start 2024-09-17T03:31:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.592+0000 7fd90e000640 1 -- start start 2024-09-17T03:31:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90e000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd908201ff0 0x7fd9080ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd908159ec0 0x7fd90815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 0x7fd9082060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080d0330 con 0x7fd908159ec0 2024-09-17T03:31:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90e000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd90806fe50 con 0x7fd908205ce0 2024-09-17T03:31:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90e000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd9080d04b0 con 0x7fd908201ff0 2024-09-17T03:31:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd908201ff0 0x7fd9080ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 0x7fd9082060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 0x7fd9082060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43034/0 (socket says 172.21.15.46:43034) 2024-09-17T03:31:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 -- 172.21.15.46:0/819130349 learned_addr learned my addr 172.21.15.46:0/819130349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd907e00640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd908159ec0 0x7fd90815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 -- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd908201ff0 msgr2=0x7fd9080ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd908201ff0 0x7fd9080ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 -- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd908159ec0 msgr2=0x7fd90815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd908159ec0 0x7fd90815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.596+0000 7fd90d600640 1 -- 172.21.15.46:0/819130349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9080cfa80 con 0x7fd908205ce0 2024-09-17T03:31:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.600+0000 7fd90d600640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 0x7fd9082060f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd900005040 tx=0x7fd9000a5e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.600+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009730 con 0x7fd908205ce0 2024-09-17T03:31:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.600+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9081593e0 con 0x7fd908205ce0 2024-09-17T03:31:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.600+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd908143870 con 0x7fd908205ce0 2024-09-17T03:31:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.600+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9000b4030 con 0x7fd908205ce0 2024-09-17T03:31:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.604+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009730 con 0x7fd908205ce0 2024-09-17T03:31:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.608+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd908075090 con 0x7fd908205ce0 2024-09-17T03:31:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.616+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fd900003ba0 con 0x7fd908205ce0 2024-09-17T03:31:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.616+0000 7fd904c00640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd8e8074930 0x7fd8e8076e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.616+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fd90012ca60 con 0x7fd908205ce0 2024-09-17T03:31:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.620+0000 7fd90cc00640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd8e8074930 0x7fd8e8076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.620+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd9000168d0 con 0x7fd908205ce0 2024-09-17T03:31:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:52.636+0000 7fd90cc00640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd8e8074930 0x7fd8e8076e40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001870 tx=0x7fd8f80017a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:53.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.236+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd9080d1010 con 0x7fd908205ce0 2024-09-17T03:31:53.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.240+0000 7fd904c00640 1 -- 172.21.15.46:0/819130349 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2445 (secure 0 0 0) 0x7fd9000f90e0 con 0x7fd908205ce0 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:31:53.247 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 671, 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:31:53.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 89, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:31:53.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 7391, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 30797896648, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 61576007680, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 711518105600, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 548396831, 2024-09-17T03:31:53.250 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 551968787, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 132, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 152 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 56, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:30:30:404049+0000", 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:31:53.251 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 6, 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246888 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:31:53.252 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 16, 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:31:25.427659+0000", 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:31:53.253 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 66.081% (36s)\n [==================..........] (remaining: 18s)", 2024-09-17T03:31:53.254 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.66080999374389648, 2024-09-17T03:31:53.254 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-total-clones": { 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Total 9 clones - average progress is 37.934% (28s)\n [==========..................] (remaining: 46s)", 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.37933999300003052, 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:53.256 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:31:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd8e8074930 msgr2=0x7fd8e8076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd8e8074930 0x7fd8e8076e40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001870 tx=0x7fd8f80017a0 comp rx=0 tx=0).stop 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 msgr2=0x7fd9082060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 0x7fd9082060f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd900005040 tx=0x7fd9000a5e00 comp rx=0 tx=0).stop 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 shutdown_connections 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd8e8074930 0x7fd8e8076e40 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001870 tx=0x7fd8f80017a0 comp rx=0 tx=0).stop 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd908205ce0 0x7fd9082060f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd908159ec0 0x7fd90815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 --2- 172.21.15.46:0/819130349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd908201ff0 0x7fd9080ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.248+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 >> 172.21.15.46:0/819130349 conn(0x7fd90806e4a0 msgr2=0x7fd908150720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.252+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 shutdown_connections 2024-09-17T03:31:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:53.252+0000 7fd90e000640 1 -- 172.21.15.46:0/819130349 wait complete. 2024-09-17T03:31:53.380 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 66.081% (36s)\n [==================..........] (remaining: 18s)', 'progress': 0.6608099937438965, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 9 clones - average progress is 37.934% (28s)\n [==========..................] (remaining: 46s)', 'progress': 0.3793399930000305, 'add_to_ceph_s': True}} 2024-09-17T03:31:53.380 INFO:tasks.cephfs.test_volumes:on_p = 0.31185001134872437 new_on_p = 0.6608099937438965 2024-09-17T03:31:53.380 INFO:tasks.cephfs.test_volumes:onpen_p = 0.3114599883556366 new_onpen_p = 0.3793399930000305 2024-09-17T03:31:53.380 INFO:tasks.cephfs.test_volumes:onpen_count = None new_onpen_count = None 2024-09-17T03:31:53.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:31:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.504+0000 7fed6f200640 1 -- 172.21.15.46:0/1932919029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed500039c0 msgr2=0x7fed50003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.504+0000 7fed6f200640 1 --2- 172.21.15.46:0/1932919029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed500039c0 0x7fed50003e10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00c710 tx=0x7fed5c01c670 comp rx=0 tx=0).stop 2024-09-17T03:31:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.504+0000 7fed6f200640 1 -- 172.21.15.46:0/1932919029 shutdown_connections 2024-09-17T03:31:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.504+0000 7fed6f200640 1 --2- 172.21.15.46:0/1932919029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed500039c0 0x7fed50003e10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.504+0000 7fed6f200640 1 -- 172.21.15.46:0/1932919029 >> 172.21.15.46:0/1932919029 conn(0x7fed640513f0 msgr2=0x7fed64051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:54.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.504+0000 7fed6f200640 1 -- 172.21.15.46:0/1932919029 shutdown_connections 2024-09-17T03:31:54.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.512+0000 7fed6f200640 1 -- 172.21.15.46:0/1932919029 wait complete. 2024-09-17T03:31:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.520+0000 7fed6f200640 1 Processor -- start 2024-09-17T03:31:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.520+0000 7fed6f200640 1 -- start start 2024-09-17T03:31:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6f200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 0x7fed64189e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed64183bf0 0x7fed640e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed64184000 0x7fed640e1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed64038d00 con 0x7fed64184000 2024-09-17T03:31:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed64038b80 con 0x7fed64183bf0 2024-09-17T03:31:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6f200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fed64038e80 con 0x7fed500039c0 2024-09-17T03:31:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 0x7fed64189e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 0x7fed64189e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:36222/0 (socket says 172.21.15.46:36222) 2024-09-17T03:31:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6de00640 1 -- 172.21.15.46:0/800575960 learned_addr learned my addr 172.21.15.46:0/800575960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:54.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6d400640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed64183bf0 0x7fed640e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:54.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.524+0000 7fed6e800640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed64184000 0x7fed640e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.528+0000 7fed6de00640 1 -- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed64183bf0 msgr2=0x7fed640e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:54.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.528+0000 7fed6de00640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed64183bf0 0x7fed640e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.528+0000 7fed6de00640 1 -- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed64184000 msgr2=0x7fed640e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.528+0000 7fed6de00640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed64184000 0x7fed640e1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.528+0000 7fed6de00640 1 -- 172.21.15.46:0/800575960 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed64049270 con 0x7fed500039c0 2024-09-17T03:31:54.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.528+0000 7fed6de00640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 0x7fed64189e30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000f00 tx=0x7fed5c003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.532+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0ad040 con 0x7fed500039c0 2024-09-17T03:31:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.532+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed5c006190 con 0x7fed500039c0 2024-09-17T03:31:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.532+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00c2e0 con 0x7fed500039c0 2024-09-17T03:31:54.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.532+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed640489d0 con 0x7fed500039c0 2024-09-17T03:31:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.532+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed64048f90 con 0x7fed500039c0 2024-09-17T03:31:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.532+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed64039370 con 0x7fed500039c0 2024-09-17T03:31:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.540+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7fed5c0b9020 con 0x7fed500039c0 2024-09-17T03:31:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.540+0000 7fed62000640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed4c074790 0x7fed4c076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.540+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fed5c12cf90 con 0x7fed500039c0 2024-09-17T03:31:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.544+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fed5c0c0020 con 0x7fed500039c0 2024-09-17T03:31:54.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.572+0000 7fed6d400640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed4c074790 0x7fed4c076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:54.596+0000 7fed6d400640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed4c074790 0x7fed4c076ca0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fed640e1f10 tx=0x7fed5800b310 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:55.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.232+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fed640e1f40 con 0x7fed500039c0 2024-09-17T03:31:55.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.236+0000 7fed62000640 1 -- 172.21.15.46:0/800575960 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2446 (secure 0 0 0) 0x7fed5c0163b0 con 0x7fed500039c0 2024-09-17T03:31:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:31:55.238 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:31:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:31:55.238 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:31:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 673, 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 89, 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:31:55.240 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:31:55.241 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 8063, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 33612275877, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 67425964032, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 705668149248, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 724404224, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 723763080, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 175, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 191 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 56, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:30:30:404049+0000", 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6, 2024-09-17T03:31:55.242 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 6, 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "c", 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 246888 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:55.243 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 16, 2024-09-17T03:31:55.244 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:31:25.427659+0000", 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": { 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "4 ongoing clones - average progress is 39.323% (46s)\n [===========.................] (remaining: 71s)", 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.39322999119758606, 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr-vol-total-clones": { 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Total 9 clones - average progress is 25.637% (38s)\n [=======.....................] (remaining: 110s)", 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": 0.25637000799179077, 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: "add_to_ceph_s": true 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:55.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:31:55.246 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.244+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed4c074790 msgr2=0x7fed4c076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed4c074790 0x7fed4c076ca0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fed640e1f10 tx=0x7fed5800b310 comp rx=0 tx=0).stop 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 msgr2=0x7fed64189e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 0x7fed64189e30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000f00 tx=0x7fed5c003ca0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 shutdown_connections 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed4c074790 0x7fed4c076ca0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed64184000 0x7fed640e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed64183bf0 0x7fed640e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 --2- 172.21.15.46:0/800575960 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed500039c0 0x7fed64189e30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 >> 172.21.15.46:0/800575960 conn(0x7fed640513f0 msgr2=0x7fed64006180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 shutdown_connections 2024-09-17T03:31:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.248+0000 7fed6f200640 1 -- 172.21.15.46:0/800575960 wait complete. 2024-09-17T03:31:55.323 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 39.323% (46s)\n [===========.................] (remaining: 71s)', 'progress': 0.39322999119758606, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 9 clones - average progress is 25.637% (38s)\n [=======.....................] (remaining: 110s)', 'progress': 0.25637000799179077, 'add_to_ceph_s': True}} 2024-09-17T03:31:55.323 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-17T03:31:55.323 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.25637000799179077; onpen_p = 0.3114599883556366 2024-09-17T03:31:55.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone52350 2024-09-17T03:31:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 -- 172.21.15.46:0/3019707626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cdd80 msgr2=0x7f05500b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 --2- 172.21.15.46:0/3019707626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cdd80 0x7f05500b3880 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0548009bc0 tx=0x7f054801eb30 comp rx=0 tx=0).stop 2024-09-17T03:31:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 -- 172.21.15.46:0/3019707626 shutdown_connections 2024-09-17T03:31:55.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 --2- 172.21.15.46:0/3019707626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cdd80 0x7f05500b3880 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 --2- 172.21.15.46:0/3019707626 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f05500cd970 0x7f05500b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 -- 172.21.15.46:0/3019707626 >> 172.21.15.46:0/3019707626 conn(0x7f05500b2040 msgr2=0x7f05500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.864+0000 7f0557400640 1 -- 172.21.15.46:0/3019707626 shutdown_connections 2024-09-17T03:31:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.868+0000 7f0557400640 1 -- 172.21.15.46:0/3019707626 wait complete. 2024-09-17T03:31:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.872+0000 7f0557400640 1 Processor -- start 2024-09-17T03:31:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.872+0000 7f0557400640 1 -- start start 2024-09-17T03:31:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.876+0000 7f0557400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cd970 0x7f055014f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.876+0000 7f0557400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05500cdd80 0x7f055013c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.876+0000 7f0554a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cd970 0x7f055014f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0554a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cd970 0x7f055014f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43074/0 (socket says 172.21.15.46:43074) 2024-09-17T03:31:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f054fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05500cdd80 0x7f055013c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f054fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05500cdd80 0x7f055013c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57044/0 (socket says 172.21.15.46:57044) 2024-09-17T03:31:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0554a00640 1 -- 172.21.15.46:0/2358319704 learned_addr learned my addr 172.21.15.46:0/2358319704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0557400640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f055015aa50 0x7f055013c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05501680e0 con 0x7f05500cdd80 2024-09-17T03:31:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0550167f60 con 0x7f05500cd970 2024-09-17T03:31:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0550168260 con 0x7f055015aa50 2024-09-17T03:31:55.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.880+0000 7f0555400640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f055015aa50 0x7f055013c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:55.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.884+0000 7f054fe00640 1 -- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f055015aa50 msgr2=0x7f055013c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.884+0000 7f054fe00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f055015aa50 0x7f055013c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.884+0000 7f054fe00640 1 -- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cd970 msgr2=0x7f055014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.884+0000 7f054fe00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cd970 0x7f055014f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.884+0000 7f054fe00640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055014bf90 con 0x7f05500cdd80 2024-09-17T03:31:55.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.884+0000 7f0554a00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05500cd970 0x7f055014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:31:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.892+0000 7f054fe00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05500cdd80 0x7f055013c250 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f053c009a80 tx=0x7f053c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:55.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.892+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c01e070 con 0x7f05500cdd80 2024-09-17T03:31:55.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.892+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f053c00bb70 con 0x7f05500cdd80 2024-09-17T03:31:55.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.892+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c019410 con 0x7f05500cdd80 2024-09-17T03:31:55.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.892+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0550150510 con 0x7f05500cdd80 2024-09-17T03:31:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.892+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0550150ad0 con 0x7f05500cdd80 2024-09-17T03:31:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.900+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052c0052a0 con 0x7f05500cdd80 2024-09-17T03:31:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.920+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 38) ==== 99172+0+0 (secure 0 0 0) 0x7f053c021070 con 0x7f05500cdd80 2024-09-17T03:31:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.924+0000 7f054cc00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05300747b0 0x7f0530076cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.924+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f053c097280 con 0x7f05500cdd80 2024-09-17T03:31:55.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.936+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f055014bf90 con 0x7f05500cdd80 2024-09-17T03:31:55.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.964+0000 7f0554a00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05300747b0 0x7f0530076cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:55.992+0000 7f0554a00640 1 --2- 172.21.15.46:0/2358319704 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f05300747b0 0x7f0530076cc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0544009950 tx=0x7f0544008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:56.608+0000 7f0557400640 1 -- 172.21.15.46:0/2358319704 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone52350", "target": ["mon-mgr", ""]}) -- 0x7f052c000bd0 con 0x7f05300747b0 2024-09-17T03:31:57.097 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:57.096+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:57.098 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:57.096+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:57.098 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:57.096+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:57.098 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:57.096+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:57.098 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:57.096+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:31:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:57.864+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f053c05f5e0 con 0x7f05500cdd80 2024-09-17T03:31:58.192 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:58.192+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:31:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.196+0000 7f054cc00640 1 -- 172.21.15.46:0/2358319704 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f052c000bd0 con 0x7f05300747b0 2024-09-17T03:31:58.206 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:31:58.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:31:58.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone79881 2024-09-17T03:31:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.756+0000 7f9bf9800640 1 -- 172.21.15.46:0/1216656936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 msgr2=0x7f9bf406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.756+0000 7f9bf9800640 1 --2- 172.21.15.46:0/1216656936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 0x7f9bf406f020 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009bc0 tx=0x7f9bec01cb70 comp rx=0 tx=0).stop 2024-09-17T03:31:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.760+0000 7f9bf9800640 1 -- 172.21.15.46:0/1216656936 shutdown_connections 2024-09-17T03:31:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.760+0000 7f9bf9800640 1 --2- 172.21.15.46:0/1216656936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 0x7f9bf406f020 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.760+0000 7f9bf9800640 1 --2- 172.21.15.46:0/1216656936 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bf415f0f0 0x7f9bf415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.760+0000 7f9bf9800640 1 -- 172.21.15.46:0/1216656936 >> 172.21.15.46:0/1216656936 conn(0x7f9bf40d06e0 msgr2=0x7f9bf40d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:31:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.768+0000 7f9bf9800640 1 -- 172.21.15.46:0/1216656936 shutdown_connections 2024-09-17T03:31:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.768+0000 7f9bf9800640 1 -- 172.21.15.46:0/1216656936 wait complete. 2024-09-17T03:31:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.780+0000 7f9bf9800640 1 Processor -- start 2024-09-17T03:31:58.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.784+0000 7f9bf9800640 1 -- start start 2024-09-17T03:31:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 0x7f9bf414ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf9800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf40ae8e0 0x7f9bf4155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf9800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bf415f0f0 0x7f9bf414d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf40ae8e0 0x7f9bf4155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf40ae8e0 0x7f9bf4155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43094/0 (socket says 172.21.15.46:43094) 2024-09-17T03:31:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf2000640 1 -- 172.21.15.46:0/3898634975 learned_addr learned my addr 172.21.15.46:0/3898634975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:31:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.788+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf416be70 con 0x7f9bf406ec10 2024-09-17T03:31:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf416bcf0 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf416bff0 con 0x7f9bf415f0f0 2024-09-17T03:31:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf2a00640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 0x7f9bf414ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf3400640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bf415f0f0 0x7f9bf414d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf2000640 1 -- 172.21.15.46:0/3898634975 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bf415f0f0 msgr2=0x7f9bf414d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf2000640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bf415f0f0 0x7f9bf414d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf2000640 1 -- 172.21.15.46:0/3898634975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 msgr2=0x7f9bf414ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:31:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf2000640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bf406ec10 0x7f9bf414ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:31:58.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.792+0000 7f9bf2000640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf414cbe0 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.796+0000 7f9bf2000640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf40ae8e0 0x7f9bf4155250 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9be400d780 tx=0x7f9be400dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.796+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4017070 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.796+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9be400de90 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.796+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4010460 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.796+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf414cec0 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.796+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf414b580 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.808+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf406f020 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.812+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f9be401a070 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.812+0000 7f9beaa00640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bd40748c0 0x7f9bd4076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:31:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.812+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f9be40934b0 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.820+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9be405fba0 con 0x7f9bf40ae8e0 2024-09-17T03:31:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.824+0000 7f9bf2a00640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bd40748c0 0x7f9bd4076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:31:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:58.852+0000 7f9bf2a00640 1 --2- 172.21.15.46:0/3898634975 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9bd40748c0 0x7f9bd4076dd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9be000b810 tx=0x7f9be0008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:31:59.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:59.356+0000 7f9bf9800640 1 -- 172.21.15.46:0/3898634975 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone79881", "target": ["mon-mgr", ""]}) -- 0x7f9bf406b540 con 0x7f9bd40748c0 2024-09-17T03:31:59.858 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:31:59.856+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:31:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:31:59.856+0000 7f9beaa00640 1 -- 172.21.15.46:0/3898634975 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f9be000b400 con 0x7f9bd40748c0 2024-09-17T03:31:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:31:59.877 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:31:59.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone50292 2024-09-17T03:32:00.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:32:00.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:32:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 -- 172.21.15.46:0/3112392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015e8c0 msgr2=0x7fb72015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 --2- 172.21.15.46:0/3112392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015e8c0 0x7fb72015ecd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009900 tx=0x7fb70c01c790 comp rx=0 tx=0).stop 2024-09-17T03:32:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 -- 172.21.15.46:0/3112392006 shutdown_connections 2024-09-17T03:32:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 --2- 172.21.15.46:0/3112392006 >> v2:172.21.15.46:3300/0 conn(0x7fb72015b2d0 0x7fb72015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 --2- 172.21.15.46:0/3112392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015e8c0 0x7fb72015ecd0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009900 tx=0x7fb70c01c790 comp rx=0 tx=0).stop 2024-09-17T03:32:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 -- 172.21.15.46:0/3112392006 >> 172.21.15.46:0/3112392006 conn(0x7fb7200b2040 msgr2=0x7fb7200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.356+0000 7fb727c00640 1 -- 172.21.15.46:0/3112392006 shutdown_connections 2024-09-17T03:32:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.360+0000 7fb727c00640 1 -- 172.21.15.46:0/3112392006 wait complete. 2024-09-17T03:32:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.360+0000 7fb727c00640 1 Processor -- start 2024-09-17T03:32:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 -- start start 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72015b2d0 0x7fb7201389c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015bc20 0x7fb720143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144150 0x7fb7201576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7200b3810 con 0x7fb720144150 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7200b3690 con 0x7fb72015b2d0 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.364+0000 7fb727c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb7200b3990 con 0x7fb72015bc20 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb71fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015bc20 0x7fb720143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb71fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015bc20 0x7fb720143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:36300/0 (socket says 172.21.15.46:36300) 2024-09-17T03:32:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb71fe00640 1 -- 172.21.15.46:0/2978844930 learned_addr learned my addr 172.21.15.46:0/2978844930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72015b2d0 0x7fb7201389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 -- 172.21.15.46:0/2978844930 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015bc20 msgr2=0x7fb720143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb72015bc20 0x7fb720143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 -- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144150 msgr2=0x7fb7201576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725c00640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144150 0x7fb7201576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144150 0x7fb7201576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 -- 172.21.15.46:0/2978844930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7200aa600 con 0x7fb72015b2d0 2024-09-17T03:32:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725c00640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144150 0x7fb7201576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:32:00.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb725200640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72015b2d0 0x7fb7201389c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0ac000 tx=0x7fb70c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:00.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0b2040 con 0x7fb72015b2d0 2024-09-17T03:32:00.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb70c0036f0 con 0x7fb72015b2d0 2024-09-17T03:32:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c003b20 con 0x7fb72015b2d0 2024-09-17T03:32:00.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.368+0000 7fb727c00640 1 -- 172.21.15.46:0/2978844930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7200aa8e0 con 0x7fb72015b2d0 2024-09-17T03:32:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.372+0000 7fb727c00640 1 -- 172.21.15.46:0/2978844930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb720138f00 con 0x7fb72015b2d0 2024-09-17T03:32:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.372+0000 7fb727c00640 1 -- 172.21.15.46:0/2978844930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72015ecd0 con 0x7fb72015b2d0 2024-09-17T03:32:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.376+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fb70c0bd060 con 0x7fb72015b2d0 2024-09-17T03:32:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.376+0000 7fb71cc00640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb700074840 0x7fb700076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.376+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fb70c0f4760 con 0x7fb72015b2d0 2024-09-17T03:32:00.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.384+0000 7fb71fe00640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb700074840 0x7fb700076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:00.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.384+0000 7fb71fe00640 1 --2- 172.21.15.46:0/2978844930 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb700074840 0x7fb700076d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb72013e2b0 tx=0x7fb710005f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:00.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.392+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb70c016410 con 0x7fb72015b2d0 2024-09-17T03:32:00.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:00.920+0000 7fb727c00640 1 -- 172.21.15.46:0/2978844930 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone50292", "target": ["mon-mgr", ""]}) -- 0x7fb7201447d0 con 0x7fb700074840 2024-09-17T03:32:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.148+0000 7fb71cc00640 1 -- 172.21.15.46:0/2978844930 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fb7201447d0 con 0x7fb700074840 2024-09-17T03:32:01.148 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:01.144+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:01.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:01.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone13923 2024-09-17T03:32:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.632+0000 7f9977200640 1 -- 172.21.15.46:0/351572003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997815f5c0 msgr2=0x7f997815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.632+0000 7f9977200640 1 --2- 172.21.15.46:0/351572003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997815f5c0 0x7f997815f9d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9970009840 tx=0x7f997001c7d0 comp rx=0 tx=0).stop 2024-09-17T03:32:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.636+0000 7f9977200640 1 -- 172.21.15.46:0/351572003 shutdown_connections 2024-09-17T03:32:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.636+0000 7f9977200640 1 --2- 172.21.15.46:0/351572003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997815f5c0 0x7f997815f9d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.636+0000 7f9977200640 1 --2- 172.21.15.46:0/351572003 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f997806dbc0 0x7f997806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.636+0000 7f9977200640 1 -- 172.21.15.46:0/351572003 >> 172.21.15.46:0/351572003 conn(0x7f99780b41f0 msgr2=0x7f99780b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.636+0000 7f9977200640 1 -- 172.21.15.46:0/351572003 shutdown_connections 2024-09-17T03:32:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.640+0000 7f9977200640 1 -- 172.21.15.46:0/351572003 wait complete. 2024-09-17T03:32:01.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.648+0000 7f9977200640 1 Processor -- start 2024-09-17T03:32:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.648+0000 7f9977200640 1 -- start start 2024-09-17T03:32:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9977200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997806dbc0 0x7f9978168620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9977200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99780d1140 0x7f99780af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9977200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f997815f5c0 0x7f99780cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9977200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9978166fe0 con 0x7f997806dbc0 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9977200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9978166e60 con 0x7f997815f5c0 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9977200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9978167160 con 0x7f99780d1140 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997806dbc0 0x7f9978168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:01.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997806dbc0 0x7f9978168620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57126/0 (socket says 172.21.15.46:57126) 2024-09-17T03:32:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 -- 172.21.15.46:0/49593296 learned_addr learned my addr 172.21.15.46:0/49593296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f996d400640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99780d1140 0x7f99780af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:01.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9976800640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f997815f5c0 0x7f99780cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 -- 172.21.15.46:0/49593296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99780d1140 msgr2=0x7f99780af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f99780d1140 0x7f99780af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 -- 172.21.15.46:0/49593296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f997815f5c0 msgr2=0x7f99780cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f997815f5c0 0x7f99780cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 -- 172.21.15.46:0/49593296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9978070dc0 con 0x7f997806dbc0 2024-09-17T03:32:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f9975e00640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f997806dbc0 0x7f9978168620 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f996800cbf0 tx=0x7f996800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968016020 con 0x7f997806dbc0 2024-09-17T03:32:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.652+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996800ab40 con 0x7f997806dbc0 2024-09-17T03:32:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.656+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968012420 con 0x7f997806dbc0 2024-09-17T03:32:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.656+0000 7f9977200640 1 -- 172.21.15.46:0/49593296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9978167580 con 0x7f997806dbc0 2024-09-17T03:32:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.656+0000 7f9977200640 1 -- 172.21.15.46:0/49593296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9978167b40 con 0x7f997806dbc0 2024-09-17T03:32:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.660+0000 7f9967400640 1 -- 172.21.15.46:0/49593296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997806dfd0 con 0x7f997806dbc0 2024-09-17T03:32:01.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.664+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f9968019020 con 0x7f997806dbc0 2024-09-17T03:32:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.668+0000 7f996ea00640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99540748f0 0x7f9954076e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.668+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f9968092fe0 con 0x7f997806dbc0 2024-09-17T03:32:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.676+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9968020070 con 0x7f997806dbc0 2024-09-17T03:32:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.688+0000 7f996d400640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99540748f0 0x7f9954076e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:01.716+0000 7f996d400640 1 --2- 172.21.15.46:0/49593296 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f99540748f0 0x7f9954076e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f99781f7960 tx=0x7f9960008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:02.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:02.340+0000 7f9967400640 1 -- 172.21.15.46:0/49593296 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone13923", "target": ["mon-mgr", ""]}) -- 0x7f997806b540 con 0x7f99540748f0 2024-09-17T03:32:02.621 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:02.620+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:02.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:02.620+0000 7f996ea00640 1 -- 172.21.15.46:0/49593296 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f996000b420 con 0x7f99540748f0 2024-09-17T03:32:02.623 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:02.655 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:02.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone77144 2024-09-17T03:32:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1bb400640 1 -- 172.21.15.46:0/1613399255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80b2070 con 0x7fd1bc0b5fd0 2024-09-17T03:32:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1c2000640 1 -- 172.21.15.46:0/1613399255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 msgr2=0x7fd1bc15b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1c2000640 1 --2- 172.21.15.46:0/1613399255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc15b7c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009900 tx=0x7fd1a801c790 comp rx=0 tx=0).stop 2024-09-17T03:32:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1c2000640 1 -- 172.21.15.46:0/1613399255 shutdown_connections 2024-09-17T03:32:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1c2000640 1 --2- 172.21.15.46:0/1613399255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc15b7c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1c2000640 1 -- 172.21.15.46:0/1613399255 >> 172.21.15.46:0/1613399255 conn(0x7fd1bc06ee20 msgr2=0x7fd1bc0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.096+0000 7fd1c2000640 1 -- 172.21.15.46:0/1613399255 shutdown_connections 2024-09-17T03:32:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.100+0000 7fd1c2000640 1 -- 172.21.15.46:0/1613399255 wait complete. 2024-09-17T03:32:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.104+0000 7fd1c2000640 1 Processor -- start 2024-09-17T03:32:03.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.108+0000 7fd1c2000640 1 -- start start 2024-09-17T03:32:03.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.108+0000 7fd1c2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1bc0b5bc0 0x7fd1bc153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:03.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.108+0000 7fd1c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc165b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1bc0b5bc0 0x7fd1bc153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1bc0b5bc0 0x7fd1bc153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:55032/0 (socket says 172.21.15.46:55032) 2024-09-17T03:32:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc165b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52688/0 (socket says 172.21.15.46:52688) 2024-09-17T03:32:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c0c00640 1 -- 172.21.15.46:0/794289007 learned_addr learned my addr 172.21.15.46:0/794289007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c2000640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1bc150f60 0x7fd1bc1f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc1690e0 con 0x7fd1bc0b5fd0 2024-09-17T03:32:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc168f60 con 0x7fd1bc0b5bc0 2024-09-17T03:32:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd1bc169260 con 0x7fd1bc150f60 2024-09-17T03:32:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c1600640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1bc150f60 0x7fd1bc1f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:03.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c1600640 1 -- 172.21.15.46:0/794289007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1bc0b5bc0 msgr2=0x7fd1bc153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:03.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c1600640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1bc0b5bc0 0x7fd1bc153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:03.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c1600640 1 -- 172.21.15.46:0/794289007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 msgr2=0x7fd1bc165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c1600640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.112+0000 7fd1c1600640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc152b50 con 0x7fd1bc150f60 2024-09-17T03:32:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1bbe00640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1bc0b5fd0 0x7fd1bc165b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1c1600640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd1bc150f60 0x7fd1bc1f4720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80ac000 tx=0x7fd1a80058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80b2040 con 0x7fd1bc150f60 2024-09-17T03:32:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc157590 con 0x7fd1bc150f60 2024-09-17T03:32:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc157b50 con 0x7fd1bc150f60 2024-09-17T03:32:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1a8004240 con 0x7fd1bc150f60 2024-09-17T03:32:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.116+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8003600 con 0x7fd1bc150f60 2024-09-17T03:32:03.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.120+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1940052a0 con 0x7fd1bc150f60 2024-09-17T03:32:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.120+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fd1a80a5850 con 0x7fd1bc150f60 2024-09-17T03:32:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.124+0000 7fd1b8c00640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd19c074840 0x7fd19c076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.124+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fd1a812cdb0 con 0x7fd1bc150f60 2024-09-17T03:32:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.136+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd1a80164d0 con 0x7fd1bc150f60 2024-09-17T03:32:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.136+0000 7fd1c0c00640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd19c074840 0x7fd19c076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.188+0000 7fd1c0c00640 1 --2- 172.21.15.46:0/794289007 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd19c074840 0x7fd19c076d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fd0 tx=0x7fd1b4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:03.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:03.628+0000 7fd1c2000640 1 -- 172.21.15.46:0/794289007 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone77144", "target": ["mon-mgr", ""]}) -- 0x7fd194000bd0 con 0x7fd19c074840 2024-09-17T03:32:03.988 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:03.988+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.020+0000 7fd1b8c00640 1 -- 172.21.15.46:0/794289007 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fd194000bd0 con 0x7fd19c074840 2024-09-17T03:32:04.026 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:04.108 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:04.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone59755 2024-09-17T03:32:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 -- 172.21.15.46:0/1897583029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006cdf0 msgr2=0x7f871006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 --2- 172.21.15.46:0/1897583029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006cdf0 0x7f871006d200 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f87100ce170 tx=0x7f870001c830 comp rx=0 tx=0).stop 2024-09-17T03:32:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 -- 172.21.15.46:0/1897583029 shutdown_connections 2024-09-17T03:32:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 --2- 172.21.15.46:0/1897583029 >> v2:172.21.15.99:3300/0 conn(0x7f871006fc90 0x7f87100700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 --2- 172.21.15.46:0/1897583029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006cdf0 0x7f871006d200 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 -- 172.21.15.46:0/1897583029 >> 172.21.15.46:0/1897583029 conn(0x7f87100b35f0 msgr2=0x7f87100b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.648+0000 7f8717800640 1 -- 172.21.15.46:0/1897583029 shutdown_connections 2024-09-17T03:32:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.652+0000 7f8717800640 1 -- 172.21.15.46:0/1897583029 wait complete. 2024-09-17T03:32:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.660+0000 7f8717800640 1 Processor -- start 2024-09-17T03:32:04.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.660+0000 7f8717800640 1 -- start start 2024-09-17T03:32:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.676+0000 7f8717800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f871006cdf0 0x7f8710148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.676+0000 7f8717800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006fc90 0x7f871014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.676+0000 7f8717800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710159130 0x7f8710150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.676+0000 7f8717800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8710069900 con 0x7f871006fc90 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.676+0000 7f8717800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8710069780 con 0x7f8710159130 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.676+0000 7f8717800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8710069a80 con 0x7f871006cdf0 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f871006cdf0 0x7f8710148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f871006cdf0 0x7f8710148020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:37108/0 (socket says 172.21.15.46:37108) 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 -- 172.21.15.46:0/1519803252 learned_addr learned my addr 172.21.15.46:0/1519803252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8715800640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710159130 0x7f8710150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:04.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f870fe00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006fc90 0x7f871014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:04.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 -- 172.21.15.46:0/1519803252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710159130 msgr2=0x7f8710150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:04.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8710159130 0x7f8710150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:04.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 -- 172.21.15.46:0/1519803252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006fc90 msgr2=0x7f871014da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:04.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871006fc90 0x7f871014da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:04.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 -- 172.21.15.46:0/1519803252 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871014cd70 con 0x7f871006cdf0 2024-09-17T03:32:04.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8714e00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f871006cdf0 0x7f8710148020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00b3e0 tx=0x7f86fc00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:04.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc015020 con 0x7f871006cdf0 2024-09-17T03:32:04.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86fc011920 con 0x7f871006cdf0 2024-09-17T03:32:04.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8717800640 1 -- 172.21.15.46:0/1519803252 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8710147540 con 0x7f871006cdf0 2024-09-17T03:32:04.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.680+0000 7f8717800640 1 -- 172.21.15.46:0/1519803252 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f871014c800 con 0x7f871006cdf0 2024-09-17T03:32:04.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.684+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc015020 con 0x7f871006cdf0 2024-09-17T03:32:04.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.684+0000 7f8706000640 1 -- 172.21.15.46:0/1519803252 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871006d200 con 0x7f871006cdf0 2024-09-17T03:32:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.688+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f86fc018020 con 0x7f871006cdf0 2024-09-17T03:32:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.692+0000 7f870cc00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f86f00749f0 0x7f86f0076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.692+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f86fc092df0 con 0x7f871006cdf0 2024-09-17T03:32:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.692+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f86fc01f020 con 0x7f871006cdf0 2024-09-17T03:32:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.716+0000 7f870fe00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f86f00749f0 0x7f86f0076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:04.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:04.772+0000 7f870fe00640 1 --2- 172.21.15.46:0/1519803252 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f86f00749f0 0x7f86f0076f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f87000af000 tx=0x7f870001b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:05.312+0000 7f8706000640 1 -- 172.21.15.46:0/1519803252 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone59755", "target": ["mon-mgr", ""]}) -- 0x7f871006b540 con 0x7f86f00749f0 2024-09-17T03:32:05.773 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:05.772+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:05.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:05.796+0000 7f870cc00640 1 -- 172.21.15.46:0/1519803252 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f8700004020 con 0x7f86f00749f0 2024-09-17T03:32:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:05.866 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:05.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone18656 2024-09-17T03:32:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.312+0000 7ffaaf800640 1 -- 172.21.15.46:0/798351112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa81553d0 msgr2=0x7ffaa81557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaa7400640 1 -- 172.21.15.46:0/798351112 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00b2070 con 0x7ffaa81553d0 2024-09-17T03:32:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaaf800640 1 --2- 172.21.15.46:0/798351112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa81553d0 0x7ffaa81557e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ffaa806edb0 tx=0x7ffaa001cb10 comp rx=0 tx=0).stop 2024-09-17T03:32:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaaf800640 1 -- 172.21.15.46:0/798351112 shutdown_connections 2024-09-17T03:32:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaaf800640 1 --2- 172.21.15.46:0/798351112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa81553d0 0x7ffaa81557e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaaf800640 1 -- 172.21.15.46:0/798351112 >> 172.21.15.46:0/798351112 conn(0x7ffaa806e1d0 msgr2=0x7ffaa806e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaaf800640 1 -- 172.21.15.46:0/798351112 shutdown_connections 2024-09-17T03:32:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.320+0000 7ffaaf800640 1 -- 172.21.15.46:0/798351112 wait complete. 2024-09-17T03:32:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.324+0000 7ffaaf800640 1 Processor -- start 2024-09-17T03:32:06.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.324+0000 7ffaaf800640 1 -- start start 2024-09-17T03:32:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaaf800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa80cd7a0 0x7ffaa81febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaaf800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa80cf210 0x7ffaa81ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:06.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaaf800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffaa81553d0 0x7ffaa81ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaaf800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa80cc6b0 con 0x7ffaa80cf210 2024-09-17T03:32:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaaf800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa80cc530 con 0x7ffaa80cd7a0 2024-09-17T03:32:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaaf800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa80cc830 con 0x7ffaa81553d0 2024-09-17T03:32:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffaa81553d0 0x7ffaa81ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffaa81553d0 0x7ffaa81ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:37118/0 (socket says 172.21.15.46:37118) 2024-09-17T03:32:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 -- 172.21.15.46:0/1350871478 learned_addr learned my addr 172.21.15.46:0/1350871478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaace00640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa80cd7a0 0x7ffaa81febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 -- 172.21.15.46:0/1350871478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa80cd7a0 msgr2=0x7ffaa81febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaa80cd7a0 0x7ffaa81febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 -- 172.21.15.46:0/1350871478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa80cf210 msgr2=0x7ffaa81ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:06.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa80cf210 0x7ffaa81ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:06.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.328+0000 7ffaad800640 1 -- 172.21.15.46:0/1350871478 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa8150b50 con 0x7ffaa81553d0 2024-09-17T03:32:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.332+0000 7ffaad800640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffaa81553d0 0x7ffaa81ff640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00985f0 tx=0x7ffaa0009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.332+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00b2040 con 0x7ffaa81553d0 2024-09-17T03:32:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.332+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaa0004b40 con 0x7ffaa81553d0 2024-09-17T03:32:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.332+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0004040 con 0x7ffaa81553d0 2024-09-17T03:32:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.332+0000 7ffaaf800640 1 -- 172.21.15.46:0/1350871478 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa8153dc0 con 0x7ffaa81553d0 2024-09-17T03:32:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.332+0000 7ffaaf800640 1 -- 172.21.15.46:0/1350871478 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa81fe0e0 con 0x7ffaa81553d0 2024-09-17T03:32:06.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.336+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7ffaa00bd020 con 0x7ffaa81553d0 2024-09-17T03:32:06.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.336+0000 7ffaaf800640 1 -- 172.21.15.46:0/1350871478 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa81557e0 con 0x7ffaa81553d0 2024-09-17T03:32:06.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.336+0000 7ffaa4c00640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa880747f0 0x7ffa88076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.336+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7ffaa012d5e0 con 0x7ffaa81553d0 2024-09-17T03:32:06.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.352+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffaa0016790 con 0x7ffaa81553d0 2024-09-17T03:32:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.372+0000 7ffaace00640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa880747f0 0x7ffa88076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:06.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:06.416+0000 7ffaace00640 1 --2- 172.21.15.46:0/1350871478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa880747f0 0x7ffa88076d00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffa94007fd0 tx=0x7ffa94009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.092+0000 7ffaaf800640 1 -- 172.21.15.46:0/1350871478 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone18656", "target": ["mon-mgr", ""]}) -- 0x7ffaa806b540 con 0x7ffa880747f0 2024-09-17T03:32:07.294 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:07.292+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:07.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.320+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1350871478 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7ffa94001d40 con 0x7ffa880747f0 2024-09-17T03:32:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:07.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:07.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone73317 2024-09-17T03:32:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.820+0000 7fd473e00640 1 -- 172.21.15.46:0/2240948401 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0e2ba0 msgr2=0x7fd46c00afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.820+0000 7fd473e00640 1 --2- 172.21.15.46:0/2240948401 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0e2ba0 0x7fd46c00afd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd46c032db0 tx=0x7fd46401d440 comp rx=0 tx=0).stop 2024-09-17T03:32:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 -- 172.21.15.46:0/2240948401 shutdown_connections 2024-09-17T03:32:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 --2- 172.21.15.46:0/2240948401 >> v2:172.21.15.46:3300/0 conn(0x7fd46c0368f0 0x7fd46c0388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 --2- 172.21.15.46:0/2240948401 >> v2:172.21.15.99:3300/0 conn(0x7fd46c005d70 0x7fd46c0363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 --2- 172.21.15.46:0/2240948401 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0e2ba0 0x7fd46c00afd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 -- 172.21.15.46:0/2240948401 >> 172.21.15.46:0/2240948401 conn(0x7fd46c0513f0 msgr2=0x7fd46c051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 -- 172.21.15.46:0/2240948401 shutdown_connections 2024-09-17T03:32:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.832+0000 7fd473e00640 1 -- 172.21.15.46:0/2240948401 wait complete. 2024-09-17T03:32:07.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.848+0000 7fd473e00640 1 Processor -- start 2024-09-17T03:32:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.848+0000 7fd473e00640 1 -- start start 2024-09-17T03:32:07.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.852+0000 7fd473e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c005d70 0x7fd46c0d93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:07.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.852+0000 7fd473e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0368f0 0x7fd46c180dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:07.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.852+0000 7fd473e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46c0e2ba0 0x7fd46c181310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.852+0000 7fd473e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c01cba0 con 0x7fd46c0e2ba0 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.852+0000 7fd473e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c01ca20 con 0x7fd46c005d70 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.852+0000 7fd473e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c01cd20 con 0x7fd46c0368f0 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0368f0 0x7fd46c180dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0368f0 0x7fd46c180dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:37132/0 (socket says 172.21.15.46:37132) 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 -- 172.21.15.46:0/485266218 learned_addr learned my addr 172.21.15.46:0/485266218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:07.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd473400640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46c0e2ba0 0x7fd46c181310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 -- 172.21.15.46:0/485266218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c005d70 msgr2=0x7fd46c0d93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd46c005d70 0x7fd46c0d93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 -- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46c0e2ba0 msgr2=0x7fd46c181310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46c0e2ba0 0x7fd46c181310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 -- 172.21.15.46:0/485266218 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c04a570 con 0x7fd46c0368f0 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd473400640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd46c0e2ba0 0x7fd46c181310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:07.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd472000640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd46c0368f0 0x7fd46c180dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e6c0 tx=0x7fd46000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:07.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46000cab0 con 0x7fd46c0368f0 2024-09-17T03:32:07.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd46000cc50 con 0x7fd46c0368f0 2024-09-17T03:32:07.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460010690 con 0x7fd46c0368f0 2024-09-17T03:32:07.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.856+0000 7fd473e00640 1 -- 172.21.15.46:0/485266218 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c0e0830 con 0x7fd46c0368f0 2024-09-17T03:32:07.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.860+0000 7fd473e00640 1 -- 172.21.15.46:0/485266218 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c0e0ad0 con 0x7fd46c0368f0 2024-09-17T03:32:07.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.860+0000 7fd473e00640 1 -- 172.21.15.46:0/485266218 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c04a000 con 0x7fd46c0368f0 2024-09-17T03:32:07.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.864+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fd460010830 con 0x7fd46c0368f0 2024-09-17T03:32:07.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.868+0000 7fd46aa00640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd454074840 0x7fd454076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:07.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.868+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fd460093ce0 con 0x7fd46c0368f0 2024-09-17T03:32:07.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.872+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd460060410 con 0x7fd46c0368f0 2024-09-17T03:32:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.916+0000 7fd472a00640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd454074840 0x7fd454076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:07.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:07.968+0000 7fd472a00640 1 --2- 172.21.15.46:0/485266218 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd454074840 0x7fd454076d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd4640ad000 tx=0x7fd46401a1e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:08.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:08.420+0000 7fd473e00640 1 -- 172.21.15.46:0/485266218 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone73317", "target": ["mon-mgr", ""]}) -- 0x7fd46c0d9910 con 0x7fd454074840 2024-09-17T03:32:08.581 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:08.580+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:08.580+0000 7fd46aa00640 1 -- 172.21.15.46:0/485266218 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fd46c0d9910 con 0x7fd454074840 2024-09-17T03:32:08.584 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:08.615 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:08.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone60228 2024-09-17T03:32:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:08.996+0000 7f2637000640 1 -- 172.21.15.46:0/2348058544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 msgr2=0x7f26280e6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:08.996+0000 7f2637000640 1 --2- 172.21.15.46:0/2348058544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 0x7f26280e6820 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f263007b050 tx=0x7f2630098950 comp rx=0 tx=0).stop 2024-09-17T03:32:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.000+0000 7f2637000640 1 -- 172.21.15.46:0/2348058544 shutdown_connections 2024-09-17T03:32:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.000+0000 7f2637000640 1 --2- 172.21.15.46:0/2348058544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 0x7f26280e6820 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.000+0000 7f2637000640 1 --2- 172.21.15.46:0/2348058544 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26280085e0 0x7f2628036ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:09.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.000+0000 7f2637000640 1 -- 172.21.15.46:0/2348058544 >> 172.21.15.46:0/2348058544 conn(0x7f262801fbc0 msgr2=0x7f2628056950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.000+0000 7f2637000640 1 -- 172.21.15.46:0/2348058544 shutdown_connections 2024-09-17T03:32:09.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.000+0000 7f2637000640 1 -- 172.21.15.46:0/2348058544 wait complete. 2024-09-17T03:32:09.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.008+0000 7f2637000640 1 Processor -- start 2024-09-17T03:32:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.008+0000 7f2637000640 1 -- start start 2024-09-17T03:32:09.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2637000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280085e0 0x7f26280e1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:09.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2637000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 0x7f2628034980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:09.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2635c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280085e0 0x7f26280e1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:09.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2635c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280085e0 0x7f26280e1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:55102/0 (socket says 172.21.15.46:55102) 2024-09-17T03:32:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2635200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 0x7f2628034980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2635200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 0x7f2628034980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52758/0 (socket says 172.21.15.46:52758) 2024-09-17T03:32:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2637000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2628057480 0x7f2628034ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2637000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26280e3960 con 0x7f2628037530 2024-09-17T03:32:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2637000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26280e37e0 con 0x7f26280085e0 2024-09-17T03:32:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2637000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f26280e3ae0 con 0x7f2628057480 2024-09-17T03:32:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2635c00640 1 -- 172.21.15.46:0/725507539 learned_addr learned my addr 172.21.15.46:0/725507539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.012+0000 7f2636600640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2628057480 0x7f2628034ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:09.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2635200640 1 -- 172.21.15.46:0/725507539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2628057480 msgr2=0x7f2628034ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:09.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2635200640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2628057480 0x7f2628034ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2635200640 1 -- 172.21.15.46:0/725507539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280085e0 msgr2=0x7f26280e1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2635200640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280085e0 0x7f26280e1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2635200640 1 -- 172.21.15.46:0/725507539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280e1650 con 0x7f2628037530 2024-09-17T03:32:09.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2635200640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2628037530 0x7f2628034980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c770 tx=0x7f261c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:09.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c007bb0 con 0x7f2628037530 2024-09-17T03:32:09.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f261c007e40 con 0x7f2628037530 2024-09-17T03:32:09.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c005280 con 0x7f2628037530 2024-09-17T03:32:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.016+0000 7f2637000640 1 -- 172.21.15.46:0/725507539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262804cc30 con 0x7f2628037530 2024-09-17T03:32:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.020+0000 7f2637000640 1 -- 172.21.15.46:0/725507539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262804d1f0 con 0x7f2628037530 2024-09-17T03:32:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.020+0000 7f2637000640 1 -- 172.21.15.46:0/725507539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2628036ff0 con 0x7f2628037530 2024-09-17T03:32:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.028+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f261c005420 con 0x7f2628037530 2024-09-17T03:32:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.032+0000 7f262d600640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26100748c0 0x7f2610076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:09.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.032+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f261c093730 con 0x7f2628037530 2024-09-17T03:32:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.040+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f261c014030 con 0x7f2628037530 2024-09-17T03:32:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.056+0000 7f2635c00640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26100748c0 0x7f2610076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:09.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.096+0000 7f2635c00640 1 --2- 172.21.15.46:0/725507539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26100748c0 0x7f2610076dd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f26240017f0 tx=0x7f2624009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:09.608+0000 7f2637000640 1 -- 172.21.15.46:0/725507539 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone60228", "target": ["mon-mgr", ""]}) -- 0x7f262804c9c0 con 0x7f26100748c0 2024-09-17T03:32:10.140 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:10.140+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.140+0000 7f262d600640 1 -- 172.21.15.46:0/725507539 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f262804c9c0 con 0x7f26100748c0 2024-09-17T03:32:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:10.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:10.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28249 2024-09-17T03:32:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.560+0000 7f240b200640 1 -- 172.21.15.46:0/1960654929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240c16f8b0 msgr2=0x7f240c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.560+0000 7f240b200640 1 --2- 172.21.15.46:0/1960654929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240c16f8b0 0x7f240c0cccb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009900 tx=0x7f23fc01c690 comp rx=0 tx=0).stop 2024-09-17T03:32:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.564+0000 7f240b200640 1 -- 172.21.15.46:0/1960654929 shutdown_connections 2024-09-17T03:32:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.564+0000 7f240b200640 1 --2- 172.21.15.46:0/1960654929 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f240c0cee60 0x7f240c1760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:10.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.564+0000 7f240b200640 1 --2- 172.21.15.46:0/1960654929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240c16f8b0 0x7f240c0cccb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:10.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.564+0000 7f240b200640 1 -- 172.21.15.46:0/1960654929 >> 172.21.15.46:0/1960654929 conn(0x7f240c0b2040 msgr2=0x7f240c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.572+0000 7f240b200640 1 -- 172.21.15.46:0/1960654929 shutdown_connections 2024-09-17T03:32:10.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.580+0000 7f240b200640 1 -- 172.21.15.46:0/1960654929 wait complete. 2024-09-17T03:32:10.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.592+0000 7f240b200640 1 Processor -- start 2024-09-17T03:32:10.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.592+0000 7f240b200640 1 -- start start 2024-09-17T03:32:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240c0cd1f0 0x7f240c16ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f240c0cee60 0x7f240c16b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240c16f8b0 0x7f240c211bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f240c178e40 con 0x7f240c0cd1f0 2024-09-17T03:32:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f240c0d00c0 con 0x7f240c16f8b0 2024-09-17T03:32:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f240c178fc0 con 0x7f240c0cee60 2024-09-17T03:32:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240c16f8b0 0x7f240c211bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f240c0cee60 0x7f240c16b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f240c0cee60 0x7f240c16b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:37172/0 (socket says 172.21.15.46:37172) 2024-09-17T03:32:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 -- 172.21.15.46:0/952583109 learned_addr learned my addr 172.21.15.46:0/952583109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 -- 172.21.15.46:0/952583109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240c16f8b0 msgr2=0x7f240c211bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 --2- 172.21.15.46:0/952583109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240c16f8b0 0x7f240c211bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 -- 172.21.15.46:0/952583109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240c0cd1f0 msgr2=0x7f240c16ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 --2- 172.21.15.46:0/952583109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240c0cd1f0 0x7f240c16ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 -- 172.21.15.46:0/952583109 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c15dbd0 con 0x7f240c0cee60 2024-09-17T03:32:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2409400640 1 --2- 172.21.15.46:0/952583109 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f240c0cee60 0x7f240c16b2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b410 tx=0x7f23f800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8015020 con 0x7f240c0cee60 2024-09-17T03:32:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f8011940 con 0x7f240c0cee60 2024-09-17T03:32:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.620+0000 7f240b200640 1 -- 172.21.15.46:0/952583109 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c0ae050 con 0x7f240c0cee60 2024-09-17T03:32:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.624+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8011d90 con 0x7f240c0cee60 2024-09-17T03:32:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.624+0000 7f240b200640 1 -- 172.21.15.46:0/952583109 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c15e350 con 0x7f240c0cee60 2024-09-17T03:32:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.624+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f23f8018020 con 0x7f240c0cee60 2024-09-17T03:32:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.628+0000 7f2402000640 1 --2- 172.21.15.46:0/952583109 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23ec074820 0x7f23ec076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.628+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f23f80933a0 con 0x7f240c0cee60 2024-09-17T03:32:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.628+0000 7f2409e00640 1 --2- 172.21.15.46:0/952583109 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23ec074820 0x7f23ec076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.628+0000 7f240b200640 1 -- 172.21.15.46:0/952583109 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c0cccb0 con 0x7f240c0cee60 2024-09-17T03:32:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.632+0000 7f2409e00640 1 --2- 172.21.15.46:0/952583109 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23ec074820 0x7f23ec076d30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0af000 tx=0x7f23fc019430 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:10.636+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23f801f020 con 0x7f240c0cee60 2024-09-17T03:32:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:11.116+0000 7f240b200640 1 -- 172.21.15.46:0/952583109 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28249", "target": ["mon-mgr", ""]}) -- 0x7f240c06b540 con 0x7f23ec074820 2024-09-17T03:32:11.398 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:11.396+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:11.396+0000 7f2402000640 1 -- 172.21.15.46:0/952583109 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f23fc0a8450 con 0x7f23ec074820 2024-09-17T03:32:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:11.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:11.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone16310 2024-09-17T03:32:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:11.992+0000 7fecd0c00640 1 -- 172.21.15.46:0/2905981427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 msgr2=0x7fecb0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:11.992+0000 7fecd0c00640 1 --2- 172.21.15.46:0/2905981427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 0x7fecb0003ea0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c260 tx=0x7fecbc01cf10 comp rx=0 tx=0).stop 2024-09-17T03:32:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.000+0000 7fecd0c00640 1 -- 172.21.15.46:0/2905981427 shutdown_connections 2024-09-17T03:32:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.000+0000 7fecd0c00640 1 --2- 172.21.15.46:0/2905981427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 0x7fecb0003ea0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.000+0000 7fecd0c00640 1 -- 172.21.15.46:0/2905981427 >> 172.21.15.46:0/2905981427 conn(0x7feccc0704e0 msgr2=0x7feccc0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.000+0000 7fecd0c00640 1 -- 172.21.15.46:0/2905981427 shutdown_connections 2024-09-17T03:32:12.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.000+0000 7fecd0c00640 1 -- 172.21.15.46:0/2905981427 wait complete. 2024-09-17T03:32:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.008+0000 7fecd0c00640 1 Processor -- start 2024-09-17T03:32:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.008+0000 7fecd0c00640 1 -- start start 2024-09-17T03:32:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.016+0000 7fecd0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 0x7feccc161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.016+0000 7fecd0c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feccc208dd0 0x7feccc154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.016+0000 7fecc9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 0x7feccc161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.016+0000 7fecc9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 0x7feccc161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52780/0 (socket says 172.21.15.46:52780) 2024-09-17T03:32:12.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.016+0000 7fecc9e00640 1 -- 172.21.15.46:0/278010846 learned_addr learned my addr 172.21.15.46:0/278010846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:12.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.020+0000 7fecd0c00640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feccc154cc0 0x7feccc1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.020+0000 7fecc9400640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feccc208dd0 0x7feccc154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.020+0000 7fecca800640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feccc154cc0 0x7feccc1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.020+0000 7fecd0c00640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feccc072180 con 0x7fecb0003a10 2024-09-17T03:32:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.020+0000 7fecd0c00640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feccc072000 con 0x7feccc154cc0 2024-09-17T03:32:12.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.020+0000 7fecd0c00640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feccc072300 con 0x7feccc208dd0 2024-09-17T03:32:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.024+0000 7fecc9400640 1 -- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feccc154cc0 msgr2=0x7feccc1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.024+0000 7fecc9400640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feccc154cc0 0x7feccc1550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:12.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.024+0000 7fecc9400640 1 -- 172.21.15.46:0/278010846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 msgr2=0x7feccc161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:12.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.024+0000 7fecc9400640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb0003a10 0x7feccc161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:12.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.024+0000 7fecc9400640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc2067b0 con 0x7feccc208dd0 2024-09-17T03:32:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecca800640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feccc154cc0 0x7feccc1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:32:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecc9400640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feccc208dd0 0x7feccc154780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feccc0779b0 tx=0x7fecb800cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:12.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8014070 con 0x7feccc208dd0 2024-09-17T03:32:12.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecd0c00640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc205c30 con 0x7feccc208dd0 2024-09-17T03:32:12.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecd0c00640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc2061f0 con 0x7feccc208dd0 2024-09-17T03:32:12.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb8004320 con 0x7feccc208dd0 2024-09-17T03:32:12.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.028+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80079a0 con 0x7feccc208dd0 2024-09-17T03:32:12.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.032+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fecb8017070 con 0x7feccc208dd0 2024-09-17T03:32:12.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.032+0000 7fecb7400640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc074ea0 con 0x7feccc208dd0 2024-09-17T03:32:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.036+0000 7fecc2000640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fecac074820 0x7fecac076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:12.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.036+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fecb8092fa0 con 0x7feccc208dd0 2024-09-17T03:32:12.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.056+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fecb805f450 con 0x7feccc208dd0 2024-09-17T03:32:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.064+0000 7fecc9e00640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fecac074820 0x7fecac076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.096+0000 7fecc9e00640 1 --2- 172.21.15.46:0/278010846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fecac074820 0x7fecac076d30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fecbc01b7e0 tx=0x7fecbc019d50 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:12.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:12.760+0000 7fecb7400640 1 -- 172.21.15.46:0/278010846 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone16310", "target": ["mon-mgr", ""]}) -- 0x7feccc2059e0 con 0x7fecac074820 2024-09-17T03:32:13.028 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:13.028+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:13.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.032+0000 7fecc2000640 1 -- 172.21.15.46:0/278010846 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7feccc2059e0 con 0x7fecac074820 2024-09-17T03:32:13.037 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:13.093 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:13.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone560911 2024-09-17T03:32:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 -- 172.21.15.46:0/2966701177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0800cf210 msgr2=0x7ff0800cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 --2- 172.21.15.46:0/2966701177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0800cf210 0x7ff0800cf620 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff070009900 tx=0x7ff07001c850 comp rx=0 tx=0).stop 2024-09-17T03:32:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 -- 172.21.15.46:0/2966701177 shutdown_connections 2024-09-17T03:32:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 --2- 172.21.15.46:0/2966701177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0800cf210 0x7ff0800cf620 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 -- 172.21.15.46:0/2966701177 >> 172.21.15.46:0/2966701177 conn(0x7ff08006e1d0 msgr2=0x7ff08006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 -- 172.21.15.46:0/2966701177 shutdown_connections 2024-09-17T03:32:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.496+0000 7ff086600640 1 -- 172.21.15.46:0/2966701177 wait complete. 2024-09-17T03:32:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.504+0000 7ff086600640 1 Processor -- start 2024-09-17T03:32:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.504+0000 7ff086600640 1 -- start start 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.504+0000 7ff086600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0800b3890 0x7ff080160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff086600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0800cd7a0 0x7ff0801449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff086600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0800cf210 0x7ff0801581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff086600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff0800cc870 con 0x7ff0800b3890 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff086600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0800cc6f0 con 0x7ff0800cd7a0 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff086600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff0800cc9f0 con 0x7ff0800cf210 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0800cf210 0x7ff0801581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0800cf210 0x7ff0801581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41684/0 (socket says 172.21.15.46:41684) 2024-09-17T03:32:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07fe00640 1 -- 172.21.15.46:0/2988890851 learned_addr learned my addr 172.21.15.46:0/2988890851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:13.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07f400640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0800b3890 0x7ff080160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:13.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07ea00640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0800cd7a0 0x7ff0801449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07f400640 1 -- 172.21.15.46:0/2988890851 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0800cf210 msgr2=0x7ff0801581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07f400640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff0800cf210 0x7ff0801581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07f400640 1 -- 172.21.15.46:0/2988890851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0800cd7a0 msgr2=0x7ff0801449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07f400640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0800cd7a0 0x7ff0801449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.508+0000 7ff07f400640 1 -- 172.21.15.46:0/2988890851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0800ab640 con 0x7ff0800b3890 2024-09-17T03:32:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.512+0000 7ff07f400640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0800b3890 0x7ff080160a80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00b410 tx=0x7ff06c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.512+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c015030 con 0x7ff0800b3890 2024-09-17T03:32:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.512+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff06c004110 con 0x7ff0800b3890 2024-09-17T03:32:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.512+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c011630 con 0x7ff0800b3890 2024-09-17T03:32:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.512+0000 7ff086600640 1 -- 172.21.15.46:0/2988890851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff080161a30 con 0x7ff0800b3890 2024-09-17T03:32:13.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.512+0000 7ff086600640 1 -- 172.21.15.46:0/2988890851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff080161ff0 con 0x7ff0800b3890 2024-09-17T03:32:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.516+0000 7ff086600640 1 -- 172.21.15.46:0/2988890851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0800ccf90 con 0x7ff0800b3890 2024-09-17T03:32:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.524+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7ff06c018070 con 0x7ff0800b3890 2024-09-17T03:32:13.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.528+0000 7ff077400640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0600748c0 0x7ff060076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.528+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7ff06c092700 con 0x7ff0800b3890 2024-09-17T03:32:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.540+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff06c01f020 con 0x7ff0800b3890 2024-09-17T03:32:13.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.572+0000 7ff07ea00640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0600748c0 0x7ff060076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:13.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:13.632+0000 7ff07ea00640 1 --2- 172.21.15.46:0/2988890851 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0600748c0 0x7ff060076dd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff0700af000 tx=0x7ff07001b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:14.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:14.100+0000 7ff086600640 1 -- 172.21.15.46:0/2988890851 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone560911", "target": ["mon-mgr", ""]}) -- 0x7ff08006b540 con 0x7ff0600748c0 2024-09-17T03:32:14.516 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:14.516+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:14.544+0000 7ff077400640 1 -- 172.21.15.46:0/2988890851 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7ff070005d60 con 0x7ff0600748c0 2024-09-17T03:32:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:32:14.602 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:32:14.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone955412 2024-09-17T03:32:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 -- 172.21.15.46:0/4183904825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc0b5bc0 msgr2=0x7f08fc0b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 --2- 172.21.15.46:0/4183904825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc0b5bc0 0x7f08fc0b5fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00afd0 tx=0x7f08ec01c340 comp rx=0 tx=0).stop 2024-09-17T03:32:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 -- 172.21.15.46:0/4183904825 shutdown_connections 2024-09-17T03:32:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 --2- 172.21.15.46:0/4183904825 >> v2:172.21.15.46:3300/0 conn(0x7f08fc15b0b0 0x7f08fc15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 --2- 172.21.15.46:0/4183904825 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc0b5bc0 0x7f08fc0b5fd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 -- 172.21.15.46:0/4183904825 >> 172.21.15.46:0/4183904825 conn(0x7f08fc06ee20 msgr2=0x7f08fc0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 -- 172.21.15.46:0/4183904825 shutdown_connections 2024-09-17T03:32:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.156+0000 7f08fbe00640 1 -- 172.21.15.46:0/4183904825 wait complete. 2024-09-17T03:32:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.168+0000 7f08fbe00640 1 Processor -- start 2024-09-17T03:32:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.168+0000 7f08fbe00640 1 -- start start 2024-09-17T03:32:15.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc06cdf0 0x7f08fc147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:15.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 0x7f08fc1ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:15.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc15b0b0 0x7f08fc1eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:15.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc168270 con 0x7f08fc06cdf0 2024-09-17T03:32:15.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fbe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1680f0 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fbe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc1683f0 con 0x7f08fc15b0b0 2024-09-17T03:32:15.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fb400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc15b0b0 0x7f08fc1eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc06cdf0 0x7f08fc147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 0x7f08fc1ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 0x7f08fc1ee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:40362/0 (socket says 172.21.15.46:40362) 2024-09-17T03:32:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 -- 172.21.15.46:0/4184495198 learned_addr learned my addr 172.21.15.46:0/4184495198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 -- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc15b0b0 msgr2=0x7f08fc1eeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc15b0b0 0x7f08fc1eeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 -- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc06cdf0 msgr2=0x7f08fc147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc06cdf0 0x7f08fc147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.172+0000 7f08fa000640 1 -- 172.21.15.46:0/4184495198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc1f94b0 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08fa000640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 0x7f08fc1ee610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c460 tx=0x7f08e800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:15.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8016020 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08e8012940 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8012d90 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08fbe00640 1 -- 172.21.15.46:0/4184495198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc1f9790 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08fbe00640 1 -- 172.21.15.46:0/4184495198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc1f9d50 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.176+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d00052a0 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.180+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f08e8019020 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.184+0000 7f08f2a00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f08dc0747f0 0x7f08dc076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:15.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.184+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f08e80931b0 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.188+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08e8020020 con 0x7f08fc0b5bc0 2024-09-17T03:32:15.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.248+0000 7f08faa00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f08dc0747f0 0x7f08dc076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.256+0000 7f08faa00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f08dc0747f0 0x7f08dc076d00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ad000 tx=0x7f08ec01ab80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:15.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:15.764+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone955412", "target": ["mon-mgr", ""]}) -- 0x7f08d0000bd0 con 0x7f08dc0747f0 2024-09-17T03:32:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.904+0000 7f08f2a00640 1 -- 172.21.15.46:0/4184495198 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f08d0000bd0 con 0x7f08dc0747f0 2024-09-17T03:32:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.912+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f08dc0747f0 msgr2=0x7f08dc076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.912+0000 7f08e7e00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f08dc0747f0 0x7f08dc076d00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ad000 tx=0x7f08ec01ab80 comp rx=0 tx=0).stop 2024-09-17T03:32:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.912+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 msgr2=0x7f08fc1ee610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.912+0000 7f08e7e00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 0x7f08fc1ee610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c460 tx=0x7f08e800c920 comp rx=0 tx=0).stop 2024-09-17T03:32:17.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 shutdown_connections 2024-09-17T03:32:17.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f08dc0747f0 0x7f08dc076d00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08fc15b0b0 0x7f08fc1eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08fc0b5bc0 0x7f08fc1ee610 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 --2- 172.21.15.46:0/4184495198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc06cdf0 0x7f08fc147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 >> 172.21.15.46:0/4184495198 conn(0x7f08fc06ee20 msgr2=0x7f08fc0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 shutdown_connections 2024-09-17T03:32:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:17.916+0000 7f08e7e00640 1 -- 172.21.15.46:0/4184495198 wait complete. 2024-09-17T03:32:17.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone579713 2024-09-17T03:32:18.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 -- 172.21.15.46:0/2627376567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f357415ef10 msgr2=0x7f357415a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:18.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 --2- 172.21.15.46:0/2627376567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f357415ef10 0x7f357415a830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35740ceb50 tx=0x7f355c01db60 comp rx=0 tx=0).stop 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 -- 172.21.15.46:0/2627376567 shutdown_connections 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 --2- 172.21.15.46:0/2627376567 >> v2:172.21.15.46:3300/0 conn(0x7f35740cd7a0 0x7f35740cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 --2- 172.21.15.46:0/2627376567 >> v2:172.21.15.99:3300/0 conn(0x7f357406ecf0 0x7f35740cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 --2- 172.21.15.46:0/2627376567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f357415ef10 0x7f357415a830 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.448+0000 7f357b600640 1 -- 172.21.15.46:0/2627376567 >> 172.21.15.46:0/2627376567 conn(0x7f357406e1d0 msgr2=0x7f357406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.452+0000 7f357b600640 1 -- 172.21.15.46:0/2627376567 shutdown_connections 2024-09-17T03:32:18.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.452+0000 7f357b600640 1 -- 172.21.15.46:0/2627376567 wait complete. 2024-09-17T03:32:18.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.460+0000 7f357b600640 1 Processor -- start 2024-09-17T03:32:18.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.460+0000 7f357b600640 1 -- start start 2024-09-17T03:32:18.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f357b600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357406ecf0 0x7f35741641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:18.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f357b600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 0x7f35740aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:18.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f357b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f357415ef10 0x7f35741477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f357b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35740b3ae0 con 0x7f357415ef10 2024-09-17T03:32:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f356be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 0x7f35740aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f356be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 0x7f35740aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41714/0 (socket says 172.21.15.46:41714) 2024-09-17T03:32:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f356be00640 1 -- 172.21.15.46:0/1985358647 learned_addr learned my addr 172.21.15.46:0/1985358647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f3578c00640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357406ecf0 0x7f35741641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35740b3960 con 0x7f357406ecf0 2024-09-17T03:32:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.464+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f35740b3c60 con 0x7f35740cd7a0 2024-09-17T03:32:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f3579600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f357415ef10 0x7f35741477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f356be00640 1 -- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357406ecf0 msgr2=0x7f35741641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:18.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f356be00640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357406ecf0 0x7f35741641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:18.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f356be00640 1 -- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f357415ef10 msgr2=0x7f35741477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:18.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f356be00640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f357415ef10 0x7f35741477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:18.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f356be00640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35740acf10 con 0x7f35740cd7a0 2024-09-17T03:32:18.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.468+0000 7f356be00640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 0x7f35740aafd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f356400d950 tx=0x7f356400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:18.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.472+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564017070 con 0x7f35740cd7a0 2024-09-17T03:32:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.472+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3574164c20 con 0x7f35740cd7a0 2024-09-17T03:32:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.472+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3574160ef0 con 0x7f35740cd7a0 2024-09-17T03:32:18.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.472+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3564009d20 con 0x7f35740cd7a0 2024-09-17T03:32:18.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.472+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640106c0 con 0x7f35740cd7a0 2024-09-17T03:32:18.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.472+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35480052a0 con 0x7f35740cd7a0 2024-09-17T03:32:18.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.476+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f356401a070 con 0x7f35740cd7a0 2024-09-17T03:32:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.480+0000 7f3571600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35500748c0 0x7f3550076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:18.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.480+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f35640931c0 con 0x7f35740cd7a0 2024-09-17T03:32:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.480+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f356405f260 con 0x7f35740cd7a0 2024-09-17T03:32:18.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.516+0000 7f3578c00640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35500748c0 0x7f3550076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:18.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:18.516+0000 7f3578c00640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35500748c0 0x7f3550076dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f355c0b0000 tx=0x7f355c01c3a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:19.068+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone579713", "target": ["mon-mgr", ""]}) -- 0x7f3548000bd0 con 0x7f35500748c0 2024-09-17T03:32:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.628+0000 7f3571600640 1 -- 172.21.15.46:0/1985358647 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3548000bd0 con 0x7f35500748c0 2024-09-17T03:32:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35500748c0 msgr2=0x7f3550076dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35500748c0 0x7f3550076dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f355c0b0000 tx=0x7f355c01c3a0 comp rx=0 tx=0).stop 2024-09-17T03:32:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 msgr2=0x7f35740aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 0x7f35740aafd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f356400d950 tx=0x7f356400de10 comp rx=0 tx=0).stop 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 shutdown_connections 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35500748c0 0x7f3550076dd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f357415ef10 0x7f35741477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35740cd7a0 0x7f35740aafd0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f356400d950 tx=0x7f356400de10 comp rx=0 tx=0).stop 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 --2- 172.21.15.46:0/1985358647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f357406ecf0 0x7f35741641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.636+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 >> 172.21.15.46:0/1985358647 conn(0x7f357406e1d0 msgr2=0x7f357406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.640+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 shutdown_connections 2024-09-17T03:32:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:20.640+0000 7f357b600640 1 -- 172.21.15.46:0/1985358647 wait complete. 2024-09-17T03:32:20.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone341414 2024-09-17T03:32:21.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 -- 172.21.15.46:0/656242216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2541557e0 msgr2=0x7ff2540ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:21.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 --2- 172.21.15.46:0/656242216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2541557e0 0x7ff2540ae8e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff254071fb0 tx=0x7ff24001c5b0 comp rx=0 tx=0).stop 2024-09-17T03:32:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 -- 172.21.15.46:0/656242216 shutdown_connections 2024-09-17T03:32:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 --2- 172.21.15.46:0/656242216 >> v2:172.21.15.46:3300/0 conn(0x7ff2540aee20 0x7ff2540b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 --2- 172.21.15.46:0/656242216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2541557e0 0x7ff2540ae8e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 -- 172.21.15.46:0/656242216 >> 172.21.15.46:0/656242216 conn(0x7ff2540b2040 msgr2=0x7ff2540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 -- 172.21.15.46:0/656242216 shutdown_connections 2024-09-17T03:32:21.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.160+0000 7ff259000640 1 -- 172.21.15.46:0/656242216 wait complete. 2024-09-17T03:32:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 Processor -- start 2024-09-17T03:32:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 -- start start 2024-09-17T03:32:21.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540aee20 0x7ff2540b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:21.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 0x7ff25414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:21.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff25415f800 0x7ff2541fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2540ccb60 con 0x7ff25415f800 2024-09-17T03:32:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2540cc9e0 con 0x7ff2540aee20 2024-09-17T03:32:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.164+0000 7ff259000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff2540ccce0 con 0x7ff2541557e0 2024-09-17T03:32:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 0x7ff25414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 0x7ff25414c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41722/0 (socket says 172.21.15.46:41722) 2024-09-17T03:32:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 -- 172.21.15.46:0/3062366582 learned_addr learned my addr 172.21.15.46:0/3062366582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251e00640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540aee20 0x7ff2540b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 -- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540aee20 msgr2=0x7ff2540b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540aee20 0x7ff2540b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:21.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 -- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff25415f800 msgr2=0x7ff2541fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff25415f800 0x7ff2541fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.168+0000 7ff251400640 1 -- 172.21.15.46:0/3062366582 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2540b1000 con 0x7ff2541557e0 2024-09-17T03:32:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.172+0000 7ff251400640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 0x7ff25414c1e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2400ad040 tx=0x7ff2400ab570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.172+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2400abaf0 con 0x7ff2541557e0 2024-09-17T03:32:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.172+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2400abd80 con 0x7ff2541557e0 2024-09-17T03:32:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.172+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240003860 con 0x7ff2541557e0 2024-09-17T03:32:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.172+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff254158cc0 con 0x7ff2541557e0 2024-09-17T03:32:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.172+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2540b0af0 con 0x7ff2541557e0 2024-09-17T03:32:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.176+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2541591e0 con 0x7ff2541557e0 2024-09-17T03:32:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.184+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7ff240003a00 con 0x7ff2541557e0 2024-09-17T03:32:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.184+0000 7ff24a000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff234074820 0x7ff234076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.184+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7ff24012da50 con 0x7ff2541557e0 2024-09-17T03:32:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.188+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff2400c7030 con 0x7ff2541557e0 2024-09-17T03:32:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.216+0000 7ff251e00640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff234074820 0x7ff234076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.232+0000 7ff251e00640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff234074820 0x7ff234076d30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff24400b810 tx=0x7ff244008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:21.792+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone341414", "target": ["mon-mgr", ""]}) -- 0x7ff25406b540 con 0x7ff234074820 2024-09-17T03:32:23.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.580+0000 7ff24a000640 1 -- 172.21.15.46:0/3062366582 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff24400bd90 con 0x7ff234074820 2024-09-17T03:32:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff234074820 msgr2=0x7ff234076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:23.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff234074820 0x7ff234076d30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff24400b810 tx=0x7ff244008040 comp rx=0 tx=0).stop 2024-09-17T03:32:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 msgr2=0x7ff25414c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 0x7ff25414c1e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2400ad040 tx=0x7ff2400ab570 comp rx=0 tx=0).stop 2024-09-17T03:32:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 shutdown_connections 2024-09-17T03:32:23.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff234074820 0x7ff234076d30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:23.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff25415f800 0x7ff2541fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:23.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2541557e0 0x7ff25414c1e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:23.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 --2- 172.21.15.46:0/3062366582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540aee20 0x7ff2540b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:23.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.596+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 >> 172.21.15.46:0/3062366582 conn(0x7ff2540b2040 msgr2=0x7ff2540b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:23.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.600+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 shutdown_connections 2024-09-17T03:32:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:23.600+0000 7ff259000640 1 -- 172.21.15.46:0/3062366582 wait complete. 2024-09-17T03:32:23.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone616815 2024-09-17T03:32:24.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.140+0000 7fe13c000640 1 -- 172.21.15.46:0/3108816019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 msgr2=0x7fe1340b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.140+0000 7fe13c000640 1 --2- 172.21.15.46:0/3108816019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 0x7fe1340b45b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe120009840 tx=0x7fe12001c7e0 comp rx=0 tx=0).stop 2024-09-17T03:32:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.140+0000 7fe13c000640 1 -- 172.21.15.46:0/3108816019 shutdown_connections 2024-09-17T03:32:24.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.140+0000 7fe13c000640 1 --2- 172.21.15.46:0/3108816019 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe13406a480 0x7fe1340cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.140+0000 7fe13c000640 1 --2- 172.21.15.46:0/3108816019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 0x7fe1340b45b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.144+0000 7fe13c000640 1 --2- 172.21.15.46:0/3108816019 >> v2:172.21.15.46:3300/0 conn(0x7fe134072b00 0x7fe1340b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.144+0000 7fe13c000640 1 -- 172.21.15.46:0/3108816019 >> 172.21.15.46:0/3108816019 conn(0x7fe1340b31f0 msgr2=0x7fe13406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.144+0000 7fe13c000640 1 -- 172.21.15.46:0/3108816019 shutdown_connections 2024-09-17T03:32:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.144+0000 7fe13c000640 1 -- 172.21.15.46:0/3108816019 wait complete. 2024-09-17T03:32:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.152+0000 7fe13c000640 1 Processor -- start 2024-09-17T03:32:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.152+0000 7fe13c000640 1 -- start start 2024-09-17T03:32:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe13c000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe13406a480 0x7fe134147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe13c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 0x7fe134153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe13c000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 0x7fe134153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe13c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe13415ee10 con 0x7fe134072b00 2024-09-17T03:32:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe13c000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe13415ec90 con 0x7fe134072f10 2024-09-17T03:32:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe13c000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe13415ef90 con 0x7fe13406a480 2024-09-17T03:32:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 0x7fe134153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 0x7fe134153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41882/0 (socket says 172.21.15.46:41882) 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 -- 172.21.15.46:0/2941874350 learned_addr learned my addr 172.21.15.46:0/2941874350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 -- 172.21.15.46:0/2941874350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe13406a480 msgr2=0x7fe134147310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe13406a480 0x7fe134147310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 -- 172.21.15.46:0/2941874350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 msgr2=0x7fe134153ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 0x7fe134153ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:24.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.156+0000 7fe138c00640 1 -- 172.21.15.46:0/2941874350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe134150b20 con 0x7fe134072b00 2024-09-17T03:32:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.160+0000 7fe138c00640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 0x7fe134153660 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe12001a410 tx=0x7fe1200a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:24.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.160+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200b2040 con 0x7fe134072b00 2024-09-17T03:32:24.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.164+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe134151020 con 0x7fe134072b00 2024-09-17T03:32:24.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.164+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13415afa0 con 0x7fe134072b00 2024-09-17T03:32:24.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.164+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1340cf6a0 con 0x7fe134072b00 2024-09-17T03:32:24.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.168+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe120000e50 con 0x7fe134072b00 2024-09-17T03:32:24.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.168+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200050c0 con 0x7fe134072b00 2024-09-17T03:32:24.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.180+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fe120005260 con 0x7fe134072b00 2024-09-17T03:32:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.188+0000 7fe131600640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1140747f0 0x7fe114076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.188+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fe12012d2a0 con 0x7fe134072b00 2024-09-17T03:32:24.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.196+0000 7fe139600640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1140747f0 0x7fe114076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.200+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe1200163f0 con 0x7fe134072b00 2024-09-17T03:32:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.208+0000 7fe139600640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1140747f0 0x7fe114076d00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe128009670 tx=0x7fe128006b90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:24.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:24.812+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone616815", "target": ["mon-mgr", ""]}) -- 0x7fe13406b540 con 0x7fe1140747f0 2024-09-17T03:32:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.772+0000 7fe131600640 1 -- 172.21.15.46:0/2941874350 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe128009280 con 0x7fe1140747f0 2024-09-17T03:32:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.780+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1140747f0 msgr2=0x7fe114076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.780+0000 7fe13c000640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1140747f0 0x7fe114076d00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe128009670 tx=0x7fe128006b90 comp rx=0 tx=0).stop 2024-09-17T03:32:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.780+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 msgr2=0x7fe134153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.780+0000 7fe13c000640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 0x7fe134153660 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe12001a410 tx=0x7fe1200a5590 comp rx=0 tx=0).stop 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 shutdown_connections 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1140747f0 0x7fe114076d00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe134072f10 0x7fe134153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe134072b00 0x7fe134153660 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 --2- 172.21.15.46:0/2941874350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe13406a480 0x7fe134147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 >> 172.21.15.46:0/2941874350 conn(0x7fe1340b31f0 msgr2=0x7fe1340b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 shutdown_connections 2024-09-17T03:32:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:25.784+0000 7fe13c000640 1 -- 172.21.15.46:0/2941874350 wait complete. 2024-09-17T03:32:25.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone838216 2024-09-17T03:32:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.308+0000 7f4c4b000640 1 -- 172.21.15.46:0/3830034472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 msgr2=0x7f4c2c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.308+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3830034472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 0x7f4c2c00b9d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4c44051dc0 tx=0x7f4c44098d00 comp rx=0 tx=0).stop 2024-09-17T03:32:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.316+0000 7f4c4b000640 1 -- 172.21.15.46:0/3830034472 shutdown_connections 2024-09-17T03:32:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.316+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3830034472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 0x7f4c2c00b9d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.316+0000 7f4c4b000640 1 -- 172.21.15.46:0/3830034472 >> 172.21.15.46:0/3830034472 conn(0x7f4c3c01d9b0 msgr2=0x7f4c3c039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:26.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.328+0000 7f4c4b000640 1 -- 172.21.15.46:0/3830034472 shutdown_connections 2024-09-17T03:32:26.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.328+0000 7f4c4b000640 1 -- 172.21.15.46:0/3830034472 wait complete. 2024-09-17T03:32:26.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.336+0000 7f4c4b000640 1 Processor -- start 2024-09-17T03:32:26.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.340+0000 7f4c4b000640 1 -- start start 2024-09-17T03:32:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4b000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 0x7f4c3c189b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4b000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c3c0df390 0x7f4c3c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:26.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 0x7f4c3c032960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:26.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c49200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c3c0df390 0x7f4c3c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:26.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c49200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c3c0df390 0x7f4c3c032420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:52500/0 (socket says 172.21.15.46:52500) 2024-09-17T03:32:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 0x7f4c3c032960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 0x7f4c3c032960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41914/0 (socket says 172.21.15.46:41914) 2024-09-17T03:32:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c49200640 1 -- 172.21.15.46:0/3671654241 learned_addr learned my addr 172.21.15.46:0/3671654241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c051740 con 0x7f4c3c0e5400 2024-09-17T03:32:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c0515c0 con 0x7f4c2c005280 2024-09-17T03:32:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.352+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c0518c0 con 0x7f4c3c0df390 2024-09-17T03:32:26.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.356+0000 7f4c4a600640 1 -- 172.21.15.46:0/3671654241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c3c0df390 msgr2=0x7f4c3c032420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:26.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.356+0000 7f4c4a600640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c3c0df390 0x7f4c3c032420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:26.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.356+0000 7f4c4a600640 1 -- 172.21.15.46:0/3671654241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 msgr2=0x7f4c3c189b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:26.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.356+0000 7f4c4a600640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 0x7f4c3c189b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.356+0000 7f4c4a600640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c048070 con 0x7f4c3c0e5400 2024-09-17T03:32:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.356+0000 7f4c4a600640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 0x7f4c3c032960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4c4406b8f0 tx=0x7f4c4406be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:26.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.360+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44127040 con 0x7f4c3c0e5400 2024-09-17T03:32:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.360+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c44121bb0 con 0x7f4c3c0e5400 2024-09-17T03:32:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.360+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44134460 con 0x7f4c3c0e5400 2024-09-17T03:32:26.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.364+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c18a390 con 0x7f4c3c0e5400 2024-09-17T03:32:26.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.364+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c18a950 con 0x7f4c3c0e5400 2024-09-17T03:32:26.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.364+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3c050c80 con 0x7f4c3c0e5400 2024-09-17T03:32:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.384+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f4c4413b060 con 0x7f4c3c0e5400 2024-09-17T03:32:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.384+0000 7f4c41600640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4c24074870 0x7f4c24076d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.392+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f4c44170cd0 con 0x7f4c3c0e5400 2024-09-17T03:32:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.396+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4c44092ca0 con 0x7f4c3c0e5400 2024-09-17T03:32:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.400+0000 7f4c49c00640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4c24074870 0x7f4c24076d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:26.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:26.408+0000 7f4c49c00640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4c24074870 0x7f4c24076d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001a40 tx=0x7f4c38000f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:27.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.004+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone838216", "target": ["mon-mgr", ""]}) -- 0x7f4c3c00b6f0 con 0x7f4c24074870 2024-09-17T03:32:27.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.896+0000 7f4c41600640 1 -- 172.21.15.46:0/3671654241 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c3c00b6f0 con 0x7f4c24074870 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4c24074870 msgr2=0x7f4c24076d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4c24074870 0x7f4c24076d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001a40 tx=0x7f4c38000f90 comp rx=0 tx=0).stop 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 msgr2=0x7f4c3c032960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 0x7f4c3c032960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4c4406b8f0 tx=0x7f4c4406be20 comp rx=0 tx=0).stop 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 shutdown_connections 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4c24074870 0x7f4c24076d80 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001a40 tx=0x7f4c38000f90 comp rx=0 tx=0).stop 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c3c0e5400 0x7f4c3c032960 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4c3c0df390 0x7f4c3c032420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 --2- 172.21.15.46:0/3671654241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c2c005280 0x7f4c3c189b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.928+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 >> 172.21.15.46:0/3671654241 conn(0x7f4c3c01d9b0 msgr2=0x7f4c3c0e3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.936+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 shutdown_connections 2024-09-17T03:32:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:27.936+0000 7f4c4b000640 1 -- 172.21.15.46:0/3671654241 wait complete. 2024-09-17T03:32:28.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone551017 2024-09-17T03:32:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.492+0000 7f55a3600640 1 -- 172.21.15.46:0/2983166131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5580005280 msgr2=0x7f558000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.492+0000 7f55a3600640 1 --2- 172.21.15.46:0/2983166131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5580005280 0x7f558000b9d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f559c0d2950 tx=0x7f559401ca20 comp rx=0 tx=0).stop 2024-09-17T03:32:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.496+0000 7f55a3600640 1 -- 172.21.15.46:0/2983166131 shutdown_connections 2024-09-17T03:32:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.496+0000 7f55a3600640 1 --2- 172.21.15.46:0/2983166131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5580005280 0x7f558000b9d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.496+0000 7f55a3600640 1 -- 172.21.15.46:0/2983166131 >> 172.21.15.46:0/2983166131 conn(0x7f559c0708c0 msgr2=0x7f559c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:28.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.504+0000 7f55a3600640 1 -- 172.21.15.46:0/2983166131 shutdown_connections 2024-09-17T03:32:28.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.508+0000 7f55a3600640 1 -- 172.21.15.46:0/2983166131 wait complete. 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.508+0000 7f55a3600640 1 Processor -- start 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.508+0000 7f55a3600640 1 -- start start 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a3600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5580005280 0x7f559c0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f559c0d0950 0x7f559c074410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a3600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 0x7f559c1f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f559c15f590 con 0x7f559c0d0950 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a3600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0d3040 con 0x7f559c074950 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a3600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f559c15f710 con 0x7f5580005280 2024-09-17T03:32:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 0x7f559c1f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 0x7f559c1f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:57636/0 (socket says 172.21.15.46:57636) 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 -- 172.21.15.46:0/2714219955 learned_addr learned my addr 172.21.15.46:0/2714219955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a0c00640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5580005280 0x7f559c0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 -- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5580005280 msgr2=0x7f559c0b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5580005280 0x7f559c0b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 -- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f559c0d0950 msgr2=0x7f559c074410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f559c0d0950 0x7f559c074410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 -- 172.21.15.46:0/2714219955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c144ec0 con 0x7f559c074950 2024-09-17T03:32:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.512+0000 7f55a1600640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 0x7f559c1f8180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f559401d2c0 tx=0x7f5594001880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.516+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940a5800 con 0x7f559c074950 2024-09-17T03:32:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.516+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55940a5a90 con 0x7f559c074950 2024-09-17T03:32:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.516+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940ad410 con 0x7f559c074950 2024-09-17T03:32:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.516+0000 7f55a3600640 1 -- 172.21.15.46:0/2714219955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c074d60 con 0x7f559c074950 2024-09-17T03:32:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.516+0000 7f55a3600640 1 -- 172.21.15.46:0/2714219955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c145410 con 0x7f559c074950 2024-09-17T03:32:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.516+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55700052a0 con 0x7f559c074950 2024-09-17T03:32:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.520+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f55940d9690 con 0x7f559c074950 2024-09-17T03:32:28.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.520+0000 7f5598c00640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f557c0747f0 0x7f557c076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:28.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.520+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f559412d400 con 0x7f559c074950 2024-09-17T03:32:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.540+0000 7f55a0c00640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f557c0747f0 0x7f557c076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.548+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f559c074d60 con 0x7f559c074950 2024-09-17T03:32:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:28.560+0000 7f55a0c00640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f557c0747f0 0x7f557c076d00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f558c001930 tx=0x7f558c006720 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:29.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:29.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone551017", "target": ["mon-mgr", ""]}) -- 0x7f5570000bd0 con 0x7f557c0747f0 2024-09-17T03:32:30.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.120+0000 7f5598c00640 1 -- 172.21.15.46:0/2714219955 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5570000bd0 con 0x7f557c0747f0 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f557c0747f0 msgr2=0x7f557c076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f557c0747f0 0x7f557c076d00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f558c001930 tx=0x7f558c006720 comp rx=0 tx=0).stop 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 msgr2=0x7f559c1f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 0x7f559c1f8180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f559401d2c0 tx=0x7f5594001880 comp rx=0 tx=0).stop 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 shutdown_connections 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f557c0747f0 0x7f557c076d00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f559c074950 0x7f559c1f8180 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f559c0d0950 0x7f559c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 --2- 172.21.15.46:0/2714219955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5580005280 0x7f559c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 >> 172.21.15.46:0/2714219955 conn(0x7f559c0708c0 msgr2=0x7f559c161e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 shutdown_connections 2024-09-17T03:32:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.132+0000 7f5592000640 1 -- 172.21.15.46:0/2714219955 wait complete. 2024-09-17T03:32:30.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone522018 2024-09-17T03:32:30.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:32:30.321 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:32:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.668+0000 7f3926200640 1 -- 172.21.15.46:0/725809134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 msgr2=0x7f39180dfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.668+0000 7f3926200640 1 --2- 172.21.15.46:0/725809134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180dfa90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f392006c230 tx=0x7f39200989f0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.668+0000 7f3926200640 1 -- 172.21.15.46:0/725809134 shutdown_connections 2024-09-17T03:32:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.668+0000 7f3926200640 1 --2- 172.21.15.46:0/725809134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180dfa90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.668+0000 7f3926200640 1 --2- 172.21.15.46:0/725809134 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 0x7f391801cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.668+0000 7f3926200640 1 -- 172.21.15.46:0/725809134 >> 172.21.15.46:0/725809134 conn(0x7f3918054970 msgr2=0x7f3918054d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.676+0000 7f3926200640 1 -- 172.21.15.46:0/725809134 shutdown_connections 2024-09-17T03:32:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.676+0000 7f3926200640 1 -- 172.21.15.46:0/725809134 wait complete. 2024-09-17T03:32:30.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.680+0000 7f3926200640 1 Processor -- start 2024-09-17T03:32:30.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.680+0000 7f3926200640 1 -- start start 2024-09-17T03:32:30.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3926200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391800b9d0 0x7f3918185380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:30.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3926200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 0x7f3918182730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:30.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3926200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180da910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3926200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39180ee9f0 con 0x7f39180df680 2024-09-17T03:32:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3924e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391800b9d0 0x7f3918185380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f391fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 0x7f3918182730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3926200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39180ee870 con 0x7f391800b9d0 2024-09-17T03:32:30.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3926200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f39180eeb70 con 0x7f391800be10 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f391fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 0x7f3918182730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:52536/0 (socket says 172.21.15.46:52536) 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f391fe00640 1 -- 172.21.15.46:0/3753200448 learned_addr learned my addr 172.21.15.46:0/3753200448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3927800640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3927800640 1 -- 172.21.15.46:0/3753200448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 msgr2=0x7f3918182730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3927800640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 0x7f3918182730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3927800640 1 -- 172.21.15.46:0/3753200448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391800b9d0 msgr2=0x7f3918185380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3927800640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391800b9d0 0x7f3918185380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.684+0000 7f3927800640 1 -- 172.21.15.46:0/3753200448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3918185cf0 con 0x7f39180df680 2024-09-17T03:32:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.688+0000 7f3927800640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180da910 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f39200779b0 tx=0x7f392007b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.688+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392013a070 con 0x7f39180df680 2024-09-17T03:32:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.688+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3918185fd0 con 0x7f39180df680 2024-09-17T03:32:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.688+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39201218d0 con 0x7f39180df680 2024-09-17T03:32:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.688+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392013a070 con 0x7f39180df680 2024-09-17T03:32:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.688+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3918182210 con 0x7f39180df680 2024-09-17T03:32:30.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.692+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391801d300 con 0x7f39180df680 2024-09-17T03:32:30.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.696+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f3920121a80 con 0x7f39180df680 2024-09-17T03:32:30.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.696+0000 7f391cc00640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39000749f0 0x7f3900076f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:30.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.700+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f39201a87d0 con 0x7f39180df680 2024-09-17T03:32:30.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.704+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39200927e0 con 0x7f39180df680 2024-09-17T03:32:30.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.716+0000 7f3924e00640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39000749f0 0x7f3900076f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:30.736+0000 7f3924e00640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39000749f0 0x7f3900076f00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3914007fd0 tx=0x7f3914007e40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.148+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone522018", "target": ["mon-mgr", ""]}) -- 0x7f3918053570 con 0x7f39000749f0 2024-09-17T03:32:31.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.464+0000 7f391cc00640 1 -- 172.21.15.46:0/3753200448 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3918053570 con 0x7f39000749f0 2024-09-17T03:32:31.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.468+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39000749f0 msgr2=0x7f3900076f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:31.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.468+0000 7f3926200640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39000749f0 0x7f3900076f00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3914007fd0 tx=0x7f3914007e40 comp rx=0 tx=0).stop 2024-09-17T03:32:31.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.468+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 msgr2=0x7f39180da910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.468+0000 7f3926200640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180da910 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f39200779b0 tx=0x7f392007b0e0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 shutdown_connections 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f39000749f0 0x7f3900076f00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39180df680 0x7f39180da910 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f391800be10 0x7f3918182730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 --2- 172.21.15.46:0/3753200448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f391800b9d0 0x7f3918185380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 >> 172.21.15.46:0/3753200448 conn(0x7f3918054970 msgr2=0x7f39180d69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 shutdown_connections 2024-09-17T03:32:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.472+0000 7f3926200640 1 -- 172.21.15.46:0/3753200448 wait complete. 2024-09-17T03:32:31.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone582719 2024-09-17T03:32:31.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 -- 172.21.15.46:0/3046733326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b6b20 msgr2=0x7fa4840b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 --2- 172.21.15.46:0/3046733326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b6b20 0x7fa4840b0920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009c80 tx=0x7fa47c01cd90 comp rx=0 tx=0).stop 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 -- 172.21.15.46:0/3046733326 shutdown_connections 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 --2- 172.21.15.46:0/3046733326 >> v2:172.21.15.46:3300/0 conn(0x7fa4841603c0 0x7fa4841607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 --2- 172.21.15.46:0/3046733326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b6b20 0x7fa4840b0920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 -- 172.21.15.46:0/3046733326 >> 172.21.15.46:0/3046733326 conn(0x7fa48409ba70 msgr2=0x7fa4840b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 -- 172.21.15.46:0/3046733326 shutdown_connections 2024-09-17T03:32:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.876+0000 7fa48b800640 1 -- 172.21.15.46:0/3046733326 wait complete. 2024-09-17T03:32:31.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.880+0000 7fa48b800640 1 Processor -- start 2024-09-17T03:32:31.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.880+0000 7fa48b800640 1 -- start start 2024-09-17T03:32:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa48b800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b53d0 0x7fa48406e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa48b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 0x7fa4841429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa48b800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841603c0 0x7fa48406cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa48b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa48416a4e0 con 0x7fa4840b6b20 2024-09-17T03:32:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa48b800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa48416a360 con 0x7fa4841603c0 2024-09-17T03:32:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa48b800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa48416a660 con 0x7fa4840b53d0 2024-09-17T03:32:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 0x7fa4841429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 0x7fa4841429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41976/0 (socket says 172.21.15.46:41976) 2024-09-17T03:32:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 -- 172.21.15.46:0/263961528 learned_addr learned my addr 172.21.15.46:0/263961528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa488e00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b53d0 0x7fa48406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa489800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841603c0 0x7fa48406cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 -- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b53d0 msgr2=0x7fa48406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:31.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b53d0 0x7fa48406e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 -- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841603c0 msgr2=0x7fa48406cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:31.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841603c0 0x7fa48406cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:31.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 -- 172.21.15.46:0/263961528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840b3f30 con 0x7fa4840b6b20 2024-09-17T03:32:31.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa483e00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 0x7fa4841429d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa47400b440 tx=0x7fa474009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.884+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474015020 con 0x7fa4840b6b20 2024-09-17T03:32:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.888+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa474011920 con 0x7fa4840b6b20 2024-09-17T03:32:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.888+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474011d50 con 0x7fa4840b6b20 2024-09-17T03:32:31.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.888+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa484166820 con 0x7fa4840b6b20 2024-09-17T03:32:31.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.888+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48413c460 con 0x7fa4840b6b20 2024-09-17T03:32:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.888+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4840b0920 con 0x7fa4840b6b20 2024-09-17T03:32:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.892+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fa474018020 con 0x7fa4840b6b20 2024-09-17T03:32:31.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.892+0000 7fa480c00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074820 0x7fa464076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:31.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.892+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fa474093130 con 0x7fa4840b6b20 2024-09-17T03:32:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.900+0000 7fa488e00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074820 0x7fa464076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.900+0000 7fa488e00640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074820 0x7fa464076d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0af000 tx=0x7fa47c01b5d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:31.904+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa47401f020 con 0x7fa4840b6b20 2024-09-17T03:32:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.276+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone582719", "target": ["mon-mgr", ""]}) -- 0x7fa48406b540 con 0x7fa464074820 2024-09-17T03:32:32.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.444+0000 7fa480c00640 1 -- 172.21.15.46:0/263961528 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa47c004ad0 con 0x7fa464074820 2024-09-17T03:32:32.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074820 msgr2=0x7fa464076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074820 0x7fa464076d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0af000 tx=0x7fa47c01b5d0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 msgr2=0x7fa4841429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 0x7fa4841429d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa47400b440 tx=0x7fa474009f90 comp rx=0 tx=0).stop 2024-09-17T03:32:32.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 shutdown_connections 2024-09-17T03:32:32.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074820 0x7fa464076d30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4841603c0 0x7fa48406cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4840b6b20 0x7fa4841429d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 --2- 172.21.15.46:0/263961528 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4840b53d0 0x7fa48406e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 >> 172.21.15.46:0/263961528 conn(0x7fa48409ba70 msgr2=0x7fa484143950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 shutdown_connections 2024-09-17T03:32:32.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.448+0000 7fa48b800640 1 -- 172.21.15.46:0/263961528 wait complete. 2024-09-17T03:32:32.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 -- 172.21.15.46:0/868764329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed401527b0 msgr2=0x7fed40074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 --2- 172.21.15.46:0/868764329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed401527b0 0x7fed40074e30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009900 tx=0x7fed2c01c820 comp rx=0 tx=0).stop 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 -- 172.21.15.46:0/868764329 shutdown_connections 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 --2- 172.21.15.46:0/868764329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed401527b0 0x7fed40074e30 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 --2- 172.21.15.46:0/868764329 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed400b5c70 0x7fed4015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 -- 172.21.15.46:0/868764329 >> 172.21.15.46:0/868764329 conn(0x7fed400b2040 msgr2=0x7fed400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:32.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 -- 172.21.15.46:0/868764329 shutdown_connections 2024-09-17T03:32:32.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.628+0000 7fed48200640 1 -- 172.21.15.46:0/868764329 wait complete. 2024-09-17T03:32:32.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 Processor -- start 2024-09-17T03:32:32.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 -- start start 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed400b4de0 0x7fed401411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 0x7fed4013d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed401527b0 0x7fed400af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed40168ca0 con 0x7fed400b5c70 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed40168b20 con 0x7fed401527b0 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed48200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fed40168e20 con 0x7fed400b4de0 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed44e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 0x7fed4013d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed44e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 0x7fed4013d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51988/0 (socket says 172.21.15.46:51988) 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed44e00640 1 -- 172.21.15.46:0/364092179 learned_addr learned my addr 172.21.15.46:0/364092179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed45800640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed400b4de0 0x7fed401411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.632+0000 7fed46200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed401527b0 0x7fed400af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed44e00640 1 -- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed400b4de0 msgr2=0x7fed401411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed44e00640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed400b4de0 0x7fed401411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed44e00640 1 -- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed401527b0 msgr2=0x7fed400af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed44e00640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed401527b0 0x7fed400af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed44e00640 1 -- 172.21.15.46:0/364092179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4013dec0 con 0x7fed400b5c70 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed44e00640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 0x7fed4013d430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009850 tx=0x7fed2c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0b2040 con 0x7fed400b5c70 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed2c005030 con 0x7fed400b5c70 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c005420 con 0x7fed400b5c70 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4013e1a0 con 0x7fed400b5c70 2024-09-17T03:32:32.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4013e760 con 0x7fed400b5c70 2024-09-17T03:32:32.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4015bb60 con 0x7fed400b5c70 2024-09-17T03:32:32.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.636+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7fed2c0bd060 con 0x7fed400b5c70 2024-09-17T03:32:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.640+0000 7fed3d600640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed200747f0 0x7fed20076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.640+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fed2c12c3f0 con 0x7fed400b5c70 2024-09-17T03:32:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.640+0000 7fed45800640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed200747f0 0x7fed20076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.640+0000 7fed45800640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed200747f0 0x7fed20076d00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fed34004630 tx=0x7fed3400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.644+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fed40075580 con 0x7fed400b5c70 2024-09-17T03:32:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.840+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fed4006b540 con 0x7fed200747f0 2024-09-17T03:32:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.840+0000 7fed3d600640 1 -- 172.21.15.46:0/364092179 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fed3400b7e0 con 0x7fed200747f0 2024-09-17T03:32:32.842 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:32:32.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:32.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:32:32.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:32.843 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed200747f0 msgr2=0x7fed20076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed200747f0 0x7fed20076d00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fed34004630 tx=0x7fed3400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 msgr2=0x7fed4013d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 0x7fed4013d430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009850 tx=0x7fed2c004040 comp rx=0 tx=0).stop 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 shutdown_connections 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fed200747f0 0x7fed20076d00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed401527b0 0x7fed400af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5c70 0x7fed4013d430 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 --2- 172.21.15.46:0/364092179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fed400b4de0 0x7fed401411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 >> 172.21.15.46:0/364092179 conn(0x7fed400b2040 msgr2=0x7fed400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 shutdown_connections 2024-09-17T03:32:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:32.844+0000 7fed48200640 1 -- 172.21.15.46:0/364092179 wait complete. 2024-09-17T03:32:32.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:32:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 -- 172.21.15.46:0/2457609645 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc1527b0 msgr2=0x7f80fc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 --2- 172.21.15.46:0/2457609645 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc1527b0 0x7f80fc074e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009900 tx=0x7f80e801c820 comp rx=0 tx=0).stop 2024-09-17T03:32:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 -- 172.21.15.46:0/2457609645 shutdown_connections 2024-09-17T03:32:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 --2- 172.21.15.46:0/2457609645 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc1527b0 0x7f80fc074e30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 --2- 172.21.15.46:0/2457609645 >> v2:172.21.15.99:3300/0 conn(0x7f80fc0b5c70 0x7f80fc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 -- 172.21.15.46:0/2457609645 >> 172.21.15.46:0/2457609645 conn(0x7f80fc0b2040 msgr2=0x7f80fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 -- 172.21.15.46:0/2457609645 shutdown_connections 2024-09-17T03:32:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.028+0000 7f8104200640 1 -- 172.21.15.46:0/2457609645 wait complete. 2024-09-17T03:32:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 Processor -- start 2024-09-17T03:32:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 -- start start 2024-09-17T03:32:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 0x7f80fc1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc0b5c70 0x7f80fc144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1527b0 0x7f80fc13f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc168bb0 con 0x7f80fc0b4de0 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc168a30 con 0x7f80fc1527b0 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8104200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f80fc168d30 con 0x7f80fc0b5c70 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 0x7f80fc1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 0x7f80fc1551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52004/0 (socket says 172.21.15.46:52004) 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 -- 172.21.15.46:0/704777703 learned_addr learned my addr 172.21.15.46:0/704777703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8100e00640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc0b5c70 0x7f80fc144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8102200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1527b0 0x7f80fc13f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 -- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc0b5c70 msgr2=0x7f80fc144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc0b5c70 0x7f80fc144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 -- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1527b0 msgr2=0x7f80fc13f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1527b0 0x7f80fc13f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 -- 172.21.15.46:0/704777703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc1451f0 con 0x7f80fc0b4de0 2024-09-17T03:32:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.032+0000 7f8101800640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 0x7f80fc1551f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f80f000e9a0 tx=0x7f80f000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f000cd80 con 0x7f80fc0b4de0 2024-09-17T03:32:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc154150 con 0x7f80fc0b4de0 2024-09-17T03:32:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f0004020 con 0x7f80fc0b4de0 2024-09-17T03:32:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0004450 con 0x7f80fc0b4de0 2024-09-17T03:32:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc154710 con 0x7f80fc0b4de0 2024-09-17T03:32:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80fc15bb60 con 0x7f80fc0b4de0 2024-09-17T03:32:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f80f0003600 con 0x7f80fc0b4de0 2024-09-17T03:32:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f80f9600640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f80dc0747f0 0x7f80dc076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f80f0093870 con 0x7f80fc0b4de0 2024-09-17T03:32:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.036+0000 7f8100e00640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f80dc0747f0 0x7f80dc076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.040+0000 7f8100e00640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f80dc0747f0 0x7f80dc076d00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f80e80af000 tx=0x7f80e801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.044+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f80f0093d10 con 0x7f80fc0b4de0 2024-09-17T03:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.244+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80fc06b540 con 0x7f80dc0747f0 2024-09-17T03:32:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.260+0000 7f80f9600640 1 -- 172.21.15.46:0/704777703 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1027 (secure 0 0 0) 0x7f80e80b3040 con 0x7f80dc0747f0 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone16310" 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone522018" 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.262 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone616815" 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone52350" 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone77144" 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone28249" 2024-09-17T03:32:33.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone59755" 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone50292" 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone73317" 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone551017" 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone582719" 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone13923" 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone60228" 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone579713" 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone18656" 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone560911" 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone838216" 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone341414" 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone955412" 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone79881" 2024-09-17T03:32:33.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:33.268 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f80dc0747f0 msgr2=0x7f80dc076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f80dc0747f0 0x7f80dc076d00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f80e80af000 tx=0x7f80e801b060 comp rx=0 tx=0).stop 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 msgr2=0x7f80fc1551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 0x7f80fc1551f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f80f000e9a0 tx=0x7f80f000ee60 comp rx=0 tx=0).stop 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 shutdown_connections 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f80dc0747f0 0x7f80dc076d00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1527b0 0x7f80fc13f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80fc0b5c70 0x7f80fc144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 --2- 172.21.15.46:0/704777703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80fc0b4de0 0x7f80fc1551f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 >> 172.21.15.46:0/704777703 conn(0x7f80fc0b2040 msgr2=0x7f80fc151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 shutdown_connections 2024-09-17T03:32:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.264+0000 7f8104200640 1 -- 172.21.15.46:0/704777703 wait complete. 2024-09-17T03:32:33.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone16310 2024-09-17T03:32:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 -- 172.21.15.46:0/3096526567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f280b5c70 msgr2=0x7f6f2815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 --2- 172.21.15.46:0/3096526567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f280b5c70 0x7f6f2815bb60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6f28070710 tx=0x7f6f1c01c800 comp rx=0 tx=0).stop 2024-09-17T03:32:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 -- 172.21.15.46:0/3096526567 shutdown_connections 2024-09-17T03:32:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 --2- 172.21.15.46:0/3096526567 >> v2:172.21.15.99:3300/0 conn(0x7f6f281527b0 0x7f6f28074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 --2- 172.21.15.46:0/3096526567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f280b5c70 0x7f6f2815bb60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 -- 172.21.15.46:0/3096526567 >> 172.21.15.46:0/3096526567 conn(0x7f6f280b2040 msgr2=0x7f6f280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 -- 172.21.15.46:0/3096526567 shutdown_connections 2024-09-17T03:32:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 -- 172.21.15.46:0/3096526567 wait complete. 2024-09-17T03:32:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 Processor -- start 2024-09-17T03:32:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.452+0000 7f6f2f400640 1 -- start start 2024-09-17T03:32:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f280b4de0 0x7f6f281fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6f280b5c70 0x7f6f2813e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 0x7f6f2813e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f28168a80 con 0x7f6f281527b0 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f28168900 con 0x7f6f280b4de0 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6f28168c00 con 0x7f6f280b5c70 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 0x7f6f2813e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f280b4de0 0x7f6f281fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 0x7f6f2813e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52022/0 (socket says 172.21.15.46:52022) 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2d400640 1 -- 172.21.15.46:0/2312193885 learned_addr learned my addr 172.21.15.46:0/2312193885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f27e00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6f280b5c70 0x7f6f2813e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 -- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6f280b5c70 msgr2=0x7f6f2813e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6f280b5c70 0x7f6f2813e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 -- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 msgr2=0x7f6f2813e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 0x7f6f2813e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 -- 172.21.15.46:0/2312193885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f28157340 con 0x7f6f280b4de0 2024-09-17T03:32:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2d400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 0x7f6f2813e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2ca00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f280b4de0 0x7f6f281fbd40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0ac000 tx=0x7f6f1c0a56e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0b2040 con 0x7f6f280b4de0 2024-09-17T03:32:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f1c0bd070 con 0x7f6f280b4de0 2024-09-17T03:32:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c003600 con 0x7f6f280b4de0 2024-09-17T03:32:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f28157620 con 0x7f6f280b4de0 2024-09-17T03:32:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f28157be0 con 0x7f6f280b4de0 2024-09-17T03:32:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.456+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2815bb60 con 0x7f6f280b4de0 2024-09-17T03:32:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.460+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f6f1c0037a0 con 0x7f6f280b4de0 2024-09-17T03:32:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.460+0000 7f6f24c00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6f08074820 0x7f6f08076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.460+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f6f1c12d8e0 con 0x7f6f280b4de0 2024-09-17T03:32:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.460+0000 7f6f27e00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6f08074820 0x7f6f08076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.460+0000 7f6f27e00640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6f08074820 0x7f6f08076d30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6f280b1000 tx=0x7f6f140091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.464+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f28075580 con 0x7f6f280b4de0 2024-09-17T03:32:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.664+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone16310", "target": ["mon-mgr", ""]}) -- 0x7f6f28141ef0 con 0x7f6f08074820 2024-09-17T03:32:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f24c00640 1 -- 172.21.15.46:0/2312193885 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f28141ef0 con 0x7f6f08074820 2024-09-17T03:32:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6f08074820 msgr2=0x7f6f08076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6f08074820 0x7f6f08076d30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6f280b1000 tx=0x7f6f140091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f280b4de0 msgr2=0x7f6f281fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f280b4de0 0x7f6f281fbd40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0ac000 tx=0x7f6f1c0a56e0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 shutdown_connections 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6f08074820 0x7f6f08076d30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f281527b0 0x7f6f2813e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6f280b5c70 0x7f6f2813e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 --2- 172.21.15.46:0/2312193885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f280b4de0 0x7f6f281fbd40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 >> 172.21.15.46:0/2312193885 conn(0x7f6f280b2040 msgr2=0x7f6f280b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 shutdown_connections 2024-09-17T03:32:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.688+0000 7f6f2f400640 1 -- 172.21.15.46:0/2312193885 wait complete. 2024-09-17T03:32:33.692 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.692+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.692 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.692+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.692 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.692+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.692 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.692+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.693 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.692+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.713 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.712+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.713 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.712+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.713 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.712+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.713 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.712+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.713 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:32:33.712+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:32:33.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone522018 2024-09-17T03:32:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 -- 172.21.15.46:0/2018260139 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 msgr2=0x7f986015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 --2- 172.21.15.46:0/2018260139 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 0x7f986015bb60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9854009840 tx=0x7f985401c760 comp rx=0 tx=0).stop 2024-09-17T03:32:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 -- 172.21.15.46:0/2018260139 shutdown_connections 2024-09-17T03:32:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 --2- 172.21.15.46:0/2018260139 >> v2:172.21.15.99:3300/0 conn(0x7f98601527b0 0x7f9860074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 --2- 172.21.15.46:0/2018260139 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 0x7f986015bb60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 -- 172.21.15.46:0/2018260139 >> 172.21.15.46:0/2018260139 conn(0x7f98600b2040 msgr2=0x7f98600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 -- 172.21.15.46:0/2018260139 shutdown_connections 2024-09-17T03:32:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 -- 172.21.15.46:0/2018260139 wait complete. 2024-09-17T03:32:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 Processor -- start 2024-09-17T03:32:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.888+0000 7f9867800640 1 -- start start 2024-09-17T03:32:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 0x7f9860158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 0x7f98601589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98601527b0 0x7f98601571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9860168c50 con 0x7f98600b4de0 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9860168ad0 con 0x7f98601527b0 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9860168dd0 con 0x7f98600b5c70 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 0x7f9860158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 0x7f9860158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52056/0 (socket says 172.21.15.46:52056) 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 -- 172.21.15.46:0/2723884836 learned_addr learned my addr 172.21.15.46:0/2723884836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9865800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98601527b0 0x7f98601571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f985fe00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 0x7f98601589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 -- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 msgr2=0x7f98601589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 0x7f98601589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 -- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98601527b0 msgr2=0x7f98601571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98601527b0 0x7f98601571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 -- 172.21.15.46:0/2723884836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860158020 con 0x7f98600b4de0 2024-09-17T03:32:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9864e00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 0x7f9860158460 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9854009790 tx=0x7f98540a5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540bb070 con 0x7f98600b4de0 2024-09-17T03:32:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9854004470 con 0x7f98600b4de0 2024-09-17T03:32:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540b2040 con 0x7f98600b4de0 2024-09-17T03:32:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9860156410 con 0x7f98600b4de0 2024-09-17T03:32:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98601569d0 con 0x7f98600b4de0 2024-09-17T03:32:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.892+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986015bb60 con 0x7f98600b4de0 2024-09-17T03:32:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.896+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f9854005290 con 0x7f98600b4de0 2024-09-17T03:32:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.896+0000 7f985cc00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9840074820 0x7f9840076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.896+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f985412c870 con 0x7f98600b4de0 2024-09-17T03:32:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.896+0000 7f985fe00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9840074820 0x7f9840076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:33.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.896+0000 7f985fe00640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9840074820 0x7f9840076d30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f98601591c0 tx=0x7f984c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:33.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:33.900+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9860075580 con 0x7f98600b4de0 2024-09-17T03:32:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.100+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone522018", "target": ["mon-mgr", ""]}) -- 0x7f98601561c0 con 0x7f9840074820 2024-09-17T03:32:34.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.772+0000 7f985cc00640 1 -- 172.21.15.46:0/2723884836 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98601561c0 con 0x7f9840074820 2024-09-17T03:32:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9840074820 msgr2=0x7f9840076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9840074820 0x7f9840076d30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f98601591c0 tx=0x7f984c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 msgr2=0x7f9860158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 0x7f9860158460 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9854009790 tx=0x7f98540a5640 comp rx=0 tx=0).stop 2024-09-17T03:32:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 shutdown_connections 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9840074820 0x7f9840076d30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98601527b0 0x7f98601571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f98600b5c70 0x7f98601589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 --2- 172.21.15.46:0/2723884836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98600b4de0 0x7f9860158460 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 >> 172.21.15.46:0/2723884836 conn(0x7f98600b2040 msgr2=0x7f98600b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 shutdown_connections 2024-09-17T03:32:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.776+0000 7f9867800640 1 -- 172.21.15.46:0/2723884836 wait complete. 2024-09-17T03:32:34.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone616815 2024-09-17T03:32:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 -- 172.21.15.46:0/1853949640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 msgr2=0x7f6264074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 --2- 172.21.15.46:0/1853949640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f6264074e30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6250009900 tx=0x7f625001c820 comp rx=0 tx=0).stop 2024-09-17T03:32:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 -- 172.21.15.46:0/1853949640 shutdown_connections 2024-09-17T03:32:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 --2- 172.21.15.46:0/1853949640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f6264074e30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 --2- 172.21.15.46:0/1853949640 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62640b5c70 0x7f626415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 -- 172.21.15.46:0/1853949640 >> 172.21.15.46:0/1853949640 conn(0x7f62640b2040 msgr2=0x7f62640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 -- 172.21.15.46:0/1853949640 shutdown_connections 2024-09-17T03:32:34.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 -- 172.21.15.46:0/1853949640 wait complete. 2024-09-17T03:32:34.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 Processor -- start 2024-09-17T03:32:34.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.980+0000 7f626c000640 1 -- start start 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62640b4de0 0x7f62641fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62640b5c70 0x7f62641f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f62641f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6264168c50 con 0x7f62641527b0 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6264168ad0 con 0x7f62640b5c70 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6264168dd0 con 0x7f62640b4de0 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f62641f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:34.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f6269600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62640b4de0 0x7f62641fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f62641f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52080/0 (socket says 172.21.15.46:52080) 2024-09-17T03:32:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 -- 172.21.15.46:0/1190468774 learned_addr learned my addr 172.21.15.46:0/1190468774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:34.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f6268c00640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62640b5c70 0x7f62641f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 -- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62640b4de0 msgr2=0x7f62641fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62640b4de0 0x7f62641fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 -- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62640b5c70 msgr2=0x7f62641f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62640b5c70 0x7f62641f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 -- 172.21.15.46:0/1190468774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62641f9a60 con 0x7f62641527b0 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626a000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f62641f9200 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c770 tx=0x7f625c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c003b80 con 0x7f62641527b0 2024-09-17T03:32:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f625c003d20 con 0x7f62641527b0 2024-09-17T03:32:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c00aab0 con 0x7f62641527b0 2024-09-17T03:32:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62641572b0 con 0x7f62641527b0 2024-09-17T03:32:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6264157870 con 0x7f62641527b0 2024-09-17T03:32:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.984+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626415bb60 con 0x7f62641527b0 2024-09-17T03:32:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.988+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f625c00ac50 con 0x7f62641527b0 2024-09-17T03:32:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.988+0000 7f6261600640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6244074820 0x7f6244076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.988+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f625c093000 con 0x7f62641527b0 2024-09-17T03:32:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.988+0000 7f6269600640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6244074820 0x7f6244076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.988+0000 7f6269600640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6244074820 0x7f6244076d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f625800bb50 tx=0x7f625800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:34.992+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f625c05f0a0 con 0x7f62641527b0 2024-09-17T03:32:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.196+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone616815", "target": ["mon-mgr", ""]}) -- 0x7f62640b63f0 con 0x7f6244074820 2024-09-17T03:32:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.216+0000 7f6261600640 1 -- 172.21.15.46:0/1190468774 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62640b63f0 con 0x7f6244074820 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6244074820 msgr2=0x7f6244076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6244074820 0x7f6244076d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f625800bb50 tx=0x7f625800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 msgr2=0x7f62641f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f62641f9200 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f625c00c770 tx=0x7f625c00cc30 comp rx=0 tx=0).stop 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 shutdown_connections 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6244074820 0x7f6244076d30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62641527b0 0x7f62641f9200 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62640b5c70 0x7f62641f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 --2- 172.21.15.46:0/1190468774 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62640b4de0 0x7f62641fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 >> 172.21.15.46:0/1190468774 conn(0x7f62640b2040 msgr2=0x7f62640b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 shutdown_connections 2024-09-17T03:32:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.220+0000 7f626c000640 1 -- 172.21.15.46:0/1190468774 wait complete. 2024-09-17T03:32:35.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone52350 2024-09-17T03:32:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.412+0000 7f62df400640 1 -- 172.21.15.46:0/263286383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 msgr2=0x7f62d8074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.412+0000 7f62df400640 1 --2- 172.21.15.46:0/263286383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8074db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009900 tx=0x7f62c801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 -- 172.21.15.46:0/263286383 shutdown_connections 2024-09-17T03:32:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 --2- 172.21.15.46:0/263286383 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62d80b3340 0x7f62d80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 --2- 172.21.15.46:0/263286383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8074db0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 -- 172.21.15.46:0/263286383 >> 172.21.15.46:0/263286383 conn(0x7f62d80b2040 msgr2=0x7f62d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 -- 172.21.15.46:0/263286383 shutdown_connections 2024-09-17T03:32:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 -- 172.21.15.46:0/263286383 wait complete. 2024-09-17T03:32:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 Processor -- start 2024-09-17T03:32:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.416+0000 7f62df400640 1 -- start start 2024-09-17T03:32:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62d80b3340 0x7f62d80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62d80cefa0 0x7f62d80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8169e10 con 0x7f62d80749a0 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8169c90 con 0x7f62d80b3340 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f62d8169f90 con 0x7f62d80cefa0 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52106/0 (socket says 172.21.15.46:52106) 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 -- 172.21.15.46:0/3174524817 learned_addr learned my addr 172.21.15.46:0/3174524817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62cfe00640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62d80b3340 0x7f62d80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dd400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62d80cefa0 0x7f62d80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 -- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62d80cefa0 msgr2=0x7f62d80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62d80cefa0 0x7f62d80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 -- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62d80b3340 msgr2=0x7f62d80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62d80b3340 0x7f62d80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 -- 172.21.15.46:0/3174524817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d815a440 con 0x7f62d80749a0 2024-09-17T03:32:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62dca00640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8159f00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009850 tx=0x7f62c80a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c80b2040 con 0x7f62d80749a0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62c80044b0 con 0x7f62d80749a0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d81419e0 con 0x7f62d80749a0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8005270 con 0x7f62d80749a0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d8141f80 con 0x7f62d80749a0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d8074db0 con 0x7f62d80749a0 2024-09-17T03:32:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.420+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) ==== 99223+0+0 (secure 0 0 0) 0x7f62c8005410 con 0x7f62d80749a0 2024-09-17T03:32:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.424+0000 7f62d5600640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f62b40747f0 0x7f62b4076d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.424+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f62c812ca20 con 0x7f62d80749a0 2024-09-17T03:32:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.424+0000 7f62cfe00640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f62b40747f0 0x7f62b4076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.424+0000 7f62cfe00640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f62b40747f0 0x7f62b4076d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f62d8146360 tx=0x7f62c0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.428+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f62c80bc060 con 0x7f62d80749a0 2024-09-17T03:32:35.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.632+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone52350", "target": ["mon-mgr", ""]}) -- 0x7f62d813b5f0 con 0x7f62b40747f0 2024-09-17T03:32:35.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.652+0000 7f62d5600640 1 -- 172.21.15.46:0/3174524817 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62c00050d0 con 0x7f62b40747f0 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f62b40747f0 msgr2=0x7f62b4076d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f62b40747f0 0x7f62b4076d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f62d8146360 tx=0x7f62c0009040 comp rx=0 tx=0).stop 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 msgr2=0x7f62d8159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8159f00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009850 tx=0x7f62c80a5810 comp rx=0 tx=0).stop 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 shutdown_connections 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f62b40747f0 0x7f62b4076d00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f62d80cefa0 0x7f62d80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62d80b3340 0x7f62d80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 --2- 172.21.15.46:0/3174524817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d80749a0 0x7f62d8159f00 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 >> 172.21.15.46:0/3174524817 conn(0x7f62d80b2040 msgr2=0x7f62d815bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 shutdown_connections 2024-09-17T03:32:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.656+0000 7f62df400640 1 -- 172.21.15.46:0/3174524817 wait complete. 2024-09-17T03:32:35.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone77144 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 -- 172.21.15.46:0/203195678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740732c0 msgr2=0x7f1c740736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 --2- 172.21.15.46:0/203195678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740732c0 0x7f1c740736d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009900 tx=0x7f1c6801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 -- 172.21.15.46:0/203195678 shutdown_connections 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 --2- 172.21.15.46:0/203195678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740732c0 0x7f1c740736d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 --2- 172.21.15.46:0/203195678 >> v2:172.21.15.99:3300/0 conn(0x7f1c740ae8e0 0x7f1c740aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 -- 172.21.15.46:0/203195678 >> 172.21.15.46:0/203195678 conn(0x7f1c740b2c40 msgr2=0x7f1c740b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.904+0000 7f1c7c200640 1 -- 172.21.15.46:0/203195678 shutdown_connections 2024-09-17T03:32:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.908+0000 7f1c7c200640 1 -- 172.21.15.46:0/203195678 wait complete. 2024-09-17T03:32:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.908+0000 7f1c7c200640 1 Processor -- start 2024-09-17T03:32:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.908+0000 7f1c7c200640 1 -- start start 2024-09-17T03:32:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7c200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c740732c0 0x7f1c74146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c740ae8e0 0x7f1c74070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7c200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 0x7f1c7414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7416c350 con 0x7f1c740ae8e0 2024-09-17T03:32:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7c200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7416c1d0 con 0x7f1c740732c0 2024-09-17T03:32:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7c200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7416c4d0 con 0x7f1c740b4630 2024-09-17T03:32:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 0x7f1c7414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 0x7f1c7414e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:38194/0 (socket says 172.21.15.46:38194) 2024-09-17T03:32:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7a200640 1 -- 172.21.15.46:0/2647026103 learned_addr learned my addr 172.21.15.46:0/2647026103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c740732c0 0x7f1c74146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c78e00640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c740ae8e0 0x7f1c74070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 -- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 msgr2=0x7f1c7414e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 0x7f1c7414e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 -- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c740ae8e0 msgr2=0x7f1c74070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c740ae8e0 0x7f1c74070790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 -- 172.21.15.46:0/2647026103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c741f9130 con 0x7f1c740732c0 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c7a200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 0x7f1c7414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c79800640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c740732c0 0x7f1c74146060 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00bd70 tx=0x7f1c6c00be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c01d070 con 0x7f1c740732c0 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c6c019610 con 0x7f1c740732c0 2024-09-17T03:32:35.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.912+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c019a80 con 0x7f1c740732c0 2024-09-17T03:32:35.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.916+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741f9410 con 0x7f1c740732c0 2024-09-17T03:32:35.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.916+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741f99d0 con 0x7f1c740732c0 2024-09-17T03:32:35.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.916+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c740736d0 con 0x7f1c740732c0 2024-09-17T03:32:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.916+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1c6c020070 con 0x7f1c740732c0 2024-09-17T03:32:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.920+0000 7f1c71600640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c58074910 0x7f1c58076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.920+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f1c6c09be30 con 0x7f1c740732c0 2024-09-17T03:32:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.920+0000 7f1c78e00640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c58074910 0x7f1c58076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.920+0000 7f1c78e00640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c58074910 0x7f1c58076e20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1c741f6c10 tx=0x7f1c64008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:35.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:35.924+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1c6c068370 con 0x7f1c740732c0 2024-09-17T03:32:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.260+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone77144", "target": ["mon-mgr", ""]}) -- 0x7f1c7406b540 con 0x7f1c58074910 2024-09-17T03:32:36.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c71600640 1 -- 172.21.15.46:0/2647026103 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1c6400b420 con 0x7f1c58074910 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c58074910 msgr2=0x7f1c58076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c58074910 0x7f1c58076e20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1c741f6c10 tx=0x7f1c64008040 comp rx=0 tx=0).stop 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c740732c0 msgr2=0x7f1c74146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c740732c0 0x7f1c74146060 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00bd70 tx=0x7f1c6c00be60 comp rx=0 tx=0).stop 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 shutdown_connections 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c58074910 0x7f1c58076e20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c740b4630 0x7f1c7414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c740ae8e0 0x7f1c74070790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 --2- 172.21.15.46:0/2647026103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c740732c0 0x7f1c74146060 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.284+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 >> 172.21.15.46:0/2647026103 conn(0x7f1c740b2c40 msgr2=0x7f1c740b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:36.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.288+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 shutdown_connections 2024-09-17T03:32:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.288+0000 7f1c7c200640 1 -- 172.21.15.46:0/2647026103 wait complete. 2024-09-17T03:32:36.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28249 2024-09-17T03:32:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 -- 172.21.15.46:0/3524718669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 msgr2=0x7f47b015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 --2- 172.21.15.46:0/3524718669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b015d480 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f47b00cf890 tx=0x7f47a001c830 comp rx=0 tx=0).stop 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 -- 172.21.15.46:0/3524718669 shutdown_connections 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 --2- 172.21.15.46:0/3524718669 >> v2:172.21.15.99:3300/0 conn(0x7f47b0074950 0x7f47b00b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 --2- 172.21.15.46:0/3524718669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b015d480 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 -- 172.21.15.46:0/3524718669 >> 172.21.15.46:0/3524718669 conn(0x7f47b00b4560 msgr2=0x7f47b00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 -- 172.21.15.46:0/3524718669 shutdown_connections 2024-09-17T03:32:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.472+0000 7f47afe00640 1 -- 172.21.15.46:0/3524718669 wait complete. 2024-09-17T03:32:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 Processor -- start 2024-09-17T03:32:36.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 -- start start 2024-09-17T03:32:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0160060 0x7f47b0152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47b014bf70 0x7f47b0152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0169dc0 con 0x7f47b01536f0 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0169c40 con 0x7f47b0160060 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47afe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f47b0169f40 con 0x7f47b014bf70 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b014b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52138/0 (socket says 172.21.15.46:52138) 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 -- 172.21.15.46:0/54509652 learned_addr learned my addr 172.21.15.46:0/54509652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47a7e00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0160060 0x7f47b0152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47af400640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47b014bf70 0x7f47b0152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 -- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47b014bf70 msgr2=0x7f47b0152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47b014bf70 0x7f47b0152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 -- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0160060 msgr2=0x7f47b0152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0160060 0x7f47b0152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47aea00640 1 -- 172.21.15.46:0/54509652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b0156650 con 0x7f47b01536f0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47a7e00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0160060 0x7f47b0152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.476+0000 7f47af400640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47b014bf70 0x7f47b0152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47aea00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b014b9d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f47a001cd10 tx=0x7f47a00a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0004440 con 0x7f47b01536f0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47a0005160 con 0x7f47b01536f0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0005590 con 0x7f47b01536f0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b0156930 con 0x7f47b01536f0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b0153ec0 con 0x7f47b01536f0 2024-09-17T03:32:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b0154120 con 0x7f47b01536f0 2024-09-17T03:32:36.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.480+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f47a00046d0 con 0x7f47b01536f0 2024-09-17T03:32:36.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.484+0000 7f47a6a00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f478c074940 0x7f478c076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.484+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f47a012d450 con 0x7f47b01536f0 2024-09-17T03:32:36.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.484+0000 7f47a7e00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f478c074940 0x7f478c076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.484+0000 7f47a7e00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f478c074940 0x7f478c076e50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f47b014c470 tx=0x7f4798009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.488+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f47a00bf070 con 0x7f47b01536f0 2024-09-17T03:32:36.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.684+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28249", "target": ["mon-mgr", ""]}) -- 0x7f47b006b540 con 0x7f478c074940 2024-09-17T03:32:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.708+0000 7f47a6a00640 1 -- 172.21.15.46:0/54509652 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4798001c00 con 0x7f478c074940 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f478c074940 msgr2=0x7f478c076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f478c074940 0x7f478c076e50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f47b014c470 tx=0x7f4798009040 comp rx=0 tx=0).stop 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 msgr2=0x7f47b014b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b014b9d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f47a001cd10 tx=0x7f47a00a6570 comp rx=0 tx=0).stop 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 shutdown_connections 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f478c074940 0x7f478c076e50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47b014bf70 0x7f47b0152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0160060 0x7f47b0152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 --2- 172.21.15.46:0/54509652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b01536f0 0x7f47b014b9d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 >> 172.21.15.46:0/54509652 conn(0x7f47b00b4560 msgr2=0x7f47b015f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 shutdown_connections 2024-09-17T03:32:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.712+0000 7f47afe00640 1 -- 172.21.15.46:0/54509652 wait complete. 2024-09-17T03:32:36.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 -- 172.21.15.46:0/3287263378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06400b5c70 msgr2=0x7f064015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 --2- 172.21.15.46:0/3287263378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06400b5c70 0x7f064015bb90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0640074b20 tx=0x7f063401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 -- 172.21.15.46:0/3287263378 shutdown_connections 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 --2- 172.21.15.46:0/3287263378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06400b5c70 0x7f064015bb90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 -- 172.21.15.46:0/3287263378 >> 172.21.15.46:0/3287263378 conn(0x7f06400b2040 msgr2=0x7f06400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 -- 172.21.15.46:0/3287263378 shutdown_connections 2024-09-17T03:32:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.964+0000 7f0647e00640 1 -- 172.21.15.46:0/3287263378 wait complete. 2024-09-17T03:32:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 Processor -- start 2024-09-17T03:32:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 -- start start 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06400b4de0 0x7f06400b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06400b5c70 0x7f064014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 0x7f06401485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0640169b90 con 0x7f06401527b0 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0640169a10 con 0x7f06400b4de0 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0640169d10 con 0x7f06400b5c70 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 0x7f06401485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 0x7f06401485b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52150/0 (socket says 172.21.15.46:52150) 2024-09-17T03:32:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 -- 172.21.15.46:0/3354293779 learned_addr learned my addr 172.21.15.46:0/3354293779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645400640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06400b4de0 0x7f06400b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0644a00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06400b5c70 0x7f064014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 -- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06400b5c70 msgr2=0x7f064014bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06400b5c70 0x7f064014bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 -- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06400b4de0 msgr2=0x7f06400b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06400b4de0 0x7f06400b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 -- 172.21.15.46:0/3354293779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06401f8b50 con 0x7f06401527b0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0645e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 0x7f06401485b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f063800c710 tx=0x7f063800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638003b80 con 0x7f06401527b0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0638003d20 con 0x7f06401527b0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063800aab0 con 0x7f06401527b0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06401f8e30 con 0x7f06401527b0 2024-09-17T03:32:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.968+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06401f93f0 con 0x7f06401527b0 2024-09-17T03:32:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.972+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064015c1b0 con 0x7f06401527b0 2024-09-17T03:32:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.972+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f063800ac50 con 0x7f06401527b0 2024-09-17T03:32:36.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.972+0000 7f063d600640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0620074910 0x7f0620076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:36.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.972+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f0638093610 con 0x7f06401527b0 2024-09-17T03:32:36.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.972+0000 7f0645400640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0620074910 0x7f0620076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.976+0000 7f0645400640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0620074910 0x7f0620076e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06340af000 tx=0x7f063401b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:36.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:36.976+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f063800ce20 con 0x7f06401527b0 2024-09-17T03:32:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.180+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0640152bc0 con 0x7f0620074910 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.192+0000 7f063d600640 1 -- 172.21.15.46:0/3354293779 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0640152bc0 con 0x7f0620074910 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:37.193 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.192+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0620074910 msgr2=0x7f0620076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.192+0000 7f0647e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0620074910 0x7f0620076e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06340af000 tx=0x7f063401b0f0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.192+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 msgr2=0x7f06401485b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.192+0000 7f0647e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 0x7f06401485b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f063800c710 tx=0x7f063800cbd0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 shutdown_connections 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0620074910 0x7f0620076e20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401527b0 0x7f06401485b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06400b5c70 0x7f064014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 --2- 172.21.15.46:0/3354293779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06400b4de0 0x7f06400b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 >> 172.21.15.46:0/3354293779 conn(0x7f06400b2040 msgr2=0x7f0640154860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 shutdown_connections 2024-09-17T03:32:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.196+0000 7f0647e00640 1 -- 172.21.15.46:0/3354293779 wait complete. 2024-09-17T03:32:37.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 -- 172.21.15.46:0/1047197174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f8072100 msgr2=0x7fa8f8155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 --2- 172.21.15.46:0/1047197174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f8072100 0x7fa8f8155b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009870 tx=0x7fa8ec01c790 comp rx=0 tx=0).stop 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 -- 172.21.15.46:0/1047197174 shutdown_connections 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 --2- 172.21.15.46:0/1047197174 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8f80b4de0 0x7fa8f80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 --2- 172.21.15.46:0/1047197174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f8072100 0x7fa8f8155b50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 -- 172.21.15.46:0/1047197174 >> 172.21.15.46:0/1047197174 conn(0x7fa8f80b2040 msgr2=0x7fa8f80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 -- 172.21.15.46:0/1047197174 shutdown_connections 2024-09-17T03:32:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 -- 172.21.15.46:0/1047197174 wait complete. 2024-09-17T03:32:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.384+0000 7fa900600640 1 Processor -- start 2024-09-17T03:32:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- start start 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 0x7fa8f8157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f80b4de0 0x7fa8f80b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8f815f0e0 0x7fa8f80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81693d0 con 0x7fa8f8072100 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8169250 con 0x7fa8f80b4de0 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f8169550 con 0x7fa8f815f0e0 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 0x7fa8f8157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 0x7fa8f8157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52158/0 (socket says 172.21.15.46:52158) 2024-09-17T03:32:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 -- 172.21.15.46:0/3233992376 learned_addr learned my addr 172.21.15.46:0/3233992376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fd200640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f80b4de0 0x7fa8f80b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fe600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8f815f0e0 0x7fa8f80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 -- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8f815f0e0 msgr2=0x7fa8f80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8f815f0e0 0x7fa8f80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 -- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f80b4de0 msgr2=0x7fa8f80b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f80b4de0 0x7fa8f80b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 -- 172.21.15.46:0/3233992376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f8075300 con 0x7fa8f8072100 2024-09-17T03:32:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fd200640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f80b4de0 0x7fa8f80b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8fdc00640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 0x7fa8f8157480 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec01cc70 tx=0x7fa8ec0a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0ac070 con 0x7fa8f8072100 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f81fcdb0 con 0x7fa8f8072100 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ec005130 con 0x7fa8f8072100 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec005540 con 0x7fa8f8072100 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f81fd090 con 0x7fa8f8072100 2024-09-17T03:32:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.388+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f8155b50 con 0x7fa8f8072100 2024-09-17T03:32:37.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.392+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fa8ec0af070 con 0x7fa8f8072100 2024-09-17T03:32:37.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.392+0000 7fa8f5600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8d8074910 0x7fa8d8076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.392+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7fa8ec12d430 con 0x7fa8f8072100 2024-09-17T03:32:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.392+0000 7fa8fd200640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8d8074910 0x7fa8d8076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.392+0000 7fa8fd200640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8d8074910 0x7fa8d8076e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80afb00 tx=0x7fa8e40091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:37.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.396+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8ec0c1030 con 0x7fa8f8072100 2024-09-17T03:32:37.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.600+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8f806b540 con 0x7fa8d8074910 2024-09-17T03:32:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.628+0000 7fa8f5600640 1 -- 172.21.15.46:0/3233992376 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8e4004140 con 0x7fa8d8074910 2024-09-17T03:32:37.631 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:37.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8d8074910 msgr2=0x7fa8d8076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8d8074910 0x7fa8d8076e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80afb00 tx=0x7fa8e40091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 msgr2=0x7fa8f8157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 0x7fa8f8157480 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec01cc70 tx=0x7fa8ec0a6f90 comp rx=0 tx=0).stop 2024-09-17T03:32:37.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 shutdown_connections 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8d8074910 0x7fa8d8076e20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8f815f0e0 0x7fa8f80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f80b4de0 0x7fa8f80b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 --2- 172.21.15.46:0/3233992376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8072100 0x7fa8f8157480 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 >> 172.21.15.46:0/3233992376 conn(0x7fa8f80b2040 msgr2=0x7fa8f815d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 shutdown_connections 2024-09-17T03:32:37.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.632+0000 7fa900600640 1 -- 172.21.15.46:0/3233992376 wait complete. 2024-09-17T03:32:37.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:32:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 -- 172.21.15.46:0/3877924283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4178003c40 msgr2=0x7f4178004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 --2- 172.21.15.46:0/3877924283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4178003c40 0x7f4178004090 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f418400c3f0 tx=0x7f418401cf00 comp rx=0 tx=0).stop 2024-09-17T03:32:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 -- 172.21.15.46:0/3877924283 shutdown_connections 2024-09-17T03:32:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 --2- 172.21.15.46:0/3877924283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4178003c40 0x7f4178004090 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 -- 172.21.15.46:0/3877924283 >> 172.21.15.46:0/3877924283 conn(0x7f41940708c0 msgr2=0x7f4194070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 -- 172.21.15.46:0/3877924283 shutdown_connections 2024-09-17T03:32:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.864+0000 7f4199a00640 1 -- 172.21.15.46:0/3877924283 wait complete. 2024-09-17T03:32:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.868+0000 7f4199a00640 1 Processor -- start 2024-09-17T03:32:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.868+0000 7f4199a00640 1 -- start start 2024-09-17T03:32:37.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4199a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 0x7f41941fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4199a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4194152710 0x7f419414c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4199a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4194154820 0x7f4194154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4199a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f419415d920 con 0x7f4178003c40 2024-09-17T03:32:37.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4199a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f419415d7a0 con 0x7f4194154820 2024-09-17T03:32:37.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4199a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4194071c00 con 0x7f4194152710 2024-09-17T03:32:37.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 0x7f41941fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 0x7f41941fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52184/0 (socket says 172.21.15.46:52184) 2024-09-17T03:32:37.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 -- 172.21.15.46:0/80191874 learned_addr learned my addr 172.21.15.46:0/80191874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:37.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4193400640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4194154820 0x7f4194154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 -- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4194152710 msgr2=0x7f419414c010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:32:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4194152710 0x7f419414c010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 -- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4194154820 msgr2=0x7f4194154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4194154820 0x7f4194154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.872+0000 7f4192a00640 1 -- 172.21.15.46:0/80191874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41941524d0 con 0x7f4178003c40 2024-09-17T03:32:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f4192a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 0x7f41941fa480 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4184098870 tx=0x7f4184001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840a5c10 con 0x7f4178003c40 2024-09-17T03:32:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41840a5db0 con 0x7f4178003c40 2024-09-17T03:32:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41840063c0 con 0x7f4178003c40 2024-09-17T03:32:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41941f9dd0 con 0x7f4178003c40 2024-09-17T03:32:37.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941521f0 con 0x7f4178003c40 2024-09-17T03:32:37.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.876+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4194161420 con 0x7f4178003c40 2024-09-17T03:32:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.880+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f41840b6030 con 0x7f4178003c40 2024-09-17T03:32:37.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.880+0000 7f418aa00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4174074990 0x7f4174076ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:37.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.880+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5738+0+0 (secure 0 0 0) 0x7f418412c5a0 con 0x7f4178003c40 2024-09-17T03:32:37.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.880+0000 7f4192000640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4174074990 0x7f4174076ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:37.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.884+0000 7f4192000640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4174074990 0x7f4174076ea0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4180005d10 tx=0x7f4180005ca0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:37.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:37.888+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4194074d60 con 0x7f4178003c40 2024-09-17T03:32:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.288+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f419406b540 con 0x7f4174074990 2024-09-17T03:32:38.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.328+0000 7f418aa00640 1 -- 172.21.15.46:0/80191874 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4180009550 con 0x7f4174074990 2024-09-17T03:32:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4174074990 msgr2=0x7f4174076ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4174074990 0x7f4174076ea0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4180005d10 tx=0x7f4180005ca0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 msgr2=0x7f41941fa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 0x7f41941fa480 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4184098870 tx=0x7f4184001360 comp rx=0 tx=0).stop 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 shutdown_connections 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4174074990 0x7f4174076ea0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4194154820 0x7f4194154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4194152710 0x7f419414c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 --2- 172.21.15.46:0/80191874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4178003c40 0x7f41941fa480 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 >> 172.21.15.46:0/80191874 conn(0x7f41940708c0 msgr2=0x7f419415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 shutdown_connections 2024-09-17T03:32:38.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.332+0000 7f4199a00640 1 -- 172.21.15.46:0/80191874 wait complete. 2024-09-17T03:32:38.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone59755 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 -- 172.21.15.46:0/3394192643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 msgr2=0x7f69c4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 --2- 172.21.15.46:0/3394192643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c4074410 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009900 tx=0x7f69ac01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 -- 172.21.15.46:0/3394192643 shutdown_connections 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 --2- 172.21.15.46:0/3394192643 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69c4074950 0x7f69c415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 --2- 172.21.15.46:0/3394192643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c4074410 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 -- 172.21.15.46:0/3394192643 >> 172.21.15.46:0/3394192643 conn(0x7f69c40708c0 msgr2=0x7f69c4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 -- 172.21.15.46:0/3394192643 shutdown_connections 2024-09-17T03:32:38.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.624+0000 7f69c9c00640 1 -- 172.21.15.46:0/3394192643 wait complete. 2024-09-17T03:32:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 Processor -- start 2024-09-17T03:32:38.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 -- start start 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c4074950 0x7f69c413a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c413b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69c4157030 0x7f69c413b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69c416a310 con 0x7f69c40b5a80 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69c416a190 con 0x7f69c4074950 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c9c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f69c416a490 con 0x7f69c4157030 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c413b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c413b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52202/0 (socket says 172.21.15.46:52202) 2024-09-17T03:32:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69bbe00640 1 -- 172.21.15.46:0/1108834370 learned_addr learned my addr 172.21.15.46:0/1108834370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c3400640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69c4157030 0x7f69c413b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.628+0000 7f69c2a00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c4074950 0x7f69c413a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69bbe00640 1 -- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69c4157030 msgr2=0x7f69c413b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69bbe00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69c4157030 0x7f69c413b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69bbe00640 1 -- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c4074950 msgr2=0x7f69c413a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69bbe00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c4074950 0x7f69c413a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69bbe00640 1 -- 172.21.15.46:0/1108834370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c415cc00 con 0x7f69c40b5a80 2024-09-17T03:32:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69bbe00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c413b430 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009850 tx=0x7f69ac0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0b2040 con 0x7f69c40b5a80 2024-09-17T03:32:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69ac0043b0 con 0x7f69c40b5a80 2024-09-17T03:32:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac005290 con 0x7f69c40b5a80 2024-09-17T03:32:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c4156090 con 0x7f69c40b5a80 2024-09-17T03:32:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c413aef0 con 0x7f69c40b5a80 2024-09-17T03:32:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c415c970 con 0x7f69c40b5a80 2024-09-17T03:32:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.632+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f69ac005430 con 0x7f69c40b5a80 2024-09-17T03:32:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.636+0000 7f69baa00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69a0074910 0x7f69a0076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.636+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f69ac12cab0 con 0x7f69c40b5a80 2024-09-17T03:32:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.636+0000 7f69c2a00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69a0074910 0x7f69a0076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.636+0000 7f69c2a00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69a0074910 0x7f69a0076e20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f69b4007fd0 tx=0x7f69b4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:38.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.636+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f69ac0164d0 con 0x7f69c40b5a80 2024-09-17T03:32:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.912+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone59755", "target": ["mon-mgr", ""]}) -- 0x7f69c406b540 con 0x7f69a0074910 2024-09-17T03:32:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.952+0000 7f69baa00640 1 -- 172.21.15.46:0/1108834370 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69b4001470 con 0x7f69a0074910 2024-09-17T03:32:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.956+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69a0074910 msgr2=0x7f69a0076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.956+0000 7f69c9c00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69a0074910 0x7f69a0076e20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f69b4007fd0 tx=0x7f69b4009040 comp rx=0 tx=0).stop 2024-09-17T03:32:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.956+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 msgr2=0x7f69c413b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.956+0000 7f69c9c00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c413b430 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009850 tx=0x7f69ac0a5710 comp rx=0 tx=0).stop 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 shutdown_connections 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f69a0074910 0x7f69a0076e20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f69c4157030 0x7f69c413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c40b5a80 0x7f69c413b430 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 --2- 172.21.15.46:0/1108834370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69c4074950 0x7f69c413a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 >> 172.21.15.46:0/1108834370 conn(0x7f69c40708c0 msgr2=0x7f69c40b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 shutdown_connections 2024-09-17T03:32:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:38.960+0000 7f69c9c00640 1 -- 172.21.15.46:0/1108834370 wait complete. 2024-09-17T03:32:39.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone50292 2024-09-17T03:32:39.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 -- 172.21.15.46:0/2137121955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd934003c40 msgr2=0x7fd934004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 --2- 172.21.15.46:0/2137121955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd934003c40 0x7fd934004090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fa0 tx=0x7fd94001cf00 comp rx=0 tx=0).stop 2024-09-17T03:32:39.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 -- 172.21.15.46:0/2137121955 shutdown_connections 2024-09-17T03:32:39.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 --2- 172.21.15.46:0/2137121955 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd934003c40 0x7fd934004090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 -- 172.21.15.46:0/2137121955 >> 172.21.15.46:0/2137121955 conn(0x7fd94c073e00 msgr2=0x7fd94c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:39.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 -- 172.21.15.46:0/2137121955 shutdown_connections 2024-09-17T03:32:39.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.288+0000 7fd954400640 1 -- 172.21.15.46:0/2137121955 wait complete. 2024-09-17T03:32:39.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 Processor -- start 2024-09-17T03:32:39.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 -- start start 2024-09-17T03:32:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd934003c40 0x7fd94c1faa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd94c14b340 0x7fd94c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd94c161590 0x7fd94c1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0d1490 con 0x7fd934003c40 2024-09-17T03:32:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0d1310 con 0x7fd94c161590 2024-09-17T03:32:39.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.292+0000 7fd954400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c0b60d0 con 0x7fd94c14b340 2024-09-17T03:32:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd934003c40 0x7fd94c1faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd934003c40 0x7fd94c1faa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52208/0 (socket says 172.21.15.46:52208) 2024-09-17T03:32:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951a00640 1 -- 172.21.15.46:0/2337787317 learned_addr learned my addr 172.21.15.46:0/2337787317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd94c14b340 0x7fd94c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd952400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd94c161590 0x7fd94c1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 -- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd94c161590 msgr2=0x7fd94c1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd94c161590 0x7fd94c1619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 -- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd934003c40 msgr2=0x7fd94c1faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd934003c40 0x7fd94c1faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 -- 172.21.15.46:0/2337787317 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c161ee0 con 0x7fd94c14b340 2024-09-17T03:32:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd951000640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd94c14b340 0x7fd94c0ce790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd94c06b990 tx=0x7fd938011730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:39.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938017070 con 0x7fd94c14b340 2024-09-17T03:32:39.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd938013640 con 0x7fd94c14b340 2024-09-17T03:32:39.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938013a50 con 0x7fd94c14b340 2024-09-17T03:32:39.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c14dbd0 con 0x7fd94c14b340 2024-09-17T03:32:39.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c14e190 con 0x7fd94c14b340 2024-09-17T03:32:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.296+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c0b6630 con 0x7fd94c14b340 2024-09-17T03:32:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.300+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fd938013bf0 con 0x7fd94c14b340 2024-09-17T03:32:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.300+0000 7fd949600640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd92c074940 0x7fd92c076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.300+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7fd9380964d0 con 0x7fd94c14b340 2024-09-17T03:32:39.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.304+0000 7fd951a00640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd92c074940 0x7fd92c076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.304+0000 7fd951a00640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd92c074940 0x7fd92c076e50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd94001b7d0 tx=0x7fd940019d40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:39.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.312+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd94c161ee0 con 0x7fd94c14b340 2024-09-17T03:32:39.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.668+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone50292", "target": ["mon-mgr", ""]}) -- 0x7fd94c14b750 con 0x7fd92c074940 2024-09-17T03:32:39.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd949600640 1 -- 172.21.15.46:0/2337787317 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd940001580 con 0x7fd92c074940 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd92c074940 msgr2=0x7fd92c076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd92c074940 0x7fd92c076e50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd94001b7d0 tx=0x7fd940019d40 comp rx=0 tx=0).stop 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd94c14b340 msgr2=0x7fd94c0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd94c14b340 0x7fd94c0ce790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd94c06b990 tx=0x7fd938011730 comp rx=0 tx=0).stop 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 shutdown_connections 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd92c074940 0x7fd92c076e50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd94c161590 0x7fd94c1619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd94c14b340 0x7fd94c0ce790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 --2- 172.21.15.46:0/2337787317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd934003c40 0x7fd94c1faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 >> 172.21.15.46:0/2337787317 conn(0x7fd94c073e00 msgr2=0x7fd94c0b8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 shutdown_connections 2024-09-17T03:32:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.688+0000 7fd954400640 1 -- 172.21.15.46:0/2337787317 wait complete. 2024-09-17T03:32:39.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone73317 2024-09-17T03:32:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 -- 172.21.15.46:0/2603282131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c003ab0 msgr2=0x7fdc0c003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2603282131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c003ab0 0x7fdc0c003f20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009b60 tx=0x7fdc1801d270 comp rx=0 tx=0).stop 2024-09-17T03:32:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 -- 172.21.15.46:0/2603282131 shutdown_connections 2024-09-17T03:32:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2603282131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc0c003ab0 0x7fdc0c003f20 secure :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009b60 tx=0x7fdc1801d270 comp rx=0 tx=0).stop 2024-09-17T03:32:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 -- 172.21.15.46:0/2603282131 >> 172.21.15.46:0/2603282131 conn(0x7fdc240b2040 msgr2=0x7fdc240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 -- 172.21.15.46:0/2603282131 shutdown_connections 2024-09-17T03:32:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.980+0000 7fdc2c600640 1 -- 172.21.15.46:0/2603282131 wait complete. 2024-09-17T03:32:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 Processor -- start 2024-09-17T03:32:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 -- start start 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc241432a0 0x7fdc241f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 0x7fdc240b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdc240b10e0 0x7fdc2413e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241608c0 con 0x7fdc241400a0 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24160740 con 0x7fdc241432a0 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc2c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdc24160a40 con 0x7fdc240b10e0 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.984+0000 7fdc29200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 0x7fdc240b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 0x7fdc240b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52236/0 (socket says 172.21.15.46:52236) 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 -- 172.21.15.46:0/2460643155 learned_addr learned my addr 172.21.15.46:0/2460643155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc2a600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdc240b10e0 0x7fdc2413e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29c00640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc241432a0 0x7fdc241f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 -- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdc240b10e0 msgr2=0x7fdc2413e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdc240b10e0 0x7fdc2413e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 -- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc241432a0 msgr2=0x7fdc241f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc241432a0 0x7fdc241f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 -- 172.21.15.46:0/2460643155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc240b14f0 con 0x7fdc241400a0 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29c00640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc241432a0 0x7fdc241f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc29200640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 0x7fdc240b0ba0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000e8f0 tx=0x7fdc1000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000cc40 con 0x7fdc241400a0 2024-09-17T03:32:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc24143840 con 0x7fdc241400a0 2024-09-17T03:32:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc10009e50 con 0x7fdc241400a0 2024-09-17T03:32:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004280 con 0x7fdc241400a0 2024-09-17T03:32:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.988+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc24143e00 con 0x7fdc241400a0 2024-09-17T03:32:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.992+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24160fa0 con 0x7fdc241400a0 2024-09-17T03:32:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.992+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fdc10005020 con 0x7fdc241400a0 2024-09-17T03:32:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.992+0000 7fdc21600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdc04074910 0x7fdc04076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:39.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.996+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7fdc100932c0 con 0x7fdc241400a0 2024-09-17T03:32:39.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:39.996+0000 7fdc29c00640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdc04074910 0x7fdc04076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.000+0000 7fdc29c00640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdc04074910 0x7fdc04076e20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdc180ae000 tx=0x7fdc1801a010 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.000+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc240b14f0 con 0x7fdc241400a0 2024-09-17T03:32:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.300+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone73317", "target": ["mon-mgr", ""]}) -- 0x7fdc24142300 con 0x7fdc04074910 2024-09-17T03:32:40.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.316+0000 7fdc21600640 1 -- 172.21.15.46:0/2460643155 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc24142300 con 0x7fdc04074910 2024-09-17T03:32:40.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdc04074910 msgr2=0x7fdc04076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdc04074910 0x7fdc04076e20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdc180ae000 tx=0x7fdc1801a010 comp rx=0 tx=0).stop 2024-09-17T03:32:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 msgr2=0x7fdc240b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 0x7fdc240b0ba0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000e8f0 tx=0x7fdc1000edb0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 shutdown_connections 2024-09-17T03:32:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdc04074910 0x7fdc04076e20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdc240b10e0 0x7fdc2413e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241400a0 0x7fdc240b0ba0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 --2- 172.21.15.46:0/2460643155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc241432a0 0x7fdc241f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 >> 172.21.15.46:0/2460643155 conn(0x7fdc240b2040 msgr2=0x7fdc2415c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 shutdown_connections 2024-09-17T03:32:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.320+0000 7fdc2c600640 1 -- 172.21.15.46:0/2460643155 wait complete. 2024-09-17T03:32:40.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone551017 2024-09-17T03:32:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 -- 172.21.15.46:0/3062380519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c160060 msgr2=0x7f856c074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 --2- 172.21.15.46:0/3062380519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c160060 0x7f856c074240 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8554009900 tx=0x7f855401c850 comp rx=0 tx=0).stop 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 -- 172.21.15.46:0/3062380519 shutdown_connections 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 --2- 172.21.15.46:0/3062380519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c160060 0x7f856c074240 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 --2- 172.21.15.46:0/3062380519 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c1536f0 0x7f856c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 -- 172.21.15.46:0/3062380519 >> 172.21.15.46:0/3062380519 conn(0x7f856c0b4560 msgr2=0x7f856c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 -- 172.21.15.46:0/3062380519 shutdown_connections 2024-09-17T03:32:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 -- 172.21.15.46:0/3062380519 wait complete. 2024-09-17T03:32:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 Processor -- start 2024-09-17T03:32:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.516+0000 7f856be00640 1 -- start start 2024-09-17T03:32:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 0x7f856c14e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c160060 0x7f856c1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f856c160470 0x7f856c14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f856c16a490 con 0x7f856c1536f0 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f856c16a310 con 0x7f856c160470 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f856c16a610 con 0x7f856c160060 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 0x7f856c14e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 0x7f856c14e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52254/0 (socket says 172.21.15.46:52254) 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 -- 172.21.15.46:0/3618086083 learned_addr learned my addr 172.21.15.46:0/3618086083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 -- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c160060 msgr2=0x7f856c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f8563e00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c160060 0x7f856c1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856b400640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f856c160470 0x7f856c14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c160060 0x7f856c1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 -- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f856c160470 msgr2=0x7f856c14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f856c160470 0x7f856c14cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 -- 172.21.15.46:0/3618086083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c148450 con 0x7f856c1536f0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f8563e00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c160060 0x7f856c1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856b400640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f856c160470 0x7f856c14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856aa00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 0x7f856c14e6e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e7f0 tx=0x7f855c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c00ccc0 con 0x7f856c1536f0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f855c00ce80 con 0x7f856c1536f0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c005280 con 0x7f856c1536f0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c1488f0 con 0x7f856c1536f0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c148eb0 con 0x7f856c1536f0 2024-09-17T03:32:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.520+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c15d480 con 0x7f856c1536f0 2024-09-17T03:32:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.524+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f855c005420 con 0x7f856c1536f0 2024-09-17T03:32:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.524+0000 7f8562a00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8548074910 0x7f8548076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.524+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f855c0938e0 con 0x7f856c1536f0 2024-09-17T03:32:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.524+0000 7f8563e00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8548074910 0x7f8548076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.524+0000 7f8563e00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8548074910 0x7f8548076e20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f85540af000 tx=0x7f855401b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:40.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.528+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f855c060490 con 0x7f856c1536f0 2024-09-17T03:32:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.728+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone551017", "target": ["mon-mgr", ""]}) -- 0x7f856c1486a0 con 0x7f8548074910 2024-09-17T03:32:40.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f8562a00640 1 -- 172.21.15.46:0/3618086083 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f856c1486a0 con 0x7f8548074910 2024-09-17T03:32:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8548074910 msgr2=0x7f8548076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8548074910 0x7f8548076e20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f85540af000 tx=0x7f855401b090 comp rx=0 tx=0).stop 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 msgr2=0x7f856c14e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 0x7f856c14e6e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e7f0 tx=0x7f855c00ecb0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 shutdown_connections 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8548074910 0x7f8548076e20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f856c160470 0x7f856c14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f856c160060 0x7f856c1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 --2- 172.21.15.46:0/3618086083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856c1536f0 0x7f856c14e6e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 >> 172.21.15.46:0/3618086083 conn(0x7f856c0b4560 msgr2=0x7f856c0b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 shutdown_connections 2024-09-17T03:32:40.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.748+0000 7f856be00640 1 -- 172.21.15.46:0/3618086083 wait complete. 2024-09-17T03:32:40.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone582719 2024-09-17T03:32:40.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 -- 172.21.15.46:0/3271562555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 msgr2=0x7f9c840b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 --2- 172.21.15.46:0/3271562555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c840b51f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009bc0 tx=0x7f9c7c01cae0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 -- 172.21.15.46:0/3271562555 shutdown_connections 2024-09-17T03:32:40.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 --2- 172.21.15.46:0/3271562555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c840b51f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 --2- 172.21.15.46:0/3271562555 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9c841527b0 0x7f9c8415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 -- 172.21.15.46:0/3271562555 >> 172.21.15.46:0/3271562555 conn(0x7f9c840b2040 msgr2=0x7f9c840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:40.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 -- 172.21.15.46:0/3271562555 shutdown_connections 2024-09-17T03:32:40.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 -- 172.21.15.46:0/3271562555 wait complete. 2024-09-17T03:32:40.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 Processor -- start 2024-09-17T03:32:40.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.944+0000 7f9c8c400640 1 -- start start 2024-09-17T03:32:40.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c84150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9c840b5c70 0x7f9c840b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c841527b0 0x7f9c840ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84168ab0 con 0x7f9c840b4de0 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84168930 con 0x7f9c841527b0 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9c84168c30 con 0x7f9c840b5c70 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c84150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c84150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52270/0 (socket says 172.21.15.46:52270) 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 -- 172.21.15.46:0/397639137 learned_addr learned my addr 172.21.15.46:0/397639137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89000640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9c840b5c70 0x7f9c840b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8a400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c841527b0 0x7f9c840ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 -- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9c840b5c70 msgr2=0x7f9c840b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9c840b5c70 0x7f9c840b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 -- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c841527b0 msgr2=0x7f9c840ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:40.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c841527b0 0x7f9c840ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:40.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 -- 172.21.15.46:0/397639137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c841fc2b0 con 0x7f9c840b4de0 2024-09-17T03:32:40.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c89a00640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c84150be0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9c78001ec0 tx=0x7f9c7800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:40.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78017070 con 0x7f9c840b4de0 2024-09-17T03:32:40.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c78003600 con 0x7f9c840b4de0 2024-09-17T03:32:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841fc590 con 0x7f9c840b4de0 2024-09-17T03:32:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003a50 con 0x7f9c840b4de0 2024-09-17T03:32:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841fcb50 con 0x7f9c840b4de0 2024-09-17T03:32:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.948+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c840b51f0 con 0x7f9c840b4de0 2024-09-17T03:32:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.952+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f9c7801a070 con 0x7f9c840b4de0 2024-09-17T03:32:40.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.952+0000 7f9c81600640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c64074910 0x7f9c64076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:40.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.952+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f9c78092250 con 0x7f9c840b4de0 2024-09-17T03:32:40.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.952+0000 7f9c89000640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c64074910 0x7f9c64076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.952+0000 7f9c89000640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c64074910 0x7f9c64076e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9c840c63c0 tx=0x7f9c7000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:40.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:40.956+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c840b5730 con 0x7f9c840b4de0 2024-09-17T03:32:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.160+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone582719", "target": ["mon-mgr", ""]}) -- 0x7f9c84151120 con 0x7f9c64074910 2024-09-17T03:32:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c81600640 1 -- 172.21.15.46:0/397639137 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c84151120 con 0x7f9c64074910 2024-09-17T03:32:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c64074910 msgr2=0x7f9c64076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c64074910 0x7f9c64076e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9c840c63c0 tx=0x7f9c7000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 msgr2=0x7f9c84150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c84150be0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9c78001ec0 tx=0x7f9c7800c890 comp rx=0 tx=0).stop 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 shutdown_connections 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c64074910 0x7f9c64076e20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c841527b0 0x7f9c840ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9c840b5c70 0x7f9c840b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 --2- 172.21.15.46:0/397639137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c840b4de0 0x7f9c84150be0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 >> 172.21.15.46:0/397639137 conn(0x7f9c840b2040 msgr2=0x7f9c841581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 shutdown_connections 2024-09-17T03:32:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.180+0000 7f9c8c400640 1 -- 172.21.15.46:0/397639137 wait complete. 2024-09-17T03:32:41.206 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone13923 2024-09-17T03:32:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 -- 172.21.15.46:0/2993113272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 msgr2=0x7f9f8006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 --2- 172.21.15.46:0/2993113272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8006d200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b6bf0 tx=0x7f9f7801caf0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 -- 172.21.15.46:0/2993113272 shutdown_connections 2024-09-17T03:32:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 --2- 172.21.15.46:0/2993113272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8006d200 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 --2- 172.21.15.46:0/2993113272 >> v2:172.21.15.99:3300/0 conn(0x7f9f80159130 0x7f9f8015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 -- 172.21.15.46:0/2993113272 >> 172.21.15.46:0/2993113272 conn(0x7f9f800b35f0 msgr2=0x7f9f800b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 -- 172.21.15.46:0/2993113272 shutdown_connections 2024-09-17T03:32:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.372+0000 7f9f86800640 1 -- 172.21.15.46:0/2993113272 wait complete. 2024-09-17T03:32:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 Processor -- start 2024-09-17T03:32:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 -- start start 2024-09-17T03:32:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f8006cdf0 0x7f9f8014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f80159130 0x7f9f8015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80168d50 con 0x7f9f800703a0 2024-09-17T03:32:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80168bd0 con 0x7f9f8006cdf0 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f86800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9f80168ed0 con 0x7f9f80159130 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.376+0000 7f9f77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f7f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f8006cdf0 0x7f9f8014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52280/0 (socket says 172.21.15.46:52280) 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 -- 172.21.15.46:0/4126841430 learned_addr learned my addr 172.21.15.46:0/4126841430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f7fe00640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f80159130 0x7f9f8015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 -- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f80159130 msgr2=0x7f9f8015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f80159130 0x7f9f8015e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 -- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f8006cdf0 msgr2=0x7f9f8014d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f8006cdf0 0x7f9f8014d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 -- 172.21.15.46:0/4126841430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f80164680 con 0x7f9f800703a0 2024-09-17T03:32:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77e00640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8015dc00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c400 tx=0x7f9f6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68011790 con 0x7f9f800703a0 2024-09-17T03:32:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f68017070 con 0x7f9f800703a0 2024-09-17T03:32:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68011cc0 con 0x7f9f800703a0 2024-09-17T03:32:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f800cf310 con 0x7f9f800703a0 2024-09-17T03:32:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f800cf8d0 con 0x7f9f800703a0 2024-09-17T03:32:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8006d200 con 0x7f9f800703a0 2024-09-17T03:32:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.380+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f9f6801c460 con 0x7f9f800703a0 2024-09-17T03:32:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.384+0000 7f9f77400640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f5c074910 0x7f9f5c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.384+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f9f68092c50 con 0x7f9f800703a0 2024-09-17T03:32:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.384+0000 7f9f7f400640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f5c074910 0x7f9f5c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.384+0000 7f9f7f400640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f5c074910 0x7f9f5c076e20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000bb50 tx=0x7f9f7000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.388+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9f8015c0d0 con 0x7f9f800703a0 2024-09-17T03:32:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.592+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone13923", "target": ["mon-mgr", ""]}) -- 0x7f9f800b3ca0 con 0x7f9f5c074910 2024-09-17T03:32:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.608+0000 7f9f77400640 1 -- 172.21.15.46:0/4126841430 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f800b3ca0 con 0x7f9f5c074910 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f5c074910 msgr2=0x7f9f5c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f5c074910 0x7f9f5c076e20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000bb50 tx=0x7f9f7000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 msgr2=0x7f9f8015dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8015dc00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c400 tx=0x7f9f6800c8c0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 shutdown_connections 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f5c074910 0x7f9f5c076e20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f80159130 0x7f9f8015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f800703a0 0x7f9f8015dc00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 --2- 172.21.15.46:0/4126841430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f8006cdf0 0x7f9f8014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 >> 172.21.15.46:0/4126841430 conn(0x7f9f800b35f0 msgr2=0x7f9f80153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 shutdown_connections 2024-09-17T03:32:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.612+0000 7f9f86800640 1 -- 172.21.15.46:0/4126841430 wait complete. 2024-09-17T03:32:41.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone60228 2024-09-17T03:32:41.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.836+0000 7f1bf9a00640 1 -- 172.21.15.46:0/3707734820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4155670 msgr2=0x7f1bf4155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.836+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/3707734820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4155670 0x7f1bf4155a80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1bf406f0c0 tx=0x7f1bdc01c770 comp rx=0 tx=0).stop 2024-09-17T03:32:41.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.840+0000 7f1bf9a00640 1 -- 172.21.15.46:0/3707734820 shutdown_connections 2024-09-17T03:32:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.840+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/3707734820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4155670 0x7f1bf4155a80 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.840+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/3707734820 >> v2:172.21.15.99:3300/0 conn(0x7f1bf4070e90 0x7f1bf415f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.840+0000 7f1bf9a00640 1 -- 172.21.15.46:0/3707734820 >> 172.21.15.46:0/3707734820 conn(0x7f1bf40d13f0 msgr2=0x7f1bf40d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.840+0000 7f1bf9a00640 1 -- 172.21.15.46:0/3707734820 shutdown_connections 2024-09-17T03:32:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.840+0000 7f1bf9a00640 1 -- 172.21.15.46:0/3707734820 wait complete. 2024-09-17T03:32:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.844+0000 7f1bf9a00640 1 Processor -- start 2024-09-17T03:32:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.844+0000 7f1bf9a00640 1 -- start start 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4070e90 0x7f1bf415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf9a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1bf40b5d50 0x7f1bf4144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf9a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 0x7f1bf415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf406d2f0 con 0x7f1bf4070e90 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf9a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf406d170 con 0x7f1bf4155670 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf9a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf406d470 con 0x7f1bf40b5d50 2024-09-17T03:32:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 0x7f1bf415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 0x7f1bf415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:60922/0 (socket says 172.21.15.46:60922) 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 -- 172.21.15.46:0/552810049 learned_addr learned my addr 172.21.15.46:0/552810049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bea000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1bf40b5d50 0x7f1bf4144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf2a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4070e90 0x7f1bf415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 -- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1bf40b5d50 msgr2=0x7f1bf4144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1bf40b5d50 0x7f1bf4144050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 -- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4070e90 msgr2=0x7f1bf415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4070e90 0x7f1bf415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 -- 172.21.15.46:0/552810049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf41595c0 con 0x7f1bf4155670 2024-09-17T03:32:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.848+0000 7f1bf3400640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 0x7f1bf415beb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00c730 tx=0x7f1bec00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.852+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec003b80 con 0x7f1bf4155670 2024-09-17T03:32:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.852+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bec003d40 con 0x7f1bf4155670 2024-09-17T03:32:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.852+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0052d0 con 0x7f1bf4155670 2024-09-17T03:32:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.852+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf4144590 con 0x7f1bf4155670 2024-09-17T03:32:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.852+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf4159da0 con 0x7f1bf4155670 2024-09-17T03:32:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.852+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf4159830 con 0x7f1bf4155670 2024-09-17T03:32:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.856+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1bec005680 con 0x7f1bf4155670 2024-09-17T03:32:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.860+0000 7f1beb400640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bd00749e0 0x7f1bd0076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:41.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.860+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f1bec093960 con 0x7f1bf4155670 2024-09-17T03:32:41.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.860+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1bf40712a0 con 0x7f1bf4155670 2024-09-17T03:32:41.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.860+0000 7f1bf2a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bd00749e0 0x7f1bd0076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:41.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:41.864+0000 7f1bf2a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bd00749e0 0x7f1bd0076ef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009b40 tx=0x7f1be40091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.292+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone60228", "target": ["mon-mgr", ""]}) -- 0x7f1bf406b540 con 0x7f1bd00749e0 2024-09-17T03:32:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.308+0000 7f1beb400640 1 -- 172.21.15.46:0/552810049 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1be4011be0 con 0x7f1bd00749e0 2024-09-17T03:32:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.308+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bd00749e0 msgr2=0x7f1bd0076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.308+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bd00749e0 0x7f1bd0076ef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009b40 tx=0x7f1be40091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 msgr2=0x7f1bf415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 0x7f1bf415beb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00c730 tx=0x7f1bec00cbf0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 shutdown_connections 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bd00749e0 0x7f1bd0076ef0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bf4155670 0x7f1bf415beb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1bf40b5d50 0x7f1bf4144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 --2- 172.21.15.46:0/552810049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bf4070e90 0x7f1bf415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 >> 172.21.15.46:0/552810049 conn(0x7f1bf40d13f0 msgr2=0x7f1bf40b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 shutdown_connections 2024-09-17T03:32:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.312+0000 7f1bf9a00640 1 -- 172.21.15.46:0/552810049 wait complete. 2024-09-17T03:32:42.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone579713 2024-09-17T03:32:42.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.608+0000 7f34eca00640 1 -- 172.21.15.46:0/2457955966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80b7220 msgr2=0x7f34e8162c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:42.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.608+0000 7f34eca00640 1 --2- 172.21.15.46:0/2457955966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80b7220 0x7f34e8162c30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009840 tx=0x7f34d001c620 comp rx=0 tx=0).stop 2024-09-17T03:32:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.608+0000 7f34eca00640 1 -- 172.21.15.46:0/2457955966 shutdown_connections 2024-09-17T03:32:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.608+0000 7f34eca00640 1 --2- 172.21.15.46:0/2457955966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80b7220 0x7f34e8162c30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.608+0000 7f34eca00640 1 -- 172.21.15.46:0/2457955966 >> 172.21.15.46:0/2457955966 conn(0x7f34e80b4560 msgr2=0x7f34e80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:42.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.612+0000 7f34eca00640 1 -- 172.21.15.46:0/2457955966 shutdown_connections 2024-09-17T03:32:42.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.612+0000 7f34eca00640 1 -- 172.21.15.46:0/2457955966 wait complete. 2024-09-17T03:32:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.612+0000 7f34eca00640 1 Processor -- start 2024-09-17T03:32:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 -- start start 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 0x7f34e8153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80d1270 0x7f34e81582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34e8155710 0x7f34e8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34e816d310 con 0x7f34e8155710 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34e816d190 con 0x7f34e80d1270 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34eca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f34e816d490 con 0x7f34e80b7220 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 0x7f34e8153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 0x7f34e8153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:36744/0 (socket says 172.21.15.46:36744) 2024-09-17T03:32:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 -- 172.21.15.46:0/733144233 learned_addr learned my addr 172.21.15.46:0/733144233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34dfe00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80d1270 0x7f34e81582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e7200640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34e8155710 0x7f34e8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 -- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80d1270 msgr2=0x7f34e81582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80d1270 0x7f34e81582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 -- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34e8155710 msgr2=0x7f34e8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34e8155710 0x7f34e8158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.616+0000 7f34e6800640 1 -- 172.21.15.46:0/733144233 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e81f95a0 con 0x7f34e80b7220 2024-09-17T03:32:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.620+0000 7f34e6800640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 0x7f34e8153b70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34d8001b60 tx=0x7f34d8014420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.620+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8001d10 con 0x7f34e80b7220 2024-09-17T03:32:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.620+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34d8014d40 con 0x7f34e80b7220 2024-09-17T03:32:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.620+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8005330 con 0x7f34e80b7220 2024-09-17T03:32:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.620+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e81f9880 con 0x7f34e80b7220 2024-09-17T03:32:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.620+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e8151fe0 con 0x7f34e80b7220 2024-09-17T03:32:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.624+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e8162c30 con 0x7f34e80b7220 2024-09-17T03:32:42.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.628+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f34d8012120 con 0x7f34e80b7220 2024-09-17T03:32:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.628+0000 7f34dea00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34c40749e0 0x7f34c4076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.628+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f34d80920d0 con 0x7f34e80b7220 2024-09-17T03:32:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.628+0000 7f34dfe00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34c40749e0 0x7f34c4076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.628+0000 7f34dfe00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34c40749e0 0x7f34c4076ef0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f34d00ae000 tx=0x7f34d001ae60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.632+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34d805e610 con 0x7f34e80b7220 2024-09-17T03:32:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:42.988+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone579713", "target": ["mon-mgr", ""]}) -- 0x7f34e806b540 con 0x7f34c40749e0 2024-09-17T03:32:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.036+0000 7f34dea00640 1 -- 172.21.15.46:0/733144233 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34d00af040 con 0x7f34c40749e0 2024-09-17T03:32:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34c40749e0 msgr2=0x7f34c4076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34c40749e0 0x7f34c4076ef0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f34d00ae000 tx=0x7f34d001ae60 comp rx=0 tx=0).stop 2024-09-17T03:32:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 msgr2=0x7f34e8153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 0x7f34e8153b70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34d8001b60 tx=0x7f34d8014420 comp rx=0 tx=0).stop 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 shutdown_connections 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34c40749e0 0x7f34c4076ef0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34e8155710 0x7f34e8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34e80d1270 0x7f34e81582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 --2- 172.21.15.46:0/733144233 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34e80b7220 0x7f34e8153b70 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34d8001b60 tx=0x7f34d8014420 comp rx=0 tx=0).stop 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 >> 172.21.15.46:0/733144233 conn(0x7f34e80b4560 msgr2=0x7f34e80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 shutdown_connections 2024-09-17T03:32:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.040+0000 7f34eca00640 1 -- 172.21.15.46:0/733144233 wait complete. 2024-09-17T03:32:43.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone18656 2024-09-17T03:32:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 -- 172.21.15.46:0/1707287950 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff4074210 msgr2=0x7f8ff415b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 --2- 172.21.15.46:0/1707287950 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff4074210 0x7f8ff415b570 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000d960 tx=0x7f8fe0020aa0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 -- 172.21.15.46:0/1707287950 shutdown_connections 2024-09-17T03:32:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 --2- 172.21.15.46:0/1707287950 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff4074210 0x7f8ff415b570 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 -- 172.21.15.46:0/1707287950 >> 172.21.15.46:0/1707287950 conn(0x7f8ff40b3b30 msgr2=0x7f8ff40733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 -- 172.21.15.46:0/1707287950 shutdown_connections 2024-09-17T03:32:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.348+0000 7f8ffb400640 1 -- 172.21.15.46:0/1707287950 wait complete. 2024-09-17T03:32:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.352+0000 7f8ffb400640 1 Processor -- start 2024-09-17T03:32:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.352+0000 7f8ffb400640 1 -- start start 2024-09-17T03:32:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ffb400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff40b57c0 0x7f8ff4205800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ffb400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 0x7f8ff4142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ffb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff420a020 0x7f8ff4143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ffb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4069990 con 0x7f8ff420a020 2024-09-17T03:32:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ffb400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4069810 con 0x7f8ff40b57c0 2024-09-17T03:32:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ffb400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff4069b10 con 0x7f8ff40cf4a0 2024-09-17T03:32:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 0x7f8ff4142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 0x7f8ff4142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:36760/0 (socket says 172.21.15.46:36760) 2024-09-17T03:32:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff3e00640 1 -- 172.21.15.46:0/60638786 learned_addr learned my addr 172.21.15.46:0/60638786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff40b57c0 0x7f8ff4205800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff9400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff420a020 0x7f8ff4143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 -- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 msgr2=0x7f8ff4142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 0x7f8ff4142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 -- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff420a020 msgr2=0x7f8ff4143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff420a020 0x7f8ff4143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 -- 172.21.15.46:0/60638786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff414ce00 con 0x7f8ff40b57c0 2024-09-17T03:32:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff3e00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 0x7f8ff4142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.356+0000 7f8ff8a00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff40b57c0 0x7f8ff4205800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b470 tx=0x7f8fe800b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.360+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8015020 con 0x7f8ff40b57c0 2024-09-17T03:32:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.360+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe8011960 con 0x7f8ff40b57c0 2024-09-17T03:32:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.360+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011db0 con 0x7f8ff40b57c0 2024-09-17T03:32:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.360+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff415a3a0 con 0x7f8ff40b57c0 2024-09-17T03:32:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.360+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff414d0a0 con 0x7f8ff40b57c0 2024-09-17T03:32:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.360+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff415b570 con 0x7f8ff40b57c0 2024-09-17T03:32:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.364+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f8fe8018020 con 0x7f8ff40b57c0 2024-09-17T03:32:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.364+0000 7f8ff0c00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8fd4074960 0x7f8fd4076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.364+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f8fe80929f0 con 0x7f8ff40b57c0 2024-09-17T03:32:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.368+0000 7f8ff3e00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8fd4074960 0x7f8fd4076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.368+0000 7f8ff3e00640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8fd4074960 0x7f8fd4076e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4150950 tx=0x7f8fe001f2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:43.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.372+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8fe801f030 con 0x7f8ff40b57c0 2024-09-17T03:32:43.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.692+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone18656", "target": ["mon-mgr", ""]}) -- 0x7f8ff406b540 con 0x7f8fd4074960 2024-09-17T03:32:43.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.704+0000 7f8ff0c00640 1 -- 172.21.15.46:0/60638786 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fe0012190 con 0x7f8fd4074960 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8fd4074960 msgr2=0x7f8fd4076e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8fd4074960 0x7f8fd4076e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4150950 tx=0x7f8fe001f2e0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff40b57c0 msgr2=0x7f8ff4205800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff40b57c0 0x7f8ff4205800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b470 tx=0x7f8fe800b930 comp rx=0 tx=0).stop 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 shutdown_connections 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8fd4074960 0x7f8fd4076e70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff420a020 0x7f8ff4143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8ff40cf4a0 0x7f8ff4142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 --2- 172.21.15.46:0/60638786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ff40b57c0 0x7f8ff4205800 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 >> 172.21.15.46:0/60638786 conn(0x7f8ff40b3b30 msgr2=0x7f8ff415ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 shutdown_connections 2024-09-17T03:32:43.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.708+0000 7f8ffb400640 1 -- 172.21.15.46:0/60638786 wait complete. 2024-09-17T03:32:43.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone560911 2024-09-17T03:32:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.984+0000 7fda66600640 1 -- 172.21.15.46:0/1505739021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006fc90 msgr2=0x7fda600700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.984+0000 7fda66600640 1 --2- 172.21.15.46:0/1505739021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006fc90 0x7fda600700a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fda58009c80 tx=0x7fda5801cd10 comp rx=0 tx=0).stop 2024-09-17T03:32:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.984+0000 7fda66600640 1 -- 172.21.15.46:0/1505739021 shutdown_connections 2024-09-17T03:32:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.984+0000 7fda66600640 1 --2- 172.21.15.46:0/1505739021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006fc90 0x7fda600700a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.984+0000 7fda66600640 1 --2- 172.21.15.46:0/1505739021 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda6006cdf0 0x7fda6006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.984+0000 7fda66600640 1 -- 172.21.15.46:0/1505739021 >> 172.21.15.46:0/1505739021 conn(0x7fda600b35f0 msgr2=0x7fda600b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.988+0000 7fda66600640 1 -- 172.21.15.46:0/1505739021 shutdown_connections 2024-09-17T03:32:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.988+0000 7fda66600640 1 -- 172.21.15.46:0/1505739021 wait complete. 2024-09-17T03:32:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.988+0000 7fda66600640 1 Processor -- start 2024-09-17T03:32:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 -- start start 2024-09-17T03:32:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 0x7fda6015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda6006fc90 0x7fda60152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60159130 0x7fda6015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda60168e40 con 0x7fda6006cdf0 2024-09-17T03:32:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda60168cc0 con 0x7fda60159130 2024-09-17T03:32:43.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.992+0000 7fda66600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fda60168fc0 con 0x7fda6006fc90 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 0x7fda6015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 0x7fda6015e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57488/0 (socket says 172.21.15.46:57488) 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 -- 172.21.15.46:0/1654118771 learned_addr learned my addr 172.21.15.46:0/1654118771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5ea00640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda6006fc90 0x7fda60152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5fe00640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60159130 0x7fda6015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 -- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda6006fc90 msgr2=0x7fda60152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda6006fc90 0x7fda60152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 -- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60159130 msgr2=0x7fda6015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:43.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60159130 0x7fda6015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:43.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 -- 172.21.15.46:0/1654118771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6015cdd0 con 0x7fda6006cdf0 2024-09-17T03:32:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda5f400640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 0x7fda6015e3a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fda5000c3c0 tx=0x7fda5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50017070 con 0x7fda6006cdf0 2024-09-17T03:32:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda500053c0 con 0x7fda6006cdf0 2024-09-17T03:32:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6015d0b0 con 0x7fda6006cdf0 2024-09-17T03:32:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50003860 con 0x7fda6006cdf0 2024-09-17T03:32:43.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:43.996+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6015d670 con 0x7fda6006cdf0 2024-09-17T03:32:43.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.000+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda380052a0 con 0x7fda6006cdf0 2024-09-17T03:32:44.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.000+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fda5001a070 con 0x7fda6006cdf0 2024-09-17T03:32:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.004+0000 7fda57400640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda40074960 0x7fda40076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:44.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.004+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7fda500921d0 con 0x7fda6006cdf0 2024-09-17T03:32:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.004+0000 7fda5ea00640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda40074960 0x7fda40076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.004+0000 7fda5ea00640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda40074960 0x7fda40076e70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0058d0 tx=0x7fda4c005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:44.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.012+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda5005ecd0 con 0x7fda6006cdf0 2024-09-17T03:32:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.300+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone560911", "target": ["mon-mgr", ""]}) -- 0x7fda38000bd0 con 0x7fda40074960 2024-09-17T03:32:44.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.316+0000 7fda57400640 1 -- 172.21.15.46:0/1654118771 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda38000bd0 con 0x7fda40074960 2024-09-17T03:32:44.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda40074960 msgr2=0x7fda40076e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda40074960 0x7fda40076e70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0058d0 tx=0x7fda4c005860 comp rx=0 tx=0).stop 2024-09-17T03:32:44.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 msgr2=0x7fda6015e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 0x7fda6015e3a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fda5000c3c0 tx=0x7fda5000c880 comp rx=0 tx=0).stop 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 shutdown_connections 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda40074960 0x7fda40076e70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60159130 0x7fda6015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda6006fc90 0x7fda60152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 --2- 172.21.15.46:0/1654118771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda6006cdf0 0x7fda6015e3a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 >> 172.21.15.46:0/1654118771 conn(0x7fda600b35f0 msgr2=0x7fda60068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 shutdown_connections 2024-09-17T03:32:44.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.320+0000 7fda66600640 1 -- 172.21.15.46:0/1654118771 wait complete. 2024-09-17T03:32:44.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone838216 2024-09-17T03:32:44.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 -- 172.21.15.46:0/2805635344 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 msgr2=0x7fe51415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 --2- 172.21.15.46:0/2805635344 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe51415bb60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe504009840 tx=0x7fe50401c760 comp rx=0 tx=0).stop 2024-09-17T03:32:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 -- 172.21.15.46:0/2805635344 shutdown_connections 2024-09-17T03:32:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 --2- 172.21.15.46:0/2805635344 >> v2:172.21.15.99:3300/0 conn(0x7fe5140b4de0 0x7fe5140b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 --2- 172.21.15.46:0/2805635344 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe51415bb60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 -- 172.21.15.46:0/2805635344 >> 172.21.15.46:0/2805635344 conn(0x7fe5140b2040 msgr2=0x7fe5140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 -- 172.21.15.46:0/2805635344 shutdown_connections 2024-09-17T03:32:44.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 -- 172.21.15.46:0/2805635344 wait complete. 2024-09-17T03:32:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 Processor -- start 2024-09-17T03:32:44.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.512+0000 7fe51c800640 1 -- start start 2024-09-17T03:32:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5140b4de0 0x7fe51414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 0x7fe5140ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe5140aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141689b0 con 0x7fe5140b5c70 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe514168830 con 0x7fe5140b4de0 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe514168b30 con 0x7fe5141527b0 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 0x7fe5140ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 0x7fe5140ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57494/0 (socket says 172.21.15.46:57494) 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 -- 172.21.15.46:0/1092724858 learned_addr learned my addr 172.21.15.46:0/1092724858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519e00640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5140b4de0 0x7fe51414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51a800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe5140aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 -- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 msgr2=0x7fe5140aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe5140aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 -- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5140b4de0 msgr2=0x7fe51414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5140b4de0 0x7fe51414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 -- 172.21.15.46:0/1092724858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5140d0040 con 0x7fe5140b5c70 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51a800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe5140aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe519400640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 0x7fe5140ae8e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe50401cc40 tx=0x7fe504005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5040b2040 con 0x7fe5140b5c70 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe504004130 con 0x7fe5140b5c70 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5141f9a90 con 0x7fe5140b5c70 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504003600 con 0x7fe5140b5c70 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe514153f80 con 0x7fe5140b5c70 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.516+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51414f520 con 0x7fe5140b5c70 2024-09-17T03:32:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.520+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fe5040bd030 con 0x7fe5140b5c70 2024-09-17T03:32:44.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.520+0000 7fe50e000640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe4f8074910 0x7fe4f8076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.520+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7fe50412ca60 con 0x7fe5140b5c70 2024-09-17T03:32:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.520+0000 7fe519e00640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe4f8074910 0x7fe4f8076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.520+0000 7fe519e00640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe4f8074910 0x7fe4f8076e20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe508007d70 tx=0x7fe508007d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.524+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe5040c4060 con 0x7fe5140b5c70 2024-09-17T03:32:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.724+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone838216", "target": ["mon-mgr", ""]}) -- 0x7fe5141f9840 con 0x7fe4f8074910 2024-09-17T03:32:44.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.744+0000 7fe50e000640 1 -- 172.21.15.46:0/1092724858 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5141f9840 con 0x7fe4f8074910 2024-09-17T03:32:44.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe4f8074910 msgr2=0x7fe4f8076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe4f8074910 0x7fe4f8076e20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe508007d70 tx=0x7fe508007d00 comp rx=0 tx=0).stop 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 msgr2=0x7fe5140ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 0x7fe5140ae8e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe50401cc40 tx=0x7fe504005ca0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 shutdown_connections 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe4f8074910 0x7fe4f8076e20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5141527b0 0x7fe5140aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5140b5c70 0x7fe5140ae8e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 --2- 172.21.15.46:0/1092724858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5140b4de0 0x7fe51414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 >> 172.21.15.46:0/1092724858 conn(0x7fe5140b2040 msgr2=0x7fe5140b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 shutdown_connections 2024-09-17T03:32:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:44.748+0000 7fe51c800640 1 -- 172.21.15.46:0/1092724858 wait complete. 2024-09-17T03:32:44.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone341414 2024-09-17T03:32:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 -- 172.21.15.46:0/3265009415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006fc90 msgr2=0x7f0c600700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 --2- 172.21.15.46:0/3265009415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006fc90 0x7f0c600700a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0c600d14d0 tx=0x7f0c5801cb10 comp rx=0 tx=0).stop 2024-09-17T03:32:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 -- 172.21.15.46:0/3265009415 shutdown_connections 2024-09-17T03:32:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 --2- 172.21.15.46:0/3265009415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006fc90 0x7f0c600700a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 --2- 172.21.15.46:0/3265009415 >> v2:172.21.15.99:3300/0 conn(0x7f0c6006cdf0 0x7f0c6006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 -- 172.21.15.46:0/3265009415 >> 172.21.15.46:0/3265009415 conn(0x7f0c600b35f0 msgr2=0x7f0c600b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 -- 172.21.15.46:0/3265009415 shutdown_connections 2024-09-17T03:32:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.048+0000 7f0c66a00640 1 -- 172.21.15.46:0/3265009415 wait complete. 2024-09-17T03:32:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.052+0000 7f0c66a00640 1 Processor -- start 2024-09-17T03:32:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.052+0000 7f0c66a00640 1 -- start start 2024-09-17T03:32:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c66a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 0x7f0c601571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c66a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c6006fc90 0x7f0c6015e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c66a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c60159130 0x7f0c600b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c66a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60168da0 con 0x7f0c6006cdf0 2024-09-17T03:32:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c66a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60168c20 con 0x7f0c60159130 2024-09-17T03:32:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c66a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0c60168f20 con 0x7f0c6006fc90 2024-09-17T03:32:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 0x7f0c601571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 0x7f0c601571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57516/0 (socket says 172.21.15.46:57516) 2024-09-17T03:32:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 -- 172.21.15.46:0/981958416 learned_addr learned my addr 172.21.15.46:0/981958416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5f400640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c6006fc90 0x7f0c6015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 -- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c6006fc90 msgr2=0x7f0c6015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c6006fc90 0x7f0c6015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 -- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c60159130 msgr2=0x7f0c600b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c60159130 0x7f0c600b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.056+0000 7f0c5fe00640 1 -- 172.21.15.46:0/981958416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c601f8f20 con 0x7f0c6006cdf0 2024-09-17T03:32:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.060+0000 7f0c5fe00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 0x7f0c601571f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000c400 tx=0x7f0c5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.060+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50011790 con 0x7f0c6006cdf0 2024-09-17T03:32:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.060+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c601f9200 con 0x7f0c6006cdf0 2024-09-17T03:32:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.060+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c50017070 con 0x7f0c6006cdf0 2024-09-17T03:32:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.060+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50011cc0 con 0x7f0c6006cdf0 2024-09-17T03:32:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.060+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c601f97c0 con 0x7f0c6006cdf0 2024-09-17T03:32:45.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.064+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6006d200 con 0x7f0c6006cdf0 2024-09-17T03:32:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.068+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f0c5001c460 con 0x7f0c6006cdf0 2024-09-17T03:32:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.072+0000 7f0c57e00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c40074910 0x7f0c40076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.072+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f0c50093790 con 0x7f0c6006cdf0 2024-09-17T03:32:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.072+0000 7f0c5f400640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c40074910 0x7f0c40076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.072+0000 7f0c5f400640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c40074910 0x7f0c40076e20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c6015de60 tx=0x7f0c4c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.076+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c500602a0 con 0x7f0c6006cdf0 2024-09-17T03:32:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.340+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone341414", "target": ["mon-mgr", ""]}) -- 0x7f0c6006b540 con 0x7f0c40074910 2024-09-17T03:32:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c57e00640 1 -- 172.21.15.46:0/981958416 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0c4c011ca0 con 0x7f0c40074910 2024-09-17T03:32:45.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c40074910 msgr2=0x7f0c40076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c40074910 0x7f0c40076e20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c6015de60 tx=0x7f0c4c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 msgr2=0x7f0c601571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 0x7f0c601571f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000c400 tx=0x7f0c5000c8c0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 shutdown_connections 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0c40074910 0x7f0c40076e20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c60159130 0x7f0c600b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0c6006fc90 0x7f0c6015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 --2- 172.21.15.46:0/981958416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c6006cdf0 0x7f0c601571f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 >> 172.21.15.46:0/981958416 conn(0x7f0c600b35f0 msgr2=0x7f0c600b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 shutdown_connections 2024-09-17T03:32:45.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.360+0000 7f0c66a00640 1 -- 172.21.15.46:0/981958416 wait complete. 2024-09-17T03:32:45.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone955412 2024-09-17T03:32:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 -- 172.21.15.46:0/2162714285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b4de0 msgr2=0x7f54d00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 --2- 172.21.15.46:0/2162714285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b4de0 0x7f54d00b51f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009bc0 tx=0x7f54c801ca00 comp rx=0 tx=0).stop 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 -- 172.21.15.46:0/2162714285 shutdown_connections 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 --2- 172.21.15.46:0/2162714285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b4de0 0x7f54d00b51f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 --2- 172.21.15.46:0/2162714285 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d01527b0 0x7f54d015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 -- 172.21.15.46:0/2162714285 >> 172.21.15.46:0/2162714285 conn(0x7f54d00b2040 msgr2=0x7f54d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 -- 172.21.15.46:0/2162714285 shutdown_connections 2024-09-17T03:32:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.552+0000 7f54d6200640 1 -- 172.21.15.46:0/2162714285 wait complete. 2024-09-17T03:32:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 Processor -- start 2024-09-17T03:32:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 -- start start 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d00b4de0 0x7f54d00cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b5c70 0x7f54d015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 0x7f54d015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54d00b4470 con 0x7f54d01527b0 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54d00b42f0 con 0x7f54d00b5c70 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f54d00b45f0 con 0x7f54d00b4de0 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 0x7f54d015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 0x7f54d015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57518/0 (socket says 172.21.15.46:57518) 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 -- 172.21.15.46:0/1335767627 learned_addr learned my addr 172.21.15.46:0/1335767627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cea00640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b5c70 0x7f54d015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cf400640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d00b4de0 0x7f54d00cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 -- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d00b4de0 msgr2=0x7f54d00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d00b4de0 0x7f54d00cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 -- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b5c70 msgr2=0x7f54d015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b5c70 0x7f54d015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 -- 172.21.15.46:0/1335767627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d01f9200 con 0x7f54d01527b0 2024-09-17T03:32:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54cfe00640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 0x7f54d015b250 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f54c801b0b0 tx=0x7f54c80092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80bb070 con 0x7f54d01527b0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c8003d70 con 0x7f54d01527b0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80b2040 con 0x7f54d01527b0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d00cd450 con 0x7f54d01527b0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.556+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d01fbf20 con 0x7f54d01527b0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.560+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d00b4b20 con 0x7f54d01527b0 2024-09-17T03:32:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.560+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f54c80be070 con 0x7f54d01527b0 2024-09-17T03:32:45.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.560+0000 7f54c7400640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54b0074910 0x7f54b0076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:45.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.560+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5794+0+0 (secure 0 0 0) 0x7f54c812c950 con 0x7f54d01527b0 2024-09-17T03:32:45.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.560+0000 7f54cf400640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54b0074910 0x7f54b0076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:45.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.560+0000 7f54cf400640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54b0074910 0x7f54b0076e20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007990 tx=0x7f54c00012e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.564+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f54c80166b0 con 0x7f54d01527b0 2024-09-17T03:32:45.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.772+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone955412", "target": ["mon-mgr", ""]}) -- 0x7f54d01fc1c0 con 0x7f54b0074910 2024-09-17T03:32:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.816+0000 7f54c7400640 1 -- 172.21.15.46:0/1335767627 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f54d01fc1c0 con 0x7f54b0074910 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54b0074910 msgr2=0x7f54b0076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54b0074910 0x7f54b0076e20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007990 tx=0x7f54c00012e0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 msgr2=0x7f54d015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 0x7f54d015b250 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f54c801b0b0 tx=0x7f54c80092a0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 shutdown_connections 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54b0074910 0x7f54b0076e20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54d01527b0 0x7f54d015b250 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54d00b5c70 0x7f54d015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 --2- 172.21.15.46:0/1335767627 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54d00b4de0 0x7f54d00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.820+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 >> 172.21.15.46:0/1335767627 conn(0x7f54d00b2040 msgr2=0x7f54d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.824+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 shutdown_connections 2024-09-17T03:32:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:45.824+0000 7f54d6200640 1 -- 172.21.15.46:0/1335767627 wait complete. 2024-09-17T03:32:45.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone79881 2024-09-17T03:32:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 -- 172.21.15.46:0/4121896029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6440b6500 msgr2=0x7fe644070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 --2- 172.21.15.46:0/4121896029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6440b6500 0x7fe644070d50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe644072ea0 tx=0x7fe63401c800 comp rx=0 tx=0).stop 2024-09-17T03:32:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 -- 172.21.15.46:0/4121896029 shutdown_connections 2024-09-17T03:32:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 --2- 172.21.15.46:0/4121896029 >> v2:172.21.15.46:3300/0 conn(0x7fe644069500 0x7fe644162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 --2- 172.21.15.46:0/4121896029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6440b6500 0x7fe644070d50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 -- 172.21.15.46:0/4121896029 >> 172.21.15.46:0/4121896029 conn(0x7fe6440b3a20 msgr2=0x7fe6440d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 -- 172.21.15.46:0/4121896029 shutdown_connections 2024-09-17T03:32:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.136+0000 7fe64a600640 1 -- 172.21.15.46:0/4121896029 wait complete. 2024-09-17T03:32:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.140+0000 7fe64a600640 1 Processor -- start 2024-09-17T03:32:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.140+0000 7fe64a600640 1 -- start start 2024-09-17T03:32:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe64a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6440b6500 0x7fe64414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe64a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 0x7fe644157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe64a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe644157e70 0x7fe6440b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe64a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe644159030 con 0x7fe6440b6500 2024-09-17T03:32:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe64a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe644158eb0 con 0x7fe64415ee60 2024-09-17T03:32:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe64a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe6441591b0 con 0x7fe644157e70 2024-09-17T03:32:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe643e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe644157e70 0x7fe6440b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe642a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 0x7fe644157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe642a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 0x7fe644157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:52116/0 (socket says 172.21.15.46:52116) 2024-09-17T03:32:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe642a00640 1 -- 172.21.15.46:0/52724461 learned_addr learned my addr 172.21.15.46:0/52724461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe643400640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6440b6500 0x7fe64414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe642a00640 1 -- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe644157e70 msgr2=0x7fe6440b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe642a00640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe644157e70 0x7fe6440b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.144+0000 7fe642a00640 1 -- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6440b6500 msgr2=0x7fe64414d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe642a00640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6440b6500 0x7fe64414d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe642a00640 1 -- 172.21.15.46:0/52724461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6441561a0 con 0x7fe64415ee60 2024-09-17T03:32:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe643400640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6440b6500 0x7fe64414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:32:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe642a00640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 0x7fe644157930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c420 tx=0x7fe63000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630017070 con 0x7fe64415ee60 2024-09-17T03:32:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6300053c0 con 0x7fe64415ee60 2024-09-17T03:32:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630003860 con 0x7fe64415ee60 2024-09-17T03:32:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe644156480 con 0x7fe64415ee60 2024-09-17T03:32:46.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.148+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe644156a40 con 0x7fe64415ee60 2024-09-17T03:32:46.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.152+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe644162830 con 0x7fe64415ee60 2024-09-17T03:32:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.156+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fe63001a070 con 0x7fe64415ee60 2024-09-17T03:32:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.156+0000 7fe63b400640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe624074960 0x7fe624076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.156+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7fe6300925d0 con 0x7fe64415ee60 2024-09-17T03:32:46.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.160+0000 7fe643400640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe624074960 0x7fe624076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.160+0000 7fe643400640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe624074960 0x7fe624076e70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe6340af000 tx=0x7fe63401b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:46.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.160+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe630021020 con 0x7fe64415ee60 2024-09-17T03:32:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.544+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone79881", "target": ["mon-mgr", ""]}) -- 0x7fe644156db0 con 0x7fe624074960 2024-09-17T03:32:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.588+0000 7fe63b400640 1 -- 172.21.15.46:0/52724461 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe644156db0 con 0x7fe624074960 2024-09-17T03:32:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe624074960 msgr2=0x7fe624076e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe624074960 0x7fe624076e70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe6340af000 tx=0x7fe63401b040 comp rx=0 tx=0).stop 2024-09-17T03:32:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 msgr2=0x7fe644157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 0x7fe644157930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe63000c420 tx=0x7fe63000c8e0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 shutdown_connections 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe624074960 0x7fe624076e70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe644157e70 0x7fe6440b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe64415ee60 0x7fe644157930 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 --2- 172.21.15.46:0/52724461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6440b6500 0x7fe64414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 >> 172.21.15.46:0/52724461 conn(0x7fe6440b3a20 msgr2=0x7fe6440705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 shutdown_connections 2024-09-17T03:32:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.592+0000 7fe64a600640 1 -- 172.21.15.46:0/52724461 wait complete. 2024-09-17T03:32:46.642 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:32:46.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:32:46.703 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:32:46.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:32:46.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:32:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 -- 172.21.15.46:0/3579755018 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815aa50 msgr2=0x7f1d980cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/3579755018 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815aa50 0x7f1d980cd260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000afd0 tx=0x7f1d9001d850 comp rx=0 tx=0).stop 2024-09-17T03:32:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 -- 172.21.15.46:0/3579755018 shutdown_connections 2024-09-17T03:32:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/3579755018 >> v2:172.21.15.99:3300/0 conn(0x7f1d980cd7a0 0x7f1d980cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/3579755018 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815aa50 0x7f1d980cd260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 -- 172.21.15.46:0/3579755018 >> 172.21.15.46:0/3579755018 conn(0x7f1d980b2040 msgr2=0x7f1d980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 -- 172.21.15.46:0/3579755018 shutdown_connections 2024-09-17T03:32:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.944+0000 7f1d9cc00640 1 -- 172.21.15.46:0/3579755018 wait complete. 2024-09-17T03:32:46.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.948+0000 7f1d9cc00640 1 Processor -- start 2024-09-17T03:32:46.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.948+0000 7f1d9cc00640 1 -- start start 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d9cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 0x7f1d981f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d9cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815aa50 0x7f1d9813e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d9cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815e4e0 0x7f1d980ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d9cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981683d0 con 0x7f1d980cd7a0 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d9cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98168250 con 0x7f1d9815aa50 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d9cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1d98168550 con 0x7f1d9815e4e0 2024-09-17T03:32:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d96800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815e4e0 0x7f1d980ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 0x7f1d981f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 0x7f1d981f9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57528/0 (socket says 172.21.15.46:57528) 2024-09-17T03:32:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 -- 172.21.15.46:0/2575098642 learned_addr learned my addr 172.21.15.46:0/2575098642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95400640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815aa50 0x7f1d9813e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 -- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815e4e0 msgr2=0x7f1d980ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815e4e0 0x7f1d980ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 -- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815aa50 msgr2=0x7f1d9813e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815aa50 0x7f1d9813e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 -- 172.21.15.46:0/2575098642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d981412b0 con 0x7f1d980cd7a0 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d96800640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815e4e0 0x7f1d980ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.952+0000 7f1d95e00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 0x7f1d981f9b00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f90 tx=0x7f1d9000ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.956+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90004e60 con 0x7f1d980cd7a0 2024-09-17T03:32:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.956+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d900a6740 con 0x7f1d980cd7a0 2024-09-17T03:32:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.956+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900a6bf0 con 0x7f1d980cd7a0 2024-09-17T03:32:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.956+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d98141590 con 0x7f1d980cd7a0 2024-09-17T03:32:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.956+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d98141b50 con 0x7f1d980cd7a0 2024-09-17T03:32:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.964+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d980cdbb0 con 0x7f1d980cd7a0 2024-09-17T03:32:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.964+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1d900bd060 con 0x7f1d980cd7a0 2024-09-17T03:32:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.964+0000 7f1d8e000640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d780749e0 0x7f1d78076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:46.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.964+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f1d9012c780 con 0x7f1d980cd7a0 2024-09-17T03:32:46.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.968+0000 7f1d95400640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d780749e0 0x7f1d78076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.968+0000 7f1d95400640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d780749e0 0x7f1d78076ef0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1d98153870 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:46.976+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1d90017440 con 0x7f1d980cd7a0 2024-09-17T03:32:47.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.348+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f1d980b2710 con 0x7f1d980cd7a0 2024-09-17T03:32:47.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.348+0000 7f1d8e000640 1 -- 172.21.15.46:0/2575098642 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v47)=0 v47) ==== 131+0+0 (secure 0 0 0) 0x7f1d900f9310 con 0x7f1d980cd7a0 2024-09-17T03:32:47.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d780749e0 msgr2=0x7f1d78076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:47.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d780749e0 0x7f1d78076ef0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1d98153870 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2024-09-17T03:32:47.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 msgr2=0x7f1d981f9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:47.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 0x7f1d981f9b00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f90 tx=0x7f1d9000ac90 comp rx=0 tx=0).stop 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 shutdown_connections 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d780749e0 0x7f1d78076ef0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d9815e4e0 0x7f1d980ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d9815aa50 0x7f1d9813e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 --2- 172.21.15.46:0/2575098642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d980cd7a0 0x7f1d981f9b00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 >> 172.21.15.46:0/2575098642 conn(0x7f1d980b2040 msgr2=0x7f1d98150490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.352+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 shutdown_connections 2024-09-17T03:32:47.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:47.356+0000 7f1d9cc00640 1 -- 172.21.15.46:0/2575098642 wait complete. 2024-09-17T03:32:47.390 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:32:47.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:32:47.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:32:47.633 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:32:47.634 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:32:47.634 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:32:47.634 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:47.648 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:32:47.648 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:47.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:32:48.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 -- 172.21.15.46:0/1646092067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 msgr2=0x7f605415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 --2- 172.21.15.46:0/1646092067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f605415aa50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6040009840 tx=0x7f604001c820 comp rx=0 tx=0).stop 2024-09-17T03:32:48.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 -- 172.21.15.46:0/1646092067 shutdown_connections 2024-09-17T03:32:48.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 --2- 172.21.15.46:0/1646092067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f605415aa50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 -- 172.21.15.46:0/1646092067 >> 172.21.15.46:0/1646092067 conn(0x7f60540b2040 msgr2=0x7f60540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 -- 172.21.15.46:0/1646092067 shutdown_connections 2024-09-17T03:32:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.012+0000 7f605b600640 1 -- 172.21.15.46:0/1646092067 wait complete. 2024-09-17T03:32:48.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 Processor -- start 2024-09-17T03:32:48.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 -- start start 2024-09-17T03:32:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f6054153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60540ce4c0 0x7f605416b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605415ebf0 0x7f605416ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6054169200 con 0x7f605415ebf0 2024-09-17T03:32:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6054169080 con 0x7f60540cdaf0 2024-09-17T03:32:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.016+0000 7f605b600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6054169380 con 0x7f60540ce4c0 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60540ce4c0 0x7f605416b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6058c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f6054153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6058c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f6054153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:52162/0 (socket says 172.21.15.46:52162) 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6058c00640 1 -- 172.21.15.46:0/2822101441 learned_addr learned my addr 172.21.15.46:0/2822101441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6059600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605415ebf0 0x7f605416ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 -- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 msgr2=0x7f6054153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f6054153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 -- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605415ebf0 msgr2=0x7f605416ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605415ebf0 0x7f605416ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 -- 172.21.15.46:0/2822101441 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60541597a0 con 0x7f60540ce4c0 2024-09-17T03:32:48.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6058c00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f6054153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:48.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6053e00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60540ce4c0 0x7f605416b4c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60400ac000 tx=0x7f60400051a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:48.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400bd070 con 0x7f60540ce4c0 2024-09-17T03:32:48.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60400b3040 con 0x7f60540ce4c0 2024-09-17T03:32:48.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400b2070 con 0x7f60540ce4c0 2024-09-17T03:32:48.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6054159a60 con 0x7f60540ce4c0 2024-09-17T03:32:48.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.020+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f605416acf0 con 0x7f60540ce4c0 2024-09-17T03:32:48.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.024+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605415aa50 con 0x7f60540ce4c0 2024-09-17T03:32:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.024+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f6040004290 con 0x7f60540ce4c0 2024-09-17T03:32:48.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.024+0000 7f6050c00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6034074910 0x7f6034076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.024+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f604012c6a0 con 0x7f60540ce4c0 2024-09-17T03:32:48.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.024+0000 7f6058c00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6034074910 0x7f6034076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.028+0000 7f6058c00640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6034074910 0x7f6034076e20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6048007fd0 tx=0x7f6048007f60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.036+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6040016410 con 0x7f60540ce4c0 2024-09-17T03:32:48.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.268+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6054075d10 con 0x7f60540ce4c0 2024-09-17T03:32:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.268+0000 7f6050c00640 1 -- 172.21.15.46:0/2822101441 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f60400f8ad0 con 0x7f60540ce4c0 2024-09-17T03:32:48.269 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:32:48.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6034074910 msgr2=0x7f6034076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6034074910 0x7f6034076e20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6048007fd0 tx=0x7f6048007f60 comp rx=0 tx=0).stop 2024-09-17T03:32:48.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60540ce4c0 msgr2=0x7f605416b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60540ce4c0 0x7f605416b4c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60400ac000 tx=0x7f60400051a0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 shutdown_connections 2024-09-17T03:32:48.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6034074910 0x7f6034076e20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f605415ebf0 0x7f605416ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f60540ce4c0 0x7f605416b4c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 --2- 172.21.15.46:0/2822101441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60540cdaf0 0x7f6054153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 >> 172.21.15.46:0/2822101441 conn(0x7f60540b2040 msgr2=0x7f605406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:48.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 shutdown_connections 2024-09-17T03:32:48.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.272+0000 7f605b600640 1 -- 172.21.15.46:0/2822101441 wait complete. 2024-09-17T03:32:48.292 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:32:48.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:32:48.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:32:48.342 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:32:48.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:32:48.342 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:32:48.396 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:32:48.397 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:32:48.397 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:48.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:32:48.407 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:32:48.407 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:32:48.407 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:48.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:32:48.451 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:32:48.451 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:32:48.451 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:48.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:32:48.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:32:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 -- 172.21.15.46:0/1229181365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cdd80 msgr2=0x7fdf300b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 --2- 172.21.15.46:0/1229181365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cdd80 0x7fdf300b3a50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800acd0 tx=0x7fdf2801def0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 -- 172.21.15.46:0/1229181365 shutdown_connections 2024-09-17T03:32:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 --2- 172.21.15.46:0/1229181365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cdd80 0x7fdf300b3a50 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 -- 172.21.15.46:0/1229181365 >> 172.21.15.46:0/1229181365 conn(0x7fdf300b2040 msgr2=0x7fdf300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 -- 172.21.15.46:0/1229181365 shutdown_connections 2024-09-17T03:32:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.652+0000 7fdf37800640 1 -- 172.21.15.46:0/1229181365 wait complete. 2024-09-17T03:32:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 Processor -- start 2024-09-17T03:32:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 -- start start 2024-09-17T03:32:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 0x7fdf3014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf300cdd80 0x7fdf3014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf3015aa50 0x7fdf30157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30169490 con 0x7fdf300cd970 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30169310 con 0x7fdf300cdd80 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdf30169610 con 0x7fdf3015aa50 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 0x7fdf3014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 0x7fdf3014f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57550/0 (socket says 172.21.15.46:57550) 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 -- 172.21.15.46:0/4108062792 learned_addr learned my addr 172.21.15.46:0/4108062792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf2fe00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf300cdd80 0x7fdf3014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf35800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf3015aa50 0x7fdf30157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 -- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf3015aa50 msgr2=0x7fdf30157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf3015aa50 0x7fdf30157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 -- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf300cdd80 msgr2=0x7fdf3014a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf300cdd80 0x7fdf3014a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 -- 172.21.15.46:0/4108062792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf300af050 con 0x7fdf300cd970 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf34e00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 0x7fdf3014f940 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c400 tx=0x7fdf2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24011790 con 0x7fdf300cd970 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf24017070 con 0x7fdf300cd970 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24011ce0 con 0x7fdf300cd970 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf30151ec0 con 0x7fdf300cd970 2024-09-17T03:32:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.656+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf301faa20 con 0x7fdf300cd970 2024-09-17T03:32:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.660+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf300b1540 con 0x7fdf300cd970 2024-09-17T03:32:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.660+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fdf2401c460 con 0x7fdf300cd970 2024-09-17T03:32:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.660+0000 7fdf2cc00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf10074910 0x7fdf10076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.660+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7fdf24092bf0 con 0x7fdf300cd970 2024-09-17T03:32:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.660+0000 7fdf2fe00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf10074910 0x7fdf10076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.660+0000 7fdf2fe00640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf10074910 0x7fdf10076e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c000c10 tx=0x7fdf1c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.664+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf2405f0b0 con 0x7fdf300cd970 2024-09-17T03:32:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.980+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf300b1260 con 0x7fdf300cd970 2024-09-17T03:32:48.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.980+0000 7fdf2cc00640 1 -- 172.21.15.46:0/4108062792 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5241 (secure 0 0 0) 0x7fdf24063ae0 con 0x7fdf300cd970 2024-09-17T03:32:48.981 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:48.981 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":56,"btime":"2024-09-17T03:30:30:404049+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:30:16.771417+0000","modified":"2024-09-17T03:30:19.787080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246888":{"gid":246888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2776262749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2776262749},{"type":"v1","addr":"172.21.15.46:6835","nonce":2776262749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246888,"qdb_cluster":[246888]},"id":6}]} 2024-09-17T03:32:48.981 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 56 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf10074910 msgr2=0x7fdf10076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf10074910 0x7fdf10076e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c000c10 tx=0x7fdf1c009040 comp rx=0 tx=0).stop 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 msgr2=0x7fdf3014f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 0x7fdf3014f940 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c400 tx=0x7fdf2400c8c0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 shutdown_connections 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdf10074910 0x7fdf10076e20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdf3015aa50 0x7fdf30157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf300cdd80 0x7fdf3014a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 --2- 172.21.15.46:0/4108062792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf300cd970 0x7fdf3014f940 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:48.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 >> 172.21.15.46:0/4108062792 conn(0x7fdf300b2040 msgr2=0x7fdf300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:48.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 shutdown_connections 2024-09-17T03:32:48.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:48.984+0000 7fdf37800640 1 -- 172.21.15.46:0/4108062792 wait complete. 2024-09-17T03:32:49.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:32:49.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 -- 172.21.15.46:0/2731560226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41741536f0 msgr2=0x7f417415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 --2- 172.21.15.46:0/2731560226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41741536f0 0x7f417415d480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4168009840 tx=0x7f416801c790 comp rx=0 tx=0).stop 2024-09-17T03:32:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 -- 172.21.15.46:0/2731560226 shutdown_connections 2024-09-17T03:32:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 --2- 172.21.15.46:0/2731560226 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4174074950 0x7f41740b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 --2- 172.21.15.46:0/2731560226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41741536f0 0x7f417415d480 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 -- 172.21.15.46:0/2731560226 >> 172.21.15.46:0/2731560226 conn(0x7f41740b4560 msgr2=0x7f41740d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 -- 172.21.15.46:0/2731560226 shutdown_connections 2024-09-17T03:32:49.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.256+0000 7f417b200640 1 -- 172.21.15.46:0/2731560226 wait complete. 2024-09-17T03:32:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 Processor -- start 2024-09-17T03:32:49.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 -- start start 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 0x7f41741fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4174160060 0x7f41740d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f417406d8a0 0x7f41741f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4174169dc0 con 0x7f41741536f0 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4174169c40 con 0x7f417406d8a0 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4174169f40 con 0x7f4174160060 2024-09-17T03:32:49.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 0x7f41741fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 0x7f41741fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57564/0 (socket says 172.21.15.46:57564) 2024-09-17T03:32:49.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 -- 172.21.15.46:0/275384393 learned_addr learned my addr 172.21.15.46:0/275384393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:49.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f417a800640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f417406d8a0 0x7f41741f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179400640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4174160060 0x7f41740d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 -- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4174160060 msgr2=0x7f41740d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4174160060 0x7f41740d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 -- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f417406d8a0 msgr2=0x7f41741f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f417406d8a0 0x7f41741f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.260+0000 7f4179e00640 1 -- 172.21.15.46:0/275384393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41740d1270 con 0x7f41741536f0 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f4179e00640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 0x7f41741fc260 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f416801cc70 tx=0x7f4168005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680be070 con 0x7f41741536f0 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41680a8a80 con 0x7f41741536f0 2024-09-17T03:32:49.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168005040 con 0x7f41741536f0 2024-09-17T03:32:49.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41741fca90 con 0x7f41741536f0 2024-09-17T03:32:49.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41741fd050 con 0x7f41741536f0 2024-09-17T03:32:49.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41740b5260 con 0x7f41741536f0 2024-09-17T03:32:49.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.264+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f41680051e0 con 0x7f41741536f0 2024-09-17T03:32:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.268+0000 7f416e000640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4158074940 0x7f4158076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.268+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f416812d120 con 0x7f41741536f0 2024-09-17T03:32:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.268+0000 7f4179400640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4158074940 0x7f4158076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.268+0000 7f4179400640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4158074940 0x7f4158076e50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4164007d70 tx=0x7f4164007d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:49.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.272+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41680ac030 con 0x7f41741536f0 2024-09-17T03:32:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.484+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f417415d710 con 0x7f41741536f0 2024-09-17T03:32:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.484+0000 7f416e000640 1 -- 172.21.15.46:0/275384393 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5241 (secure 0 0 0) 0x7f4168016410 con 0x7f41741536f0 2024-09-17T03:32:49.486 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:49.486 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":56,"btime":"2024-09-17T03:30:30:404049+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:30:16.771417+0000","modified":"2024-09-17T03:30:19.787080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246888":{"gid":246888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2776262749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2776262749},{"type":"v1","addr":"172.21.15.46:6835","nonce":2776262749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246888,"qdb_cluster":[246888]},"id":6}]} 2024-09-17T03:32:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 56 2024-09-17T03:32:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4158074940 msgr2=0x7f4158076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4158074940 0x7f4158076e50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4164007d70 tx=0x7f4164007d00 comp rx=0 tx=0).stop 2024-09-17T03:32:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 msgr2=0x7f41741fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 0x7f41741fc260 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f416801cc70 tx=0x7f4168005db0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 shutdown_connections 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4158074940 0x7f4158076e50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f417406d8a0 0x7f41741f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4174160060 0x7f41740d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 --2- 172.21.15.46:0/275384393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41741536f0 0x7f41741fc260 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 >> 172.21.15.46:0/275384393 conn(0x7f41740b4560 msgr2=0x7f41740715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 shutdown_connections 2024-09-17T03:32:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.488+0000 7f417b200640 1 -- 172.21.15.46:0/275384393 wait complete. 2024-09-17T03:32:49.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 -- 172.21.15.46:0/3190276586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8160d50 msgr2=0x7f85b8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 --2- 172.21.15.46:0/3190276586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8160d50 0x7f85b8074410 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009840 tx=0x7f85a001c820 comp rx=0 tx=0).stop 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 -- 172.21.15.46:0/3190276586 shutdown_connections 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 --2- 172.21.15.46:0/3190276586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8160d50 0x7f85b8074410 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 --2- 172.21.15.46:0/3190276586 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8157030 0x7f85b80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 -- 172.21.15.46:0/3190276586 >> 172.21.15.46:0/3190276586 conn(0x7f85b80708c0 msgr2=0x7f85b8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 -- 172.21.15.46:0/3190276586 shutdown_connections 2024-09-17T03:32:49.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 -- 172.21.15.46:0/3190276586 wait complete. 2024-09-17T03:32:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 Processor -- start 2024-09-17T03:32:49.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.668+0000 7f85bce00640 1 -- start start 2024-09-17T03:32:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8157030 0x7f85b8148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b8160d50 0x7f85b8151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 0x7f85b815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85b816a820 con 0x7f85b8161160 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85b816a6a0 con 0x7f85b8160d50 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f85b816a9a0 con 0x7f85b8157030 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 0x7f85b815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 0x7f85b815bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57596/0 (socket says 172.21.15.46:57596) 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 -- 172.21.15.46:0/835338213 learned_addr learned my addr 172.21.15.46:0/835338213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b5e00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8157030 0x7f85b8148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85ad400640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b8160d50 0x7f85b8151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 -- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8157030 msgr2=0x7f85b8148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8157030 0x7f85b8148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 -- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b8160d50 msgr2=0x7f85b8151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b8160d50 0x7f85b8151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 -- 172.21.15.46:0/835338213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b806ec70 con 0x7f85b8161160 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85b6800640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 0x7f85b815bac0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c730 tx=0x7f85b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0003b80 con 0x7f85b8161160 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85b0003d40 con 0x7f85b8161160 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b806e3d0 con 0x7f85b8161160 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00052d0 con 0x7f85b8161160 2024-09-17T03:32:49.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b806e990 con 0x7f85b8161160 2024-09-17T03:32:49.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.672+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b8152590 con 0x7f85b8161160 2024-09-17T03:32:49.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.676+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f85b0005680 con 0x7f85b8161160 2024-09-17T03:32:49.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.676+0000 7f85aea00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8594074910 0x7f8594076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:49.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.676+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f85b0092e90 con 0x7f85b8161160 2024-09-17T03:32:49.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.676+0000 7f85b5e00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8594074910 0x7f8594076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:49.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.676+0000 7f85b5e00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8594074910 0x7f8594076e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009b40 tx=0x7f85a80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.680+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f85b005f2d0 con 0x7f85b8161160 2024-09-17T03:32:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.852+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85b80d2260 con 0x7f85b8161160 2024-09-17T03:32:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85aea00640 1 -- 172.21.15.46:0/835338213 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16800 (secure 0 0 0) 0x7f85b0063d00 con 0x7f85b8161160 2024-09-17T03:32:49.856 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:49.856 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:32:46.102335+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:30:14.515457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T03:30:15.740742+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:32:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8594074910 msgr2=0x7f8594076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8594074910 0x7f8594076e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009b40 tx=0x7f85a80091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 msgr2=0x7f85b815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 0x7f85b815bac0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f85b000c730 tx=0x7f85b000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 shutdown_connections 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8594074910 0x7f8594076e20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b8161160 0x7f85b815bac0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85b8160d50 0x7f85b8151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 --2- 172.21.15.46:0/835338213 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85b8157030 0x7f85b8148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 >> 172.21.15.46:0/835338213 conn(0x7f85b80708c0 msgr2=0x7f85b80d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 shutdown_connections 2024-09-17T03:32:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:49.856+0000 7f85bce00640 1 -- 172.21.15.46:0/835338213 wait complete. 2024-09-17T03:32:49.879 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:32:49.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:32:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 -- 172.21.15.46:0/365790886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 msgr2=0x7f72e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 --2- 172.21.15.46:0/365790886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e015bb60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009840 tx=0x7f72d001c6b0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 -- 172.21.15.46:0/365790886 shutdown_connections 2024-09-17T03:32:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 --2- 172.21.15.46:0/365790886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72e00b4de0 0x7f72e00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 --2- 172.21.15.46:0/365790886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e015bb60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 -- 172.21.15.46:0/365790886 >> 172.21.15.46:0/365790886 conn(0x7f72e00b2040 msgr2=0x7f72e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 -- 172.21.15.46:0/365790886 shutdown_connections 2024-09-17T03:32:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.036+0000 7f72e7000640 1 -- 172.21.15.46:0/365790886 wait complete. 2024-09-17T03:32:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 Processor -- start 2024-09-17T03:32:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 -- start start 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72e00b4de0 0x7f72e00af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e00b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 0x7f72e00b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72e015d2f0 con 0x7f72e01527b0 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72e015d170 con 0x7f72e00b5c70 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.040+0000 7f72e7000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f72e015d470 con 0x7f72e00b4de0 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 0x7f72e00b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 0x7f72e00b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57598/0 (socket says 172.21.15.46:57598) 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 -- 172.21.15.46:0/1585712102 learned_addr learned my addr 172.21.15.46:0/1585712102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72dfe00640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72e00b4de0 0x7f72e00af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72d7e00640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e00b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 -- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72e00b4de0 msgr2=0x7f72e00af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72e00b4de0 0x7f72e00af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 -- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 msgr2=0x7f72e00b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e00b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 -- 172.21.15.46:0/1585712102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e01f8e20 con 0x7f72e01527b0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72d7e00640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e00b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e5000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 0x7f72e00b15d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f72d800ec00 tx=0x7f72d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8003b80 con 0x7f72e01527b0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72d8003e10 con 0x7f72e01527b0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004280 con 0x7f72e01527b0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e01f9100 con 0x7f72e01527b0 2024-09-17T03:32:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e01f96c0 con 0x7f72e01527b0 2024-09-17T03:32:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.044+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e00b51f0 con 0x7f72e01527b0 2024-09-17T03:32:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.048+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f72d8004420 con 0x7f72e01527b0 2024-09-17T03:32:50.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.048+0000 7f72dcc00640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72bc074940 0x7f72bc076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.048+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f72d8093d30 con 0x7f72e01527b0 2024-09-17T03:32:50.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.048+0000 7f72dfe00640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72bc074940 0x7f72bc076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.048+0000 7f72dfe00640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72bc074940 0x7f72bc076e50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f72d00af000 tx=0x7f72d001aef0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:50.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.052+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f72d8060840 con 0x7f72e01527b0 2024-09-17T03:32:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.264+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e00aee60 con 0x7f72e01527b0 2024-09-17T03:32:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.264+0000 7f72dcc00640 1 -- 172.21.15.46:0/1585712102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5241 (secure 0 0 0) 0x7f72d8060140 con 0x7f72e01527b0 2024-09-17T03:32:50.266 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:50.266 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":56,"btime":"2024-09-17T03:30:30:404049+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:30:16.771417+0000","modified":"2024-09-17T03:30:19.787080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246888":{"gid":246888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2776262749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2776262749},{"type":"v1","addr":"172.21.15.46:6835","nonce":2776262749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246888,"qdb_cluster":[246888]},"id":6}]} 2024-09-17T03:32:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 56 2024-09-17T03:32:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72bc074940 msgr2=0x7f72bc076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72bc074940 0x7f72bc076e50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f72d00af000 tx=0x7f72d001aef0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 msgr2=0x7f72e00b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 0x7f72e00b15d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f72d800ec00 tx=0x7f72d800c6a0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 shutdown_connections 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f72bc074940 0x7f72bc076e50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e01527b0 0x7f72e00b15d0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e00b5c70 0x7f72e00b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 --2- 172.21.15.46:0/1585712102 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72e00b4de0 0x7f72e00af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 >> 172.21.15.46:0/1585712102 conn(0x7f72e00b2040 msgr2=0x7f72e0158190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 shutdown_connections 2024-09-17T03:32:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.268+0000 7f72e7000640 1 -- 172.21.15.46:0/1585712102 wait complete. 2024-09-17T03:32:50.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:32:50.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 -- 172.21.15.46:0/3555467516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 msgr2=0x7f168815b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 --2- 172.21.15.46:0/3555467516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f168815b9e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1688071130 tx=0x7f168001cb80 comp rx=0 tx=0).stop 2024-09-17T03:32:50.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 -- 172.21.15.46:0/3555467516 shutdown_connections 2024-09-17T03:32:50.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 --2- 172.21.15.46:0/3555467516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f168815b9e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 --2- 172.21.15.46:0/3555467516 >> v2:172.21.15.99:3300/0 conn(0x7f168815eeb0 0x7f16880cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 -- 172.21.15.46:0/3555467516 >> 172.21.15.46:0/3555467516 conn(0x7f16880b2040 msgr2=0x7f16880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:50.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 -- 172.21.15.46:0/3555467516 shutdown_connections 2024-09-17T03:32:50.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.448+0000 7f168fc00640 1 -- 172.21.15.46:0/3555467516 wait complete. 2024-09-17T03:32:50.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 Processor -- start 2024-09-17T03:32:50.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 -- start start 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16880cf620 0x7f168813ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f1688153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f168815eeb0 0x7f16881539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1688169290 con 0x7f168815e190 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1688169110 con 0x7f168815eeb0 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1688169410 con 0x7f16880cf620 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f1688153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f1688153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57612/0 (socket says 172.21.15.46:57612) 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 -- 172.21.15.46:0/24533165 learned_addr learned my addr 172.21.15.46:0/24533165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168dc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f168815eeb0 0x7f16881539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168d200640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16880cf620 0x7f168813ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 -- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16880cf620 msgr2=0x7f168813ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16880cf620 0x7f168813ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 -- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f168815eeb0 msgr2=0x7f16881539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f168815eeb0 0x7f16881539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 -- 172.21.15.46:0/24533165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1688154b40 con 0x7f168815e190 2024-09-17T03:32:50.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f167fe00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f1688153470 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1670001ec0 tx=0x7f167000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:50.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670017070 con 0x7f168815e190 2024-09-17T03:32:50.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1670003600 con 0x7f168815e190 2024-09-17T03:32:50.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16881f98e0 con 0x7f168815e190 2024-09-17T03:32:50.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670003a50 con 0x7f168815e190 2024-09-17T03:32:50.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.452+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881f9ea0 con 0x7f168815e190 2024-09-17T03:32:50.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.456+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16881f9630 con 0x7f168815e190 2024-09-17T03:32:50.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.456+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f167001a070 con 0x7f168815e190 2024-09-17T03:32:50.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.456+0000 7f1685600640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1664074940 0x7f1664076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.456+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f16700920f0 con 0x7f168815e190 2024-09-17T03:32:50.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.456+0000 7f168d200640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1664074940 0x7f1664076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.456+0000 7f168d200640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1664074940 0x7f1664076e50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1678005fd0 tx=0x7f167800b390 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:50.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.460+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f167005e730 con 0x7f168815e190 2024-09-17T03:32:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.676+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16880cf0e0 con 0x7f168815e190 2024-09-17T03:32:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.676+0000 7f1685600640 1 -- 172.21.15.46:0/24533165 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5241 (secure 0 0 0) 0x7f1670063160 con 0x7f168815e190 2024-09-17T03:32:50.677 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:50.677 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":56,"btime":"2024-09-17T03:30:30:404049+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:30:16.771417+0000","modified":"2024-09-17T03:30:19.787080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":246888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_246888":{"gid":246888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2776262749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2776262749},{"type":"v1","addr":"172.21.15.46:6835","nonce":2776262749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":246888,"qdb_cluster":[246888]},"id":6}]} 2024-09-17T03:32:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 56 2024-09-17T03:32:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1664074940 msgr2=0x7f1664076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1664074940 0x7f1664076e50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1678005fd0 tx=0x7f167800b390 comp rx=0 tx=0).stop 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 msgr2=0x7f1688153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f1688153470 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1670001ec0 tx=0x7f167000c890 comp rx=0 tx=0).stop 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 shutdown_connections 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1664074940 0x7f1664076e50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f168815eeb0 0x7f16881539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f168815e190 0x7f1688153470 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 --2- 172.21.15.46:0/24533165 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16880cf620 0x7f168813ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 >> 172.21.15.46:0/24533165 conn(0x7f16880b2040 msgr2=0x7f168813d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 shutdown_connections 2024-09-17T03:32:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.680+0000 7f168fc00640 1 -- 172.21.15.46:0/24533165 wait complete. 2024-09-17T03:32:50.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 -- 172.21.15.46:0/4281318264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 msgr2=0x7f0f50004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 --2- 172.21.15.46:0/4281318264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f50004c20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009b40 tx=0x7f0f5401d270 comp rx=0 tx=0).stop 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f71800640 1 -- 172.21.15.46:0/4281318264 reap_dead start 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 -- 172.21.15.46:0/4281318264 shutdown_connections 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 -- 172.21.15.46:0/4281318264 >> 172.21.15.46:0/4281318264 conn(0x7f0f6c0708c0 msgr2=0x7f0f6c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 -- 172.21.15.46:0/4281318264 shutdown_connections 2024-09-17T03:32:50.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 -- 172.21.15.46:0/4281318264 wait complete. 2024-09-17T03:32:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 Processor -- start 2024-09-17T03:32:50.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.860+0000 7f0f73800640 1 -- start start 2024-09-17T03:32:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f6c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c14aec0 0x7f0f6c1f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f6c1f9840 0x7f0f6c1f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d2f50 con 0x7f0f50004810 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d2dd0 con 0x7f0f6c14aec0 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d30d0 con 0x7f0f6c1f9840 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f6c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f6c1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57628/0 (socket says 172.21.15.46:57628) 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 -- 172.21.15.46:0/2624088574 learned_addr learned my addr 172.21.15.46:0/2624088574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f63e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c14aec0 0x7f0f6c1f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f71800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f6c1f9840 0x7f0f6c1f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 -- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f6c1f9840 msgr2=0x7f0f6c1f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f6c1f9840 0x7f0f6c1f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 -- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c14aec0 msgr2=0x7f0f6c1f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c14aec0 0x7f0f6c1f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 -- 172.21.15.46:0/2624088574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c1f9c50 con 0x7f0f50004810 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f71800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f6c1f9840 0x7f0f6c1f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f63e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c14aec0 0x7f0f6c1f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f70e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f6c1571c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c004680 tx=0x7f0f5c012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c013920 con 0x7f0f50004810 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f5c013ac0 con 0x7f0f50004810 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c010480 con 0x7f0f50004810 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c157ba0 con 0x7f0f50004810 2024-09-17T03:32:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c158160 con 0x7f0f50004810 2024-09-17T03:32:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.864+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c15fe50 con 0x7f0f50004810 2024-09-17T03:32:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.868+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f0f5c010620 con 0x7f0f50004810 2024-09-17T03:32:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.868+0000 7f0f69600640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f48074910 0x7f0f48076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.868+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f0f5c096c80 con 0x7f0f50004810 2024-09-17T03:32:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.868+0000 7f0f63e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f48074910 0x7f0f48076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.868+0000 7f0f63e00640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f48074910 0x7f0f48076e20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0f540ad000 tx=0x7f0f5401a010 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:50.872+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0f6c1f9c50 con 0x7f0f50004810 2024-09-17T03:32:51.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.048+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f6c0b5700 con 0x7f0f50004810 2024-09-17T03:32:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.048+0000 7f0f69600640 1 -- 172.21.15.46:0/2624088574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16800 (secure 0 0 0) 0x7f0f6c0d3310 con 0x7f0f50004810 2024-09-17T03:32:51.049 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:51.049 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:32:46.102335+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:30:14.515457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T03:30:15.740742+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.048+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f48074910 msgr2=0x7f0f48076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.048+0000 7f0f73800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f48074910 0x7f0f48076e20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0f540ad000 tx=0x7f0f5401a010 comp rx=0 tx=0).stop 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.048+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 msgr2=0x7f0f6c1571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.048+0000 7f0f73800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f6c1571c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c004680 tx=0x7f0f5c012870 comp rx=0 tx=0).stop 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 shutdown_connections 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f48074910 0x7f0f48076e20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f6c1f9840 0x7f0f6c1f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f6c14aec0 0x7f0f6c1f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 --2- 172.21.15.46:0/2624088574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f50004810 0x7f0f6c1571c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 >> 172.21.15.46:0/2624088574 conn(0x7f0f6c0708c0 msgr2=0x7f0f6c074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 shutdown_connections 2024-09-17T03:32:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.052+0000 7f0f73800640 1 -- 172.21.15.46:0/2624088574 wait complete. 2024-09-17T03:32:51.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 -- 172.21.15.46:0/4293722615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a4072360 msgr2=0x7f97a4072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 --2- 172.21.15.46:0/4293722615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a4072360 0x7f97a4072770 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9790009900 tx=0x7f979001c8a0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 -- 172.21.15.46:0/4293722615 shutdown_connections 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 --2- 172.21.15.46:0/4293722615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a4072360 0x7f97a4072770 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 --2- 172.21.15.46:0/4293722615 >> v2:172.21.15.99:3300/0 conn(0x7f97a415c250 0x7f97a415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 --2- 172.21.15.46:0/4293722615 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a41591f0 0x7f97a415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 -- 172.21.15.46:0/4293722615 >> 172.21.15.46:0/4293722615 conn(0x7f97a406d920 msgr2=0x7f97a40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 -- 172.21.15.46:0/4293722615 shutdown_connections 2024-09-17T03:32:51.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 -- 172.21.15.46:0/4293722615 wait complete. 2024-09-17T03:32:51.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 Processor -- start 2024-09-17T03:32:51.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.232+0000 7f97aa000640 1 -- start start 2024-09-17T03:32:51.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a4072360 0x7f97a4145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 0x7f97a4162fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a415c250 0x7f97a40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40cd840 con 0x7f97a41591f0 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40cd6c0 con 0x7f97a415c250 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f97a40cd9c0 con 0x7f97a4072360 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 0x7f97a4162fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 0x7f97a4162fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57644/0 (socket says 172.21.15.46:57644) 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 -- 172.21.15.46:0/3412557193 learned_addr learned my addr 172.21.15.46:0/3412557193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a8c00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a4072360 0x7f97a4145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a9600640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a415c250 0x7f97a40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 -- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a4072360 msgr2=0x7f97a4145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a4072360 0x7f97a4145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 -- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a415c250 msgr2=0x7f97a40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a415c250 0x7f97a40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 -- 172.21.15.46:0/3412557193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a41f8e50 con 0x7f97a41591f0 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a9600640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a415c250 0x7f97a40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a3e00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 0x7f97a4162fe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9790009850 tx=0x7f97900a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97900b2040 con 0x7f97a41591f0 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97900043b0 con 0x7f97a41591f0 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a41f8b50 con 0x7f97a41591f0 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790005270 con 0x7f97a41591f0 2024-09-17T03:32:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a41f9410 con 0x7f97a41591f0 2024-09-17T03:32:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.236+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a41f9ad0 con 0x7f97a41591f0 2024-09-17T03:32:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.240+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f9790005410 con 0x7f97a41591f0 2024-09-17T03:32:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.240+0000 7f97a0c00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9784074910 0x7f9784076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.240+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5766+0+0 (secure 0 0 0) 0x7f979012c740 con 0x7f97a41591f0 2024-09-17T03:32:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.240+0000 7f97a8c00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9784074910 0x7f9784076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.240+0000 7f97a8c00640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9784074910 0x7f9784076e20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9798004630 tx=0x7f979800b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:51.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.244+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f97a41f9ad0 con 0x7f97a41591f0 2024-09-17T03:32:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.452+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f97a415f180 con 0x7f97a41591f0 2024-09-17T03:32:51.454 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:32:51.452+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:32:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.456+0000 7f97a0c00640 1 -- 172.21.15.46:0/3412557193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) ==== 161+0+0 (secure 0 0 0) 0x7f97900bc060 con 0x7f97a41591f0 2024-09-17T03:32:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9784074910 msgr2=0x7f9784076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9784074910 0x7f9784076e20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9798004630 tx=0x7f979800b260 comp rx=0 tx=0).stop 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 msgr2=0x7f97a4162fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 0x7f97a4162fe0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9790009850 tx=0x7f97900a5690 comp rx=0 tx=0).stop 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 shutdown_connections 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9784074910 0x7f9784076e20 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9798004630 tx=0x7f979800b260 comp rx=0 tx=0).stop 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97a415c250 0x7f97a40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97a41591f0 0x7f97a4162fe0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 --2- 172.21.15.46:0/3412557193 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f97a4072360 0x7f97a4145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 >> 172.21.15.46:0/3412557193 conn(0x7f97a406d920 msgr2=0x7f97a415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 shutdown_connections 2024-09-17T03:32:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.460+0000 7f97aa000640 1 -- 172.21.15.46:0/3412557193 wait complete. 2024-09-17T03:32:51.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:32:51.613 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 -- 172.21.15.46:0/501582338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078072100 msgr2=0x7f9078155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 --2- 172.21.15.46:0/501582338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078072100 0x7f9078155b50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9068009840 tx=0x7f906801c790 comp rx=0 tx=0).stop 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 -- 172.21.15.46:0/501582338 shutdown_connections 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 --2- 172.21.15.46:0/501582338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078072100 0x7f9078155b50 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 -- 172.21.15.46:0/501582338 >> 172.21.15.46:0/501582338 conn(0x7f90780b2040 msgr2=0x7f90780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 -- 172.21.15.46:0/501582338 shutdown_connections 2024-09-17T03:32:51.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 -- 172.21.15.46:0/501582338 wait complete. 2024-09-17T03:32:51.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 Processor -- start 2024-09-17T03:32:51.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.660+0000 7f907ec00640 1 -- start start 2024-09-17T03:32:51.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9078072100 0x7f90781456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 0x7f907814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90780b4de0 0x7f9078151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9078169f70 con 0x7f9078073a90 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9078169df0 con 0x7f90780b4de0 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f907816a0f0 con 0x7f9078072100 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 0x7f907814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 0x7f907814fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57682/0 (socket says 172.21.15.46:57682) 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 -- 172.21.15.46:0/2864316799 learned_addr learned my addr 172.21.15.46:0/2864316799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f9077e00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9078072100 0x7f90781456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907cc00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90780b4de0 0x7f9078151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 -- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9078072100 msgr2=0x7f90781456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9078072100 0x7f90781456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 -- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90780b4de0 msgr2=0x7f9078151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90780b4de0 0x7f9078151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:51.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 -- 172.21.15.46:0/2864316799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9078145c20 con 0x7f9078073a90 2024-09-17T03:32:51.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f906fe00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 0x7f907814fb70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f906000c3c0 tx=0x7f906000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:51.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060011790 con 0x7f9078073a90 2024-09-17T03:32:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9060017070 con 0x7f9078073a90 2024-09-17T03:32:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9078150a00 con 0x7f9078073a90 2024-09-17T03:32:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060011d00 con 0x7f9078073a90 2024-09-17T03:32:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907814f370 con 0x7f9078073a90 2024-09-17T03:32:51.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.664+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9078155b50 con 0x7f9078073a90 2024-09-17T03:32:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.668+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f906003b6f0 con 0x7f9078073a90 2024-09-17T03:32:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.668+0000 7f9074c00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9054074940 0x7f9054076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.668+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5824+0+0 (secure 0 0 0) 0x7f9060092c20 con 0x7f9078073a90 2024-09-17T03:32:51.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.668+0000 7f9077e00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9054074940 0x7f9054076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:51.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.668+0000 7f9077e00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9054074940 0x7f9054076e50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f90680af000 tx=0x7f906801afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:51.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.672+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f906005f180 con 0x7f9078073a90 2024-09-17T03:32:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:51.880+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f90780b3740 con 0x7f9078073a90 2024-09-17T03:32:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.460+0000 7f9074c00640 1 -- 172.21.15.46:0/2864316799 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7f9060063bb0 con 0x7f9078073a90 2024-09-17T03:32:52.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9054074940 msgr2=0x7f9054076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:52.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9054074940 0x7f9054076e50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f90680af000 tx=0x7f906801afd0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 msgr2=0x7f907814fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:52.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 0x7f907814fb70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f906000c3c0 tx=0x7f906000c880 comp rx=0 tx=0).stop 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 shutdown_connections 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9054074940 0x7f9054076e50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90780b4de0 0x7f9078151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078073a90 0x7f907814fb70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 --2- 172.21.15.46:0/2864316799 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9078072100 0x7f90781456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 >> 172.21.15.46:0/2864316799 conn(0x7f90780b2040 msgr2=0x7f9078075e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 shutdown_connections 2024-09-17T03:32:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.464+0000 7f907ec00640 1 -- 172.21.15.46:0/2864316799 wait complete. 2024-09-17T03:32:52.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:32:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108fe00640 1 -- 172.21.15.46:0/2382375719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 msgr2=0x7f1070004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108fe00640 1 --2- 172.21.15.46:0/2382375719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f1070004c20 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f107400bbb0 tx=0x7f107401ce40 comp rx=0 tx=0).stop 2024-09-17T03:32:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108de00640 1 -- 172.21.15.46:0/2382375719 reap_dead start 2024-09-17T03:32:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108fe00640 1 -- 172.21.15.46:0/2382375719 shutdown_connections 2024-09-17T03:32:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108fe00640 1 -- 172.21.15.46:0/2382375719 >> 172.21.15.46:0/2382375719 conn(0x7f10880b2040 msgr2=0x7f10880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108fe00640 1 -- 172.21.15.46:0/2382375719 shutdown_connections 2024-09-17T03:32:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.660+0000 7f108fe00640 1 -- 172.21.15.46:0/2382375719 wait complete. 2024-09-17T03:32:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 Processor -- start 2024-09-17T03:32:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 -- start start 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f10881e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10881e5a00 0x7f10880ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10881e5e10 0x7f10880aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f108815cf30 con 0x7f1070004810 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f108815cdb0 con 0x7f10881e5e10 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f108815d0b0 con 0x7f10881e5a00 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f10881e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f10881e8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59608/0 (socket says 172.21.15.46:59608) 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 -- 172.21.15.46:0/2787330240 learned_addr learned my addr 172.21.15.46:0/2787330240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108ca00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10881e5a00 0x7f10880ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108de00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10881e5e10 0x7f10880aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 -- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10881e5a00 msgr2=0x7f10880ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10881e5a00 0x7f10880ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 -- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10881e5e10 msgr2=0x7f10880aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10881e5e10 0x7f10880aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 -- 172.21.15.46:0/2787330240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10881f9f00 con 0x7f1070004810 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108d400640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f10881e8550 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f107c011650 tx=0x7f107c011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c017070 con 0x7f1070004810 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f107c00fbc0 con 0x7f1070004810 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c013450 con 0x7f1070004810 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10881fa1e0 con 0x7f1070004810 2024-09-17T03:32:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.664+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10881fa7a0 con 0x7f1070004810 2024-09-17T03:32:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.668+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10880768c0 con 0x7f1070004810 2024-09-17T03:32:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.668+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f107c0135f0 con 0x7f1070004810 2024-09-17T03:32:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.668+0000 7f1085600640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1068074940 0x7f1068076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.668+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5824+0+0 (secure 0 0 0) 0x7f107c096eb0 con 0x7f1070004810 2024-09-17T03:32:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.668+0000 7f108ca00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1068074940 0x7f1068076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.672+0000 7f108ca00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1068074940 0x7f1068076e50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f10740b0000 tx=0x7f1074019be0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.672+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f10880b5730 con 0x7f1070004810 2024-09-17T03:32:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:52.848+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f10881594d0 con 0x7f1070004810 2024-09-17T03:32:53.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.464+0000 7f1085600640 1 -- 172.21.15.46:0/2787330240 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f107c063950 con 0x7f1070004810 2024-09-17T03:32:53.465 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:32:53.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1068074940 msgr2=0x7f1068076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:53.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1068074940 0x7f1068076e50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f10740b0000 tx=0x7f1074019be0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 msgr2=0x7f10881e8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f10881e8550 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f107c011650 tx=0x7f107c011b10 comp rx=0 tx=0).stop 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 shutdown_connections 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1068074940 0x7f1068076e50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10881e5e10 0x7f10880aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10881e5a00 0x7f10880ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 --2- 172.21.15.46:0/2787330240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070004810 0x7f10881e8550 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 >> 172.21.15.46:0/2787330240 conn(0x7f10880b2040 msgr2=0x7f10880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:53.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 shutdown_connections 2024-09-17T03:32:53.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.472+0000 7f108fe00640 1 -- 172.21.15.46:0/2787330240 wait complete. 2024-09-17T03:32:53.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:32:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 -- 172.21.15.46:0/3201285814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 msgr2=0x7f4698004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 --2- 172.21.15.46:0/3201285814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f4698004c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f469c001860 tx=0x7f469c01c850 comp rx=0 tx=0).stop 2024-09-17T03:32:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 -- 172.21.15.46:0/3201285814 shutdown_connections 2024-09-17T03:32:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 --2- 172.21.15.46:0/3201285814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f4698004c40 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 -- 172.21.15.46:0/3201285814 >> 172.21.15.46:0/3201285814 conn(0x7f46b40b2040 msgr2=0x7f46b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 -- 172.21.15.46:0/3201285814 shutdown_connections 2024-09-17T03:32:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.672+0000 7f46bac00640 1 -- 172.21.15.46:0/3201285814 wait complete. 2024-09-17T03:32:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 Processor -- start 2024-09-17T03:32:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 -- start start 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f46b40aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40af4d0 0x7f46b414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b4154e80 0x7f46b4155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40b5690 con 0x7f4698004810 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40b5510 con 0x7f46b40af4d0 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f46b415f180 con 0x7f46b4154e80 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f46b40aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f46b40aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59624/0 (socket says 172.21.15.46:59624) 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 -- 172.21.15.46:0/1843819476 learned_addr learned my addr 172.21.15.46:0/1843819476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46ab400640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40af4d0 0x7f46b414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b8c00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b4154e80 0x7f46b4155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 -- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b4154e80 msgr2=0x7f46b4155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b4154e80 0x7f46b4155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 -- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40af4d0 msgr2=0x7f46b414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40af4d0 0x7f46b414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 -- 172.21.15.46:0/1843819476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41557d0 con 0x7f4698004810 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b3e00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f46b40aae50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011a40 tx=0x7f46a4011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a401e070 con 0x7f4698004810 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a4004460 con 0x7f4698004810 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b4156c40 con 0x7f4698004810 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400fc00 con 0x7f4698004810 2024-09-17T03:32:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.676+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b40aa600 con 0x7f4698004810 2024-09-17T03:32:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.680+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b415f6e0 con 0x7f4698004810 2024-09-17T03:32:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.680+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f46a4021070 con 0x7f4698004810 2024-09-17T03:32:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.680+0000 7f46b0c00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4690074940 0x7f4690076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.680+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5386+0+0 (secure 0 0 0) 0x7f46a4093830 con 0x7f4698004810 2024-09-17T03:32:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.680+0000 7f46ab400640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4690074940 0x7f4690076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.680+0000 7f46ab400640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4690074940 0x7f4690076e50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f469c001830 tx=0x7f469c019690 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:53.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.684+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f46b41557d0 con 0x7f4698004810 2024-09-17T03:32:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:53.864+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46b415eb00 con 0x7f4698004810 2024-09-17T03:32:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.468+0000 7f46b0c00640 1 -- 172.21.15.46:0/1843819476 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f46a405fc70 con 0x7f4698004810 2024-09-17T03:32:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4690074940 msgr2=0x7f4690076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4690074940 0x7f4690076e50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f469c001830 tx=0x7f469c019690 comp rx=0 tx=0).stop 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 msgr2=0x7f46b40aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f46b40aae50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011a40 tx=0x7f46a4011e40 comp rx=0 tx=0).stop 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 shutdown_connections 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4690074940 0x7f4690076e50 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f469c001830 tx=0x7f469c019690 comp rx=0 tx=0).stop 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f46b4154e80 0x7f46b4155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b40af4d0 0x7f46b414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 --2- 172.21.15.46:0/1843819476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4698004810 0x7f46b40aae50 secure :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011a40 tx=0x7f46a4011e40 comp rx=0 tx=0).stop 2024-09-17T03:32:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.476+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 >> 172.21.15.46:0/1843819476 conn(0x7f46b40b2040 msgr2=0x7f46b40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:54.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.480+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 shutdown_connections 2024-09-17T03:32:54.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.480+0000 7f46bac00640 1 -- 172.21.15.46:0/1843819476 wait complete. 2024-09-17T03:32:54.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:32:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15f4e00640 1 -- 172.21.15.46:0/3222992037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 msgr2=0x7f15d4005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15f4e00640 1 --2- 172.21.15.46:0/3222992037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15d4005250 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c560 tx=0x7f15e801d260 comp rx=0 tx=0).stop 2024-09-17T03:32:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15ee800640 1 -- 172.21.15.46:0/3222992037 reap_dead start 2024-09-17T03:32:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15f4e00640 1 -- 172.21.15.46:0/3222992037 shutdown_connections 2024-09-17T03:32:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15f4e00640 1 -- 172.21.15.46:0/3222992037 >> 172.21.15.46:0/3222992037 conn(0x7f15f00b2040 msgr2=0x7f15f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15f4e00640 1 -- 172.21.15.46:0/3222992037 shutdown_connections 2024-09-17T03:32:54.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.684+0000 7f15f4e00640 1 -- 172.21.15.46:0/3222992037 wait complete. 2024-09-17T03:32:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 Processor -- start 2024-09-17T03:32:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 -- start start 2024-09-17T03:32:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15f01e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15f01e2410 0x7f15f01e2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15f0155060 0x7f15f0155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15f015d040 con 0x7f15d4004da0 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15f015cec0 con 0x7f15f01e2410 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15f4e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f15f015d1c0 con 0x7f15f0155060 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15f01e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15f01e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59632/0 (socket says 172.21.15.46:59632) 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.688+0000 7f15ede00640 1 -- 172.21.15.46:0/1131702760 learned_addr learned my addr 172.21.15.46:0/1131702760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ed400640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15f01e2410 0x7f15f01e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ee800640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15f0155060 0x7f15f0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ede00640 1 -- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15f0155060 msgr2=0x7f15f0155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ede00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15f0155060 0x7f15f0155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ede00640 1 -- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15f01e2410 msgr2=0x7f15f01e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ede00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15f01e2410 0x7f15f01e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ede00640 1 -- 172.21.15.46:0/1131702760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f01559b0 con 0x7f15d4004da0 2024-09-17T03:32:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ed400640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15f01e2410 0x7f15f01e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15ede00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15f01e6a90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f15e0005f50 tx=0x7f15e0011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005320 con 0x7f15d4004da0 2024-09-17T03:32:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e0018020 con 0x7f15d4004da0 2024-09-17T03:32:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0003860 con 0x7f15d4004da0 2024-09-17T03:32:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f01e3180 con 0x7f15d4004da0 2024-09-17T03:32:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f0144070 con 0x7f15d4004da0 2024-09-17T03:32:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f0074e30 con 0x7f15d4004da0 2024-09-17T03:32:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.692+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f15e0003a00 con 0x7f15d4004da0 2024-09-17T03:32:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.696+0000 7f15e6000640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15d0074940 0x7f15d0076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.696+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f15e0096710 con 0x7f15d4004da0 2024-09-17T03:32:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.696+0000 7f15ed400640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15d0074940 0x7f15d0076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.696+0000 7f15ed400640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15d0074940 0x7f15d0076e50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f15f00b4820 tx=0x7f15dc0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.700+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f15e0022020 con 0x7f15d4004da0 2024-09-17T03:32:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.872+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f15f0154720 con 0x7f15d4004da0 2024-09-17T03:32:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.876+0000 7f15e6000640 1 -- 172.21.15.46:0/1131702760 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7f15f01559b0 con 0x7f15d4004da0 2024-09-17T03:32:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15d0074940 msgr2=0x7f15d0076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15d0074940 0x7f15d0076e50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f15f00b4820 tx=0x7f15dc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 msgr2=0x7f15f01e6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15f01e6a90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f15e0005f50 tx=0x7f15e0011800 comp rx=0 tx=0).stop 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 shutdown_connections 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f15d0074940 0x7f15d0076e50 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f15f00b4820 tx=0x7f15dc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f15f0155060 0x7f15f0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15f01e2410 0x7f15f01e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 --2- 172.21.15.46:0/1131702760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4004da0 0x7f15f01e6a90 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 >> 172.21.15.46:0/1131702760 conn(0x7f15f00b2040 msgr2=0x7f15f00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 shutdown_connections 2024-09-17T03:32:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:54.880+0000 7f15f4e00640 1 -- 172.21.15.46:0/1131702760 wait complete. 2024-09-17T03:32:54.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-17T03:32:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 -- 172.21.15.46:0/3067179906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b5c70 msgr2=0x7f75ec15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 --2- 172.21.15.46:0/3067179906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b5c70 0x7f75ec15bb60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009840 tx=0x7f75dc01c7f0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 -- 172.21.15.46:0/3067179906 shutdown_connections 2024-09-17T03:32:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 --2- 172.21.15.46:0/3067179906 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75ec0b4de0 0x7f75ec0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 --2- 172.21.15.46:0/3067179906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b5c70 0x7f75ec15bb60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 -- 172.21.15.46:0/3067179906 >> 172.21.15.46:0/3067179906 conn(0x7f75ec0b2040 msgr2=0x7f75ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 -- 172.21.15.46:0/3067179906 shutdown_connections 2024-09-17T03:32:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 -- 172.21.15.46:0/3067179906 wait complete. 2024-09-17T03:32:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 Processor -- start 2024-09-17T03:32:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.060+0000 7f75f1600640 1 -- start start 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b4de0 0x7f75ec140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 0x7f75ec0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75ec1527b0 0x7f75ec0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec168980 con 0x7f75ec0b5c70 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec168800 con 0x7f75ec0b4de0 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec168b00 con 0x7f75ec1527b0 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 0x7f75ec0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 0x7f75ec0ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59646/0 (socket says 172.21.15.46:59646) 2024-09-17T03:32:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b4de0 0x7f75ec140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e2000640 1 -- 172.21.15.46:0/215183026 learned_addr learned my addr 172.21.15.46:0/215183026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eb400640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75ec1527b0 0x7f75ec0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 -- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75ec1527b0 msgr2=0x7f75ec0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75ec1527b0 0x7f75ec0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 -- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 msgr2=0x7f75ec0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 0x7f75ec0ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 -- 172.21.15.46:0/215183026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec1412c0 con 0x7f75ec0b4de0 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e2000640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 0x7f75ec0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75eaa00640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b4de0 0x7f75ec140d80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0ac000 tx=0x7f75dc0a56b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0bb070 con 0x7f75ec0b4de0 2024-09-17T03:32:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75dc0043b0 con 0x7f75ec0b4de0 2024-09-17T03:32:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b2040 con 0x7f75ec0b4de0 2024-09-17T03:32:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec153b40 con 0x7f75ec0b4de0 2024-09-17T03:32:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec154100 con 0x7f75ec0b4de0 2024-09-17T03:32:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.064+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec0b51f0 con 0x7f75ec0b4de0 2024-09-17T03:32:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.068+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f75dc003860 con 0x7f75ec0b4de0 2024-09-17T03:32:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.068+0000 7f75e3400640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f75c8074910 0x7f75c8076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.068+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f75dc12cd20 con 0x7f75ec0b4de0 2024-09-17T03:32:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.068+0000 7f75e2000640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f75c8074910 0x7f75c8076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.068+0000 7f75e2000640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f75c8074910 0x7f75c8076e20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f75d4000f70 tx=0x7f75d4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.072+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f75dc0c4020 con 0x7f75ec0b4de0 2024-09-17T03:32:55.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.340+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f75ec0b2790 con 0x7f75ec0b4de0 2024-09-17T03:32:55.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75e3400640 1 -- 172.21.15.46:0/215183026 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v552) ==== 174+0+0 (secure 0 0 0) 0x7f75dc0fee20 con 0x7f75ec0b4de0 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f75c8074910 msgr2=0x7f75c8076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f75c8074910 0x7f75c8076e20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f75d4000f70 tx=0x7f75d4009040 comp rx=0 tx=0).stop 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b4de0 msgr2=0x7f75ec140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b4de0 0x7f75ec140d80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0ac000 tx=0x7f75dc0a56b0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 shutdown_connections 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f75c8074910 0x7f75c8076e20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75ec1527b0 0x7f75ec0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec0b5c70 0x7f75ec0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 --2- 172.21.15.46:0/215183026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75ec0b4de0 0x7f75ec140d80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 >> 172.21.15.46:0/215183026 conn(0x7f75ec0b2040 msgr2=0x7f75ec15a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 shutdown_connections 2024-09-17T03:32:55.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.476+0000 7f75f1600640 1 -- 172.21.15.46:0/215183026 wait complete. 2024-09-17T03:32:55.499 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-17T03:32:55.500 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-17T03:32:55.500 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:32:55.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 -- 172.21.15.46:0/855077768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb01527b0 msgr2=0x7f1cb015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/855077768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb01527b0 0x7f1cb015bb60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009900 tx=0x7f1c9801c6d0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 -- 172.21.15.46:0/855077768 shutdown_connections 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/855077768 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 0x7f1cb00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/855077768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb01527b0 0x7f1cb015bb60 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 -- 172.21.15.46:0/855077768 >> 172.21.15.46:0/855077768 conn(0x7f1cb00b2040 msgr2=0x7f1cb00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 -- 172.21.15.46:0/855077768 shutdown_connections 2024-09-17T03:32:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 -- 172.21.15.46:0/855077768 wait complete. 2024-09-17T03:32:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.660+0000 7f1cb4e00640 1 Processor -- start 2024-09-17T03:32:55.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 -- start start 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 0x7f1cb0159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 0x7f1cb01499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb01527b0 0x7f1cb01531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0168a20 con 0x7f1cb00b5c70 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb01688a0 con 0x7f1cb01527b0 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb0168ba0 con 0x7f1cb00b4de0 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 0x7f1cb01499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 0x7f1cb01499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59660/0 (socket says 172.21.15.46:59660) 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 -- 172.21.15.46:0/3989633660 learned_addr learned my addr 172.21.15.46:0/3989633660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cae800640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb01527b0 0x7f1cb01531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cade00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 0x7f1cb0159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 -- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 msgr2=0x7f1cb0159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 0x7f1cb0159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 -- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb01527b0 msgr2=0x7f1cb01531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb01527b0 0x7f1cb01531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 -- 172.21.15.46:0/3989633660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb01fc4d0 con 0x7f1cb00b5c70 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cae800640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb01527b0 0x7f1cb01531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cade00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 0x7f1cb0159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca7e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 0x7f1cb01499d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009850 tx=0x7f1c980a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980ba020 con 0x7f1cb00b5c70 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c98004430 con 0x7f1cb00b5c70 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980b2040 con 0x7f1cb00b5c70 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01fc7b0 con 0x7f1cb00b5c70 2024-09-17T03:32:55.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.664+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01fcd70 con 0x7f1cb00b5c70 2024-09-17T03:32:55.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.668+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb00b51f0 con 0x7f1cb00b5c70 2024-09-17T03:32:55.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.668+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1c980bd070 con 0x7f1cb00b5c70 2024-09-17T03:32:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.668+0000 7f1ca6000640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074940 0x7f1c8c076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.668+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f1c9812c7e0 con 0x7f1cb00b5c70 2024-09-17T03:32:55.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.668+0000 7f1cade00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074940 0x7f1c8c076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:55.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.668+0000 7f1cade00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074940 0x7f1c8c076e50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00098b0 tx=0x7f1ca0008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:55.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.672+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1cb00b5730 con 0x7f1cb00b5c70 2024-09-17T03:32:55.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cb01594d0 con 0x7f1cb00b5c70 2024-09-17T03:32:55.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1ca6000640 1 -- 172.21.15.46:0/3989633660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+12997 (secure 0 0 0) 0x7f1c980f9320 con 0x7f1cb00b5c70 2024-09-17T03:32:55.848 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:55.848 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:32:54.467687+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2776262749":"2024-09-18T03:32:51.457091+0000","172.21.15.46:6834/2776262749":"2024-09-18T03:32:51.457091+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:32:55.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074940 msgr2=0x7f1c8c076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074940 0x7f1c8c076e50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00098b0 tx=0x7f1ca0008040 comp rx=0 tx=0).stop 2024-09-17T03:32:55.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 msgr2=0x7f1cb01499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:55.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 0x7f1cb01499d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009850 tx=0x7f1c980a5620 comp rx=0 tx=0).stop 2024-09-17T03:32:55.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 shutdown_connections 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074940 0x7f1c8c076e50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb01527b0 0x7f1cb01531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00b5c70 0x7f1cb01499d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 --2- 172.21.15.46:0/3989633660 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb00b4de0 0x7f1cb0159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 >> 172.21.15.46:0/3989633660 conn(0x7f1cb00b2040 msgr2=0x7f1cb00b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 shutdown_connections 2024-09-17T03:32:55.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:55.848+0000 7f1cb4e00640 1 -- 172.21.15.46:0/3989633660 wait complete. 2024-09-17T03:32:55.871 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:32:55.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:32:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 -- 172.21.15.46:0/2710566348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68160d50 msgr2=0x7f5f68074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 --2- 172.21.15.46:0/2710566348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68160d50 0x7f5f68074410 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f5f680713e0 tx=0x7f5f5401c830 comp rx=0 tx=0).stop 2024-09-17T03:32:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 -- 172.21.15.46:0/2710566348 shutdown_connections 2024-09-17T03:32:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 --2- 172.21.15.46:0/2710566348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68160d50 0x7f5f68074410 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 --2- 172.21.15.46:0/2710566348 >> v2:172.21.15.99:3300/0 conn(0x7f5f68157030 0x7f5f680b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 -- 172.21.15.46:0/2710566348 >> 172.21.15.46:0/2710566348 conn(0x7f5f680708c0 msgr2=0x7f5f68070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 -- 172.21.15.46:0/2710566348 shutdown_connections 2024-09-17T03:32:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.032+0000 7f5f70000640 1 -- 172.21.15.46:0/2710566348 wait complete. 2024-09-17T03:32:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 Processor -- start 2024-09-17T03:32:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 -- start start 2024-09-17T03:32:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f68157030 0x7f5f6806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f68160d50 0x7f5f68141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68161160 0x7f5f68150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6816a740 con 0x7f5f68161160 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6816a5c0 con 0x7f5f68160d50 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f70000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6816a8c0 con 0x7f5f68157030 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f6e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68161160 0x7f5f68150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f6d600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f68157030 0x7f5f6806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f6d600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f68157030 0x7f5f6806cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:60026/0 (socket says 172.21.15.46:60026) 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f6cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f68160d50 0x7f5f68141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.036+0000 7f5f6d600640 1 -- 172.21.15.46:0/906919504 learned_addr learned my addr 172.21.15.46:0/906919504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f6e000640 1 -- 172.21.15.46:0/906919504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f68157030 msgr2=0x7f5f6806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f6e000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f68157030 0x7f5f6806cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f6e000640 1 -- 172.21.15.46:0/906919504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f68160d50 msgr2=0x7f5f68141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f6e000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f68160d50 0x7f5f68141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f6e000640 1 -- 172.21.15.46:0/906919504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681fd630 con 0x7f5f68161160 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f6e000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68161160 0x7f5f68150640 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c760 tx=0x7f5f6000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60003b80 con 0x7f5f68161160 2024-09-17T03:32:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f60003d20 con 0x7f5f68161160 2024-09-17T03:32:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681fd910 con 0x7f5f68161160 2024-09-17T03:32:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60005330 con 0x7f5f68161160 2024-09-17T03:32:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f68150120 con 0x7f5f68161160 2024-09-17T03:32:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f681fd3a0 con 0x7f5f68161160 2024-09-17T03:32:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.040+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f5f600054d0 con 0x7f5f68161160 2024-09-17T03:32:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.044+0000 7f5f65600640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f48074940 0x7f5f48076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.044+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f5f60093560 con 0x7f5f68161160 2024-09-17T03:32:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.044+0000 7f5f6d600640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f48074940 0x7f5f48076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.044+0000 7f5f6d600640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f48074940 0x7f5f48076e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c005fd0 tx=0x7f5f5c00b390 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:56.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.044+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5f6005faa0 con 0x7f5f68161160 2024-09-17T03:32:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f680d2260 con 0x7f5f68161160 2024-09-17T03:32:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f65600640 1 -- 172.21.15.46:0/906919504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f5f600644d0 con 0x7f5f68161160 2024-09-17T03:32:56.220 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f48074940 msgr2=0x7f5f48076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f48074940 0x7f5f48076e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c005fd0 tx=0x7f5f5c00b390 comp rx=0 tx=0).stop 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68161160 msgr2=0x7f5f68150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68161160 0x7f5f68150640 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c760 tx=0x7f5f6000cc20 comp rx=0 tx=0).stop 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 shutdown_connections 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5f48074940 0x7f5f48076e50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68161160 0x7f5f68150640 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f68160d50 0x7f5f68141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 --2- 172.21.15.46:0/906919504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5f68157030 0x7f5f6806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 >> 172.21.15.46:0/906919504 conn(0x7f5f680708c0 msgr2=0x7f5f681523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 shutdown_connections 2024-09-17T03:32:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.220+0000 7f5f70000640 1 -- 172.21.15.46:0/906919504 wait complete. 2024-09-17T03:32:56.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-17T03:32:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 -- 172.21.15.46:0/3255467597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c0b54f0 msgr2=0x7ff62c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 --2- 172.21.15.46:0/3255467597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c0b54f0 0x7ff62c15bb60 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff624009bc0 tx=0x7ff62401cae0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 -- 172.21.15.46:0/3255467597 shutdown_connections 2024-09-17T03:32:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 --2- 172.21.15.46:0/3255467597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c0b54f0 0x7ff62c15bb60 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 -- 172.21.15.46:0/3255467597 >> 172.21.15.46:0/3255467597 conn(0x7ff62c0b2040 msgr2=0x7ff62c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 -- 172.21.15.46:0/3255467597 shutdown_connections 2024-09-17T03:32:56.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.400+0000 7ff631200640 1 -- 172.21.15.46:0/3255467597 wait complete. 2024-09-17T03:32:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 Processor -- start 2024-09-17T03:32:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 -- start start 2024-09-17T03:32:56.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff62c0b5c70 0x7ff62c0ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 0x7ff62c1f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62c13d690 0x7ff62c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c169b30 con 0x7ff62c1527b0 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1699b0 con 0x7ff62c13d690 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff62c169cb0 con 0x7ff62c0b5c70 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 0x7ff62c1f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 0x7ff62c1f9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59686/0 (socket says 172.21.15.46:59686) 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 -- 172.21.15.46:0/558044297 learned_addr learned my addr 172.21.15.46:0/558044297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff629e00640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff62c0b5c70 0x7ff62c0ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff62a800640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62c13d690 0x7ff62c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 -- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff62c0b5c70 msgr2=0x7ff62c0ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff62c0b5c70 0x7ff62c0ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 -- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62c13d690 msgr2=0x7ff62c0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62c13d690 0x7ff62c0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 -- 172.21.15.46:0/558044297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c1fa4a0 con 0x7ff62c1527b0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff623e00640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 0x7ff62c1f9f60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c3c0 tx=0x7ff61400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614017070 con 0x7ff62c1527b0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6140053c0 con 0x7ff62c1527b0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614003860 con 0x7ff62c1527b0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c13e130 con 0x7ff62c1527b0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.404+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1f9760 con 0x7ff62c1527b0 2024-09-17T03:32:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.408+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c1f9550 con 0x7ff62c1527b0 2024-09-17T03:32:56.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.408+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7ff61401a070 con 0x7ff62c1527b0 2024-09-17T03:32:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.408+0000 7ff622000640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff608074910 0x7ff608076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:56.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.408+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7ff6140921d0 con 0x7ff62c1527b0 2024-09-17T03:32:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.408+0000 7ff629e00640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff608074910 0x7ff608076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.408+0000 7ff629e00640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff608074910 0x7ff608076e20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff61c001870 tx=0x7ff61c001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:56.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.412+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff62c1fa4a0 con 0x7ff62c1527b0 2024-09-17T03:32:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:56.652+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7ff62c15bb60 con 0x7ff62c1527b0 2024-09-17T03:32:57.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.480+0000 7ff622000640 1 -- 172.21.15.46:0/558044297 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v554) ==== 181+0+0 (secure 0 0 0) 0x7ff61405e690 con 0x7ff62c1527b0 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff608074910 msgr2=0x7ff608076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff608074910 0x7ff608076e20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff61c001870 tx=0x7ff61c001380 comp rx=0 tx=0).stop 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 msgr2=0x7ff62c1f9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 0x7ff62c1f9f60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c3c0 tx=0x7ff61400c880 comp rx=0 tx=0).stop 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 shutdown_connections 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff608074910 0x7ff608076e20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff62c13d690 0x7ff62c0aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff62c1527b0 0x7ff62c1f9f60 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 --2- 172.21.15.46:0/558044297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff62c0b5c70 0x7ff62c0ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 >> 172.21.15.46:0/558044297 conn(0x7ff62c0b2040 msgr2=0x7ff62c15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 shutdown_connections 2024-09-17T03:32:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.484+0000 7ff631200640 1 -- 172.21.15.46:0/558044297 wait complete. 2024-09-17T03:32:57.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:32:57.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 -- 172.21.15.46:0/3953475546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140cefa0 msgr2=0x7fe2140cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 --2- 172.21.15.46:0/3953475546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140cefa0 0x7fe2140cf3b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe200009840 tx=0x7fe20001c820 comp rx=0 tx=0).stop 2024-09-17T03:32:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 -- 172.21.15.46:0/3953475546 shutdown_connections 2024-09-17T03:32:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 --2- 172.21.15.46:0/3953475546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140cefa0 0x7fe2140cf3b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 -- 172.21.15.46:0/3953475546 >> 172.21.15.46:0/3953475546 conn(0x7fe2140b2040 msgr2=0x7fe2140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 -- 172.21.15.46:0/3953475546 shutdown_connections 2024-09-17T03:32:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.672+0000 7fe21a800640 1 -- 172.21.15.46:0/3953475546 wait complete. 2024-09-17T03:32:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 Processor -- start 2024-09-17T03:32:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 -- start start 2024-09-17T03:32:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2140749a0 0x7fe214159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 0x7fe214160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2140cefa0 0x7fe214160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe21416b350 con 0x7fe2140b3340 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe21416b1d0 con 0x7fe2140cefa0 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe21a800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe21416b4d0 con 0x7fe2140749a0 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 0x7fe214160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe213400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2140749a0 0x7fe214159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 0x7fe214160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59706/0 (socket says 172.21.15.46:59706) 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 -- 172.21.15.46:0/3036233425 learned_addr learned my addr 172.21.15.46:0/3036233425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe213e00640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2140cefa0 0x7fe214160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 -- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2140749a0 msgr2=0x7fe214159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2140749a0 0x7fe214159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 -- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2140cefa0 msgr2=0x7fe214160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2140cefa0 0x7fe214160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.676+0000 7fe212a00640 1 -- 172.21.15.46:0/3036233425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21415e650 con 0x7fe2140b3340 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe212a00640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 0x7fe214160250 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe200009790 tx=0x7fe200004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2000b2040 con 0x7fe2140b3340 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe200005030 con 0x7fe2140b3340 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200005440 con 0x7fe2140b3340 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21415e930 con 0x7fe2140b3340 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21415eef0 con 0x7fe2140b3340 2024-09-17T03:32:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2140cf3b0 con 0x7fe2140b3340 2024-09-17T03:32:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fe2000bd060 con 0x7fe2140b3340 2024-09-17T03:32:57.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.680+0000 7fe20b400640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1f4074910 0x7fe1f4076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:57.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.684+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7fe20012c440 con 0x7fe2140b3340 2024-09-17T03:32:57.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.684+0000 7fe213400640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1f4074910 0x7fe1f4076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:57.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.684+0000 7fe213400640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1f4074910 0x7fe1f4076e20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe204007fd0 tx=0x7fe204007ea0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.688+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe200016410 con 0x7fe2140b3340 2024-09-17T03:32:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.860+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe2140aea10 con 0x7fe2140b3340 2024-09-17T03:32:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.860+0000 7fe20b400640 1 -- 172.21.15.46:0/3036233425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v48) ==== 59+0+2294 (secure 0 0 0) 0x7fe2000f8870 con 0x7fe2140b3340 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:32:57.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:32:57.863 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:32:57.864 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:32:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1f4074910 msgr2=0x7fe1f4076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1f4074910 0x7fe1f4076e20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe204007fd0 tx=0x7fe204007ea0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 msgr2=0x7fe214160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 0x7fe214160250 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe200009790 tx=0x7fe200004040 comp rx=0 tx=0).stop 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 shutdown_connections 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1f4074910 0x7fe1f4076e20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2140cefa0 0x7fe214160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2140b3340 0x7fe214160250 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 --2- 172.21.15.46:0/3036233425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe2140749a0 0x7fe214159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 >> 172.21.15.46:0/3036233425 conn(0x7fe2140b2040 msgr2=0x7fe2140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 shutdown_connections 2024-09-17T03:32:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:57.864+0000 7fe21a800640 1 -- 172.21.15.46:0/3036233425 wait complete. 2024-09-17T03:32:57.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:32:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 -- 172.21.15.46:0/2711372870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0b3340 msgr2=0x7f139c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 --2- 172.21.15.46:0/2711372870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0b3340 0x7f139c0b3750 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1394009bc0 tx=0x7f139401cba0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 -- 172.21.15.46:0/2711372870 shutdown_connections 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 --2- 172.21.15.46:0/2711372870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0b3340 0x7f139c0b3750 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 --2- 172.21.15.46:0/2711372870 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f139c0cefa0 0x7f139c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 -- 172.21.15.46:0/2711372870 >> 172.21.15.46:0/2711372870 conn(0x7f139c0b2040 msgr2=0x7f139c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 -- 172.21.15.46:0/2711372870 shutdown_connections 2024-09-17T03:32:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.044+0000 7f13a3c00640 1 -- 172.21.15.46:0/2711372870 wait complete. 2024-09-17T03:32:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 Processor -- start 2024-09-17T03:32:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 -- start start 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0749a0 0x7f139c138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 0x7f139c1fc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f139c0cefa0 0x7f139c160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f139c169dc0 con 0x7f139c0b3340 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f139c169c40 con 0x7f139c0749a0 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a3c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f139c169f40 con 0x7f139c0cefa0 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f139be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 0x7f139c1fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f139be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 0x7f139c1fc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59716/0 (socket says 172.21.15.46:59716) 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f139be00640 1 -- 172.21.15.46:0/1584810059 learned_addr learned my addr 172.21.15.46:0/1584810059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a1c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f139c0cefa0 0x7f139c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f13a1200640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0749a0 0x7f139c138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f139be00640 1 -- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f139c0cefa0 msgr2=0x7f139c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f139be00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f139c0cefa0 0x7f139c160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.048+0000 7f139be00640 1 -- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0749a0 msgr2=0x7f139c138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f139be00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0749a0 0x7f139c138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f139be00640 1 -- 172.21.15.46:0/1584810059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c1fcdf0 con 0x7f139c0b3340 2024-09-17T03:32:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f139be00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 0x7f139c1fc090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f138800e7c0 tx=0x7f138800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138800cc60 con 0x7f139c0b3340 2024-09-17T03:32:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f138800ce20 con 0x7f139c0b3340 2024-09-17T03:32:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139c1f8a20 con 0x7f139c0b3340 2024-09-17T03:32:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388005280 con 0x7f139c0b3340 2024-09-17T03:32:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139c1f8fe0 con 0x7f139c0b3340 2024-09-17T03:32:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139c0b3750 con 0x7f139c0b3340 2024-09-17T03:32:58.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1388005420 con 0x7f139c0b3340 2024-09-17T03:32:58.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.052+0000 7f1398c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f137c074910 0x7f137c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.056+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f1388093720 con 0x7f139c0b3340 2024-09-17T03:32:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.056+0000 7f13a1200640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f137c074910 0x7f137c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.056+0000 7f13a1200640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f137c074910 0x7f137c076e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1390001870 tx=0x7f1390001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.060+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f138805fc80 con 0x7f139c0b3340 2024-09-17T03:32:58.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.236+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f139c074db0 con 0x7f139c0b3340 2024-09-17T03:32:58.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.240+0000 7f1398c00640 1 -- 172.21.15.46:0/1584810059 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f13880646b0 con 0x7f139c0b3340 2024-09-17T03:32:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.240+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f137c074910 msgr2=0x7f137c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.240+0000 7f13a3c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f137c074910 0x7f137c076e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1390001870 tx=0x7f1390001380 comp rx=0 tx=0).stop 2024-09-17T03:32:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 msgr2=0x7f139c1fc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 0x7f139c1fc090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f138800e7c0 tx=0x7f138800ec80 comp rx=0 tx=0).stop 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 shutdown_connections 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f137c074910 0x7f137c076e20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f139c0cefa0 0x7f139c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139c0b3340 0x7f139c1fc090 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 --2- 172.21.15.46:0/1584810059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f139c0749a0 0x7f139c138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 >> 172.21.15.46:0/1584810059 conn(0x7f139c0b2040 msgr2=0x7f139c13ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 shutdown_connections 2024-09-17T03:32:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.244+0000 7f13a3c00640 1 -- 172.21.15.46:0/1584810059 wait complete. 2024-09-17T03:32:58.263 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:32:58.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:32:58.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:32:58.496 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:32:58.497 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:32:58.497 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:58.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:32:58.506 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:32:58.506 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:58.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:32:58.516 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:32:58.516 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:58.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:32:58.559 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:32:58.559 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:32:58.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:32:58.569 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:32:58.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:32:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 -- 172.21.15.46:0/1441638464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778004810 msgr2=0x7f3778004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 --2- 172.21.15.46:0/1441638464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3778004810 0x7f3778004c20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f3790071fd0 tx=0x7f377c01d1e0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3796600640 1 -- 172.21.15.46:0/1441638464 reap_dead start 2024-09-17T03:32:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 -- 172.21.15.46:0/1441638464 shutdown_connections 2024-09-17T03:32:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 -- 172.21.15.46:0/1441638464 >> 172.21.15.46:0/1441638464 conn(0x7f37900b2040 msgr2=0x7f37900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 -- 172.21.15.46:0/1441638464 shutdown_connections 2024-09-17T03:32:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 -- 172.21.15.46:0/1441638464 wait complete. 2024-09-17T03:32:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 Processor -- start 2024-09-17T03:32:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.772+0000 7f3798600640 1 -- start start 2024-09-17T03:32:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3778004810 0x7f379015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 0x7f37900b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37900ae960 0x7f37900aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3790166900 con 0x7f37900b3ea0 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3790166780 con 0x7f37900ae960 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3790166a80 con 0x7f3778004810 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 0x7f37900b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 0x7f37900b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59736/0 (socket says 172.21.15.46:59736) 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 -- 172.21.15.46:0/2607262599 learned_addr learned my addr 172.21.15.46:0/2607262599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3796600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37900ae960 0x7f37900aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795c00640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3778004810 0x7f379015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 -- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3778004810 msgr2=0x7f379015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3778004810 0x7f379015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 -- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37900ae960 msgr2=0x7f37900aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37900ae960 0x7f37900aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 -- 172.21.15.46:0/2607262599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37900af2b0 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3796600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37900ae960 0x7f37900aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3795200640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 0x7f37900b42b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f377c0ae040 tx=0x7f377c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c0bb030 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f377c004880 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c004c90 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379015e470 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37901f7d20 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.776+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379015e1e0 con 0x7f37900b3ea0 2024-09-17T03:32:58.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.780+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f377c0be030 con 0x7f37900b3ea0 2024-09-17T03:32:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.780+0000 7f378d600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3770074940 0x7f3770076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.780+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f377c12e510 con 0x7f37900b3ea0 2024-09-17T03:32:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.780+0000 7f3795c00640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3770074940 0x7f3770076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:58.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.780+0000 7f3795c00640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3770074940 0x7f3770076e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3784000fc0 tx=0x7f3784001120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:58.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.784+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f377c016f20 con 0x7f37900b3ea0 2024-09-17T03:32:58.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37900b51f0 con 0x7f37900b3ea0 2024-09-17T03:32:58.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f378d600640 1 -- 172.21.15.46:0/2607262599 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+3919 (secure 0 0 0) 0x7f377c0fb020 con 0x7f37900b3ea0 2024-09-17T03:32:58.992 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:58.992 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":58,"btime":"2024-09-17T03:32:52:459796+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246119,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2500131889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":2500131889},{"type":"v1","addr":"172.21.15.99:6835","nonce":2500131889}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":246122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3417429186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3417429186},{"type":"v1","addr":"172.21.15.99:6834","nonce":3417429186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":246661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2645704859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2645704859},{"type":"v1","addr":"172.21.15.46:6837","nonce":2645704859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":247599,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3357865815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3357865815},{"type":"v1","addr":"172.21.15.46:6835","nonce":3357865815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[]} 2024-09-17T03:32:58.993 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 58 2024-09-17T03:32:58.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3770074940 msgr2=0x7f3770076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3770074940 0x7f3770076e50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3784000fc0 tx=0x7f3784001120 comp rx=0 tx=0).stop 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 msgr2=0x7f37900b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 0x7f37900b42b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f377c0ae040 tx=0x7f377c005020 comp rx=0 tx=0).stop 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 shutdown_connections 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3770074940 0x7f3770076e50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37900ae960 0x7f37900aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37900b3ea0 0x7f37900b42b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 --2- 172.21.15.46:0/2607262599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3778004810 0x7f379015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 >> 172.21.15.46:0/2607262599 conn(0x7f37900b2040 msgr2=0x7f37900b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 shutdown_connections 2024-09-17T03:32:58.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:58.992+0000 7f3798600640 1 -- 172.21.15.46:0/2607262599 wait complete. 2024-09-17T03:32:59.015 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:32:59.015 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:32:59.016 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:32:59.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:32:59.016 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:32:59.016 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:32:59.017 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:32:59.017 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:32:59.017 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:32:59.017 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:32:59.017 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:32:59.017 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:32:59.017 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:32:59.017 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:32:59.018 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:32:59.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:32:59.029 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:32:59.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:32:59.032 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:32:59.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:32:59.033 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:32:59.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:32:59.038 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:32:59.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:32:59.080 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:32:59.081 DEBUG:teuthology.parallel:result is None 2024-09-17T03:32:59.082 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:32:59.082 DEBUG:teuthology.parallel:result is None 2024-09-17T03:32:59.085 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:32:59.085 DEBUG:teuthology.parallel:result is None 2024-09-17T03:32:59.087 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:32:59.087 DEBUG:teuthology.parallel:result is None 2024-09-17T03:32:59.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:32:59.185 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:32:59.185 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:32:59.187 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:32:59.187 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:32:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 -- 172.21.15.46:0/2641656717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0074b20 msgr2=0x7f1cb015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 --2- 172.21.15.46:0/2641656717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0074b20 0x7f1cb015cf90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009bc0 tx=0x7f1ca801cba0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 -- 172.21.15.46:0/2641656717 shutdown_connections 2024-09-17T03:32:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 --2- 172.21.15.46:0/2641656717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0074b20 0x7f1cb015cf90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 --2- 172.21.15.46:0/2641656717 >> v2:172.21.15.99:3300/0 conn(0x7f1cb0157030 0x7f1cb00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 -- 172.21.15.46:0/2641656717 >> 172.21.15.46:0/2641656717 conn(0x7f1cb00708c0 msgr2=0x7f1cb0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 -- 172.21.15.46:0/2641656717 shutdown_connections 2024-09-17T03:32:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 -- 172.21.15.46:0/2641656717 wait complete. 2024-09-17T03:32:59.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 Processor -- start 2024-09-17T03:32:59.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.256+0000 7f1cb5200640 1 -- start start 2024-09-17T03:32:59.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 0x7f1cb015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0160d50 0x7f1cb013dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb015e950 0x7f1cb015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb016a740 con 0x7f1cb0157030 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb016a5c0 con 0x7f1cb015e950 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb016a8c0 con 0x7f1cb0160d50 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 0x7f1cb015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0160d50 0x7f1cb013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 0x7f1cb015c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59766/0 (socket says 172.21.15.46:59766) 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb015e950 0x7f1cb015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cade00640 1 -- 172.21.15.46:0/3244080588 learned_addr learned my addr 172.21.15.46:0/3244080588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 -- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0160d50 msgr2=0x7f1cb013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0160d50 0x7f1cb013dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 -- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 msgr2=0x7f1cb015c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 0x7f1cb015c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 -- 172.21.15.46:0/3244080588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb015f2a0 con 0x7f1cb015e950 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cade00640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 0x7f1cb015c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1ca7e00640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0160d50 0x7f1cb013dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cae800640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb015e950 0x7f1cb015ed60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80ac000 tx=0x7f1ca8005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80b2040 con 0x7f1cb015e950 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ca8004130 con 0x7f1cb015e950 2024-09-17T03:32:59.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8004540 con 0x7f1cb015e950 2024-09-17T03:32:59.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01516d0 con 0x7f1cb015e950 2024-09-17T03:32:59.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb0153600 con 0x7f1cb015e950 2024-09-17T03:32:59.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.260+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb015cf90 con 0x7f1cb015e950 2024-09-17T03:32:59.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.264+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1ca80bd060 con 0x7f1cb015e950 2024-09-17T03:32:59.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.264+0000 7f1ca6000640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074910 0x7f1c8c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.264+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f1ca812c720 con 0x7f1cb015e950 2024-09-17T03:32:59.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.264+0000 7f1cade00640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074910 0x7f1c8c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.264+0000 7f1cade00640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074910 0x7f1c8c076e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009680 tx=0x7f1ca00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.268+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1cb015f2a0 con 0x7f1cb015e950 2024-09-17T03:32:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.452+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1cb00d2260 con 0x7f1cb015e950 2024-09-17T03:32:59.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1ca6000640 1 -- 172.21.15.46:0/3244080588 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) ==== 97+0+0 (secure 0 0 0) 0x7f1ca8016790 con 0x7f1cb015e950 2024-09-17T03:32:59.504 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:32:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074910 msgr2=0x7f1c8c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074910 0x7f1c8c076e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009680 tx=0x7f1ca00091e0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb015e950 msgr2=0x7f1cb015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb015e950 0x7f1cb015ed60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80ac000 tx=0x7f1ca8005c40 comp rx=0 tx=0).stop 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 shutdown_connections 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c8c074910 0x7f1c8c076e20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cb015e950 0x7f1cb015ed60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cb0160d50 0x7f1cb013dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 --2- 172.21.15.46:0/3244080588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0157030 0x7f1cb015c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 >> 172.21.15.46:0/3244080588 conn(0x7f1cb00708c0 msgr2=0x7f1cb014c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 shutdown_connections 2024-09-17T03:32:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.504+0000 7f1cb5200640 1 -- 172.21.15.46:0/3244080588 wait complete. 2024-09-17T03:32:59.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:32:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.684+0000 7f58f1800640 1 -- 172.21.15.46:0/982579341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 msgr2=0x7f58ec070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.684+0000 7f58f1800640 1 --2- 172.21.15.46:0/982579341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec070690 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009840 tx=0x7f58e401c7f0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- 172.21.15.46:0/982579341 shutdown_connections 2024-09-17T03:32:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 --2- 172.21.15.46:0/982579341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec070690 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- 172.21.15.46:0/982579341 >> 172.21.15.46:0/982579341 conn(0x7f58ec0b35f0 msgr2=0x7f58ec0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- 172.21.15.46:0/982579341 shutdown_connections 2024-09-17T03:32:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- 172.21.15.46:0/982579341 wait complete. 2024-09-17T03:32:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 Processor -- start 2024-09-17T03:32:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- start start 2024-09-17T03:32:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58ec15c7b0 0x7f58ec14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec1fb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58ec158d10 0x7f58ec1fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec16af20 con 0x7f58ec160af0 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec16ada0 con 0x7f58ec158d10 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.688+0000 7f58f1800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec16b0a0 con 0x7f58ec15c7b0 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec1fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec1fb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59786/0 (socket says 172.21.15.46:59786) 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 -- 172.21.15.46:0/2061614190 learned_addr learned my addr 172.21.15.46:0/2061614190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58eb400640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58ec158d10 0x7f58ec1fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58eaa00640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58ec15c7b0 0x7f58ec14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 -- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58ec15c7b0 msgr2=0x7f58ec14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58ec15c7b0 0x7f58ec14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 -- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58ec158d10 msgr2=0x7f58ec1fbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58ec158d10 0x7f58ec1fbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 -- 172.21.15.46:0/2061614190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec0affc0 con 0x7f58ec160af0 2024-09-17T03:32:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58eaa00640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58ec15c7b0 0x7f58ec14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:32:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e2000640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec1fb830 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f58d400cbb0 tx=0x7f58d400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4016020 con 0x7f58ec160af0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d400ab40 con 0x7f58ec160af0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d400ce30 con 0x7f58ec160af0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec1fc640 con 0x7f58ec160af0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec1515f0 con 0x7f58ec160af0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ec070690 con 0x7f58ec160af0 2024-09-17T03:32:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.692+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f58d4019020 con 0x7f58ec160af0 2024-09-17T03:32:59.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.696+0000 7f58e3400640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f58c8074910 0x7f58c8076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:32:59.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.696+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f58d4092e00 con 0x7f58ec160af0 2024-09-17T03:32:59.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.696+0000 7f58eaa00640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f58c8074910 0x7f58c8076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:32:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.696+0000 7f58eaa00640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f58c8074910 0x7f58c8076e20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f58dc000f70 tx=0x7f58dc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:32:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.700+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f58d4020030 con 0x7f58ec160af0 2024-09-17T03:32:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.928+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f58ec1fc310 con 0x7f58ec160af0 2024-09-17T03:32:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58e3400640 1 -- 172.21.15.46:0/2061614190 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) ==== 80+0+5304 (secure 0 0 0) 0x7f58d405f910 con 0x7f58ec160af0 2024-09-17T03:32:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:32:59.931 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:32:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:32:59.931 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:32:59.932 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:32:59.933 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.934 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:32:59.935 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.936 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.937 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.938 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.939 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:32:59.942 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:32:59.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:32:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:32:59.945 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:32:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f58c8074910 msgr2=0x7f58c8076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f58c8074910 0x7f58c8076e20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f58dc000f70 tx=0x7f58dc009040 comp rx=0 tx=0).stop 2024-09-17T03:32:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 msgr2=0x7f58ec1fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:32:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec1fb830 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f58d400cbb0 tx=0x7f58d400cbe0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 shutdown_connections 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f58c8074910 0x7f58c8076e20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58ec158d10 0x7f58ec1fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ec160af0 0x7f58ec1fb830 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 --2- 172.21.15.46:0/2061614190 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f58ec15c7b0 0x7f58ec14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 >> 172.21.15.46:0/2061614190 conn(0x7f58ec0b35f0 msgr2=0x7f58ec0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 shutdown_connections 2024-09-17T03:32:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:32:59.932+0000 7f58f1800640 1 -- 172.21.15.46:0/2061614190 wait complete. 2024-09-17T03:32:59.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e3e200640 1 -- 172.21.15.46:0/3818131517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e1c003a10 msgr2=0x7f1e1c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e3e200640 1 --2- 172.21.15.46:0/3818131517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e1c003a10 0x7f1e1c003e80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1e380719d0 tx=0x7f1e2801d2a0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e37e00640 1 -- 172.21.15.46:0/3818131517 reap_dead start 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e3e200640 1 -- 172.21.15.46:0/3818131517 shutdown_connections 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e3e200640 1 -- 172.21.15.46:0/3818131517 >> 172.21.15.46:0/3818131517 conn(0x7f1e380b2040 msgr2=0x7f1e380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e3e200640 1 -- 172.21.15.46:0/3818131517 shutdown_connections 2024-09-17T03:33:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.112+0000 7f1e3e200640 1 -- 172.21.15.46:0/3818131517 wait complete. 2024-09-17T03:33:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 Processor -- start 2024-09-17T03:33:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 -- start start 2024-09-17T03:33:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e1c003a10 0x7f1e381fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e381f9730 0x7f1e3815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 0x7f1e38159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38166910 con 0x7f1e381509b0 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38166790 con 0x7f1e1c003a10 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1e38166a90 con 0x7f1e381f9730 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 0x7f1e38159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 0x7f1e38159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59802/0 (socket says 172.21.15.46:59802) 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 -- 172.21.15.46:0/4286615381 learned_addr learned my addr 172.21.15.46:0/4286615381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37400640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e1c003a10 0x7f1e381fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e2fe00640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e381f9730 0x7f1e3815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 -- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e381f9730 msgr2=0x7f1e3815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e381f9730 0x7f1e3815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 -- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e1c003a10 msgr2=0x7f1e381fcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e1c003a10 0x7f1e381fcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 -- 172.21.15.46:0/4286615381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3815ae20 con 0x7f1e381509b0 2024-09-17T03:33:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e37e00640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 0x7f1e38159720 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1e38156170 tx=0x7f1e30006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30006910 con 0x7f1e381509b0 2024-09-17T03:33:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3000fc80 con 0x7f1e381509b0 2024-09-17T03:33:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e381f8c50 con 0x7f1e381509b0 2024-09-17T03:33:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30011690 con 0x7f1e381509b0 2024-09-17T03:33:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.116+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e38154ed0 con 0x7f1e381509b0 2024-09-17T03:33:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.120+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e38155130 con 0x7f1e381509b0 2024-09-17T03:33:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.120+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f1e30011830 con 0x7f1e381509b0 2024-09-17T03:33:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.120+0000 7f1e2f400640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e14074910 0x7f1e14076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.120+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f1e30093880 con 0x7f1e381509b0 2024-09-17T03:33:00.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.120+0000 7f1e37400640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e14074910 0x7f1e14076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.124+0000 7f1e37400640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e14074910 0x7f1e14076e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1e280ae000 tx=0x7f1e2801a040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.124+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e3005fcb0 con 0x7f1e381509b0 2024-09-17T03:33:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1e38074e30 con 0x7f1e381509b0 2024-09-17T03:33:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e2f400640 1 -- 172.21.15.46:0/4286615381 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) ==== 84+0+65 (secure 0 0 0) 0x7f1e300646e0 con 0x7f1e381509b0 2024-09-17T03:33:00.304 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:00.304 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:33:00.304 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:33:00.305 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:33:00.305 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:33:00.305 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:00.305 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:33:00.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e14074910 msgr2=0x7f1e14076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e14074910 0x7f1e14076e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1e280ae000 tx=0x7f1e2801a040 comp rx=0 tx=0).stop 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 msgr2=0x7f1e38159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 0x7f1e38159720 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1e38156170 tx=0x7f1e30006720 comp rx=0 tx=0).stop 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 shutdown_connections 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e14074910 0x7f1e14076e20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e381509b0 0x7f1e38159720 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e381f9730 0x7f1e3815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 --2- 172.21.15.46:0/4286615381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e1c003a10 0x7f1e381fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 >> 172.21.15.46:0/4286615381 conn(0x7f1e380b2040 msgr2=0x7f1e380b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 shutdown_connections 2024-09-17T03:33:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.304+0000 7f1e3e200640 1 -- 172.21.15.46:0/4286615381 wait complete. 2024-09-17T03:33:00.327 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:33:00.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:33:00.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:33:00.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:33:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 -- 172.21.15.46:0/2012840486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 msgr2=0x7fa9800b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 --2- 172.21.15.46:0/2012840486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa9800b51f0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa9800703e0 tx=0x7fa97801ccc0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 -- 172.21.15.46:0/2012840486 shutdown_connections 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 --2- 172.21.15.46:0/2012840486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa9800b51f0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 --2- 172.21.15.46:0/2012840486 >> v2:172.21.15.99:3300/0 conn(0x7fa9801527b0 0x7fa980074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 --2- 172.21.15.46:0/2012840486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9800b5c70 0x7fa98015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 -- 172.21.15.46:0/2012840486 >> 172.21.15.46:0/2012840486 conn(0x7fa9800b2040 msgr2=0x7fa9800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 -- 172.21.15.46:0/2012840486 shutdown_connections 2024-09-17T03:33:00.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 -- 172.21.15.46:0/2012840486 wait complete. 2024-09-17T03:33:00.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 Processor -- start 2024-09-17T03:33:00.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.492+0000 7fa985800640 1 -- start start 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa985800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa980148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa985800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9800b5c70 0x7fa9800ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa985800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9801527b0 0x7fa9800aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa985800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa98015f6b0 con 0x7fa9800b4de0 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa985800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa98015f530 con 0x7fa9800b5c70 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa985800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa98015f830 con 0x7fa9801527b0 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa980148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa980148030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59826/0 (socket says 172.21.15.46:59826) 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 -- 172.21.15.46:0/2947985588 learned_addr learned my addr 172.21.15.46:0/2947985588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97f400640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9801527b0 0x7fa9800aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa977e00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9800b5c70 0x7fa9800ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 -- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9801527b0 msgr2=0x7fa9800aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9801527b0 0x7fa9800aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 -- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9800b5c70 msgr2=0x7fa9800ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9800b5c70 0x7fa9800ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 -- 172.21.15.46:0/2947985588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9800cf900 con 0x7fa9800b4de0 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97f400640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9801527b0 0x7fa9800aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa977e00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9800b5c70 0x7fa9800ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.496+0000 7fa97ea00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa980148030 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa97000b410 tx=0x7fa97000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.500+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970015020 con 0x7fa9800b4de0 2024-09-17T03:33:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.500+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa980152dd0 con 0x7fa9800b4de0 2024-09-17T03:33:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.500+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa980153390 con 0x7fa9800b4de0 2024-09-17T03:33:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.500+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa970011830 con 0x7fa9800b4de0 2024-09-17T03:33:00.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.500+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970011c40 con 0x7fa9800b4de0 2024-09-17T03:33:00.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.500+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9580052a0 con 0x7fa9800b4de0 2024-09-17T03:33:00.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.504+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fa970018020 con 0x7fa9800b4de0 2024-09-17T03:33:00.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.504+0000 7fa976a00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa95c074910 0x7fa95c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:00.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.504+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7fa970092ea0 con 0x7fa9800b4de0 2024-09-17T03:33:00.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.504+0000 7fa977e00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa95c074910 0x7fa95c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:00.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.504+0000 7fa977e00640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa95c074910 0x7fa95c076e20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa9800cf6c0 tx=0x7fa968005eb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:00.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.512+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa97001f020 con 0x7fa9800b4de0 2024-09-17T03:33:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:00.688+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa958005860 con 0x7fa9800b4de0 2024-09-17T03:33:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.516+0000 7fa976a00640 1 -- 172.21.15.46:0/2947985588 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) ==== 138+0+0 (secure 0 0 0) 0x7fa97005f9b0 con 0x7fa9800b4de0 2024-09-17T03:33:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:33:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa95c074910 msgr2=0x7fa95c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa95c074910 0x7fa95c076e20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa9800cf6c0 tx=0x7fa968005eb0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 msgr2=0x7fa980148030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa980148030 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa97000b410 tx=0x7fa97000b8d0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 shutdown_connections 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa95c074910 0x7fa95c076e20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9801527b0 0x7fa9800aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9800b5c70 0x7fa9800ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 --2- 172.21.15.46:0/2947985588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800b4de0 0x7fa980148030 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 >> 172.21.15.46:0/2947985588 conn(0x7fa9800b2040 msgr2=0x7fa9800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 shutdown_connections 2024-09-17T03:33:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.520+0000 7fa985800640 1 -- 172.21.15.46:0/2947985588 wait complete. 2024-09-17T03:33:01.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:33:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 -- 172.21.15.46:0/2448943360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a800b4de0 msgr2=0x7f2a800b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 --2- 172.21.15.46:0/2448943360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a800b4de0 0x7f2a800b51f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009c80 tx=0x7f2a7c01cce0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 -- 172.21.15.46:0/2448943360 shutdown_connections 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 --2- 172.21.15.46:0/2448943360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a800b4de0 0x7f2a800b51f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 --2- 172.21.15.46:0/2448943360 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a801527b0 0x7f2a8015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 -- 172.21.15.46:0/2448943360 >> 172.21.15.46:0/2448943360 conn(0x7f2a800b2040 msgr2=0x7f2a800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 -- 172.21.15.46:0/2448943360 shutdown_connections 2024-09-17T03:33:01.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.764+0000 7f2a88800640 1 -- 172.21.15.46:0/2448943360 wait complete. 2024-09-17T03:33:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 Processor -- start 2024-09-17T03:33:01.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 -- start start 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a800b4de0 0x7f2a800ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 0x7f2a8013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801527b0 0x7f2a800b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a80168ab0 con 0x7f2a800b5c70 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a80168930 con 0x7f2a801527b0 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2a80168c30 con 0x7f2a800b4de0 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 0x7f2a8013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 0x7f2a8013fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59832/0 (socket says 172.21.15.46:59832) 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 -- 172.21.15.46:0/507036816 learned_addr learned my addr 172.21.15.46:0/507036816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85e00640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a800b4de0 0x7f2a800ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a86800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801527b0 0x7f2a800b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 -- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a800b4de0 msgr2=0x7f2a800ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a800b4de0 0x7f2a800ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 -- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801527b0 msgr2=0x7f2a800b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801527b0 0x7f2a800b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 -- 172.21.15.46:0/507036816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a800af150 con 0x7f2a800b5c70 2024-09-17T03:33:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a85400640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 0x7f2a8013fcf0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000e740 tx=0x7f2a7000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000ccc0 con 0x7f2a800b5c70 2024-09-17T03:33:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a800af430 con 0x7f2a800b5c70 2024-09-17T03:33:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a7000ee50 con 0x7f2a800b5c70 2024-09-17T03:33:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70010670 con 0x7f2a800b5c70 2024-09-17T03:33:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.768+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a800ac4d0 con 0x7f2a800b5c70 2024-09-17T03:33:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.772+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a800b51f0 con 0x7f2a800b5c70 2024-09-17T03:33:01.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.772+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f2a70010810 con 0x7f2a800b5c70 2024-09-17T03:33:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.776+0000 7f2a7a000640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a64074910 0x7f2a64076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.776+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5227+0+0 (secure 0 0 0) 0x7f2a70094170 con 0x7f2a800b5c70 2024-09-17T03:33:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.776+0000 7f2a85e00640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a64074910 0x7f2a64076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.776+0000 7f2a85e00640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a64074910 0x7f2a64076e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2a74007fd0 tx=0x7f2a74009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:01.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.776+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2a70060640 con 0x7f2a800b5c70 2024-09-17T03:33:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:01.952+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2a8015a790 con 0x7f2a800b5c70 2024-09-17T03:33:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.520+0000 7f2a7a000640 1 -- 172.21.15.46:0/507036816 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) ==== 170+0+0 (secure 0 0 0) 0x7f2a70065070 con 0x7f2a800b5c70 2024-09-17T03:33:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:33:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a64074910 msgr2=0x7f2a64076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a64074910 0x7f2a64076e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2a74007fd0 tx=0x7f2a74009040 comp rx=0 tx=0).stop 2024-09-17T03:33:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 msgr2=0x7f2a8013fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 0x7f2a8013fcf0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000e740 tx=0x7f2a7000ec00 comp rx=0 tx=0).stop 2024-09-17T03:33:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 shutdown_connections 2024-09-17T03:33:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2a64074910 0x7f2a64076e20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a801527b0 0x7f2a800b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a800b5c70 0x7f2a8013fcf0 secure :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000e740 tx=0x7f2a7000ec00 comp rx=0 tx=0).stop 2024-09-17T03:33:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 --2- 172.21.15.46:0/507036816 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2a800b4de0 0x7f2a800ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 >> 172.21.15.46:0/507036816 conn(0x7f2a800b2040 msgr2=0x7f2a801581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.524+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 shutdown_connections 2024-09-17T03:33:02.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.528+0000 7f2a88800640 1 -- 172.21.15.46:0/507036816 wait complete. 2024-09-17T03:33:02.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:33:02.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 -- 172.21.15.46:0/501745472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40749a0 msgr2=0x7fcec4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:02.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 --2- 172.21.15.46:0/501745472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40749a0 0x7fcec4074db0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009840 tx=0x7fceb401c820 comp rx=0 tx=0).stop 2024-09-17T03:33:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 -- 172.21.15.46:0/501745472 shutdown_connections 2024-09-17T03:33:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 --2- 172.21.15.46:0/501745472 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcec40b3340 0x7fcec40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 --2- 172.21.15.46:0/501745472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40749a0 0x7fcec4074db0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 -- 172.21.15.46:0/501745472 >> 172.21.15.46:0/501745472 conn(0x7fcec40b2040 msgr2=0x7fcec40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 -- 172.21.15.46:0/501745472 shutdown_connections 2024-09-17T03:33:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.732+0000 7fceca400640 1 -- 172.21.15.46:0/501745472 wait complete. 2024-09-17T03:33:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 Processor -- start 2024-09-17T03:33:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 -- start start 2024-09-17T03:33:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40749a0 0x7fcec4159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcec40b3340 0x7fcec40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 0x7fcec40aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4169b80 con 0x7fcec40cefa0 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4169a00 con 0x7fcec40749a0 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fceca400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcec4169d00 con 0x7fcec40b3340 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 0x7fcec40aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 0x7fcec40aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56934/0 (socket says 172.21.15.46:56934) 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fcec3e00640 1 -- 172.21.15.46:0/493945135 learned_addr learned my addr 172.21.15.46:0/493945135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fcebbe00640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcec40b3340 0x7fcec40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.736+0000 7fcec3400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40749a0 0x7fcec4159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3e00640 1 -- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcec40b3340 msgr2=0x7fcec40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3e00640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcec40b3340 0x7fcec40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3e00640 1 -- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40749a0 msgr2=0x7fcec4159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3e00640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40749a0 0x7fcec4159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3e00640 1 -- 172.21.15.46:0/493945135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec413fa40 con 0x7fcec40cefa0 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40749a0 0x7fcec4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcec3e00640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 0x7fcec40aee20 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00ea80 tx=0x7fcebc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc004b30 con 0x7fcec40cefa0 2024-09-17T03:33:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcebc004cf0 con 0x7fcec40cefa0 2024-09-17T03:33:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec413f760 con 0x7fcec40cefa0 2024-09-17T03:33:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc005280 con 0x7fcec40cefa0 2024-09-17T03:33:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec413d450 con 0x7fcec40cefa0 2024-09-17T03:33:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.740+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec4074db0 con 0x7fcec40cefa0 2024-09-17T03:33:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.744+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fcebc005420 con 0x7fcec40cefa0 2024-09-17T03:33:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.744+0000 7fcebb400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcea0074910 0x7fcea0076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.744+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5609+0+0 (secure 0 0 0) 0x7fcebc093fa0 con 0x7fcec40cefa0 2024-09-17T03:33:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.744+0000 7fcec3400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcea0074910 0x7fcea0076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.744+0000 7fcec3400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcea0074910 0x7fcea0076e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fceb40af000 tx=0x7fceb401b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:02.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.748+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcec413fa40 con 0x7fcec40cefa0 2024-09-17T03:33:02.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:02.956+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcec413f4f0 con 0x7fcec40cefa0 2024-09-17T03:33:03.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.532+0000 7fcebb400640 1 -- 172.21.15.46:0/493945135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v61) ==== 174+0+0 (secure 0 0 0) 0x7fcebc0604e0 con 0x7fcec40cefa0 2024-09-17T03:33:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 14 and data pool 15 2024-09-17T03:33:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcea0074910 msgr2=0x7fcea0076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcea0074910 0x7fcea0076e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fceb40af000 tx=0x7fceb401b060 comp rx=0 tx=0).stop 2024-09-17T03:33:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 msgr2=0x7fcec40aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 0x7fcec40aee20 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00ea80 tx=0x7fcebc00ef40 comp rx=0 tx=0).stop 2024-09-17T03:33:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 shutdown_connections 2024-09-17T03:33:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcea0074910 0x7fcea0076e20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec40cefa0 0x7fcec40aee20 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcec40b3340 0x7fcec40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 --2- 172.21.15.46:0/493945135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40749a0 0x7fcec4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 >> 172.21.15.46:0/493945135 conn(0x7fcec40b2040 msgr2=0x7fcec40b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.536+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 shutdown_connections 2024-09-17T03:33:03.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.540+0000 7fceca400640 1 -- 172.21.15.46:0/493945135 wait complete. 2024-09-17T03:33:03.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:33:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 -- 172.21.15.46:0/2371498314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 msgr2=0x7f9d38074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2371498314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d38074240 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9d380cf5e0 tx=0x7f9d2001c830 comp rx=0 tx=0).stop 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 -- 172.21.15.46:0/2371498314 shutdown_connections 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2371498314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d38074240 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2371498314 >> v2:172.21.15.99:3300/0 conn(0x7f9d38160470 0x7f9d38074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2371498314 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 0x7f9d3815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 -- 172.21.15.46:0/2371498314 >> 172.21.15.46:0/2371498314 conn(0x7f9d380b4560 msgr2=0x7f9d380d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 -- 172.21.15.46:0/2371498314 shutdown_connections 2024-09-17T03:33:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.748+0000 7f9d37e00640 1 -- 172.21.15.46:0/2371498314 wait complete. 2024-09-17T03:33:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 Processor -- start 2024-09-17T03:33:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 -- start start 2024-09-17T03:33:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 0x7f9d38150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d3814c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d38160470 0x7f9d3814c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38071db0 con 0x7f9d38160060 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38071c30 con 0x7f9d38160470 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9d38071f30 con 0x7f9d381536f0 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d3814c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d3814c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56964/0 (socket says 172.21.15.46:56964) 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d36a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 0x7f9d38150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d2e000640 1 -- 172.21.15.46:0/2413207772 learned_addr learned my addr 172.21.15.46:0/2413207772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.752+0000 7f9d37400640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d38160470 0x7f9d3814c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2e000640 1 -- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 msgr2=0x7f9d38150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2e000640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 0x7f9d38150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2e000640 1 -- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d38160470 msgr2=0x7f9d3814c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2e000640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d38160470 0x7f9d3814c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2e000640 1 -- 172.21.15.46:0/2413207772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d381f6940 con 0x7f9d38160060 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d36a00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 0x7f9d38150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2e000640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d3814c450 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9d200b0040 tx=0x7f9d200a6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20004450 con 0x7f9d38160060 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d20005020 con 0x7f9d38160060 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20005490 con 0x7f9d38160060 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d381f6bb0 con 0x7f9d38160060 2024-09-17T03:33:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d381f7170 con 0x7f9d38160060 2024-09-17T03:33:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38072450 con 0x7f9d38160060 2024-09-17T03:33:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.756+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f9d200be060 con 0x7f9d38160060 2024-09-17T03:33:03.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.760+0000 7f9d2f400640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9d14074910 0x7f9d14076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:03.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.760+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f9d2012d130 con 0x7f9d38160060 2024-09-17T03:33:03.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.760+0000 7f9d36a00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9d14074910 0x7f9d14076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:03.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.760+0000 7f9d36a00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9d14074910 0x7f9d14076e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d28011690 tx=0x7f9d28009500 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:03.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.764+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9d200164b0 con 0x7f9d38160060 2024-09-17T03:33:03.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.936+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d38157a20 con 0x7f9d38160060 2024-09-17T03:33:03.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.936+0000 7f9d2f400640 1 -- 172.21.15.46:0/2413207772 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16656 (secure 0 0 0) 0x7f9d200f9c70 con 0x7f9d38160060 2024-09-17T03:33:03.938 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:03.938 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":101,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:33:02.961538+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:33:00.702058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T03:33:01.968680+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:33:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9d14074910 msgr2=0x7f9d14076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9d14074910 0x7f9d14076e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9d28011690 tx=0x7f9d28009500 comp rx=0 tx=0).stop 2024-09-17T03:33:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 msgr2=0x7f9d3814c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d3814c450 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9d200b0040 tx=0x7f9d200a6540 comp rx=0 tx=0).stop 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 shutdown_connections 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9d14074910 0x7f9d14076e20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d38160470 0x7f9d3814c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d38160060 0x7f9d3814c450 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 --2- 172.21.15.46:0/2413207772 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9d381536f0 0x7f9d38150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 >> 172.21.15.46:0/2413207772 conn(0x7f9d380b4560 msgr2=0x7f9d38151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 shutdown_connections 2024-09-17T03:33:03.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:03.940+0000 7f9d37e00640 1 -- 172.21.15.46:0/2413207772 wait complete. 2024-09-17T03:33:03.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 -- 172.21.15.46:0/3040376005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 msgr2=0x7f85d0070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 --2- 172.21.15.46:0/3040376005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0070690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009bf0 tx=0x7f85c801cbb0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 -- 172.21.15.46:0/3040376005 shutdown_connections 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 --2- 172.21.15.46:0/3040376005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0070690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 -- 172.21.15.46:0/3040376005 >> 172.21.15.46:0/3040376005 conn(0x7f85d00b35f0 msgr2=0x7f85d00b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 -- 172.21.15.46:0/3040376005 shutdown_connections 2024-09-17T03:33:04.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 -- 172.21.15.46:0/3040376005 wait complete. 2024-09-17T03:33:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 Processor -- start 2024-09-17T03:33:04.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.120+0000 7f85d7200640 1 -- start start 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d015c7b0 0x7f85d01fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 0x7f85d015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85d016af70 con 0x7f85d015a730 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85d016adf0 con 0x7f85d015c7b0 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f85d016b0f0 con 0x7f85d0160af0 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 0x7f85d015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 0x7f85d015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56978/0 (socket says 172.21.15.46:56978) 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 -- 172.21.15.46:0/683846886 learned_addr learned my addr 172.21.15.46:0/683846886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85cfe00640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d015c7b0 0x7f85d01fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85c7e00640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 -- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 msgr2=0x7f85d0140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 -- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d015c7b0 msgr2=0x7f85d01fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d015c7b0 0x7f85d01fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 -- 172.21.15.46:0/683846886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d015e950 con 0x7f85d015a730 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85c7e00640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d5200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 0x7f85d015d860 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f85c80197c0 tx=0x7f85c8005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b2040 con 0x7f85d015a730 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85c8004b40 con 0x7f85d015a730 2024-09-17T03:33:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8003fe0 con 0x7f85d015a730 2024-09-17T03:33:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d0140aa0 con 0x7f85d015a730 2024-09-17T03:33:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d015eea0 con 0x7f85d015a730 2024-09-17T03:33:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.124+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0070690 con 0x7f85d015a730 2024-09-17T03:33:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.128+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f85c80be060 con 0x7f85d015a730 2024-09-17T03:33:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.128+0000 7f85ccc00640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85ac074910 0x7f85ac076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.128+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f85c812cbc0 con 0x7f85d015a730 2024-09-17T03:33:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.128+0000 7f85cfe00640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85ac074910 0x7f85ac076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.128+0000 7f85cfe00640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85ac074910 0x7f85ac076e20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007fd0 tx=0x7f85c00013a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.132+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f85c80167d0 con 0x7f85d015a730 2024-09-17T03:33:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.308+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85d0158ef0 con 0x7f85d015a730 2024-09-17T03:33:04.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.308+0000 7f85ccc00640 1 -- 172.21.15.46:0/683846886 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16656 (secure 0 0 0) 0x7f85c80f96d0 con 0x7f85d015a730 2024-09-17T03:33:04.309 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:04.309 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":101,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:33:02.961538+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:33:00.702058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T03:33:01.968680+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:33:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85ac074910 msgr2=0x7f85ac076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85ac074910 0x7f85ac076e20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007fd0 tx=0x7f85c00013a0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 msgr2=0x7f85d015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 0x7f85d015d860 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f85c80197c0 tx=0x7f85c8005aa0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 shutdown_connections 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85ac074910 0x7f85ac076e20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85d015a730 0x7f85d015d860 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f85d0160af0 0x7f85d0140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 --2- 172.21.15.46:0/683846886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d015c7b0 0x7f85d01fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 >> 172.21.15.46:0/683846886 conn(0x7f85d00b35f0 msgr2=0x7f85d00716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 shutdown_connections 2024-09-17T03:33:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.312+0000 7f85d7200640 1 -- 172.21.15.46:0/683846886 wait complete. 2024-09-17T03:33:04.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:33:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 -- 172.21.15.46:0/773816613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d41536f0 msgr2=0x7fc1d415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 --2- 172.21.15.46:0/773816613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d41536f0 0x7fc1d415d480 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009bc0 tx=0x7fc1c401cb10 comp rx=0 tx=0).stop 2024-09-17T03:33:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 -- 172.21.15.46:0/773816613 shutdown_connections 2024-09-17T03:33:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 --2- 172.21.15.46:0/773816613 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1d4074950 0x7fc1d40b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 --2- 172.21.15.46:0/773816613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d41536f0 0x7fc1d415d480 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 -- 172.21.15.46:0/773816613 >> 172.21.15.46:0/773816613 conn(0x7fc1d40b4560 msgr2=0x7fc1d40d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 -- 172.21.15.46:0/773816613 shutdown_connections 2024-09-17T03:33:04.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.496+0000 7fc1d9000640 1 -- 172.21.15.46:0/773816613 wait complete. 2024-09-17T03:33:04.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 Processor -- start 2024-09-17T03:33:04.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 -- start start 2024-09-17T03:33:04.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1d41536f0 0x7fc1d414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1d4160060 0x7fc1d41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 0x7fc1d4156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4169e60 con 0x7fc1d4156280 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4169ce0 con 0x7fc1d4160060 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d4169fe0 con 0x7fc1d41536f0 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 0x7fc1d4156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 0x7fc1d4156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56996/0 (socket says 172.21.15.46:56996) 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 -- 172.21.15.46:0/1371993881 learned_addr learned my addr 172.21.15.46:0/1371993881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3400640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1d41536f0 0x7fc1d414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1caa00640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1d4160060 0x7fc1d41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 -- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1d41536f0 msgr2=0x7fc1d414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1d41536f0 0x7fc1d414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 -- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1d4160060 msgr2=0x7fc1d41549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1d4160060 0x7fc1d41549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 -- 172.21.15.46:0/1371993881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d4156bd0 con 0x7fc1d4156280 2024-09-17T03:33:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d3e00640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 0x7fc1d4156690 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00d660 tx=0x7fc1cc00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc017070 con 0x7fc1d4156280 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1cc005310 con 0x7fc1d4156280 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d414d530 con 0x7fc1d4156280 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009820 con 0x7fc1d4156280 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.500+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d41558b0 con 0x7fc1d4156280 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.504+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d40b5260 con 0x7fc1d4156280 2024-09-17T03:33:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.504+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fc1cc0099c0 con 0x7fc1d4156280 2024-09-17T03:33:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.504+0000 7fc1cbe00640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1b0074910 0x7fc1b0076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.504+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fc1cc093010 con 0x7fc1d4156280 2024-09-17T03:33:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.504+0000 7fc1d3400640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1b0074910 0x7fc1b0076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.504+0000 7fc1d3400640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1b0074910 0x7fc1b0076e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40af000 tx=0x7fc1c401b350 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.508+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc1d4156bd0 con 0x7fc1d4156280 2024-09-17T03:33:04.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:04.728+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc1d4155360 con 0x7fc1d4156280 2024-09-17T03:33:05.511 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-09-17T03:33:05.512+0000 7fe79d800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:33:05.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.564+0000 7fc1cbe00640 1 -- 172.21.15.46:0/1371993881 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) ==== 118+0+0 (secure 0 0 0) 0x7fc1cc05fb20 con 0x7fc1d4156280 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1b0074910 msgr2=0x7fc1b0076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1b0074910 0x7fc1b0076e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40af000 tx=0x7fc1c401b350 comp rx=0 tx=0).stop 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 msgr2=0x7fc1d4156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 0x7fc1d4156690 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00d660 tx=0x7fc1cc00db20 comp rx=0 tx=0).stop 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 shutdown_connections 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1b0074910 0x7fc1b0076e20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d4156280 0x7fc1d4156690 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1d4160060 0x7fc1d41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 --2- 172.21.15.46:0/1371993881 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1d41536f0 0x7fc1d414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 >> 172.21.15.46:0/1371993881 conn(0x7fc1d40b4560 msgr2=0x7fc1d415ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 shutdown_connections 2024-09-17T03:33:05.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.568+0000 7fc1d9000640 1 -- 172.21.15.46:0/1371993881 wait complete. 2024-09-17T03:33:05.593 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:33:05.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:33:05.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41f1000640 1 -- 172.21.15.46:0/3319744421 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec0b4ce0 msgr2=0x7f41d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41f1000640 1 --2- 172.21.15.46:0/3319744421 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec0b4ce0 0x7f41d0005210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41e400afa0 tx=0x7f41e401e410 comp rx=0 tx=0).stop 2024-09-17T03:33:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41ea800640 1 -- 172.21.15.46:0/3319744421 reap_dead start 2024-09-17T03:33:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41f1000640 1 -- 172.21.15.46:0/3319744421 shutdown_connections 2024-09-17T03:33:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41f1000640 1 -- 172.21.15.46:0/3319744421 >> 172.21.15.46:0/3319744421 conn(0x7f41ec0b2040 msgr2=0x7f41ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41f1000640 1 -- 172.21.15.46:0/3319744421 shutdown_connections 2024-09-17T03:33:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.760+0000 7f41f1000640 1 -- 172.21.15.46:0/3319744421 wait complete. 2024-09-17T03:33:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 Processor -- start 2024-09-17T03:33:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 -- start start 2024-09-17T03:33:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 0x7f41ec1f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ec0ae540 0x7f41ec0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec1f6eb0 0x7f41ec1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec0b3910 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec0b3790 con 0x7f41ec0ae540 2024-09-17T03:33:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41f1000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec0b3a90 con 0x7f41ec1f6eb0 2024-09-17T03:33:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41e9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 0x7f41ec1f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41e9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 0x7f41ec1f0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57026/0 (socket says 172.21.15.46:57026) 2024-09-17T03:33:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.764+0000 7f41e9e00640 1 -- 172.21.15.46:0/3844052545 learned_addr learned my addr 172.21.15.46:0/3844052545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e1400640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ec0ae540 0x7f41ec0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41ea800640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec1f6eb0 0x7f41ec1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e9e00640 1 -- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec1f6eb0 msgr2=0x7f41ec1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e9e00640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec1f6eb0 0x7f41ec1f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e9e00640 1 -- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ec0ae540 msgr2=0x7f41ec0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e9e00640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ec0ae540 0x7f41ec0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e9e00640 1 -- 172.21.15.46:0/3844052545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec0aa770 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e9e00640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 0x7f41ec1f0ef0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f41dc006fa0 tx=0x7f41dc011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc013720 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41dc0138c0 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec0aaad0 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc013cd0 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec0ab090 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec0b5610 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.768+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f41dc01b400 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.772+0000 7f41e2a00640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41c8074940 0x7f41c8076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.772+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f41dc096510 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.772+0000 7f41e1400640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41c8074940 0x7f41c8076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.772+0000 7f41e1400640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41c8074940 0x7f41c8076e50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f41d4001230 tx=0x7f41d4001120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:05.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.776+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41dc017030 con 0x7f41ec0b4ce0 2024-09-17T03:33:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:05.984+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f41ec075570 con 0x7f41ec0b4ce0 2024-09-17T03:33:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.564+0000 7f41e2a00640 1 -- 172.21.15.46:0/3844052545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) ==== 122+0+0 (secure 0 0 0) 0x7f41dc0630a0 con 0x7f41ec0b4ce0 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41c8074940 msgr2=0x7f41c8076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41c8074940 0x7f41c8076e50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f41d4001230 tx=0x7f41d4001120 comp rx=0 tx=0).stop 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 msgr2=0x7f41ec1f0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 0x7f41ec1f0ef0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f41dc006fa0 tx=0x7f41dc011970 comp rx=0 tx=0).stop 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 shutdown_connections 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f41c8074940 0x7f41c8076e50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41ec1f6eb0 0x7f41ec1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41ec0ae540 0x7f41ec0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 --2- 172.21.15.46:0/3844052545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41ec0b4ce0 0x7f41ec1f0ef0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 >> 172.21.15.46:0/3844052545 conn(0x7f41ec0b2040 msgr2=0x7f41ec0b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 shutdown_connections 2024-09-17T03:33:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.568+0000 7f41f1000640 1 -- 172.21.15.46:0/3844052545 wait complete. 2024-09-17T03:33:06.594 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:33:06.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 -- 172.21.15.46:0/1327047500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40157030 msgr2=0x7f8d400b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 --2- 172.21.15.46:0/1327047500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40157030 0x7f8d400b5a80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009840 tx=0x7f8d3001c820 comp rx=0 tx=0).stop 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 -- 172.21.15.46:0/1327047500 shutdown_connections 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 --2- 172.21.15.46:0/1327047500 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40074b20 0x7f8d4015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 --2- 172.21.15.46:0/1327047500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40157030 0x7f8d400b5a80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 -- 172.21.15.46:0/1327047500 >> 172.21.15.46:0/1327047500 conn(0x7f8d400708c0 msgr2=0x7f8d40070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 -- 172.21.15.46:0/1327047500 shutdown_connections 2024-09-17T03:33:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.756+0000 7f8d44e00640 1 -- 172.21.15.46:0/1327047500 wait complete. 2024-09-17T03:33:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 Processor -- start 2024-09-17T03:33:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 -- start start 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40157030 0x7f8d40154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40160d50 0x7f8d401f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 0x7f8d401f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4016a740 con 0x7f8d40154df0 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4016a5c0 con 0x7f8d40160d50 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d44e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4016a8c0 con 0x7f8d40157030 2024-09-17T03:33:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 0x7f8d401f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 0x7f8d401f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57042/0 (socket says 172.21.15.46:57042) 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d3e800640 1 -- 172.21.15.46:0/535926322 learned_addr learned my addr 172.21.15.46:0/535926322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d3de00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40157030 0x7f8d40154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.760+0000 7f8d37e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40160d50 0x7f8d401f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3e800640 1 -- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40157030 msgr2=0x7f8d40154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3e800640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40157030 0x7f8d40154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3e800640 1 -- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40160d50 msgr2=0x7f8d401f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3e800640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40160d50 0x7f8d401f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3e800640 1 -- 172.21.15.46:0/535926322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401f80c0 con 0x7f8d40154df0 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3de00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40157030 0x7f8d40154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d3e800640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 0x7f8d401f79e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e970 tx=0x7f8d3800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38004d30 con 0x7f8d40154df0 2024-09-17T03:33:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d3800cb20 con 0x7f8d40154df0 2024-09-17T03:33:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4014af00 con 0x7f8d40154df0 2024-09-17T03:33:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38005020 con 0x7f8d40154df0 2024-09-17T03:33:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4014b4c0 con 0x7f8d40154df0 2024-09-17T03:33:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4015cf90 con 0x7f8d40154df0 2024-09-17T03:33:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f8d380051c0 con 0x7f8d40154df0 2024-09-17T03:33:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.764+0000 7f8d36000640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d1c074940 0x7f8d1c076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.768+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f8d380938d0 con 0x7f8d40154df0 2024-09-17T03:33:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.768+0000 7f8d3de00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d1c074940 0x7f8d1c076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.768+0000 7f8d3de00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d1c074940 0x7f8d1c076e50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8d300af000 tx=0x7f8d3001b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.772+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d38016030 con 0x7f8d40154df0 2024-09-17T03:33:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.980+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d4015f590 con 0x7f8d40154df0 2024-09-17T03:33:06.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.980+0000 7f8d36000640 1 -- 172.21.15.46:0/535926322 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5238 (secure 0 0 0) 0x7f8d380603e0 con 0x7f8d40154df0 2024-09-17T03:33:06.981 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:06.981 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"btime":"2024-09-17T03:33:06:567020+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:33:02.961494+0000","modified":"2024-09-17T03:33:05.989948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":247692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_247692":{"gid":247692,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4088974288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4088974288},{"type":"v1","addr":"172.21.15.46:6836","nonce":4088974288}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":247692,"qdb_cluster":[247692]},"id":7}]} 2024-09-17T03:33:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 65 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d1c074940 msgr2=0x7f8d1c076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d1c074940 0x7f8d1c076e50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8d300af000 tx=0x7f8d3001b060 comp rx=0 tx=0).stop 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 msgr2=0x7f8d401f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 0x7f8d401f79e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e970 tx=0x7f8d3800ee30 comp rx=0 tx=0).stop 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 shutdown_connections 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d1c074940 0x7f8d1c076e50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40154df0 0x7f8d401f79e0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40160d50 0x7f8d401f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 --2- 172.21.15.46:0/535926322 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d40157030 0x7f8d40154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 >> 172.21.15.46:0/535926322 conn(0x7f8d400708c0 msgr2=0x7f8d4015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 shutdown_connections 2024-09-17T03:33:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:06.984+0000 7f8d44e00640 1 -- 172.21.15.46:0/535926322 wait complete. 2024-09-17T03:33:07.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:33:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 -- 172.21.15.46:0/1076622156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec072100 msgr2=0x7f64ec072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1076622156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec072100 0x7f64ec072510 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009bc0 tx=0x7f64e401ca80 comp rx=0 tx=0).stop 2024-09-17T03:33:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 -- 172.21.15.46:0/1076622156 shutdown_connections 2024-09-17T03:33:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1076622156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec072100 0x7f64ec072510 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1076622156 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64ec0cefa0 0x7f64ec0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 -- 172.21.15.46:0/1076622156 >> 172.21.15.46:0/1076622156 conn(0x7f64ec0b2040 msgr2=0x7f64ec0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 -- 172.21.15.46:0/1076622156 shutdown_connections 2024-09-17T03:33:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.164+0000 7f64f1a00640 1 -- 172.21.15.46:0/1076622156 wait complete. 2024-09-17T03:33:07.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 Processor -- start 2024-09-17T03:33:07.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 -- start start 2024-09-17T03:33:07.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64ec072100 0x7f64ec13bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 0x7f64ec145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec15cbb0 0x7f64ec145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec169ed0 con 0x7f64ec0cefa0 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec169d50 con 0x7f64ec15cbb0 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec16a050 con 0x7f64ec072100 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 0x7f64ec145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 0x7f64ec145250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57052/0 (socket says 172.21.15.46:57052) 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 -- 172.21.15.46:0/1669569916 learned_addr learned my addr 172.21.15.46:0/1669569916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64eb400640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec15cbb0 0x7f64ec145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64eaa00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64ec072100 0x7f64ec13bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 -- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64ec072100 msgr2=0x7f64ec13bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64ec072100 0x7f64ec13bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 -- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec15cbb0 msgr2=0x7f64ec145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec15cbb0 0x7f64ec145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 -- 172.21.15.46:0/1669569916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec155490 con 0x7f64ec0cefa0 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64eb400640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec15cbb0 0x7f64ec145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:07.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64ea000640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 0x7f64ec145250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f64d800c760 tx=0x7f64d800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:07.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8006aa0 con 0x7f64ec0cefa0 2024-09-17T03:33:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d8006c40 con 0x7f64ec0cefa0 2024-09-17T03:33:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec155770 con 0x7f64ec0cefa0 2024-09-17T03:33:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8007290 con 0x7f64ec0cefa0 2024-09-17T03:33:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.168+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec155d30 con 0x7f64ec0cefa0 2024-09-17T03:33:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.172+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec155200 con 0x7f64ec0cefa0 2024-09-17T03:33:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.172+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f64d8007430 con 0x7f64ec0cefa0 2024-09-17T03:33:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.172+0000 7f64e2a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64cc074940 0x7f64cc076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.172+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f64d8092a60 con 0x7f64ec0cefa0 2024-09-17T03:33:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.172+0000 7f64eaa00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64cc074940 0x7f64cc076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.172+0000 7f64eaa00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64cc074940 0x7f64cc076e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009ad0 tx=0x7f64dc0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.176+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f64ec0cf8f0 con 0x7f64ec0cefa0 2024-09-17T03:33:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.352+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64ec072510 con 0x7f64ec0cefa0 2024-09-17T03:33:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.352+0000 7f64e2a00640 1 -- 172.21.15.46:0/1669569916 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16643 (secure 0 0 0) 0x7f64d805f570 con 0x7f64ec0cefa0 2024-09-17T03:33:07.354 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:07.354 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":102,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:33:04.534225+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:33:00.702058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T03:33:01.968680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:33:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64cc074940 msgr2=0x7f64cc076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64cc074940 0x7f64cc076e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009ad0 tx=0x7f64dc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 msgr2=0x7f64ec145250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 0x7f64ec145250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f64d800c760 tx=0x7f64d800cc20 comp rx=0 tx=0).stop 2024-09-17T03:33:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 shutdown_connections 2024-09-17T03:33:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64cc074940 0x7f64cc076e50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64ec15cbb0 0x7f64ec145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64ec0cefa0 0x7f64ec145250 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 --2- 172.21.15.46:0/1669569916 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64ec072100 0x7f64ec13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 >> 172.21.15.46:0/1669569916 conn(0x7f64ec0b2040 msgr2=0x7f64ec13f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 shutdown_connections 2024-09-17T03:33:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.356+0000 7f64f1a00640 1 -- 172.21.15.46:0/1669569916 wait complete. 2024-09-17T03:33:07.377 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:33:07.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:33:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 -- 172.21.15.46:0/2682499909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 msgr2=0x7f2bfc1678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 --2- 172.21.15.46:0/2682499909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc1678c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009bc0 tx=0x7f2bf401cba0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 -- 172.21.15.46:0/2682499909 shutdown_connections 2024-09-17T03:33:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 --2- 172.21.15.46:0/2682499909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc1678c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 -- 172.21.15.46:0/2682499909 >> 172.21.15.46:0/2682499909 conn(0x7f2bfc0b2040 msgr2=0x7f2bfc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:07.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 -- 172.21.15.46:0/2682499909 shutdown_connections 2024-09-17T03:33:07.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.536+0000 7f2c04600640 1 -- 172.21.15.46:0/2682499909 wait complete. 2024-09-17T03:33:07.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 Processor -- start 2024-09-17T03:33:07.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 -- start start 2024-09-17T03:33:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bfc0749a0 0x7f2bfc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bfc0cefa0 0x7f2bfc0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc16af70 con 0x7f2bfc1654a0 2024-09-17T03:33:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc16adf0 con 0x7f2bfc0749a0 2024-09-17T03:33:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2bfc16b0f0 con 0x7f2bfc0cefa0 2024-09-17T03:33:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57066/0 (socket says 172.21.15.46:57066) 2024-09-17T03:33:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 -- 172.21.15.46:0/3686013451 learned_addr learned my addr 172.21.15.46:0/3686013451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c01200640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bfc0cefa0 0x7f2bfc0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c01c00640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bfc0749a0 0x7f2bfc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 -- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bfc0cefa0 msgr2=0x7f2bfc0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bfc0cefa0 0x7f2bfc0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 -- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bfc0749a0 msgr2=0x7f2bfc159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bfc0749a0 0x7f2bfc159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 -- 172.21.15.46:0/3686013451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc166b10 con 0x7f2bfc1654a0 2024-09-17T03:33:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c01c00640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bfc0749a0 0x7f2bfc159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c02600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc0cd3b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40197b0 tx=0x7f2bf40a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40bb070 con 0x7f2bfc1654a0 2024-09-17T03:33:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bf4004d40 con 0x7f2bfc1654a0 2024-09-17T03:33:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc166df0 con 0x7f2bfc1654a0 2024-09-17T03:33:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40b2040 con 0x7f2bfc1654a0 2024-09-17T03:33:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.540+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc1673b0 con 0x7f2bfc1654a0 2024-09-17T03:33:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.544+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc074db0 con 0x7f2bfc1654a0 2024-09-17T03:33:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.544+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f2bf40be070 con 0x7f2bfc1654a0 2024-09-17T03:33:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.544+0000 7f2bf9600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bdc074910 0x7f2bdc076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.544+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2bf412cbd0 con 0x7f2bfc1654a0 2024-09-17T03:33:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.544+0000 7f2c01c00640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bdc074910 0x7f2bdc076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.548+0000 7f2c01c00640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bdc074910 0x7f2bdc076e20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00046c0 tx=0x7f2bf0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.548+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2bf40a5e20 con 0x7f2bfc1654a0 2024-09-17T03:33:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.720+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bfc0b2790 con 0x7f2bdc074910 2024-09-17T03:33:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.728+0000 7f2bf9600640 1 -- 172.21.15.46:0/3686013451 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251442 (secure 0 0 0) 0x7f2bfc0b2790 con 0x7f2bdc074910 2024-09-17T03:33:07.730 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bdc074910 msgr2=0x7f2bdc076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bdc074910 0x7f2bdc076e20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00046c0 tx=0x7f2bf0009040 comp rx=0 tx=0).stop 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 msgr2=0x7f2bfc0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc0cd3b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40197b0 tx=0x7f2bf40a5ae0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 shutdown_connections 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bdc074910 0x7f2bdc076e20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc1654a0 0x7f2bfc0cd3b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bfc0cefa0 0x7f2bfc0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 --2- 172.21.15.46:0/3686013451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bfc0749a0 0x7f2bfc159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 >> 172.21.15.46:0/3686013451 conn(0x7f2bfc0b2040 msgr2=0x7f2bfc160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 shutdown_connections 2024-09-17T03:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.732+0000 7f2c04600640 1 -- 172.21.15.46:0/3686013451 wait complete. 2024-09-17T03:33:07.752 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-17T03:33:07.511563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":143,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1018000,"kb_used_data":10964,"kb_used_omap":12,"kb_used_meta":1006835,"kb_avail":753956720,"statfs":{"total":773094113280,"available":772051681280,"internally_reserved":0,"allocated":11227136,"data_stored":5497360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1030999600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991341"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539642+0000","last_change":"2024-09-17T03:33:03.581362+0000","last_active":"2024-09-17T03:33:04.539642+0000","last_peered":"2024-09-17T03:33:04.539642+0000","last_clean":"2024-09-17T03:33:04.539642+0000","last_became_active":"2024-09-17T03:33:03.580475+0000","last_became_peered":"2024-09-17T03:33:03.580475+0000","last_unstale":"2024-09-17T03:33:04.539642+0000","last_undegraded":"2024-09-17T03:33:04.539642+0000","last_fullsized":"2024-09-17T03:33:04.539642+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:11.925403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420228+0000","last_change":"2024-09-17T03:33:03.572549+0000","last_active":"2024-09-17T03:33:05.420228+0000","last_peered":"2024-09-17T03:33:05.420228+0000","last_clean":"2024-09-17T03:33:05.420228+0000","last_became_active":"2024-09-17T03:33:03.572232+0000","last_became_peered":"2024-09-17T03:33:03.572232+0000","last_unstale":"2024-09-17T03:33:05.420228+0000","last_undegraded":"2024-09-17T03:33:05.420228+0000","last_fullsized":"2024-09-17T03:33:05.420228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:46.104711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539783+0000","last_change":"2024-09-17T03:33:03.580570+0000","last_active":"2024-09-17T03:33:04.539783+0000","last_peered":"2024-09-17T03:33:04.539783+0000","last_clean":"2024-09-17T03:33:04.539783+0000","last_became_active":"2024-09-17T03:33:03.580241+0000","last_became_peered":"2024-09-17T03:33:03.580241+0000","last_unstale":"2024-09-17T03:33:04.539783+0000","last_undegraded":"2024-09-17T03:33:04.539783+0000","last_fullsized":"2024-09-17T03:33:04.539783+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:36.303054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541593+0000","last_change":"2024-09-17T03:33:03.551187+0000","last_active":"2024-09-17T03:33:04.541593+0000","last_peered":"2024-09-17T03:33:04.541593+0000","last_clean":"2024-09-17T03:33:04.541593+0000","last_became_active":"2024-09-17T03:33:03.550854+0000","last_became_peered":"2024-09-17T03:33:03.550854+0000","last_unstale":"2024-09-17T03:33:04.541593+0000","last_undegraded":"2024-09-17T03:33:04.541593+0000","last_fullsized":"2024-09-17T03:33:04.541593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:32.888001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427161+0000","last_change":"2024-09-17T03:33:03.562221+0000","last_active":"2024-09-17T03:33:05.427161+0000","last_peered":"2024-09-17T03:33:05.427161+0000","last_clean":"2024-09-17T03:33:05.427161+0000","last_became_active":"2024-09-17T03:33:03.561816+0000","last_became_peered":"2024-09-17T03:33:03.561816+0000","last_unstale":"2024-09-17T03:33:05.427161+0000","last_undegraded":"2024-09-17T03:33:05.427161+0000","last_fullsized":"2024-09-17T03:33:05.427161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.528483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547618+0000","last_change":"2024-09-17T03:33:03.580482+0000","last_active":"2024-09-17T03:33:04.547618+0000","last_peered":"2024-09-17T03:33:04.547618+0000","last_clean":"2024-09-17T03:33:04.547618+0000","last_became_active":"2024-09-17T03:33:03.579513+0000","last_became_peered":"2024-09-17T03:33:03.579513+0000","last_unstale":"2024-09-17T03:33:04.547618+0000","last_undegraded":"2024-09-17T03:33:04.547618+0000","last_fullsized":"2024-09-17T03:33:04.547618+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:22.208855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547571+0000","last_change":"2024-09-17T03:33:03.576842+0000","last_active":"2024-09-17T03:33:04.547571+0000","last_peered":"2024-09-17T03:33:04.547571+0000","last_clean":"2024-09-17T03:33:04.547571+0000","last_became_active":"2024-09-17T03:33:03.576572+0000","last_became_peered":"2024-09-17T03:33:03.576572+0000","last_unstale":"2024-09-17T03:33:04.547571+0000","last_undegraded":"2024-09-17T03:33:04.547571+0000","last_fullsized":"2024-09-17T03:33:04.547571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420470+0000","last_change":"2024-09-17T03:33:03.564312+0000","last_active":"2024-09-17T03:33:05.420470+0000","last_peered":"2024-09-17T03:33:05.420470+0000","last_clean":"2024-09-17T03:33:05.420470+0000","last_became_active":"2024-09-17T03:33:03.563925+0000","last_became_peered":"2024-09-17T03:33:03.563925+0000","last_unstale":"2024-09-17T03:33:05.420470+0000","last_undegraded":"2024-09-17T03:33:05.420470+0000","last_fullsized":"2024-09-17T03:33:05.420470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:00.239782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216739+0000","last_change":"2024-09-17T03:33:03.556879+0000","last_active":"2024-09-17T03:33:05.216739+0000","last_peered":"2024-09-17T03:33:05.216739+0000","last_clean":"2024-09-17T03:33:05.216739+0000","last_became_active":"2024-09-17T03:33:03.556579+0000","last_became_peered":"2024-09-17T03:33:03.556579+0000","last_unstale":"2024-09-17T03:33:05.216739+0000","last_undegraded":"2024-09-17T03:33:05.216739+0000","last_fullsized":"2024-09-17T03:33:05.216739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:00.787049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426696+0000","last_change":"2024-09-17T03:33:03.590158+0000","last_active":"2024-09-17T03:33:05.426696+0000","last_peered":"2024-09-17T03:33:05.426696+0000","last_clean":"2024-09-17T03:33:05.426696+0000","last_became_active":"2024-09-17T03:33:03.589751+0000","last_became_peered":"2024-09-17T03:33:03.589751+0000","last_unstale":"2024-09-17T03:33:05.426696+0000","last_undegraded":"2024-09-17T03:33:05.426696+0000","last_fullsized":"2024-09-17T03:33:05.426696+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:08.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548057+0000","last_change":"2024-09-17T03:33:03.559214+0000","last_active":"2024-09-17T03:33:04.548057+0000","last_peered":"2024-09-17T03:33:04.548057+0000","last_clean":"2024-09-17T03:33:04.548057+0000","last_became_active":"2024-09-17T03:33:03.558880+0000","last_became_peered":"2024-09-17T03:33:03.558880+0000","last_unstale":"2024-09-17T03:33:04.548057+0000","last_undegraded":"2024-09-17T03:33:04.548057+0000","last_fullsized":"2024-09-17T03:33:04.548057+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:32.880704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216475+0000","last_change":"2024-09-17T03:33:03.595048+0000","last_active":"2024-09-17T03:33:05.216475+0000","last_peered":"2024-09-17T03:33:05.216475+0000","last_clean":"2024-09-17T03:33:05.216475+0000","last_became_active":"2024-09-17T03:33:03.594649+0000","last_became_peered":"2024-09-17T03:33:03.594649+0000","last_unstale":"2024-09-17T03:33:05.216475+0000","last_undegraded":"2024-09-17T03:33:05.216475+0000","last_fullsized":"2024-09-17T03:33:05.216475+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:26.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541461+0000","last_change":"2024-09-17T03:33:03.546430+0000","last_active":"2024-09-17T03:33:04.541461+0000","last_peered":"2024-09-17T03:33:04.541461+0000","last_clean":"2024-09-17T03:33:04.541461+0000","last_became_active":"2024-09-17T03:33:03.546006+0000","last_became_peered":"2024-09-17T03:33:03.546006+0000","last_unstale":"2024-09-17T03:33:04.541461+0000","last_undegraded":"2024-09-17T03:33:04.541461+0000","last_fullsized":"2024-09-17T03:33:04.541461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535651+0000","last_change":"2024-09-17T03:33:02.535651+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535651+0000","last_undegraded":"2024-09-17T03:33:02.535651+0000","last_fullsized":"2024-09-17T03:33:02.535651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540556+0000","last_change":"2024-09-17T03:33:03.580895+0000","last_active":"2024-09-17T03:33:04.540556+0000","last_peered":"2024-09-17T03:33:04.540556+0000","last_clean":"2024-09-17T03:33:04.540556+0000","last_became_active":"2024-09-17T03:33:03.580561+0000","last_became_peered":"2024-09-17T03:33:03.580561+0000","last_unstale":"2024-09-17T03:33:04.540556+0000","last_undegraded":"2024-09-17T03:33:04.540556+0000","last_fullsized":"2024-09-17T03:33:04.540556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:59.846145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547563+0000","last_change":"2024-09-17T03:33:03.573694+0000","last_active":"2024-09-17T03:33:04.547563+0000","last_peered":"2024-09-17T03:33:04.547563+0000","last_clean":"2024-09-17T03:33:04.547563+0000","last_became_active":"2024-09-17T03:33:03.573199+0000","last_became_peered":"2024-09-17T03:33:03.573199+0000","last_unstale":"2024-09-17T03:33:04.547563+0000","last_undegraded":"2024-09-17T03:33:04.547563+0000","last_fullsized":"2024-09-17T03:33:04.547563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:14.793531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420606+0000","last_change":"2024-09-17T03:33:03.570549+0000","last_active":"2024-09-17T03:33:05.420606+0000","last_peered":"2024-09-17T03:33:05.420606+0000","last_clean":"2024-09-17T03:33:05.420606+0000","last_became_active":"2024-09-17T03:33:03.570244+0000","last_became_peered":"2024-09-17T03:33:03.570244+0000","last_unstale":"2024-09-17T03:33:05.420606+0000","last_undegraded":"2024-09-17T03:33:05.420606+0000","last_fullsized":"2024-09-17T03:33:05.420606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:57.566540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541670+0000","last_change":"2024-09-17T03:33:03.558416+0000","last_active":"2024-09-17T03:33:04.541670+0000","last_peered":"2024-09-17T03:33:04.541670+0000","last_clean":"2024-09-17T03:33:04.541670+0000","last_became_active":"2024-09-17T03:33:03.558033+0000","last_became_peered":"2024-09-17T03:33:03.558033+0000","last_unstale":"2024-09-17T03:33:04.541670+0000","last_undegraded":"2024-09-17T03:33:04.541670+0000","last_fullsized":"2024-09-17T03:33:04.541670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:07.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535461+0000","last_change":"2024-09-17T03:33:02.535461+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535461+0000","last_undegraded":"2024-09-17T03:33:02.535461+0000","last_fullsized":"2024-09-17T03:33:02.535461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216563+0000","last_change":"2024-09-17T03:33:03.556063+0000","last_active":"2024-09-17T03:33:05.216563+0000","last_peered":"2024-09-17T03:33:05.216563+0000","last_clean":"2024-09-17T03:33:05.216563+0000","last_became_active":"2024-09-17T03:33:03.555571+0000","last_became_peered":"2024-09-17T03:33:03.555571+0000","last_unstale":"2024-09-17T03:33:05.216563+0000","last_undegraded":"2024-09-17T03:33:05.216563+0000","last_fullsized":"2024-09-17T03:33:05.216563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:58.787401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534384+0000","last_change":"2024-09-17T03:33:02.534384+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534384+0000","last_undegraded":"2024-09-17T03:33:02.534384+0000","last_fullsized":"2024-09-17T03:33:02.534384+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420061+0000","last_change":"2024-09-17T03:33:03.572171+0000","last_active":"2024-09-17T03:33:05.420061+0000","last_peered":"2024-09-17T03:33:05.420061+0000","last_clean":"2024-09-17T03:33:05.420061+0000","last_became_active":"2024-09-17T03:33:03.571829+0000","last_became_peered":"2024-09-17T03:33:03.571829+0000","last_unstale":"2024-09-17T03:33:05.420061+0000","last_undegraded":"2024-09-17T03:33:05.420061+0000","last_fullsized":"2024-09-17T03:33:05.420061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:38.888214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419929+0000","last_change":"2024-09-17T03:33:03.572010+0000","last_active":"2024-09-17T03:33:05.419929+0000","last_peered":"2024-09-17T03:33:05.419929+0000","last_clean":"2024-09-17T03:33:05.419929+0000","last_became_active":"2024-09-17T03:33:03.571696+0000","last_became_peered":"2024-09-17T03:33:03.571696+0000","last_unstale":"2024-09-17T03:33:05.419929+0000","last_undegraded":"2024-09-17T03:33:05.419929+0000","last_fullsized":"2024-09-17T03:33:05.419929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.483130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547565+0000","last_change":"2024-09-17T03:33:03.589869+0000","last_active":"2024-09-17T03:33:04.547565+0000","last_peered":"2024-09-17T03:33:04.547565+0000","last_clean":"2024-09-17T03:33:04.547565+0000","last_became_active":"2024-09-17T03:33:03.589493+0000","last_became_peered":"2024-09-17T03:33:03.589493+0000","last_unstale":"2024-09-17T03:33:04.547565+0000","last_undegraded":"2024-09-17T03:33:04.547565+0000","last_fullsized":"2024-09-17T03:33:04.547565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.449496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426760+0000","last_change":"2024-09-17T03:33:03.576923+0000","last_active":"2024-09-17T03:33:05.426760+0000","last_peered":"2024-09-17T03:33:05.426760+0000","last_clean":"2024-09-17T03:33:05.426760+0000","last_became_active":"2024-09-17T03:33:03.576548+0000","last_became_peered":"2024-09-17T03:33:03.576548+0000","last_unstale":"2024-09-17T03:33:05.426760+0000","last_undegraded":"2024-09-17T03:33:05.426760+0000","last_fullsized":"2024-09-17T03:33:05.426760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:08.997404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420450+0000","last_change":"2024-09-17T03:33:03.583037+0000","last_active":"2024-09-17T03:33:05.420450+0000","last_peered":"2024-09-17T03:33:05.420450+0000","last_clean":"2024-09-17T03:33:05.420450+0000","last_became_active":"2024-09-17T03:33:03.582714+0000","last_became_peered":"2024-09-17T03:33:03.582714+0000","last_unstale":"2024-09-17T03:33:05.420450+0000","last_undegraded":"2024-09-17T03:33:05.420450+0000","last_fullsized":"2024-09-17T03:33:05.420450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:08.781666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548160+0000","last_change":"2024-09-17T03:33:03.567964+0000","last_active":"2024-09-17T03:33:04.548160+0000","last_peered":"2024-09-17T03:33:04.548160+0000","last_clean":"2024-09-17T03:33:04.548160+0000","last_became_active":"2024-09-17T03:33:03.567632+0000","last_became_peered":"2024-09-17T03:33:03.567632+0000","last_unstale":"2024-09-17T03:33:04.548160+0000","last_undegraded":"2024-09-17T03:33:04.548160+0000","last_fullsized":"2024-09-17T03:33:04.548160+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:39.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426889+0000","last_change":"2024-09-17T03:33:03.559826+0000","last_active":"2024-09-17T03:33:05.426889+0000","last_peered":"2024-09-17T03:33:05.426889+0000","last_clean":"2024-09-17T03:33:05.426889+0000","last_became_active":"2024-09-17T03:33:03.559504+0000","last_became_peered":"2024-09-17T03:33:03.559504+0000","last_unstale":"2024-09-17T03:33:05.426889+0000","last_undegraded":"2024-09-17T03:33:05.426889+0000","last_fullsized":"2024-09-17T03:33:05.426889+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:54.833105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420363+0000","last_change":"2024-09-17T03:33:03.580138+0000","last_active":"2024-09-17T03:33:05.420363+0000","last_peered":"2024-09-17T03:33:05.420363+0000","last_clean":"2024-09-17T03:33:05.420363+0000","last_became_active":"2024-09-17T03:33:03.579782+0000","last_became_peered":"2024-09-17T03:33:03.579782+0000","last_unstale":"2024-09-17T03:33:05.420363+0000","last_undegraded":"2024-09-17T03:33:05.420363+0000","last_fullsized":"2024-09-17T03:33:05.420363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:07.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.533906+0000","last_change":"2024-09-17T03:33:02.533906+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.533906+0000","last_undegraded":"2024-09-17T03:33:02.533906+0000","last_fullsized":"2024-09-17T03:33:02.533906+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541093+0000","last_change":"2024-09-17T03:33:03.565826+0000","last_active":"2024-09-17T03:33:04.541093+0000","last_peered":"2024-09-17T03:33:04.541093+0000","last_clean":"2024-09-17T03:33:04.541093+0000","last_became_active":"2024-09-17T03:33:03.557053+0000","last_became_peered":"2024-09-17T03:33:03.557053+0000","last_unstale":"2024-09-17T03:33:04.541093+0000","last_undegraded":"2024-09-17T03:33:04.541093+0000","last_fullsized":"2024-09-17T03:33:04.541093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:54.693438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539233+0000","last_change":"2024-09-17T03:33:03.581059+0000","last_active":"2024-09-17T03:33:04.539233+0000","last_peered":"2024-09-17T03:33:04.539233+0000","last_clean":"2024-09-17T03:33:04.539233+0000","last_became_active":"2024-09-17T03:33:03.580721+0000","last_became_peered":"2024-09-17T03:33:03.580721+0000","last_unstale":"2024-09-17T03:33:04.539233+0000","last_undegraded":"2024-09-17T03:33:04.539233+0000","last_fullsized":"2024-09-17T03:33:04.539233+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:12.767168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534628+0000","last_change":"2024-09-17T03:33:02.534628+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534628+0000","last_undegraded":"2024-09-17T03:33:02.534628+0000","last_fullsized":"2024-09-17T03:33:02.534628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420603+0000","last_change":"2024-09-17T03:33:02.543361+0000","last_active":"2024-09-17T03:33:05.420603+0000","last_peered":"2024-09-17T03:33:05.420603+0000","last_clean":"2024-09-17T03:33:05.420603+0000","last_became_active":"2024-09-17T03:33:02.543063+0000","last_became_peered":"2024-09-17T03:33:02.543063+0000","last_unstale":"2024-09-17T03:33:05.420603+0000","last_undegraded":"2024-09-17T03:33:05.420603+0000","last_fullsized":"2024-09-17T03:33:05.420603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:01.651709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541863+0000","last_change":"2024-09-17T03:33:03.556411+0000","last_active":"2024-09-17T03:33:04.541863+0000","last_peered":"2024-09-17T03:33:04.541863+0000","last_clean":"2024-09-17T03:33:04.541863+0000","last_became_active":"2024-09-17T03:33:03.555405+0000","last_became_peered":"2024-09-17T03:33:03.555405+0000","last_unstale":"2024-09-17T03:33:04.541863+0000","last_undegraded":"2024-09-17T03:33:04.541863+0000","last_fullsized":"2024-09-17T03:33:04.541863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:44.098643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547818+0000","last_change":"2024-09-17T03:33:02.563716+0000","last_active":"2024-09-17T03:33:04.547818+0000","last_peered":"2024-09-17T03:33:04.547818+0000","last_clean":"2024-09-17T03:33:04.547818+0000","last_became_active":"2024-09-17T03:33:02.563400+0000","last_became_peered":"2024-09-17T03:33:02.563400+0000","last_unstale":"2024-09-17T03:33:04.547818+0000","last_undegraded":"2024-09-17T03:33:04.547818+0000","last_fullsized":"2024-09-17T03:33:04.547818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540439+0000","last_change":"2024-09-17T03:33:03.587192+0000","last_active":"2024-09-17T03:33:04.540439+0000","last_peered":"2024-09-17T03:33:04.540439+0000","last_clean":"2024-09-17T03:33:04.540439+0000","last_became_active":"2024-09-17T03:33:03.586800+0000","last_became_peered":"2024-09-17T03:33:03.586800+0000","last_unstale":"2024-09-17T03:33:04.540439+0000","last_undegraded":"2024-09-17T03:33:04.540439+0000","last_fullsized":"2024-09-17T03:33:04.540439+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:04.696195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419965+0000","last_change":"2024-09-17T03:33:02.555846+0000","last_active":"2024-09-17T03:33:05.419965+0000","last_peered":"2024-09-17T03:33:05.419965+0000","last_clean":"2024-09-17T03:33:05.419965+0000","last_became_active":"2024-09-17T03:33:02.554535+0000","last_became_peered":"2024-09-17T03:33:02.554535+0000","last_unstale":"2024-09-17T03:33:05.419965+0000","last_undegraded":"2024-09-17T03:33:05.419965+0000","last_fullsized":"2024-09-17T03:33:05.419965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:20.329251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216814+0000","last_change":"2024-09-17T03:33:03.576310+0000","last_active":"2024-09-17T03:33:05.216814+0000","last_peered":"2024-09-17T03:33:05.216814+0000","last_clean":"2024-09-17T03:33:05.216814+0000","last_became_active":"2024-09-17T03:33:03.575941+0000","last_became_peered":"2024-09-17T03:33:03.575941+0000","last_unstale":"2024-09-17T03:33:05.216814+0000","last_undegraded":"2024-09-17T03:33:05.216814+0000","last_fullsized":"2024-09-17T03:33:05.216814+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:19.596987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419816+0000","last_change":"2024-09-17T03:33:02.539022+0000","last_active":"2024-09-17T03:33:05.419816+0000","last_peered":"2024-09-17T03:33:05.419816+0000","last_clean":"2024-09-17T03:33:05.419816+0000","last_became_active":"2024-09-17T03:33:02.538700+0000","last_became_peered":"2024-09-17T03:33:02.538700+0000","last_unstale":"2024-09-17T03:33:05.419816+0000","last_undegraded":"2024-09-17T03:33:05.419816+0000","last_fullsized":"2024-09-17T03:33:05.419816+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:31.500503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426743+0000","last_change":"2024-09-17T03:33:03.569261+0000","last_active":"2024-09-17T03:33:05.426743+0000","last_peered":"2024-09-17T03:33:05.426743+0000","last_clean":"2024-09-17T03:33:05.426743+0000","last_became_active":"2024-09-17T03:33:03.568877+0000","last_became_peered":"2024-09-17T03:33:03.568877+0000","last_unstale":"2024-09-17T03:33:05.426743+0000","last_undegraded":"2024-09-17T03:33:05.426743+0000","last_fullsized":"2024-09-17T03:33:05.426743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:04.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539368+0000","last_change":"2024-09-17T03:33:02.550110+0000","last_active":"2024-09-17T03:33:04.539368+0000","last_peered":"2024-09-17T03:33:04.539368+0000","last_clean":"2024-09-17T03:33:04.539368+0000","last_became_active":"2024-09-17T03:33:02.549795+0000","last_became_peered":"2024-09-17T03:33:02.549795+0000","last_unstale":"2024-09-17T03:33:04.539368+0000","last_undegraded":"2024-09-17T03:33:04.539368+0000","last_fullsized":"2024-09-17T03:33:04.539368+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:47.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547977+0000","last_change":"2024-09-17T03:33:03.564062+0000","last_active":"2024-09-17T03:33:04.547977+0000","last_peered":"2024-09-17T03:33:04.547977+0000","last_clean":"2024-09-17T03:33:04.547977+0000","last_became_active":"2024-09-17T03:33:03.563650+0000","last_became_peered":"2024-09-17T03:33:03.563650+0000","last_unstale":"2024-09-17T03:33:04.547977+0000","last_undegraded":"2024-09-17T03:33:04.547977+0000","last_fullsized":"2024-09-17T03:33:04.547977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:08.738111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419621+0000","last_change":"2024-09-17T03:33:02.549221+0000","last_active":"2024-09-17T03:33:05.419621+0000","last_peered":"2024-09-17T03:33:05.419621+0000","last_clean":"2024-09-17T03:33:05.419621+0000","last_became_active":"2024-09-17T03:33:02.548861+0000","last_became_peered":"2024-09-17T03:33:02.548861+0000","last_unstale":"2024-09-17T03:33:05.419621+0000","last_undegraded":"2024-09-17T03:33:05.419621+0000","last_fullsized":"2024-09-17T03:33:05.419621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540576+0000","last_change":"2024-09-17T03:33:03.581594+0000","last_active":"2024-09-17T03:33:04.540576+0000","last_peered":"2024-09-17T03:33:04.540576+0000","last_clean":"2024-09-17T03:33:04.540576+0000","last_became_active":"2024-09-17T03:33:03.581295+0000","last_became_peered":"2024-09-17T03:33:03.581295+0000","last_unstale":"2024-09-17T03:33:04.540576+0000","last_undegraded":"2024-09-17T03:33:04.540576+0000","last_fullsized":"2024-09-17T03:33:04.540576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:02.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426395+0000","last_change":"2024-09-17T03:33:02.555268+0000","last_active":"2024-09-17T03:33:05.426395+0000","last_peered":"2024-09-17T03:33:05.426395+0000","last_clean":"2024-09-17T03:33:05.426395+0000","last_became_active":"2024-09-17T03:33:02.554769+0000","last_became_peered":"2024-09-17T03:33:02.554769+0000","last_unstale":"2024-09-17T03:33:05.426395+0000","last_undegraded":"2024-09-17T03:33:05.426395+0000","last_fullsized":"2024-09-17T03:33:05.426395+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:38.916564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215985+0000","last_change":"2024-09-17T03:33:03.570252+0000","last_active":"2024-09-17T03:33:05.215985+0000","last_peered":"2024-09-17T03:33:05.215985+0000","last_clean":"2024-09-17T03:33:05.215985+0000","last_became_active":"2024-09-17T03:33:03.569899+0000","last_became_peered":"2024-09-17T03:33:03.569899+0000","last_unstale":"2024-09-17T03:33:05.215985+0000","last_undegraded":"2024-09-17T03:33:05.215985+0000","last_fullsized":"2024-09-17T03:33:05.215985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:27.095812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539125+0000","last_change":"2024-09-17T03:33:02.551914+0000","last_active":"2024-09-17T03:33:04.539125+0000","last_peered":"2024-09-17T03:33:04.539125+0000","last_clean":"2024-09-17T03:33:04.539125+0000","last_became_active":"2024-09-17T03:33:02.551576+0000","last_became_peered":"2024-09-17T03:33:02.551576+0000","last_unstale":"2024-09-17T03:33:04.539125+0000","last_undegraded":"2024-09-17T03:33:04.539125+0000","last_fullsized":"2024-09-17T03:33:04.539125+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:46.988395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420593+0000","last_change":"2024-09-17T03:33:03.558995+0000","last_active":"2024-09-17T03:33:05.420593+0000","last_peered":"2024-09-17T03:33:05.420593+0000","last_clean":"2024-09-17T03:33:05.420593+0000","last_became_active":"2024-09-17T03:33:03.558645+0000","last_became_peered":"2024-09-17T03:33:03.558645+0000","last_unstale":"2024-09-17T03:33:05.420593+0000","last_undegraded":"2024-09-17T03:33:05.420593+0000","last_fullsized":"2024-09-17T03:33:05.420593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:28.537475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539936+0000","last_change":"2024-09-17T03:33:02.556533+0000","last_active":"2024-09-17T03:33:04.539936+0000","last_peered":"2024-09-17T03:33:04.539936+0000","last_clean":"2024-09-17T03:33:04.539936+0000","last_became_active":"2024-09-17T03:33:02.556134+0000","last_became_peered":"2024-09-17T03:33:02.556134+0000","last_unstale":"2024-09-17T03:33:04.539936+0000","last_undegraded":"2024-09-17T03:33:04.539936+0000","last_fullsized":"2024-09-17T03:33:04.539936+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:08.763261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216780+0000","last_change":"2024-09-17T03:33:03.575625+0000","last_active":"2024-09-17T03:33:05.216780+0000","last_peered":"2024-09-17T03:33:05.216780+0000","last_clean":"2024-09-17T03:33:05.216780+0000","last_became_active":"2024-09-17T03:33:03.575239+0000","last_became_peered":"2024-09-17T03:33:03.575239+0000","last_unstale":"2024-09-17T03:33:05.216780+0000","last_undegraded":"2024-09-17T03:33:05.216780+0000","last_fullsized":"2024-09-17T03:33:05.216780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:57.701386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541519+0000","last_change":"2024-09-17T03:33:02.547247+0000","last_active":"2024-09-17T03:33:04.541519+0000","last_peered":"2024-09-17T03:33:04.541519+0000","last_clean":"2024-09-17T03:33:04.541519+0000","last_became_active":"2024-09-17T03:33:02.546932+0000","last_became_peered":"2024-09-17T03:33:02.546932+0000","last_unstale":"2024-09-17T03:33:04.541519+0000","last_undegraded":"2024-09-17T03:33:04.541519+0000","last_fullsized":"2024-09-17T03:33:04.541519+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.962710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427462+0000","last_change":"2024-09-17T03:33:03.569370+0000","last_active":"2024-09-17T03:33:05.427462+0000","last_peered":"2024-09-17T03:33:05.427462+0000","last_clean":"2024-09-17T03:33:05.427462+0000","last_became_active":"2024-09-17T03:33:03.569034+0000","last_became_peered":"2024-09-17T03:33:03.569034+0000","last_unstale":"2024-09-17T03:33:05.427462+0000","last_undegraded":"2024-09-17T03:33:05.427462+0000","last_fullsized":"2024-09-17T03:33:05.427462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:36.003958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548375+0000","last_change":"2024-09-17T03:33:02.557519+0000","last_active":"2024-09-17T03:33:04.548375+0000","last_peered":"2024-09-17T03:33:04.548375+0000","last_clean":"2024-09-17T03:33:04.548375+0000","last_became_active":"2024-09-17T03:33:02.557143+0000","last_became_peered":"2024-09-17T03:33:02.557143+0000","last_unstale":"2024-09-17T03:33:04.548375+0000","last_undegraded":"2024-09-17T03:33:04.548375+0000","last_fullsized":"2024-09-17T03:33:04.548375+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.298939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548299+0000","last_change":"2024-09-17T03:33:03.567932+0000","last_active":"2024-09-17T03:33:04.548299+0000","last_peered":"2024-09-17T03:33:04.548299+0000","last_clean":"2024-09-17T03:33:04.548299+0000","last_became_active":"2024-09-17T03:33:03.567611+0000","last_became_peered":"2024-09-17T03:33:03.567611+0000","last_unstale":"2024-09-17T03:33:04.548299+0000","last_undegraded":"2024-09-17T03:33:04.548299+0000","last_fullsized":"2024-09-17T03:33:04.548299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:15.466755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216540+0000","last_change":"2024-09-17T03:33:02.552966+0000","last_active":"2024-09-17T03:33:05.216540+0000","last_peered":"2024-09-17T03:33:05.216540+0000","last_clean":"2024-09-17T03:33:05.216540+0000","last_became_active":"2024-09-17T03:33:02.552610+0000","last_became_peered":"2024-09-17T03:33:02.552610+0000","last_unstale":"2024-09-17T03:33:05.216540+0000","last_undegraded":"2024-09-17T03:33:05.216540+0000","last_fullsized":"2024-09-17T03:33:05.216540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:57.033583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541014+0000","last_change":"2024-09-17T03:33:03.551601+0000","last_active":"2024-09-17T03:33:04.541014+0000","last_peered":"2024-09-17T03:33:04.541014+0000","last_clean":"2024-09-17T03:33:04.541014+0000","last_became_active":"2024-09-17T03:33:03.551024+0000","last_became_peered":"2024-09-17T03:33:03.551024+0000","last_unstale":"2024-09-17T03:33:04.541014+0000","last_undegraded":"2024-09-17T03:33:04.541014+0000","last_fullsized":"2024-09-17T03:33:04.541014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.061549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.547749+0000","last_change":"2024-09-17T03:33:02.539385+0000","last_active":"2024-09-17T03:33:02.547749+0000","last_peered":"2024-09-17T03:33:02.547749+0000","last_clean":"2024-09-17T03:33:02.547749+0000","last_became_active":"2024-09-17T03:33:02.539072+0000","last_became_peered":"2024-09-17T03:33:02.539072+0000","last_unstale":"2024-09-17T03:33:02.547749+0000","last_undegraded":"2024-09-17T03:33:02.547749+0000","last_fullsized":"2024-09-17T03:33:02.547749+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:33.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539523+0000","last_change":"2024-09-17T03:33:03.581118+0000","last_active":"2024-09-17T03:33:04.539523+0000","last_peered":"2024-09-17T03:33:04.539523+0000","last_clean":"2024-09-17T03:33:04.539523+0000","last_became_active":"2024-09-17T03:33:03.580758+0000","last_became_peered":"2024-09-17T03:33:03.580758+0000","last_unstale":"2024-09-17T03:33:04.539523+0000","last_undegraded":"2024-09-17T03:33:04.539523+0000","last_fullsized":"2024-09-17T03:33:04.539523+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:53.643044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.555053+0000","last_change":"2024-09-17T03:33:02.553011+0000","last_active":"2024-09-17T03:33:02.555053+0000","last_peered":"2024-09-17T03:33:02.555053+0000","last_clean":"2024-09-17T03:33:02.555053+0000","last_became_active":"2024-09-17T03:33:02.552649+0000","last_became_peered":"2024-09-17T03:33:02.552649+0000","last_unstale":"2024-09-17T03:33:02.555053+0000","last_undegraded":"2024-09-17T03:33:02.555053+0000","last_fullsized":"2024-09-17T03:33:02.555053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:31.641522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215951+0000","last_change":"2024-09-17T03:33:03.570295+0000","last_active":"2024-09-17T03:33:05.215951+0000","last_peered":"2024-09-17T03:33:05.215951+0000","last_clean":"2024-09-17T03:33:05.215951+0000","last_became_active":"2024-09-17T03:33:03.569927+0000","last_became_peered":"2024-09-17T03:33:03.569927+0000","last_unstale":"2024-09-17T03:33:05.215951+0000","last_undegraded":"2024-09-17T03:33:05.215951+0000","last_fullsized":"2024-09-17T03:33:05.215951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:36.691869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215879+0000","last_change":"2024-09-17T03:33:02.553261+0000","last_active":"2024-09-17T03:33:05.215879+0000","last_peered":"2024-09-17T03:33:05.215879+0000","last_clean":"2024-09-17T03:33:05.215879+0000","last_became_active":"2024-09-17T03:33:02.551020+0000","last_became_peered":"2024-09-17T03:33:02.551020+0000","last_unstale":"2024-09-17T03:33:05.215879+0000","last_undegraded":"2024-09-17T03:33:05.215879+0000","last_fullsized":"2024-09-17T03:33:05.215879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:01.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.531873+0000","last_change":"2024-09-17T03:33:02.531873+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.531873+0000","last_undegraded":"2024-09-17T03:33:02.531873+0000","last_fullsized":"2024-09-17T03:33:02.531873+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539246+0000","last_change":"2024-09-17T03:33:02.553687+0000","last_active":"2024-09-17T03:33:04.539246+0000","last_peered":"2024-09-17T03:33:04.539246+0000","last_clean":"2024-09-17T03:33:04.539246+0000","last_became_active":"2024-09-17T03:33:02.552355+0000","last_became_peered":"2024-09-17T03:33:02.552355+0000","last_unstale":"2024-09-17T03:33:04.539246+0000","last_undegraded":"2024-09-17T03:33:04.539246+0000","last_fullsized":"2024-09-17T03:33:04.539246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:35.198181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427004+0000","last_change":"2024-09-17T03:33:03.558620+0000","last_active":"2024-09-17T03:33:05.427004+0000","last_peered":"2024-09-17T03:33:05.427004+0000","last_clean":"2024-09-17T03:33:05.427004+0000","last_became_active":"2024-09-17T03:33:03.558274+0000","last_became_peered":"2024-09-17T03:33:03.558274+0000","last_unstale":"2024-09-17T03:33:05.427004+0000","last_undegraded":"2024-09-17T03:33:05.427004+0000","last_fullsized":"2024-09-17T03:33:05.427004+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216404+0000","last_change":"2024-09-17T03:33:02.550762+0000","last_active":"2024-09-17T03:33:05.216404+0000","last_peered":"2024-09-17T03:33:05.216404+0000","last_clean":"2024-09-17T03:33:05.216404+0000","last_became_active":"2024-09-17T03:33:02.550405+0000","last_became_peered":"2024-09-17T03:33:02.550405+0000","last_unstale":"2024-09-17T03:33:05.216404+0000","last_undegraded":"2024-09-17T03:33:05.216404+0000","last_fullsized":"2024-09-17T03:33:05.216404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426813+0000","last_change":"2024-09-17T03:33:03.572043+0000","last_active":"2024-09-17T03:33:05.426813+0000","last_peered":"2024-09-17T03:33:05.426813+0000","last_clean":"2024-09-17T03:33:05.426813+0000","last_became_active":"2024-09-17T03:33:03.570999+0000","last_became_peered":"2024-09-17T03:33:03.570999+0000","last_unstale":"2024-09-17T03:33:05.426813+0000","last_undegraded":"2024-09-17T03:33:05.426813+0000","last_fullsized":"2024-09-17T03:33:05.426813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.832675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426583+0000","last_change":"2024-09-17T03:33:02.546791+0000","last_active":"2024-09-17T03:33:05.426583+0000","last_peered":"2024-09-17T03:33:05.426583+0000","last_clean":"2024-09-17T03:33:05.426583+0000","last_became_active":"2024-09-17T03:33:02.546444+0000","last_became_peered":"2024-09-17T03:33:02.546444+0000","last_unstale":"2024-09-17T03:33:05.426583+0000","last_undegraded":"2024-09-17T03:33:05.426583+0000","last_fullsized":"2024-09-17T03:33:05.426583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.093303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548398+0000","last_change":"2024-09-17T03:33:03.561978+0000","last_active":"2024-09-17T03:33:04.548398+0000","last_peered":"2024-09-17T03:33:04.548398+0000","last_clean":"2024-09-17T03:33:04.548398+0000","last_became_active":"2024-09-17T03:33:03.561642+0000","last_became_peered":"2024-09-17T03:33:03.561642+0000","last_unstale":"2024-09-17T03:33:04.548398+0000","last_undegraded":"2024-09-17T03:33:04.548398+0000","last_fullsized":"2024-09-17T03:33:04.548398+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419887+0000","last_change":"2024-09-17T03:33:02.540951+0000","last_active":"2024-09-17T03:33:05.419887+0000","last_peered":"2024-09-17T03:33:05.419887+0000","last_clean":"2024-09-17T03:33:05.419887+0000","last_became_active":"2024-09-17T03:33:02.540486+0000","last_became_peered":"2024-09-17T03:33:02.540486+0000","last_unstale":"2024-09-17T03:33:05.419887+0000","last_undegraded":"2024-09-17T03:33:05.419887+0000","last_fullsized":"2024-09-17T03:33:05.419887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:48.319608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530771+0000","last_change":"2024-09-17T03:33:02.530771+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530771+0000","last_undegraded":"2024-09-17T03:33:02.530771+0000","last_fullsized":"2024-09-17T03:33:02.530771+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419837+0000","last_change":"2024-09-17T03:33:02.550968+0000","last_active":"2024-09-17T03:33:05.419837+0000","last_peered":"2024-09-17T03:33:05.419837+0000","last_clean":"2024-09-17T03:33:05.419837+0000","last_became_active":"2024-09-17T03:33:02.550661+0000","last_became_peered":"2024-09-17T03:33:02.550661+0000","last_unstale":"2024-09-17T03:33:05.419837+0000","last_undegraded":"2024-09-17T03:33:05.419837+0000","last_fullsized":"2024-09-17T03:33:05.419837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:42.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530834+0000","last_change":"2024-09-17T03:33:02.530834+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530834+0000","last_undegraded":"2024-09-17T03:33:02.530834+0000","last_fullsized":"2024-09-17T03:33:02.530834+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426662+0000","last_change":"2024-09-17T03:33:02.556914+0000","last_active":"2024-09-17T03:33:05.426662+0000","last_peered":"2024-09-17T03:33:05.426662+0000","last_clean":"2024-09-17T03:33:05.426662+0000","last_became_active":"2024-09-17T03:33:02.556176+0000","last_became_peered":"2024-09-17T03:33:02.556176+0000","last_unstale":"2024-09-17T03:33:05.426662+0000","last_undegraded":"2024-09-17T03:33:05.426662+0000","last_fullsized":"2024-09-17T03:33:05.426662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:19.803397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530656+0000","last_change":"2024-09-17T03:33:02.530656+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530656+0000","last_undegraded":"2024-09-17T03:33:02.530656+0000","last_fullsized":"2024-09-17T03:33:02.530656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426418+0000","last_change":"2024-09-17T03:33:02.554769+0000","last_active":"2024-09-17T03:33:05.426418+0000","last_peered":"2024-09-17T03:33:05.426418+0000","last_clean":"2024-09-17T03:33:05.426418+0000","last_became_active":"2024-09-17T03:33:02.554148+0000","last_became_peered":"2024-09-17T03:33:02.554148+0000","last_unstale":"2024-09-17T03:33:05.426418+0000","last_undegraded":"2024-09-17T03:33:05.426418+0000","last_fullsized":"2024-09-17T03:33:05.426418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:47:37.685954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426425+0000","last_change":"2024-09-17T03:33:03.587346+0000","last_active":"2024-09-17T03:33:05.426425+0000","last_peered":"2024-09-17T03:33:05.426425+0000","last_clean":"2024-09-17T03:33:05.426425+0000","last_became_active":"2024-09-17T03:33:03.587021+0000","last_became_peered":"2024-09-17T03:33:03.587021+0000","last_unstale":"2024-09-17T03:33:05.426425+0000","last_undegraded":"2024-09-17T03:33:05.426425+0000","last_fullsized":"2024-09-17T03:33:05.426425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:16.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419857+0000","last_change":"2024-09-17T03:33:02.552916+0000","last_active":"2024-09-17T03:33:05.419857+0000","last_peered":"2024-09-17T03:33:05.419857+0000","last_clean":"2024-09-17T03:33:05.419857+0000","last_became_active":"2024-09-17T03:33:02.551707+0000","last_became_peered":"2024-09-17T03:33:02.551707+0000","last_unstale":"2024-09-17T03:33:05.419857+0000","last_undegraded":"2024-09-17T03:33:05.419857+0000","last_fullsized":"2024-09-17T03:33:05.419857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:28.925660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420744+0000","last_change":"2024-09-17T03:33:03.572163+0000","last_active":"2024-09-17T03:33:05.420744+0000","last_peered":"2024-09-17T03:33:05.420744+0000","last_clean":"2024-09-17T03:33:05.420744+0000","last_became_active":"2024-09-17T03:33:03.571635+0000","last_became_peered":"2024-09-17T03:33:03.571635+0000","last_unstale":"2024-09-17T03:33:05.420744+0000","last_undegraded":"2024-09-17T03:33:05.420744+0000","last_fullsized":"2024-09-17T03:33:05.420744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:54.558593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427248+0000","last_change":"2024-09-17T03:33:02.547235+0000","last_active":"2024-09-17T03:33:05.427248+0000","last_peered":"2024-09-17T03:33:05.427248+0000","last_clean":"2024-09-17T03:33:05.427248+0000","last_became_active":"2024-09-17T03:33:02.546057+0000","last_became_peered":"2024-09-17T03:33:02.546057+0000","last_unstale":"2024-09-17T03:33:05.427248+0000","last_undegraded":"2024-09-17T03:33:05.427248+0000","last_fullsized":"2024-09-17T03:33:05.427248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:57.628934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539866+0000","last_change":"2024-09-17T03:33:03.580070+0000","last_active":"2024-09-17T03:33:04.539866+0000","last_peered":"2024-09-17T03:33:04.539866+0000","last_clean":"2024-09-17T03:33:04.539866+0000","last_became_active":"2024-09-17T03:33:03.579536+0000","last_became_peered":"2024-09-17T03:33:03.579536+0000","last_unstale":"2024-09-17T03:33:04.539866+0000","last_undegraded":"2024-09-17T03:33:04.539866+0000","last_fullsized":"2024-09-17T03:33:04.539866+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:15.857981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420763+0000","last_change":"2024-09-17T03:33:02.549424+0000","last_active":"2024-09-17T03:33:05.420763+0000","last_peered":"2024-09-17T03:33:05.420763+0000","last_clean":"2024-09-17T03:33:05.420763+0000","last_became_active":"2024-09-17T03:33:02.548964+0000","last_became_peered":"2024-09-17T03:33:02.548964+0000","last_unstale":"2024-09-17T03:33:05.420763+0000","last_undegraded":"2024-09-17T03:33:05.420763+0000","last_fullsized":"2024-09-17T03:33:05.420763+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:18.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540218+0000","last_change":"2024-09-17T03:33:03.583513+0000","last_active":"2024-09-17T03:33:04.540218+0000","last_peered":"2024-09-17T03:33:04.540218+0000","last_clean":"2024-09-17T03:33:04.540218+0000","last_became_active":"2024-09-17T03:33:03.583186+0000","last_became_peered":"2024-09-17T03:33:03.583186+0000","last_unstale":"2024-09-17T03:33:04.540218+0000","last_undegraded":"2024-09-17T03:33:04.540218+0000","last_fullsized":"2024-09-17T03:33:04.540218+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:49.099577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420208+0000","last_change":"2024-09-17T03:33:02.550099+0000","last_active":"2024-09-17T03:33:05.420208+0000","last_peered":"2024-09-17T03:33:05.420208+0000","last_clean":"2024-09-17T03:33:05.420208+0000","last_became_active":"2024-09-17T03:33:02.549761+0000","last_became_peered":"2024-09-17T03:33:02.549761+0000","last_unstale":"2024-09-17T03:33:05.420208+0000","last_undegraded":"2024-09-17T03:33:05.420208+0000","last_fullsized":"2024-09-17T03:33:05.420208+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:58.843636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426573+0000","last_change":"2024-09-17T03:33:03.561146+0000","last_active":"2024-09-17T03:33:05.426573+0000","last_peered":"2024-09-17T03:33:05.426573+0000","last_clean":"2024-09-17T03:33:05.426573+0000","last_became_active":"2024-09-17T03:33:03.560840+0000","last_became_peered":"2024-09-17T03:33:03.560840+0000","last_unstale":"2024-09-17T03:33:05.426573+0000","last_undegraded":"2024-09-17T03:33:05.426573+0000","last_fullsized":"2024-09-17T03:33:05.426573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:39.264426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426977+0000","last_change":"2024-09-17T03:33:02.550579+0000","last_active":"2024-09-17T03:33:05.426977+0000","last_peered":"2024-09-17T03:33:05.426977+0000","last_clean":"2024-09-17T03:33:05.426977+0000","last_became_active":"2024-09-17T03:33:02.550238+0000","last_became_peered":"2024-09-17T03:33:02.550238+0000","last_unstale":"2024-09-17T03:33:05.426977+0000","last_undegraded":"2024-09-17T03:33:05.426977+0000","last_fullsized":"2024-09-17T03:33:05.426977+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:42.312117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.528893+0000","last_change":"2024-09-17T03:33:02.528893+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.528893+0000","last_undegraded":"2024-09-17T03:33:02.528893+0000","last_fullsized":"2024-09-17T03:33:02.528893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420094+0000","last_change":"2024-09-17T03:33:02.554041+0000","last_active":"2024-09-17T03:33:05.420094+0000","last_peered":"2024-09-17T03:33:05.420094+0000","last_clean":"2024-09-17T03:33:05.420094+0000","last_became_active":"2024-09-17T03:33:02.553699+0000","last_became_peered":"2024-09-17T03:33:02.553699+0000","last_unstale":"2024-09-17T03:33:05.420094+0000","last_undegraded":"2024-09-17T03:33:05.420094+0000","last_fullsized":"2024-09-17T03:33:05.420094+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216039+0000","last_change":"2024-09-17T03:33:03.563718+0000","last_active":"2024-09-17T03:33:05.216039+0000","last_peered":"2024-09-17T03:33:05.216039+0000","last_clean":"2024-09-17T03:33:05.216039+0000","last_became_active":"2024-09-17T03:33:03.563409+0000","last_became_peered":"2024-09-17T03:33:03.563409+0000","last_unstale":"2024-09-17T03:33:05.216039+0000","last_undegraded":"2024-09-17T03:33:05.216039+0000","last_fullsized":"2024-09-17T03:33:05.216039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:38.241017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426089+0000","last_change":"2024-09-17T03:33:02.556605+0000","last_active":"2024-09-17T03:33:05.426089+0000","last_peered":"2024-09-17T03:33:05.426089+0000","last_clean":"2024-09-17T03:33:05.426089+0000","last_became_active":"2024-09-17T03:33:02.556146+0000","last_became_peered":"2024-09-17T03:33:02.556146+0000","last_unstale":"2024-09-17T03:33:05.426089+0000","last_undegraded":"2024-09-17T03:33:05.426089+0000","last_fullsized":"2024-09-17T03:33:05.426089+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:36.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530028+0000","last_change":"2024-09-17T03:33:02.530028+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530028+0000","last_undegraded":"2024-09-17T03:33:02.530028+0000","last_fullsized":"2024-09-17T03:33:02.530028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540887+0000","last_change":"2024-09-17T03:33:02.547114+0000","last_active":"2024-09-17T03:33:04.540887+0000","last_peered":"2024-09-17T03:33:04.540887+0000","last_clean":"2024-09-17T03:33:04.540887+0000","last_became_active":"2024-09-17T03:33:02.546778+0000","last_became_peered":"2024-09-17T03:33:02.546778+0000","last_unstale":"2024-09-17T03:33:04.540887+0000","last_undegraded":"2024-09-17T03:33:04.540887+0000","last_fullsized":"2024-09-17T03:33:04.540887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:19.611348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426304+0000","last_change":"2024-09-17T03:33:03.561358+0000","last_active":"2024-09-17T03:33:05.426304+0000","last_peered":"2024-09-17T03:33:05.426304+0000","last_clean":"2024-09-17T03:33:05.426304+0000","last_became_active":"2024-09-17T03:33:03.561027+0000","last_became_peered":"2024-09-17T03:33:03.561027+0000","last_unstale":"2024-09-17T03:33:05.426304+0000","last_undegraded":"2024-09-17T03:33:05.426304+0000","last_fullsized":"2024-09-17T03:33:05.426304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:04.965218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.553304+0000","last_change":"2024-09-17T03:33:02.544930+0000","last_active":"2024-09-17T03:33:02.553304+0000","last_peered":"2024-09-17T03:33:02.553304+0000","last_clean":"2024-09-17T03:33:02.553304+0000","last_became_active":"2024-09-17T03:33:02.542640+0000","last_became_peered":"2024-09-17T03:33:02.542640+0000","last_unstale":"2024-09-17T03:33:02.553304+0000","last_undegraded":"2024-09-17T03:33:02.553304+0000","last_fullsized":"2024-09-17T03:33:02.553304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:12.140218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530977+0000","last_change":"2024-09-17T03:33:02.530977+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530977+0000","last_undegraded":"2024-09-17T03:33:02.530977+0000","last_fullsized":"2024-09-17T03:33:02.530977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547713+0000","last_change":"2024-09-17T03:33:02.558759+0000","last_active":"2024-09-17T03:33:04.547713+0000","last_peered":"2024-09-17T03:33:04.547713+0000","last_clean":"2024-09-17T03:33:04.547713+0000","last_became_active":"2024-09-17T03:33:02.558503+0000","last_became_peered":"2024-09-17T03:33:02.558503+0000","last_unstale":"2024-09-17T03:33:04.547713+0000","last_undegraded":"2024-09-17T03:33:04.547713+0000","last_fullsized":"2024-09-17T03:33:04.547713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547648+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:33:04.547648+0000","last_peered":"2024-09-17T03:33:04.547648+0000","last_clean":"2024-09-17T03:33:04.547648+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:33:04.547648+0000","last_undegraded":"2024-09-17T03:33:04.547648+0000","last_fullsized":"2024-09-17T03:33:04.547648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127344,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244496,"statfs":{"total":96636764160,"available":96506363904,"internally_reserved":0,"allocated":1855488,"data_stored":1132666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136484,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235356,"statfs":{"total":96636764160,"available":96497004544,"internally_reserved":0,"allocated":1261568,"data_stored":542838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144816,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227024,"statfs":{"total":96636764160,"available":96488472576,"internally_reserved":0,"allocated":1273856,"data_stored":543922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130924,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240916,"statfs":{"total":96636764160,"available":96502697984,"internally_reserved":0,"allocated":1269760,"data_stored":543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136452,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235388,"statfs":{"total":96636764160,"available":96497037312,"internally_reserved":0,"allocated":1216512,"data_stored":522836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119016,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252824,"statfs":{"total":96636764160,"available":96514891776,"internally_reserved":0,"allocated":1851392,"data_stored":1133136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82324,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289516,"statfs":{"total":96636764160,"available":96552464384,"internally_reserved":0,"allocated":1245184,"data_stored":535956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140640,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231200,"statfs":{"total":96636764160,"available":96492748800,"internally_reserved":0,"allocated":1253376,"data_stored":542186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:33:07.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 -- 172.21.15.46:0/3095495921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 msgr2=0x7fca8c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 --2- 172.21.15.46:0/3095495921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fca8c003ed0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fca98009b40 tx=0x7fca9801d250 comp rx=0 tx=0).stop 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 -- 172.21.15.46:0/3095495921 shutdown_connections 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 --2- 172.21.15.46:0/3095495921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fca8c003ed0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 -- 172.21.15.46:0/3095495921 >> 172.21.15.46:0/3095495921 conn(0x7fcaa80b2040 msgr2=0x7fcaa80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 -- 172.21.15.46:0/3095495921 shutdown_connections 2024-09-17T03:33:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.956+0000 7fcaafc00640 1 -- 172.21.15.46:0/3095495921 wait complete. 2024-09-17T03:33:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 Processor -- start 2024-09-17T03:33:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 -- start start 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fcaa81f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaa8159eb0 0x7fcaa8145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcaa81459b0 0x7fcaa81f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80b3ae0 con 0x7fca8c003a60 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80b3960 con 0x7fcaa8159eb0 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa80b3c60 con 0x7fcaa81459b0 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fcaa81f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fcaa81f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57076/0 (socket says 172.21.15.46:57076) 2024-09-17T03:33:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 -- 172.21.15.46:0/873030036 learned_addr learned my addr 172.21.15.46:0/873030036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaadc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcaa81459b0 0x7fcaa81f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fca9fe00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaa8159eb0 0x7fcaa8145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 -- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcaa81459b0 msgr2=0x7fcaa81f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcaa81459b0 0x7fcaa81f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 -- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaa8159eb0 msgr2=0x7fcaa8145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaa8159eb0 0x7fcaa8145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 -- 172.21.15.46:0/873030036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa80730d0 con 0x7fca8c003a60 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fca9fe00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaa8159eb0 0x7fcaa8145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaadc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcaa81459b0 0x7fcaa81f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaad200640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fcaa81f6ad0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fca9801d080 tx=0x7fca980050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980bb030 con 0x7fca8c003a60 2024-09-17T03:33:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca98004900 con 0x7fca8c003a60 2024-09-17T03:33:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98004d50 con 0x7fca8c003a60 2024-09-17T03:33:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa8145dc0 con 0x7fca8c003a60 2024-09-17T03:33:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.960+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81f7010 con 0x7fca8c003a60 2024-09-17T03:33:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.964+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa80b41c0 con 0x7fca8c003a60 2024-09-17T03:33:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.964+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fca980be030 con 0x7fca8c003a60 2024-09-17T03:33:07.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.964+0000 7fcaa5600640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fca84074910 0x7fca84076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.964+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fca9812e8c0 con 0x7fca8c003a60 2024-09-17T03:33:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.964+0000 7fca9fe00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fca84074910 0x7fca84076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:07.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.964+0000 7fca9fe00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fca84074910 0x7fca84076e20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fca90006fd0 tx=0x7fca90006b90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:07.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:07.968+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcaa8145dc0 con 0x7fca8c003a60 2024-09-17T03:33:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.140+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcaa806b540 con 0x7fca84074910 2024-09-17T03:33:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.144+0000 7fcaa5600640 1 -- 172.21.15.46:0/873030036 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251442 (secure 0 0 0) 0x7fca90012500 con 0x7fca84074910 2024-09-17T03:33:08.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:08.150 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:33:08.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fca84074910 msgr2=0x7fca84076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fca84074910 0x7fca84076e20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fca90006fd0 tx=0x7fca90006b90 comp rx=0 tx=0).stop 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 msgr2=0x7fcaa81f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fcaa81f6ad0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fca9801d080 tx=0x7fca980050a0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 shutdown_connections 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fca84074910 0x7fca84076e20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcaa81459b0 0x7fcaa81f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcaa8159eb0 0x7fcaa8145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 --2- 172.21.15.46:0/873030036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c003a60 0x7fcaa81f6ad0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 >> 172.21.15.46:0/873030036 conn(0x7fcaa80b2040 msgr2=0x7fcaa80b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 shutdown_connections 2024-09-17T03:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.148+0000 7fcaafc00640 1 -- 172.21.15.46:0/873030036 wait complete. 2024-09-17T03:33:08.171 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-17T03:33:07.511563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":143,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1018000,"kb_used_data":10964,"kb_used_omap":12,"kb_used_meta":1006835,"kb_avail":753956720,"statfs":{"total":773094113280,"available":772051681280,"internally_reserved":0,"allocated":11227136,"data_stored":5497360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1030999600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991341"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539642+0000","last_change":"2024-09-17T03:33:03.581362+0000","last_active":"2024-09-17T03:33:04.539642+0000","last_peered":"2024-09-17T03:33:04.539642+0000","last_clean":"2024-09-17T03:33:04.539642+0000","last_became_active":"2024-09-17T03:33:03.580475+0000","last_became_peered":"2024-09-17T03:33:03.580475+0000","last_unstale":"2024-09-17T03:33:04.539642+0000","last_undegraded":"2024-09-17T03:33:04.539642+0000","last_fullsized":"2024-09-17T03:33:04.539642+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:11.925403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420228+0000","last_change":"2024-09-17T03:33:03.572549+0000","last_active":"2024-09-17T03:33:05.420228+0000","last_peered":"2024-09-17T03:33:05.420228+0000","last_clean":"2024-09-17T03:33:05.420228+0000","last_became_active":"2024-09-17T03:33:03.572232+0000","last_became_peered":"2024-09-17T03:33:03.572232+0000","last_unstale":"2024-09-17T03:33:05.420228+0000","last_undegraded":"2024-09-17T03:33:05.420228+0000","last_fullsized":"2024-09-17T03:33:05.420228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:46.104711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539783+0000","last_change":"2024-09-17T03:33:03.580570+0000","last_active":"2024-09-17T03:33:04.539783+0000","last_peered":"2024-09-17T03:33:04.539783+0000","last_clean":"2024-09-17T03:33:04.539783+0000","last_became_active":"2024-09-17T03:33:03.580241+0000","last_became_peered":"2024-09-17T03:33:03.580241+0000","last_unstale":"2024-09-17T03:33:04.539783+0000","last_undegraded":"2024-09-17T03:33:04.539783+0000","last_fullsized":"2024-09-17T03:33:04.539783+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:36.303054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541593+0000","last_change":"2024-09-17T03:33:03.551187+0000","last_active":"2024-09-17T03:33:04.541593+0000","last_peered":"2024-09-17T03:33:04.541593+0000","last_clean":"2024-09-17T03:33:04.541593+0000","last_became_active":"2024-09-17T03:33:03.550854+0000","last_became_peered":"2024-09-17T03:33:03.550854+0000","last_unstale":"2024-09-17T03:33:04.541593+0000","last_undegraded":"2024-09-17T03:33:04.541593+0000","last_fullsized":"2024-09-17T03:33:04.541593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:32.888001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427161+0000","last_change":"2024-09-17T03:33:03.562221+0000","last_active":"2024-09-17T03:33:05.427161+0000","last_peered":"2024-09-17T03:33:05.427161+0000","last_clean":"2024-09-17T03:33:05.427161+0000","last_became_active":"2024-09-17T03:33:03.561816+0000","last_became_peered":"2024-09-17T03:33:03.561816+0000","last_unstale":"2024-09-17T03:33:05.427161+0000","last_undegraded":"2024-09-17T03:33:05.427161+0000","last_fullsized":"2024-09-17T03:33:05.427161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.528483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547618+0000","last_change":"2024-09-17T03:33:03.580482+0000","last_active":"2024-09-17T03:33:04.547618+0000","last_peered":"2024-09-17T03:33:04.547618+0000","last_clean":"2024-09-17T03:33:04.547618+0000","last_became_active":"2024-09-17T03:33:03.579513+0000","last_became_peered":"2024-09-17T03:33:03.579513+0000","last_unstale":"2024-09-17T03:33:04.547618+0000","last_undegraded":"2024-09-17T03:33:04.547618+0000","last_fullsized":"2024-09-17T03:33:04.547618+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:22.208855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547571+0000","last_change":"2024-09-17T03:33:03.576842+0000","last_active":"2024-09-17T03:33:04.547571+0000","last_peered":"2024-09-17T03:33:04.547571+0000","last_clean":"2024-09-17T03:33:04.547571+0000","last_became_active":"2024-09-17T03:33:03.576572+0000","last_became_peered":"2024-09-17T03:33:03.576572+0000","last_unstale":"2024-09-17T03:33:04.547571+0000","last_undegraded":"2024-09-17T03:33:04.547571+0000","last_fullsized":"2024-09-17T03:33:04.547571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420470+0000","last_change":"2024-09-17T03:33:03.564312+0000","last_active":"2024-09-17T03:33:05.420470+0000","last_peered":"2024-09-17T03:33:05.420470+0000","last_clean":"2024-09-17T03:33:05.420470+0000","last_became_active":"2024-09-17T03:33:03.563925+0000","last_became_peered":"2024-09-17T03:33:03.563925+0000","last_unstale":"2024-09-17T03:33:05.420470+0000","last_undegraded":"2024-09-17T03:33:05.420470+0000","last_fullsized":"2024-09-17T03:33:05.420470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:00.239782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216739+0000","last_change":"2024-09-17T03:33:03.556879+0000","last_active":"2024-09-17T03:33:05.216739+0000","last_peered":"2024-09-17T03:33:05.216739+0000","last_clean":"2024-09-17T03:33:05.216739+0000","last_became_active":"2024-09-17T03:33:03.556579+0000","last_became_peered":"2024-09-17T03:33:03.556579+0000","last_unstale":"2024-09-17T03:33:05.216739+0000","last_undegraded":"2024-09-17T03:33:05.216739+0000","last_fullsized":"2024-09-17T03:33:05.216739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:00.787049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426696+0000","last_change":"2024-09-17T03:33:03.590158+0000","last_active":"2024-09-17T03:33:05.426696+0000","last_peered":"2024-09-17T03:33:05.426696+0000","last_clean":"2024-09-17T03:33:05.426696+0000","last_became_active":"2024-09-17T03:33:03.589751+0000","last_became_peered":"2024-09-17T03:33:03.589751+0000","last_unstale":"2024-09-17T03:33:05.426696+0000","last_undegraded":"2024-09-17T03:33:05.426696+0000","last_fullsized":"2024-09-17T03:33:05.426696+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:08.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548057+0000","last_change":"2024-09-17T03:33:03.559214+0000","last_active":"2024-09-17T03:33:04.548057+0000","last_peered":"2024-09-17T03:33:04.548057+0000","last_clean":"2024-09-17T03:33:04.548057+0000","last_became_active":"2024-09-17T03:33:03.558880+0000","last_became_peered":"2024-09-17T03:33:03.558880+0000","last_unstale":"2024-09-17T03:33:04.548057+0000","last_undegraded":"2024-09-17T03:33:04.548057+0000","last_fullsized":"2024-09-17T03:33:04.548057+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:32.880704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216475+0000","last_change":"2024-09-17T03:33:03.595048+0000","last_active":"2024-09-17T03:33:05.216475+0000","last_peered":"2024-09-17T03:33:05.216475+0000","last_clean":"2024-09-17T03:33:05.216475+0000","last_became_active":"2024-09-17T03:33:03.594649+0000","last_became_peered":"2024-09-17T03:33:03.594649+0000","last_unstale":"2024-09-17T03:33:05.216475+0000","last_undegraded":"2024-09-17T03:33:05.216475+0000","last_fullsized":"2024-09-17T03:33:05.216475+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:26.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541461+0000","last_change":"2024-09-17T03:33:03.546430+0000","last_active":"2024-09-17T03:33:04.541461+0000","last_peered":"2024-09-17T03:33:04.541461+0000","last_clean":"2024-09-17T03:33:04.541461+0000","last_became_active":"2024-09-17T03:33:03.546006+0000","last_became_peered":"2024-09-17T03:33:03.546006+0000","last_unstale":"2024-09-17T03:33:04.541461+0000","last_undegraded":"2024-09-17T03:33:04.541461+0000","last_fullsized":"2024-09-17T03:33:04.541461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535651+0000","last_change":"2024-09-17T03:33:02.535651+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535651+0000","last_undegraded":"2024-09-17T03:33:02.535651+0000","last_fullsized":"2024-09-17T03:33:02.535651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540556+0000","last_change":"2024-09-17T03:33:03.580895+0000","last_active":"2024-09-17T03:33:04.540556+0000","last_peered":"2024-09-17T03:33:04.540556+0000","last_clean":"2024-09-17T03:33:04.540556+0000","last_became_active":"2024-09-17T03:33:03.580561+0000","last_became_peered":"2024-09-17T03:33:03.580561+0000","last_unstale":"2024-09-17T03:33:04.540556+0000","last_undegraded":"2024-09-17T03:33:04.540556+0000","last_fullsized":"2024-09-17T03:33:04.540556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:59.846145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547563+0000","last_change":"2024-09-17T03:33:03.573694+0000","last_active":"2024-09-17T03:33:04.547563+0000","last_peered":"2024-09-17T03:33:04.547563+0000","last_clean":"2024-09-17T03:33:04.547563+0000","last_became_active":"2024-09-17T03:33:03.573199+0000","last_became_peered":"2024-09-17T03:33:03.573199+0000","last_unstale":"2024-09-17T03:33:04.547563+0000","last_undegraded":"2024-09-17T03:33:04.547563+0000","last_fullsized":"2024-09-17T03:33:04.547563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:14.793531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420606+0000","last_change":"2024-09-17T03:33:03.570549+0000","last_active":"2024-09-17T03:33:05.420606+0000","last_peered":"2024-09-17T03:33:05.420606+0000","last_clean":"2024-09-17T03:33:05.420606+0000","last_became_active":"2024-09-17T03:33:03.570244+0000","last_became_peered":"2024-09-17T03:33:03.570244+0000","last_unstale":"2024-09-17T03:33:05.420606+0000","last_undegraded":"2024-09-17T03:33:05.420606+0000","last_fullsized":"2024-09-17T03:33:05.420606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:57.566540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541670+0000","last_change":"2024-09-17T03:33:03.558416+0000","last_active":"2024-09-17T03:33:04.541670+0000","last_peered":"2024-09-17T03:33:04.541670+0000","last_clean":"2024-09-17T03:33:04.541670+0000","last_became_active":"2024-09-17T03:33:03.558033+0000","last_became_peered":"2024-09-17T03:33:03.558033+0000","last_unstale":"2024-09-17T03:33:04.541670+0000","last_undegraded":"2024-09-17T03:33:04.541670+0000","last_fullsized":"2024-09-17T03:33:04.541670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:07.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535461+0000","last_change":"2024-09-17T03:33:02.535461+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535461+0000","last_undegraded":"2024-09-17T03:33:02.535461+0000","last_fullsized":"2024-09-17T03:33:02.535461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216563+0000","last_change":"2024-09-17T03:33:03.556063+0000","last_active":"2024-09-17T03:33:05.216563+0000","last_peered":"2024-09-17T03:33:05.216563+0000","last_clean":"2024-09-17T03:33:05.216563+0000","last_became_active":"2024-09-17T03:33:03.555571+0000","last_became_peered":"2024-09-17T03:33:03.555571+0000","last_unstale":"2024-09-17T03:33:05.216563+0000","last_undegraded":"2024-09-17T03:33:05.216563+0000","last_fullsized":"2024-09-17T03:33:05.216563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:58.787401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534384+0000","last_change":"2024-09-17T03:33:02.534384+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534384+0000","last_undegraded":"2024-09-17T03:33:02.534384+0000","last_fullsized":"2024-09-17T03:33:02.534384+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420061+0000","last_change":"2024-09-17T03:33:03.572171+0000","last_active":"2024-09-17T03:33:05.420061+0000","last_peered":"2024-09-17T03:33:05.420061+0000","last_clean":"2024-09-17T03:33:05.420061+0000","last_became_active":"2024-09-17T03:33:03.571829+0000","last_became_peered":"2024-09-17T03:33:03.571829+0000","last_unstale":"2024-09-17T03:33:05.420061+0000","last_undegraded":"2024-09-17T03:33:05.420061+0000","last_fullsized":"2024-09-17T03:33:05.420061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:38.888214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419929+0000","last_change":"2024-09-17T03:33:03.572010+0000","last_active":"2024-09-17T03:33:05.419929+0000","last_peered":"2024-09-17T03:33:05.419929+0000","last_clean":"2024-09-17T03:33:05.419929+0000","last_became_active":"2024-09-17T03:33:03.571696+0000","last_became_peered":"2024-09-17T03:33:03.571696+0000","last_unstale":"2024-09-17T03:33:05.419929+0000","last_undegraded":"2024-09-17T03:33:05.419929+0000","last_fullsized":"2024-09-17T03:33:05.419929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.483130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547565+0000","last_change":"2024-09-17T03:33:03.589869+0000","last_active":"2024-09-17T03:33:04.547565+0000","last_peered":"2024-09-17T03:33:04.547565+0000","last_clean":"2024-09-17T03:33:04.547565+0000","last_became_active":"2024-09-17T03:33:03.589493+0000","last_became_peered":"2024-09-17T03:33:03.589493+0000","last_unstale":"2024-09-17T03:33:04.547565+0000","last_undegraded":"2024-09-17T03:33:04.547565+0000","last_fullsized":"2024-09-17T03:33:04.547565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.449496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426760+0000","last_change":"2024-09-17T03:33:03.576923+0000","last_active":"2024-09-17T03:33:05.426760+0000","last_peered":"2024-09-17T03:33:05.426760+0000","last_clean":"2024-09-17T03:33:05.426760+0000","last_became_active":"2024-09-17T03:33:03.576548+0000","last_became_peered":"2024-09-17T03:33:03.576548+0000","last_unstale":"2024-09-17T03:33:05.426760+0000","last_undegraded":"2024-09-17T03:33:05.426760+0000","last_fullsized":"2024-09-17T03:33:05.426760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:08.997404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420450+0000","last_change":"2024-09-17T03:33:03.583037+0000","last_active":"2024-09-17T03:33:05.420450+0000","last_peered":"2024-09-17T03:33:05.420450+0000","last_clean":"2024-09-17T03:33:05.420450+0000","last_became_active":"2024-09-17T03:33:03.582714+0000","last_became_peered":"2024-09-17T03:33:03.582714+0000","last_unstale":"2024-09-17T03:33:05.420450+0000","last_undegraded":"2024-09-17T03:33:05.420450+0000","last_fullsized":"2024-09-17T03:33:05.420450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:08.781666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548160+0000","last_change":"2024-09-17T03:33:03.567964+0000","last_active":"2024-09-17T03:33:04.548160+0000","last_peered":"2024-09-17T03:33:04.548160+0000","last_clean":"2024-09-17T03:33:04.548160+0000","last_became_active":"2024-09-17T03:33:03.567632+0000","last_became_peered":"2024-09-17T03:33:03.567632+0000","last_unstale":"2024-09-17T03:33:04.548160+0000","last_undegraded":"2024-09-17T03:33:04.548160+0000","last_fullsized":"2024-09-17T03:33:04.548160+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:39.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426889+0000","last_change":"2024-09-17T03:33:03.559826+0000","last_active":"2024-09-17T03:33:05.426889+0000","last_peered":"2024-09-17T03:33:05.426889+0000","last_clean":"2024-09-17T03:33:05.426889+0000","last_became_active":"2024-09-17T03:33:03.559504+0000","last_became_peered":"2024-09-17T03:33:03.559504+0000","last_unstale":"2024-09-17T03:33:05.426889+0000","last_undegraded":"2024-09-17T03:33:05.426889+0000","last_fullsized":"2024-09-17T03:33:05.426889+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:54.833105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420363+0000","last_change":"2024-09-17T03:33:03.580138+0000","last_active":"2024-09-17T03:33:05.420363+0000","last_peered":"2024-09-17T03:33:05.420363+0000","last_clean":"2024-09-17T03:33:05.420363+0000","last_became_active":"2024-09-17T03:33:03.579782+0000","last_became_peered":"2024-09-17T03:33:03.579782+0000","last_unstale":"2024-09-17T03:33:05.420363+0000","last_undegraded":"2024-09-17T03:33:05.420363+0000","last_fullsized":"2024-09-17T03:33:05.420363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:07.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.533906+0000","last_change":"2024-09-17T03:33:02.533906+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.533906+0000","last_undegraded":"2024-09-17T03:33:02.533906+0000","last_fullsized":"2024-09-17T03:33:02.533906+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541093+0000","last_change":"2024-09-17T03:33:03.565826+0000","last_active":"2024-09-17T03:33:04.541093+0000","last_peered":"2024-09-17T03:33:04.541093+0000","last_clean":"2024-09-17T03:33:04.541093+0000","last_became_active":"2024-09-17T03:33:03.557053+0000","last_became_peered":"2024-09-17T03:33:03.557053+0000","last_unstale":"2024-09-17T03:33:04.541093+0000","last_undegraded":"2024-09-17T03:33:04.541093+0000","last_fullsized":"2024-09-17T03:33:04.541093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:54.693438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539233+0000","last_change":"2024-09-17T03:33:03.581059+0000","last_active":"2024-09-17T03:33:04.539233+0000","last_peered":"2024-09-17T03:33:04.539233+0000","last_clean":"2024-09-17T03:33:04.539233+0000","last_became_active":"2024-09-17T03:33:03.580721+0000","last_became_peered":"2024-09-17T03:33:03.580721+0000","last_unstale":"2024-09-17T03:33:04.539233+0000","last_undegraded":"2024-09-17T03:33:04.539233+0000","last_fullsized":"2024-09-17T03:33:04.539233+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:12.767168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534628+0000","last_change":"2024-09-17T03:33:02.534628+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534628+0000","last_undegraded":"2024-09-17T03:33:02.534628+0000","last_fullsized":"2024-09-17T03:33:02.534628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420603+0000","last_change":"2024-09-17T03:33:02.543361+0000","last_active":"2024-09-17T03:33:05.420603+0000","last_peered":"2024-09-17T03:33:05.420603+0000","last_clean":"2024-09-17T03:33:05.420603+0000","last_became_active":"2024-09-17T03:33:02.543063+0000","last_became_peered":"2024-09-17T03:33:02.543063+0000","last_unstale":"2024-09-17T03:33:05.420603+0000","last_undegraded":"2024-09-17T03:33:05.420603+0000","last_fullsized":"2024-09-17T03:33:05.420603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:01.651709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541863+0000","last_change":"2024-09-17T03:33:03.556411+0000","last_active":"2024-09-17T03:33:04.541863+0000","last_peered":"2024-09-17T03:33:04.541863+0000","last_clean":"2024-09-17T03:33:04.541863+0000","last_became_active":"2024-09-17T03:33:03.555405+0000","last_became_peered":"2024-09-17T03:33:03.555405+0000","last_unstale":"2024-09-17T03:33:04.541863+0000","last_undegraded":"2024-09-17T03:33:04.541863+0000","last_fullsized":"2024-09-17T03:33:04.541863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:44.098643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547818+0000","last_change":"2024-09-17T03:33:02.563716+0000","last_active":"2024-09-17T03:33:04.547818+0000","last_peered":"2024-09-17T03:33:04.547818+0000","last_clean":"2024-09-17T03:33:04.547818+0000","last_became_active":"2024-09-17T03:33:02.563400+0000","last_became_peered":"2024-09-17T03:33:02.563400+0000","last_unstale":"2024-09-17T03:33:04.547818+0000","last_undegraded":"2024-09-17T03:33:04.547818+0000","last_fullsized":"2024-09-17T03:33:04.547818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540439+0000","last_change":"2024-09-17T03:33:03.587192+0000","last_active":"2024-09-17T03:33:04.540439+0000","last_peered":"2024-09-17T03:33:04.540439+0000","last_clean":"2024-09-17T03:33:04.540439+0000","last_became_active":"2024-09-17T03:33:03.586800+0000","last_became_peered":"2024-09-17T03:33:03.586800+0000","last_unstale":"2024-09-17T03:33:04.540439+0000","last_undegraded":"2024-09-17T03:33:04.540439+0000","last_fullsized":"2024-09-17T03:33:04.540439+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:04.696195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419965+0000","last_change":"2024-09-17T03:33:02.555846+0000","last_active":"2024-09-17T03:33:05.419965+0000","last_peered":"2024-09-17T03:33:05.419965+0000","last_clean":"2024-09-17T03:33:05.419965+0000","last_became_active":"2024-09-17T03:33:02.554535+0000","last_became_peered":"2024-09-17T03:33:02.554535+0000","last_unstale":"2024-09-17T03:33:05.419965+0000","last_undegraded":"2024-09-17T03:33:05.419965+0000","last_fullsized":"2024-09-17T03:33:05.419965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:20.329251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216814+0000","last_change":"2024-09-17T03:33:03.576310+0000","last_active":"2024-09-17T03:33:05.216814+0000","last_peered":"2024-09-17T03:33:05.216814+0000","last_clean":"2024-09-17T03:33:05.216814+0000","last_became_active":"2024-09-17T03:33:03.575941+0000","last_became_peered":"2024-09-17T03:33:03.575941+0000","last_unstale":"2024-09-17T03:33:05.216814+0000","last_undegraded":"2024-09-17T03:33:05.216814+0000","last_fullsized":"2024-09-17T03:33:05.216814+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:19.596987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419816+0000","last_change":"2024-09-17T03:33:02.539022+0000","last_active":"2024-09-17T03:33:05.419816+0000","last_peered":"2024-09-17T03:33:05.419816+0000","last_clean":"2024-09-17T03:33:05.419816+0000","last_became_active":"2024-09-17T03:33:02.538700+0000","last_became_peered":"2024-09-17T03:33:02.538700+0000","last_unstale":"2024-09-17T03:33:05.419816+0000","last_undegraded":"2024-09-17T03:33:05.419816+0000","last_fullsized":"2024-09-17T03:33:05.419816+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:31.500503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426743+0000","last_change":"2024-09-17T03:33:03.569261+0000","last_active":"2024-09-17T03:33:05.426743+0000","last_peered":"2024-09-17T03:33:05.426743+0000","last_clean":"2024-09-17T03:33:05.426743+0000","last_became_active":"2024-09-17T03:33:03.568877+0000","last_became_peered":"2024-09-17T03:33:03.568877+0000","last_unstale":"2024-09-17T03:33:05.426743+0000","last_undegraded":"2024-09-17T03:33:05.426743+0000","last_fullsized":"2024-09-17T03:33:05.426743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:04.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539368+0000","last_change":"2024-09-17T03:33:02.550110+0000","last_active":"2024-09-17T03:33:04.539368+0000","last_peered":"2024-09-17T03:33:04.539368+0000","last_clean":"2024-09-17T03:33:04.539368+0000","last_became_active":"2024-09-17T03:33:02.549795+0000","last_became_peered":"2024-09-17T03:33:02.549795+0000","last_unstale":"2024-09-17T03:33:04.539368+0000","last_undegraded":"2024-09-17T03:33:04.539368+0000","last_fullsized":"2024-09-17T03:33:04.539368+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:47.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547977+0000","last_change":"2024-09-17T03:33:03.564062+0000","last_active":"2024-09-17T03:33:04.547977+0000","last_peered":"2024-09-17T03:33:04.547977+0000","last_clean":"2024-09-17T03:33:04.547977+0000","last_became_active":"2024-09-17T03:33:03.563650+0000","last_became_peered":"2024-09-17T03:33:03.563650+0000","last_unstale":"2024-09-17T03:33:04.547977+0000","last_undegraded":"2024-09-17T03:33:04.547977+0000","last_fullsized":"2024-09-17T03:33:04.547977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:08.738111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419621+0000","last_change":"2024-09-17T03:33:02.549221+0000","last_active":"2024-09-17T03:33:05.419621+0000","last_peered":"2024-09-17T03:33:05.419621+0000","last_clean":"2024-09-17T03:33:05.419621+0000","last_became_active":"2024-09-17T03:33:02.548861+0000","last_became_peered":"2024-09-17T03:33:02.548861+0000","last_unstale":"2024-09-17T03:33:05.419621+0000","last_undegraded":"2024-09-17T03:33:05.419621+0000","last_fullsized":"2024-09-17T03:33:05.419621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540576+0000","last_change":"2024-09-17T03:33:03.581594+0000","last_active":"2024-09-17T03:33:04.540576+0000","last_peered":"2024-09-17T03:33:04.540576+0000","last_clean":"2024-09-17T03:33:04.540576+0000","last_became_active":"2024-09-17T03:33:03.581295+0000","last_became_peered":"2024-09-17T03:33:03.581295+0000","last_unstale":"2024-09-17T03:33:04.540576+0000","last_undegraded":"2024-09-17T03:33:04.540576+0000","last_fullsized":"2024-09-17T03:33:04.540576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:02.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426395+0000","last_change":"2024-09-17T03:33:02.555268+0000","last_active":"2024-09-17T03:33:05.426395+0000","last_peered":"2024-09-17T03:33:05.426395+0000","last_clean":"2024-09-17T03:33:05.426395+0000","last_became_active":"2024-09-17T03:33:02.554769+0000","last_became_peered":"2024-09-17T03:33:02.554769+0000","last_unstale":"2024-09-17T03:33:05.426395+0000","last_undegraded":"2024-09-17T03:33:05.426395+0000","last_fullsized":"2024-09-17T03:33:05.426395+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:38.916564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215985+0000","last_change":"2024-09-17T03:33:03.570252+0000","last_active":"2024-09-17T03:33:05.215985+0000","last_peered":"2024-09-17T03:33:05.215985+0000","last_clean":"2024-09-17T03:33:05.215985+0000","last_became_active":"2024-09-17T03:33:03.569899+0000","last_became_peered":"2024-09-17T03:33:03.569899+0000","last_unstale":"2024-09-17T03:33:05.215985+0000","last_undegraded":"2024-09-17T03:33:05.215985+0000","last_fullsized":"2024-09-17T03:33:05.215985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:27.095812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539125+0000","last_change":"2024-09-17T03:33:02.551914+0000","last_active":"2024-09-17T03:33:04.539125+0000","last_peered":"2024-09-17T03:33:04.539125+0000","last_clean":"2024-09-17T03:33:04.539125+0000","last_became_active":"2024-09-17T03:33:02.551576+0000","last_became_peered":"2024-09-17T03:33:02.551576+0000","last_unstale":"2024-09-17T03:33:04.539125+0000","last_undegraded":"2024-09-17T03:33:04.539125+0000","last_fullsized":"2024-09-17T03:33:04.539125+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:46.988395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420593+0000","last_change":"2024-09-17T03:33:03.558995+0000","last_active":"2024-09-17T03:33:05.420593+0000","last_peered":"2024-09-17T03:33:05.420593+0000","last_clean":"2024-09-17T03:33:05.420593+0000","last_became_active":"2024-09-17T03:33:03.558645+0000","last_became_peered":"2024-09-17T03:33:03.558645+0000","last_unstale":"2024-09-17T03:33:05.420593+0000","last_undegraded":"2024-09-17T03:33:05.420593+0000","last_fullsized":"2024-09-17T03:33:05.420593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:28.537475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539936+0000","last_change":"2024-09-17T03:33:02.556533+0000","last_active":"2024-09-17T03:33:04.539936+0000","last_peered":"2024-09-17T03:33:04.539936+0000","last_clean":"2024-09-17T03:33:04.539936+0000","last_became_active":"2024-09-17T03:33:02.556134+0000","last_became_peered":"2024-09-17T03:33:02.556134+0000","last_unstale":"2024-09-17T03:33:04.539936+0000","last_undegraded":"2024-09-17T03:33:04.539936+0000","last_fullsized":"2024-09-17T03:33:04.539936+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:08.763261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216780+0000","last_change":"2024-09-17T03:33:03.575625+0000","last_active":"2024-09-17T03:33:05.216780+0000","last_peered":"2024-09-17T03:33:05.216780+0000","last_clean":"2024-09-17T03:33:05.216780+0000","last_became_active":"2024-09-17T03:33:03.575239+0000","last_became_peered":"2024-09-17T03:33:03.575239+0000","last_unstale":"2024-09-17T03:33:05.216780+0000","last_undegraded":"2024-09-17T03:33:05.216780+0000","last_fullsized":"2024-09-17T03:33:05.216780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:57.701386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541519+0000","last_change":"2024-09-17T03:33:02.547247+0000","last_active":"2024-09-17T03:33:04.541519+0000","last_peered":"2024-09-17T03:33:04.541519+0000","last_clean":"2024-09-17T03:33:04.541519+0000","last_became_active":"2024-09-17T03:33:02.546932+0000","last_became_peered":"2024-09-17T03:33:02.546932+0000","last_unstale":"2024-09-17T03:33:04.541519+0000","last_undegraded":"2024-09-17T03:33:04.541519+0000","last_fullsized":"2024-09-17T03:33:04.541519+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.962710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427462+0000","last_change":"2024-09-17T03:33:03.569370+0000","last_active":"2024-09-17T03:33:05.427462+0000","last_peered":"2024-09-17T03:33:05.427462+0000","last_clean":"2024-09-17T03:33:05.427462+0000","last_became_active":"2024-09-17T03:33:03.569034+0000","last_became_peered":"2024-09-17T03:33:03.569034+0000","last_unstale":"2024-09-17T03:33:05.427462+0000","last_undegraded":"2024-09-17T03:33:05.427462+0000","last_fullsized":"2024-09-17T03:33:05.427462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:36.003958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548375+0000","last_change":"2024-09-17T03:33:02.557519+0000","last_active":"2024-09-17T03:33:04.548375+0000","last_peered":"2024-09-17T03:33:04.548375+0000","last_clean":"2024-09-17T03:33:04.548375+0000","last_became_active":"2024-09-17T03:33:02.557143+0000","last_became_peered":"2024-09-17T03:33:02.557143+0000","last_unstale":"2024-09-17T03:33:04.548375+0000","last_undegraded":"2024-09-17T03:33:04.548375+0000","last_fullsized":"2024-09-17T03:33:04.548375+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.298939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548299+0000","last_change":"2024-09-17T03:33:03.567932+0000","last_active":"2024-09-17T03:33:04.548299+0000","last_peered":"2024-09-17T03:33:04.548299+0000","last_clean":"2024-09-17T03:33:04.548299+0000","last_became_active":"2024-09-17T03:33:03.567611+0000","last_became_peered":"2024-09-17T03:33:03.567611+0000","last_unstale":"2024-09-17T03:33:04.548299+0000","last_undegraded":"2024-09-17T03:33:04.548299+0000","last_fullsized":"2024-09-17T03:33:04.548299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:15.466755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216540+0000","last_change":"2024-09-17T03:33:02.552966+0000","last_active":"2024-09-17T03:33:05.216540+0000","last_peered":"2024-09-17T03:33:05.216540+0000","last_clean":"2024-09-17T03:33:05.216540+0000","last_became_active":"2024-09-17T03:33:02.552610+0000","last_became_peered":"2024-09-17T03:33:02.552610+0000","last_unstale":"2024-09-17T03:33:05.216540+0000","last_undegraded":"2024-09-17T03:33:05.216540+0000","last_fullsized":"2024-09-17T03:33:05.216540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:57.033583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541014+0000","last_change":"2024-09-17T03:33:03.551601+0000","last_active":"2024-09-17T03:33:04.541014+0000","last_peered":"2024-09-17T03:33:04.541014+0000","last_clean":"2024-09-17T03:33:04.541014+0000","last_became_active":"2024-09-17T03:33:03.551024+0000","last_became_peered":"2024-09-17T03:33:03.551024+0000","last_unstale":"2024-09-17T03:33:04.541014+0000","last_undegraded":"2024-09-17T03:33:04.541014+0000","last_fullsized":"2024-09-17T03:33:04.541014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.061549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.547749+0000","last_change":"2024-09-17T03:33:02.539385+0000","last_active":"2024-09-17T03:33:02.547749+0000","last_peered":"2024-09-17T03:33:02.547749+0000","last_clean":"2024-09-17T03:33:02.547749+0000","last_became_active":"2024-09-17T03:33:02.539072+0000","last_became_peered":"2024-09-17T03:33:02.539072+0000","last_unstale":"2024-09-17T03:33:02.547749+0000","last_undegraded":"2024-09-17T03:33:02.547749+0000","last_fullsized":"2024-09-17T03:33:02.547749+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:33.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539523+0000","last_change":"2024-09-17T03:33:03.581118+0000","last_active":"2024-09-17T03:33:04.539523+0000","last_peered":"2024-09-17T03:33:04.539523+0000","last_clean":"2024-09-17T03:33:04.539523+0000","last_became_active":"2024-09-17T03:33:03.580758+0000","last_became_peered":"2024-09-17T03:33:03.580758+0000","last_unstale":"2024-09-17T03:33:04.539523+0000","last_undegraded":"2024-09-17T03:33:04.539523+0000","last_fullsized":"2024-09-17T03:33:04.539523+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:53.643044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.555053+0000","last_change":"2024-09-17T03:33:02.553011+0000","last_active":"2024-09-17T03:33:02.555053+0000","last_peered":"2024-09-17T03:33:02.555053+0000","last_clean":"2024-09-17T03:33:02.555053+0000","last_became_active":"2024-09-17T03:33:02.552649+0000","last_became_peered":"2024-09-17T03:33:02.552649+0000","last_unstale":"2024-09-17T03:33:02.555053+0000","last_undegraded":"2024-09-17T03:33:02.555053+0000","last_fullsized":"2024-09-17T03:33:02.555053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:31.641522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215951+0000","last_change":"2024-09-17T03:33:03.570295+0000","last_active":"2024-09-17T03:33:05.215951+0000","last_peered":"2024-09-17T03:33:05.215951+0000","last_clean":"2024-09-17T03:33:05.215951+0000","last_became_active":"2024-09-17T03:33:03.569927+0000","last_became_peered":"2024-09-17T03:33:03.569927+0000","last_unstale":"2024-09-17T03:33:05.215951+0000","last_undegraded":"2024-09-17T03:33:05.215951+0000","last_fullsized":"2024-09-17T03:33:05.215951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:36.691869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215879+0000","last_change":"2024-09-17T03:33:02.553261+0000","last_active":"2024-09-17T03:33:05.215879+0000","last_peered":"2024-09-17T03:33:05.215879+0000","last_clean":"2024-09-17T03:33:05.215879+0000","last_became_active":"2024-09-17T03:33:02.551020+0000","last_became_peered":"2024-09-17T03:33:02.551020+0000","last_unstale":"2024-09-17T03:33:05.215879+0000","last_undegraded":"2024-09-17T03:33:05.215879+0000","last_fullsized":"2024-09-17T03:33:05.215879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:01.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.531873+0000","last_change":"2024-09-17T03:33:02.531873+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.531873+0000","last_undegraded":"2024-09-17T03:33:02.531873+0000","last_fullsized":"2024-09-17T03:33:02.531873+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539246+0000","last_change":"2024-09-17T03:33:02.553687+0000","last_active":"2024-09-17T03:33:04.539246+0000","last_peered":"2024-09-17T03:33:04.539246+0000","last_clean":"2024-09-17T03:33:04.539246+0000","last_became_active":"2024-09-17T03:33:02.552355+0000","last_became_peered":"2024-09-17T03:33:02.552355+0000","last_unstale":"2024-09-17T03:33:04.539246+0000","last_undegraded":"2024-09-17T03:33:04.539246+0000","last_fullsized":"2024-09-17T03:33:04.539246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:35.198181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427004+0000","last_change":"2024-09-17T03:33:03.558620+0000","last_active":"2024-09-17T03:33:05.427004+0000","last_peered":"2024-09-17T03:33:05.427004+0000","last_clean":"2024-09-17T03:33:05.427004+0000","last_became_active":"2024-09-17T03:33:03.558274+0000","last_became_peered":"2024-09-17T03:33:03.558274+0000","last_unstale":"2024-09-17T03:33:05.427004+0000","last_undegraded":"2024-09-17T03:33:05.427004+0000","last_fullsized":"2024-09-17T03:33:05.427004+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216404+0000","last_change":"2024-09-17T03:33:02.550762+0000","last_active":"2024-09-17T03:33:05.216404+0000","last_peered":"2024-09-17T03:33:05.216404+0000","last_clean":"2024-09-17T03:33:05.216404+0000","last_became_active":"2024-09-17T03:33:02.550405+0000","last_became_peered":"2024-09-17T03:33:02.550405+0000","last_unstale":"2024-09-17T03:33:05.216404+0000","last_undegraded":"2024-09-17T03:33:05.216404+0000","last_fullsized":"2024-09-17T03:33:05.216404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426813+0000","last_change":"2024-09-17T03:33:03.572043+0000","last_active":"2024-09-17T03:33:05.426813+0000","last_peered":"2024-09-17T03:33:05.426813+0000","last_clean":"2024-09-17T03:33:05.426813+0000","last_became_active":"2024-09-17T03:33:03.570999+0000","last_became_peered":"2024-09-17T03:33:03.570999+0000","last_unstale":"2024-09-17T03:33:05.426813+0000","last_undegraded":"2024-09-17T03:33:05.426813+0000","last_fullsized":"2024-09-17T03:33:05.426813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.832675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426583+0000","last_change":"2024-09-17T03:33:02.546791+0000","last_active":"2024-09-17T03:33:05.426583+0000","last_peered":"2024-09-17T03:33:05.426583+0000","last_clean":"2024-09-17T03:33:05.426583+0000","last_became_active":"2024-09-17T03:33:02.546444+0000","last_became_peered":"2024-09-17T03:33:02.546444+0000","last_unstale":"2024-09-17T03:33:05.426583+0000","last_undegraded":"2024-09-17T03:33:05.426583+0000","last_fullsized":"2024-09-17T03:33:05.426583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.093303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548398+0000","last_change":"2024-09-17T03:33:03.561978+0000","last_active":"2024-09-17T03:33:04.548398+0000","last_peered":"2024-09-17T03:33:04.548398+0000","last_clean":"2024-09-17T03:33:04.548398+0000","last_became_active":"2024-09-17T03:33:03.561642+0000","last_became_peered":"2024-09-17T03:33:03.561642+0000","last_unstale":"2024-09-17T03:33:04.548398+0000","last_undegraded":"2024-09-17T03:33:04.548398+0000","last_fullsized":"2024-09-17T03:33:04.548398+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419887+0000","last_change":"2024-09-17T03:33:02.540951+0000","last_active":"2024-09-17T03:33:05.419887+0000","last_peered":"2024-09-17T03:33:05.419887+0000","last_clean":"2024-09-17T03:33:05.419887+0000","last_became_active":"2024-09-17T03:33:02.540486+0000","last_became_peered":"2024-09-17T03:33:02.540486+0000","last_unstale":"2024-09-17T03:33:05.419887+0000","last_undegraded":"2024-09-17T03:33:05.419887+0000","last_fullsized":"2024-09-17T03:33:05.419887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:48.319608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530771+0000","last_change":"2024-09-17T03:33:02.530771+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530771+0000","last_undegraded":"2024-09-17T03:33:02.530771+0000","last_fullsized":"2024-09-17T03:33:02.530771+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419837+0000","last_change":"2024-09-17T03:33:02.550968+0000","last_active":"2024-09-17T03:33:05.419837+0000","last_peered":"2024-09-17T03:33:05.419837+0000","last_clean":"2024-09-17T03:33:05.419837+0000","last_became_active":"2024-09-17T03:33:02.550661+0000","last_became_peered":"2024-09-17T03:33:02.550661+0000","last_unstale":"2024-09-17T03:33:05.419837+0000","last_undegraded":"2024-09-17T03:33:05.419837+0000","last_fullsized":"2024-09-17T03:33:05.419837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:42.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530834+0000","last_change":"2024-09-17T03:33:02.530834+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530834+0000","last_undegraded":"2024-09-17T03:33:02.530834+0000","last_fullsized":"2024-09-17T03:33:02.530834+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426662+0000","last_change":"2024-09-17T03:33:02.556914+0000","last_active":"2024-09-17T03:33:05.426662+0000","last_peered":"2024-09-17T03:33:05.426662+0000","last_clean":"2024-09-17T03:33:05.426662+0000","last_became_active":"2024-09-17T03:33:02.556176+0000","last_became_peered":"2024-09-17T03:33:02.556176+0000","last_unstale":"2024-09-17T03:33:05.426662+0000","last_undegraded":"2024-09-17T03:33:05.426662+0000","last_fullsized":"2024-09-17T03:33:05.426662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:19.803397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530656+0000","last_change":"2024-09-17T03:33:02.530656+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530656+0000","last_undegraded":"2024-09-17T03:33:02.530656+0000","last_fullsized":"2024-09-17T03:33:02.530656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426418+0000","last_change":"2024-09-17T03:33:02.554769+0000","last_active":"2024-09-17T03:33:05.426418+0000","last_peered":"2024-09-17T03:33:05.426418+0000","last_clean":"2024-09-17T03:33:05.426418+0000","last_became_active":"2024-09-17T03:33:02.554148+0000","last_became_peered":"2024-09-17T03:33:02.554148+0000","last_unstale":"2024-09-17T03:33:05.426418+0000","last_undegraded":"2024-09-17T03:33:05.426418+0000","last_fullsized":"2024-09-17T03:33:05.426418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:47:37.685954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426425+0000","last_change":"2024-09-17T03:33:03.587346+0000","last_active":"2024-09-17T03:33:05.426425+0000","last_peered":"2024-09-17T03:33:05.426425+0000","last_clean":"2024-09-17T03:33:05.426425+0000","last_became_active":"2024-09-17T03:33:03.587021+0000","last_became_peered":"2024-09-17T03:33:03.587021+0000","last_unstale":"2024-09-17T03:33:05.426425+0000","last_undegraded":"2024-09-17T03:33:05.426425+0000","last_fullsized":"2024-09-17T03:33:05.426425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:16.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419857+0000","last_change":"2024-09-17T03:33:02.552916+0000","last_active":"2024-09-17T03:33:05.419857+0000","last_peered":"2024-09-17T03:33:05.419857+0000","last_clean":"2024-09-17T03:33:05.419857+0000","last_became_active":"2024-09-17T03:33:02.551707+0000","last_became_peered":"2024-09-17T03:33:02.551707+0000","last_unstale":"2024-09-17T03:33:05.419857+0000","last_undegraded":"2024-09-17T03:33:05.419857+0000","last_fullsized":"2024-09-17T03:33:05.419857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:28.925660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420744+0000","last_change":"2024-09-17T03:33:03.572163+0000","last_active":"2024-09-17T03:33:05.420744+0000","last_peered":"2024-09-17T03:33:05.420744+0000","last_clean":"2024-09-17T03:33:05.420744+0000","last_became_active":"2024-09-17T03:33:03.571635+0000","last_became_peered":"2024-09-17T03:33:03.571635+0000","last_unstale":"2024-09-17T03:33:05.420744+0000","last_undegraded":"2024-09-17T03:33:05.420744+0000","last_fullsized":"2024-09-17T03:33:05.420744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:54.558593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427248+0000","last_change":"2024-09-17T03:33:02.547235+0000","last_active":"2024-09-17T03:33:05.427248+0000","last_peered":"2024-09-17T03:33:05.427248+0000","last_clean":"2024-09-17T03:33:05.427248+0000","last_became_active":"2024-09-17T03:33:02.546057+0000","last_became_peered":"2024-09-17T03:33:02.546057+0000","last_unstale":"2024-09-17T03:33:05.427248+0000","last_undegraded":"2024-09-17T03:33:05.427248+0000","last_fullsized":"2024-09-17T03:33:05.427248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:57.628934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539866+0000","last_change":"2024-09-17T03:33:03.580070+0000","last_active":"2024-09-17T03:33:04.539866+0000","last_peered":"2024-09-17T03:33:04.539866+0000","last_clean":"2024-09-17T03:33:04.539866+0000","last_became_active":"2024-09-17T03:33:03.579536+0000","last_became_peered":"2024-09-17T03:33:03.579536+0000","last_unstale":"2024-09-17T03:33:04.539866+0000","last_undegraded":"2024-09-17T03:33:04.539866+0000","last_fullsized":"2024-09-17T03:33:04.539866+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:15.857981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420763+0000","last_change":"2024-09-17T03:33:02.549424+0000","last_active":"2024-09-17T03:33:05.420763+0000","last_peered":"2024-09-17T03:33:05.420763+0000","last_clean":"2024-09-17T03:33:05.420763+0000","last_became_active":"2024-09-17T03:33:02.548964+0000","last_became_peered":"2024-09-17T03:33:02.548964+0000","last_unstale":"2024-09-17T03:33:05.420763+0000","last_undegraded":"2024-09-17T03:33:05.420763+0000","last_fullsized":"2024-09-17T03:33:05.420763+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:18.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540218+0000","last_change":"2024-09-17T03:33:03.583513+0000","last_active":"2024-09-17T03:33:04.540218+0000","last_peered":"2024-09-17T03:33:04.540218+0000","last_clean":"2024-09-17T03:33:04.540218+0000","last_became_active":"2024-09-17T03:33:03.583186+0000","last_became_peered":"2024-09-17T03:33:03.583186+0000","last_unstale":"2024-09-17T03:33:04.540218+0000","last_undegraded":"2024-09-17T03:33:04.540218+0000","last_fullsized":"2024-09-17T03:33:04.540218+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:49.099577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420208+0000","last_change":"2024-09-17T03:33:02.550099+0000","last_active":"2024-09-17T03:33:05.420208+0000","last_peered":"2024-09-17T03:33:05.420208+0000","last_clean":"2024-09-17T03:33:05.420208+0000","last_became_active":"2024-09-17T03:33:02.549761+0000","last_became_peered":"2024-09-17T03:33:02.549761+0000","last_unstale":"2024-09-17T03:33:05.420208+0000","last_undegraded":"2024-09-17T03:33:05.420208+0000","last_fullsized":"2024-09-17T03:33:05.420208+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:58.843636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426573+0000","last_change":"2024-09-17T03:33:03.561146+0000","last_active":"2024-09-17T03:33:05.426573+0000","last_peered":"2024-09-17T03:33:05.426573+0000","last_clean":"2024-09-17T03:33:05.426573+0000","last_became_active":"2024-09-17T03:33:03.560840+0000","last_became_peered":"2024-09-17T03:33:03.560840+0000","last_unstale":"2024-09-17T03:33:05.426573+0000","last_undegraded":"2024-09-17T03:33:05.426573+0000","last_fullsized":"2024-09-17T03:33:05.426573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:39.264426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426977+0000","last_change":"2024-09-17T03:33:02.550579+0000","last_active":"2024-09-17T03:33:05.426977+0000","last_peered":"2024-09-17T03:33:05.426977+0000","last_clean":"2024-09-17T03:33:05.426977+0000","last_became_active":"2024-09-17T03:33:02.550238+0000","last_became_peered":"2024-09-17T03:33:02.550238+0000","last_unstale":"2024-09-17T03:33:05.426977+0000","last_undegraded":"2024-09-17T03:33:05.426977+0000","last_fullsized":"2024-09-17T03:33:05.426977+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:42.312117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.528893+0000","last_change":"2024-09-17T03:33:02.528893+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.528893+0000","last_undegraded":"2024-09-17T03:33:02.528893+0000","last_fullsized":"2024-09-17T03:33:02.528893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420094+0000","last_change":"2024-09-17T03:33:02.554041+0000","last_active":"2024-09-17T03:33:05.420094+0000","last_peered":"2024-09-17T03:33:05.420094+0000","last_clean":"2024-09-17T03:33:05.420094+0000","last_became_active":"2024-09-17T03:33:02.553699+0000","last_became_peered":"2024-09-17T03:33:02.553699+0000","last_unstale":"2024-09-17T03:33:05.420094+0000","last_undegraded":"2024-09-17T03:33:05.420094+0000","last_fullsized":"2024-09-17T03:33:05.420094+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216039+0000","last_change":"2024-09-17T03:33:03.563718+0000","last_active":"2024-09-17T03:33:05.216039+0000","last_peered":"2024-09-17T03:33:05.216039+0000","last_clean":"2024-09-17T03:33:05.216039+0000","last_became_active":"2024-09-17T03:33:03.563409+0000","last_became_peered":"2024-09-17T03:33:03.563409+0000","last_unstale":"2024-09-17T03:33:05.216039+0000","last_undegraded":"2024-09-17T03:33:05.216039+0000","last_fullsized":"2024-09-17T03:33:05.216039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:38.241017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426089+0000","last_change":"2024-09-17T03:33:02.556605+0000","last_active":"2024-09-17T03:33:05.426089+0000","last_peered":"2024-09-17T03:33:05.426089+0000","last_clean":"2024-09-17T03:33:05.426089+0000","last_became_active":"2024-09-17T03:33:02.556146+0000","last_became_peered":"2024-09-17T03:33:02.556146+0000","last_unstale":"2024-09-17T03:33:05.426089+0000","last_undegraded":"2024-09-17T03:33:05.426089+0000","last_fullsized":"2024-09-17T03:33:05.426089+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:36.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530028+0000","last_change":"2024-09-17T03:33:02.530028+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530028+0000","last_undegraded":"2024-09-17T03:33:02.530028+0000","last_fullsized":"2024-09-17T03:33:02.530028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540887+0000","last_change":"2024-09-17T03:33:02.547114+0000","last_active":"2024-09-17T03:33:04.540887+0000","last_peered":"2024-09-17T03:33:04.540887+0000","last_clean":"2024-09-17T03:33:04.540887+0000","last_became_active":"2024-09-17T03:33:02.546778+0000","last_became_peered":"2024-09-17T03:33:02.546778+0000","last_unstale":"2024-09-17T03:33:04.540887+0000","last_undegraded":"2024-09-17T03:33:04.540887+0000","last_fullsized":"2024-09-17T03:33:04.540887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:19.611348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426304+0000","last_change":"2024-09-17T03:33:03.561358+0000","last_active":"2024-09-17T03:33:05.426304+0000","last_peered":"2024-09-17T03:33:05.426304+0000","last_clean":"2024-09-17T03:33:05.426304+0000","last_became_active":"2024-09-17T03:33:03.561027+0000","last_became_peered":"2024-09-17T03:33:03.561027+0000","last_unstale":"2024-09-17T03:33:05.426304+0000","last_undegraded":"2024-09-17T03:33:05.426304+0000","last_fullsized":"2024-09-17T03:33:05.426304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:04.965218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.553304+0000","last_change":"2024-09-17T03:33:02.544930+0000","last_active":"2024-09-17T03:33:02.553304+0000","last_peered":"2024-09-17T03:33:02.553304+0000","last_clean":"2024-09-17T03:33:02.553304+0000","last_became_active":"2024-09-17T03:33:02.542640+0000","last_became_peered":"2024-09-17T03:33:02.542640+0000","last_unstale":"2024-09-17T03:33:02.553304+0000","last_undegraded":"2024-09-17T03:33:02.553304+0000","last_fullsized":"2024-09-17T03:33:02.553304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:12.140218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530977+0000","last_change":"2024-09-17T03:33:02.530977+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530977+0000","last_undegraded":"2024-09-17T03:33:02.530977+0000","last_fullsized":"2024-09-17T03:33:02.530977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547713+0000","last_change":"2024-09-17T03:33:02.558759+0000","last_active":"2024-09-17T03:33:04.547713+0000","last_peered":"2024-09-17T03:33:04.547713+0000","last_clean":"2024-09-17T03:33:04.547713+0000","last_became_active":"2024-09-17T03:33:02.558503+0000","last_became_peered":"2024-09-17T03:33:02.558503+0000","last_unstale":"2024-09-17T03:33:04.547713+0000","last_undegraded":"2024-09-17T03:33:04.547713+0000","last_fullsized":"2024-09-17T03:33:04.547713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547648+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:33:04.547648+0000","last_peered":"2024-09-17T03:33:04.547648+0000","last_clean":"2024-09-17T03:33:04.547648+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:33:04.547648+0000","last_undegraded":"2024-09-17T03:33:04.547648+0000","last_fullsized":"2024-09-17T03:33:04.547648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127344,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244496,"statfs":{"total":96636764160,"available":96506363904,"internally_reserved":0,"allocated":1855488,"data_stored":1132666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136484,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235356,"statfs":{"total":96636764160,"available":96497004544,"internally_reserved":0,"allocated":1261568,"data_stored":542838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144816,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227024,"statfs":{"total":96636764160,"available":96488472576,"internally_reserved":0,"allocated":1273856,"data_stored":543922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130924,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240916,"statfs":{"total":96636764160,"available":96502697984,"internally_reserved":0,"allocated":1269760,"data_stored":543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136452,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235388,"statfs":{"total":96636764160,"available":96497037312,"internally_reserved":0,"allocated":1216512,"data_stored":522836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119016,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252824,"statfs":{"total":96636764160,"available":96514891776,"internally_reserved":0,"allocated":1851392,"data_stored":1133136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82324,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289516,"statfs":{"total":96636764160,"available":96552464384,"internally_reserved":0,"allocated":1245184,"data_stored":535956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140640,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231200,"statfs":{"total":96636764160,"available":96492748800,"internally_reserved":0,"allocated":1253376,"data_stored":542186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:33:08.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:33:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 -- 172.21.15.46:0/2623316600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a0005260 msgr2=0x7f12a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 --2- 172.21.15.46:0/2623316600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a0005260 0x7f12a000b9d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009e90 tx=0x7f12b401d5f0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 -- 172.21.15.46:0/2623316600 shutdown_connections 2024-09-17T03:33:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 --2- 172.21.15.46:0/2623316600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a0005260 0x7f12a000b9d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 -- 172.21.15.46:0/2623316600 >> 172.21.15.46:0/2623316600 conn(0x7f12bc0b2040 msgr2=0x7f12bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 -- 172.21.15.46:0/2623316600 shutdown_connections 2024-09-17T03:33:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.376+0000 7f12c1200640 1 -- 172.21.15.46:0/2623316600 wait complete. 2024-09-17T03:33:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 Processor -- start 2024-09-17T03:33:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 -- start start 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12a0005260 0x7f12bc15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 0x7f12bc1f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc1f5fc0 0x7f12bc1f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc075190 con 0x7f12bc15f440 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc075010 con 0x7f12bc1f5fc0 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12c1200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc075310 con 0x7f12a0005260 2024-09-17T03:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 0x7f12bc1f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 0x7f12bc1f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57090/0 (socket says 172.21.15.46:57090) 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 -- 172.21.15.46:0/920195176 learned_addr learned my addr 172.21.15.46:0/920195176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b9e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12a0005260 0x7f12bc15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12ba800640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc1f5fc0 0x7f12bc1f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 -- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12a0005260 msgr2=0x7f12bc15ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12a0005260 0x7f12bc15ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 -- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc1f5fc0 msgr2=0x7f12bc1f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc1f5fc0 0x7f12bc1f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 -- 172.21.15.46:0/920195176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc15f850 con 0x7f12bc15f440 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b9e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12a0005260 0x7f12bc15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.380+0000 7f12b3e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 0x7f12bc1f5a80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f12a4004fd0 tx=0x7f12a400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4015020 con 0x7f12bc15f440 2024-09-17T03:33:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12a40109a0 con 0x7f12bc15f440 2024-09-17T03:33:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4010df0 con 0x7f12bc15f440 2024-09-17T03:33:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc13d550 con 0x7f12bc15f440 2024-09-17T03:33:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc13db10 con 0x7f12bc15f440 2024-09-17T03:33:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc0cf650 con 0x7f12bc15f440 2024-09-17T03:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f12a4018020 con 0x7f12bc15f440 2024-09-17T03:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b2000640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298074940 0x7f1298076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f12a4093450 con 0x7f12bc15f440 2024-09-17T03:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.384+0000 7f12b9e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298074940 0x7f1298076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.388+0000 7f12b9e00640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298074940 0x7f1298076e50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f12ac000c10 tx=0x7f12ac006b80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:08.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.388+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12a401f020 con 0x7f12bc15f440 2024-09-17T03:33:08.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.560+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12bc06b540 con 0x7f1298074940 2024-09-17T03:33:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.568+0000 7f12b2000640 1 -- 172.21.15.46:0/920195176 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251442 (secure 0 0 0) 0x7f12ac016070 con 0x7f1298074940 2024-09-17T03:33:08.570 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298074940 msgr2=0x7f1298076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298074940 0x7f1298076e50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f12ac000c10 tx=0x7f12ac006b80 comp rx=0 tx=0).stop 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 msgr2=0x7f12bc1f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 0x7f12bc1f5a80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f12a4004fd0 tx=0x7f12a400ea20 comp rx=0 tx=0).stop 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 shutdown_connections 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1298074940 0x7f1298076e50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12bc1f5fc0 0x7f12bc1f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12bc15f440 0x7f12bc1f5a80 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 --2- 172.21.15.46:0/920195176 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12a0005260 0x7f12bc15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 >> 172.21.15.46:0/920195176 conn(0x7f12bc0b2040 msgr2=0x7f12bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 shutdown_connections 2024-09-17T03:33:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.572+0000 7f12c1200640 1 -- 172.21.15.46:0/920195176 wait complete. 2024-09-17T03:33:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-17T03:33:07.511563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":143,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1018000,"kb_used_data":10964,"kb_used_omap":12,"kb_used_meta":1006835,"kb_avail":753956720,"statfs":{"total":773094113280,"available":772051681280,"internally_reserved":0,"allocated":11227136,"data_stored":5497360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1030999600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991341"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539642+0000","last_change":"2024-09-17T03:33:03.581362+0000","last_active":"2024-09-17T03:33:04.539642+0000","last_peered":"2024-09-17T03:33:04.539642+0000","last_clean":"2024-09-17T03:33:04.539642+0000","last_became_active":"2024-09-17T03:33:03.580475+0000","last_became_peered":"2024-09-17T03:33:03.580475+0000","last_unstale":"2024-09-17T03:33:04.539642+0000","last_undegraded":"2024-09-17T03:33:04.539642+0000","last_fullsized":"2024-09-17T03:33:04.539642+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:11.925403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420228+0000","last_change":"2024-09-17T03:33:03.572549+0000","last_active":"2024-09-17T03:33:05.420228+0000","last_peered":"2024-09-17T03:33:05.420228+0000","last_clean":"2024-09-17T03:33:05.420228+0000","last_became_active":"2024-09-17T03:33:03.572232+0000","last_became_peered":"2024-09-17T03:33:03.572232+0000","last_unstale":"2024-09-17T03:33:05.420228+0000","last_undegraded":"2024-09-17T03:33:05.420228+0000","last_fullsized":"2024-09-17T03:33:05.420228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:46.104711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539783+0000","last_change":"2024-09-17T03:33:03.580570+0000","last_active":"2024-09-17T03:33:04.539783+0000","last_peered":"2024-09-17T03:33:04.539783+0000","last_clean":"2024-09-17T03:33:04.539783+0000","last_became_active":"2024-09-17T03:33:03.580241+0000","last_became_peered":"2024-09-17T03:33:03.580241+0000","last_unstale":"2024-09-17T03:33:04.539783+0000","last_undegraded":"2024-09-17T03:33:04.539783+0000","last_fullsized":"2024-09-17T03:33:04.539783+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:36.303054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541593+0000","last_change":"2024-09-17T03:33:03.551187+0000","last_active":"2024-09-17T03:33:04.541593+0000","last_peered":"2024-09-17T03:33:04.541593+0000","last_clean":"2024-09-17T03:33:04.541593+0000","last_became_active":"2024-09-17T03:33:03.550854+0000","last_became_peered":"2024-09-17T03:33:03.550854+0000","last_unstale":"2024-09-17T03:33:04.541593+0000","last_undegraded":"2024-09-17T03:33:04.541593+0000","last_fullsized":"2024-09-17T03:33:04.541593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:32.888001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427161+0000","last_change":"2024-09-17T03:33:03.562221+0000","last_active":"2024-09-17T03:33:05.427161+0000","last_peered":"2024-09-17T03:33:05.427161+0000","last_clean":"2024-09-17T03:33:05.427161+0000","last_became_active":"2024-09-17T03:33:03.561816+0000","last_became_peered":"2024-09-17T03:33:03.561816+0000","last_unstale":"2024-09-17T03:33:05.427161+0000","last_undegraded":"2024-09-17T03:33:05.427161+0000","last_fullsized":"2024-09-17T03:33:05.427161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.528483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547618+0000","last_change":"2024-09-17T03:33:03.580482+0000","last_active":"2024-09-17T03:33:04.547618+0000","last_peered":"2024-09-17T03:33:04.547618+0000","last_clean":"2024-09-17T03:33:04.547618+0000","last_became_active":"2024-09-17T03:33:03.579513+0000","last_became_peered":"2024-09-17T03:33:03.579513+0000","last_unstale":"2024-09-17T03:33:04.547618+0000","last_undegraded":"2024-09-17T03:33:04.547618+0000","last_fullsized":"2024-09-17T03:33:04.547618+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:22.208855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547571+0000","last_change":"2024-09-17T03:33:03.576842+0000","last_active":"2024-09-17T03:33:04.547571+0000","last_peered":"2024-09-17T03:33:04.547571+0000","last_clean":"2024-09-17T03:33:04.547571+0000","last_became_active":"2024-09-17T03:33:03.576572+0000","last_became_peered":"2024-09-17T03:33:03.576572+0000","last_unstale":"2024-09-17T03:33:04.547571+0000","last_undegraded":"2024-09-17T03:33:04.547571+0000","last_fullsized":"2024-09-17T03:33:04.547571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420470+0000","last_change":"2024-09-17T03:33:03.564312+0000","last_active":"2024-09-17T03:33:05.420470+0000","last_peered":"2024-09-17T03:33:05.420470+0000","last_clean":"2024-09-17T03:33:05.420470+0000","last_became_active":"2024-09-17T03:33:03.563925+0000","last_became_peered":"2024-09-17T03:33:03.563925+0000","last_unstale":"2024-09-17T03:33:05.420470+0000","last_undegraded":"2024-09-17T03:33:05.420470+0000","last_fullsized":"2024-09-17T03:33:05.420470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:00.239782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216739+0000","last_change":"2024-09-17T03:33:03.556879+0000","last_active":"2024-09-17T03:33:05.216739+0000","last_peered":"2024-09-17T03:33:05.216739+0000","last_clean":"2024-09-17T03:33:05.216739+0000","last_became_active":"2024-09-17T03:33:03.556579+0000","last_became_peered":"2024-09-17T03:33:03.556579+0000","last_unstale":"2024-09-17T03:33:05.216739+0000","last_undegraded":"2024-09-17T03:33:05.216739+0000","last_fullsized":"2024-09-17T03:33:05.216739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:00.787049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426696+0000","last_change":"2024-09-17T03:33:03.590158+0000","last_active":"2024-09-17T03:33:05.426696+0000","last_peered":"2024-09-17T03:33:05.426696+0000","last_clean":"2024-09-17T03:33:05.426696+0000","last_became_active":"2024-09-17T03:33:03.589751+0000","last_became_peered":"2024-09-17T03:33:03.589751+0000","last_unstale":"2024-09-17T03:33:05.426696+0000","last_undegraded":"2024-09-17T03:33:05.426696+0000","last_fullsized":"2024-09-17T03:33:05.426696+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:08.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548057+0000","last_change":"2024-09-17T03:33:03.559214+0000","last_active":"2024-09-17T03:33:04.548057+0000","last_peered":"2024-09-17T03:33:04.548057+0000","last_clean":"2024-09-17T03:33:04.548057+0000","last_became_active":"2024-09-17T03:33:03.558880+0000","last_became_peered":"2024-09-17T03:33:03.558880+0000","last_unstale":"2024-09-17T03:33:04.548057+0000","last_undegraded":"2024-09-17T03:33:04.548057+0000","last_fullsized":"2024-09-17T03:33:04.548057+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:32.880704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216475+0000","last_change":"2024-09-17T03:33:03.595048+0000","last_active":"2024-09-17T03:33:05.216475+0000","last_peered":"2024-09-17T03:33:05.216475+0000","last_clean":"2024-09-17T03:33:05.216475+0000","last_became_active":"2024-09-17T03:33:03.594649+0000","last_became_peered":"2024-09-17T03:33:03.594649+0000","last_unstale":"2024-09-17T03:33:05.216475+0000","last_undegraded":"2024-09-17T03:33:05.216475+0000","last_fullsized":"2024-09-17T03:33:05.216475+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:26.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541461+0000","last_change":"2024-09-17T03:33:03.546430+0000","last_active":"2024-09-17T03:33:04.541461+0000","last_peered":"2024-09-17T03:33:04.541461+0000","last_clean":"2024-09-17T03:33:04.541461+0000","last_became_active":"2024-09-17T03:33:03.546006+0000","last_became_peered":"2024-09-17T03:33:03.546006+0000","last_unstale":"2024-09-17T03:33:04.541461+0000","last_undegraded":"2024-09-17T03:33:04.541461+0000","last_fullsized":"2024-09-17T03:33:04.541461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535651+0000","last_change":"2024-09-17T03:33:02.535651+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535651+0000","last_undegraded":"2024-09-17T03:33:02.535651+0000","last_fullsized":"2024-09-17T03:33:02.535651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540556+0000","last_change":"2024-09-17T03:33:03.580895+0000","last_active":"2024-09-17T03:33:04.540556+0000","last_peered":"2024-09-17T03:33:04.540556+0000","last_clean":"2024-09-17T03:33:04.540556+0000","last_became_active":"2024-09-17T03:33:03.580561+0000","last_became_peered":"2024-09-17T03:33:03.580561+0000","last_unstale":"2024-09-17T03:33:04.540556+0000","last_undegraded":"2024-09-17T03:33:04.540556+0000","last_fullsized":"2024-09-17T03:33:04.540556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:59.846145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547563+0000","last_change":"2024-09-17T03:33:03.573694+0000","last_active":"2024-09-17T03:33:04.547563+0000","last_peered":"2024-09-17T03:33:04.547563+0000","last_clean":"2024-09-17T03:33:04.547563+0000","last_became_active":"2024-09-17T03:33:03.573199+0000","last_became_peered":"2024-09-17T03:33:03.573199+0000","last_unstale":"2024-09-17T03:33:04.547563+0000","last_undegraded":"2024-09-17T03:33:04.547563+0000","last_fullsized":"2024-09-17T03:33:04.547563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:14.793531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420606+0000","last_change":"2024-09-17T03:33:03.570549+0000","last_active":"2024-09-17T03:33:05.420606+0000","last_peered":"2024-09-17T03:33:05.420606+0000","last_clean":"2024-09-17T03:33:05.420606+0000","last_became_active":"2024-09-17T03:33:03.570244+0000","last_became_peered":"2024-09-17T03:33:03.570244+0000","last_unstale":"2024-09-17T03:33:05.420606+0000","last_undegraded":"2024-09-17T03:33:05.420606+0000","last_fullsized":"2024-09-17T03:33:05.420606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:57.566540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541670+0000","last_change":"2024-09-17T03:33:03.558416+0000","last_active":"2024-09-17T03:33:04.541670+0000","last_peered":"2024-09-17T03:33:04.541670+0000","last_clean":"2024-09-17T03:33:04.541670+0000","last_became_active":"2024-09-17T03:33:03.558033+0000","last_became_peered":"2024-09-17T03:33:03.558033+0000","last_unstale":"2024-09-17T03:33:04.541670+0000","last_undegraded":"2024-09-17T03:33:04.541670+0000","last_fullsized":"2024-09-17T03:33:04.541670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:07.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535461+0000","last_change":"2024-09-17T03:33:02.535461+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535461+0000","last_undegraded":"2024-09-17T03:33:02.535461+0000","last_fullsized":"2024-09-17T03:33:02.535461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216563+0000","last_change":"2024-09-17T03:33:03.556063+0000","last_active":"2024-09-17T03:33:05.216563+0000","last_peered":"2024-09-17T03:33:05.216563+0000","last_clean":"2024-09-17T03:33:05.216563+0000","last_became_active":"2024-09-17T03:33:03.555571+0000","last_became_peered":"2024-09-17T03:33:03.555571+0000","last_unstale":"2024-09-17T03:33:05.216563+0000","last_undegraded":"2024-09-17T03:33:05.216563+0000","last_fullsized":"2024-09-17T03:33:05.216563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:58.787401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534384+0000","last_change":"2024-09-17T03:33:02.534384+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534384+0000","last_undegraded":"2024-09-17T03:33:02.534384+0000","last_fullsized":"2024-09-17T03:33:02.534384+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420061+0000","last_change":"2024-09-17T03:33:03.572171+0000","last_active":"2024-09-17T03:33:05.420061+0000","last_peered":"2024-09-17T03:33:05.420061+0000","last_clean":"2024-09-17T03:33:05.420061+0000","last_became_active":"2024-09-17T03:33:03.571829+0000","last_became_peered":"2024-09-17T03:33:03.571829+0000","last_unstale":"2024-09-17T03:33:05.420061+0000","last_undegraded":"2024-09-17T03:33:05.420061+0000","last_fullsized":"2024-09-17T03:33:05.420061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:38.888214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419929+0000","last_change":"2024-09-17T03:33:03.572010+0000","last_active":"2024-09-17T03:33:05.419929+0000","last_peered":"2024-09-17T03:33:05.419929+0000","last_clean":"2024-09-17T03:33:05.419929+0000","last_became_active":"2024-09-17T03:33:03.571696+0000","last_became_peered":"2024-09-17T03:33:03.571696+0000","last_unstale":"2024-09-17T03:33:05.419929+0000","last_undegraded":"2024-09-17T03:33:05.419929+0000","last_fullsized":"2024-09-17T03:33:05.419929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.483130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547565+0000","last_change":"2024-09-17T03:33:03.589869+0000","last_active":"2024-09-17T03:33:04.547565+0000","last_peered":"2024-09-17T03:33:04.547565+0000","last_clean":"2024-09-17T03:33:04.547565+0000","last_became_active":"2024-09-17T03:33:03.589493+0000","last_became_peered":"2024-09-17T03:33:03.589493+0000","last_unstale":"2024-09-17T03:33:04.547565+0000","last_undegraded":"2024-09-17T03:33:04.547565+0000","last_fullsized":"2024-09-17T03:33:04.547565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.449496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426760+0000","last_change":"2024-09-17T03:33:03.576923+0000","last_active":"2024-09-17T03:33:05.426760+0000","last_peered":"2024-09-17T03:33:05.426760+0000","last_clean":"2024-09-17T03:33:05.426760+0000","last_became_active":"2024-09-17T03:33:03.576548+0000","last_became_peered":"2024-09-17T03:33:03.576548+0000","last_unstale":"2024-09-17T03:33:05.426760+0000","last_undegraded":"2024-09-17T03:33:05.426760+0000","last_fullsized":"2024-09-17T03:33:05.426760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:08.997404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420450+0000","last_change":"2024-09-17T03:33:03.583037+0000","last_active":"2024-09-17T03:33:05.420450+0000","last_peered":"2024-09-17T03:33:05.420450+0000","last_clean":"2024-09-17T03:33:05.420450+0000","last_became_active":"2024-09-17T03:33:03.582714+0000","last_became_peered":"2024-09-17T03:33:03.582714+0000","last_unstale":"2024-09-17T03:33:05.420450+0000","last_undegraded":"2024-09-17T03:33:05.420450+0000","last_fullsized":"2024-09-17T03:33:05.420450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:08.781666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548160+0000","last_change":"2024-09-17T03:33:03.567964+0000","last_active":"2024-09-17T03:33:04.548160+0000","last_peered":"2024-09-17T03:33:04.548160+0000","last_clean":"2024-09-17T03:33:04.548160+0000","last_became_active":"2024-09-17T03:33:03.567632+0000","last_became_peered":"2024-09-17T03:33:03.567632+0000","last_unstale":"2024-09-17T03:33:04.548160+0000","last_undegraded":"2024-09-17T03:33:04.548160+0000","last_fullsized":"2024-09-17T03:33:04.548160+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:39.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426889+0000","last_change":"2024-09-17T03:33:03.559826+0000","last_active":"2024-09-17T03:33:05.426889+0000","last_peered":"2024-09-17T03:33:05.426889+0000","last_clean":"2024-09-17T03:33:05.426889+0000","last_became_active":"2024-09-17T03:33:03.559504+0000","last_became_peered":"2024-09-17T03:33:03.559504+0000","last_unstale":"2024-09-17T03:33:05.426889+0000","last_undegraded":"2024-09-17T03:33:05.426889+0000","last_fullsized":"2024-09-17T03:33:05.426889+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:54.833105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420363+0000","last_change":"2024-09-17T03:33:03.580138+0000","last_active":"2024-09-17T03:33:05.420363+0000","last_peered":"2024-09-17T03:33:05.420363+0000","last_clean":"2024-09-17T03:33:05.420363+0000","last_became_active":"2024-09-17T03:33:03.579782+0000","last_became_peered":"2024-09-17T03:33:03.579782+0000","last_unstale":"2024-09-17T03:33:05.420363+0000","last_undegraded":"2024-09-17T03:33:05.420363+0000","last_fullsized":"2024-09-17T03:33:05.420363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:07.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.533906+0000","last_change":"2024-09-17T03:33:02.533906+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.533906+0000","last_undegraded":"2024-09-17T03:33:02.533906+0000","last_fullsized":"2024-09-17T03:33:02.533906+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541093+0000","last_change":"2024-09-17T03:33:03.565826+0000","last_active":"2024-09-17T03:33:04.541093+0000","last_peered":"2024-09-17T03:33:04.541093+0000","last_clean":"2024-09-17T03:33:04.541093+0000","last_became_active":"2024-09-17T03:33:03.557053+0000","last_became_peered":"2024-09-17T03:33:03.557053+0000","last_unstale":"2024-09-17T03:33:04.541093+0000","last_undegraded":"2024-09-17T03:33:04.541093+0000","last_fullsized":"2024-09-17T03:33:04.541093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:54.693438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539233+0000","last_change":"2024-09-17T03:33:03.581059+0000","last_active":"2024-09-17T03:33:04.539233+0000","last_peered":"2024-09-17T03:33:04.539233+0000","last_clean":"2024-09-17T03:33:04.539233+0000","last_became_active":"2024-09-17T03:33:03.580721+0000","last_became_peered":"2024-09-17T03:33:03.580721+0000","last_unstale":"2024-09-17T03:33:04.539233+0000","last_undegraded":"2024-09-17T03:33:04.539233+0000","last_fullsized":"2024-09-17T03:33:04.539233+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:12.767168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534628+0000","last_change":"2024-09-17T03:33:02.534628+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534628+0000","last_undegraded":"2024-09-17T03:33:02.534628+0000","last_fullsized":"2024-09-17T03:33:02.534628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420603+0000","last_change":"2024-09-17T03:33:02.543361+0000","last_active":"2024-09-17T03:33:05.420603+0000","last_peered":"2024-09-17T03:33:05.420603+0000","last_clean":"2024-09-17T03:33:05.420603+0000","last_became_active":"2024-09-17T03:33:02.543063+0000","last_became_peered":"2024-09-17T03:33:02.543063+0000","last_unstale":"2024-09-17T03:33:05.420603+0000","last_undegraded":"2024-09-17T03:33:05.420603+0000","last_fullsized":"2024-09-17T03:33:05.420603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:01.651709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541863+0000","last_change":"2024-09-17T03:33:03.556411+0000","last_active":"2024-09-17T03:33:04.541863+0000","last_peered":"2024-09-17T03:33:04.541863+0000","last_clean":"2024-09-17T03:33:04.541863+0000","last_became_active":"2024-09-17T03:33:03.555405+0000","last_became_peered":"2024-09-17T03:33:03.555405+0000","last_unstale":"2024-09-17T03:33:04.541863+0000","last_undegraded":"2024-09-17T03:33:04.541863+0000","last_fullsized":"2024-09-17T03:33:04.541863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:44.098643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547818+0000","last_change":"2024-09-17T03:33:02.563716+0000","last_active":"2024-09-17T03:33:04.547818+0000","last_peered":"2024-09-17T03:33:04.547818+0000","last_clean":"2024-09-17T03:33:04.547818+0000","last_became_active":"2024-09-17T03:33:02.563400+0000","last_became_peered":"2024-09-17T03:33:02.563400+0000","last_unstale":"2024-09-17T03:33:04.547818+0000","last_undegraded":"2024-09-17T03:33:04.547818+0000","last_fullsized":"2024-09-17T03:33:04.547818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540439+0000","last_change":"2024-09-17T03:33:03.587192+0000","last_active":"2024-09-17T03:33:04.540439+0000","last_peered":"2024-09-17T03:33:04.540439+0000","last_clean":"2024-09-17T03:33:04.540439+0000","last_became_active":"2024-09-17T03:33:03.586800+0000","last_became_peered":"2024-09-17T03:33:03.586800+0000","last_unstale":"2024-09-17T03:33:04.540439+0000","last_undegraded":"2024-09-17T03:33:04.540439+0000","last_fullsized":"2024-09-17T03:33:04.540439+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:04.696195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419965+0000","last_change":"2024-09-17T03:33:02.555846+0000","last_active":"2024-09-17T03:33:05.419965+0000","last_peered":"2024-09-17T03:33:05.419965+0000","last_clean":"2024-09-17T03:33:05.419965+0000","last_became_active":"2024-09-17T03:33:02.554535+0000","last_became_peered":"2024-09-17T03:33:02.554535+0000","last_unstale":"2024-09-17T03:33:05.419965+0000","last_undegraded":"2024-09-17T03:33:05.419965+0000","last_fullsized":"2024-09-17T03:33:05.419965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:20.329251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216814+0000","last_change":"2024-09-17T03:33:03.576310+0000","last_active":"2024-09-17T03:33:05.216814+0000","last_peered":"2024-09-17T03:33:05.216814+0000","last_clean":"2024-09-17T03:33:05.216814+0000","last_became_active":"2024-09-17T03:33:03.575941+0000","last_became_peered":"2024-09-17T03:33:03.575941+0000","last_unstale":"2024-09-17T03:33:05.216814+0000","last_undegraded":"2024-09-17T03:33:05.216814+0000","last_fullsized":"2024-09-17T03:33:05.216814+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:19.596987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419816+0000","last_change":"2024-09-17T03:33:02.539022+0000","last_active":"2024-09-17T03:33:05.419816+0000","last_peered":"2024-09-17T03:33:05.419816+0000","last_clean":"2024-09-17T03:33:05.419816+0000","last_became_active":"2024-09-17T03:33:02.538700+0000","last_became_peered":"2024-09-17T03:33:02.538700+0000","last_unstale":"2024-09-17T03:33:05.419816+0000","last_undegraded":"2024-09-17T03:33:05.419816+0000","last_fullsized":"2024-09-17T03:33:05.419816+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:31.500503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426743+0000","last_change":"2024-09-17T03:33:03.569261+0000","last_active":"2024-09-17T03:33:05.426743+0000","last_peered":"2024-09-17T03:33:05.426743+0000","last_clean":"2024-09-17T03:33:05.426743+0000","last_became_active":"2024-09-17T03:33:03.568877+0000","last_became_peered":"2024-09-17T03:33:03.568877+0000","last_unstale":"2024-09-17T03:33:05.426743+0000","last_undegraded":"2024-09-17T03:33:05.426743+0000","last_fullsized":"2024-09-17T03:33:05.426743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:04.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539368+0000","last_change":"2024-09-17T03:33:02.550110+0000","last_active":"2024-09-17T03:33:04.539368+0000","last_peered":"2024-09-17T03:33:04.539368+0000","last_clean":"2024-09-17T03:33:04.539368+0000","last_became_active":"2024-09-17T03:33:02.549795+0000","last_became_peered":"2024-09-17T03:33:02.549795+0000","last_unstale":"2024-09-17T03:33:04.539368+0000","last_undegraded":"2024-09-17T03:33:04.539368+0000","last_fullsized":"2024-09-17T03:33:04.539368+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:47.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547977+0000","last_change":"2024-09-17T03:33:03.564062+0000","last_active":"2024-09-17T03:33:04.547977+0000","last_peered":"2024-09-17T03:33:04.547977+0000","last_clean":"2024-09-17T03:33:04.547977+0000","last_became_active":"2024-09-17T03:33:03.563650+0000","last_became_peered":"2024-09-17T03:33:03.563650+0000","last_unstale":"2024-09-17T03:33:04.547977+0000","last_undegraded":"2024-09-17T03:33:04.547977+0000","last_fullsized":"2024-09-17T03:33:04.547977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:08.738111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419621+0000","last_change":"2024-09-17T03:33:02.549221+0000","last_active":"2024-09-17T03:33:05.419621+0000","last_peered":"2024-09-17T03:33:05.419621+0000","last_clean":"2024-09-17T03:33:05.419621+0000","last_became_active":"2024-09-17T03:33:02.548861+0000","last_became_peered":"2024-09-17T03:33:02.548861+0000","last_unstale":"2024-09-17T03:33:05.419621+0000","last_undegraded":"2024-09-17T03:33:05.419621+0000","last_fullsized":"2024-09-17T03:33:05.419621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540576+0000","last_change":"2024-09-17T03:33:03.581594+0000","last_active":"2024-09-17T03:33:04.540576+0000","last_peered":"2024-09-17T03:33:04.540576+0000","last_clean":"2024-09-17T03:33:04.540576+0000","last_became_active":"2024-09-17T03:33:03.581295+0000","last_became_peered":"2024-09-17T03:33:03.581295+0000","last_unstale":"2024-09-17T03:33:04.540576+0000","last_undegraded":"2024-09-17T03:33:04.540576+0000","last_fullsized":"2024-09-17T03:33:04.540576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:02.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426395+0000","last_change":"2024-09-17T03:33:02.555268+0000","last_active":"2024-09-17T03:33:05.426395+0000","last_peered":"2024-09-17T03:33:05.426395+0000","last_clean":"2024-09-17T03:33:05.426395+0000","last_became_active":"2024-09-17T03:33:02.554769+0000","last_became_peered":"2024-09-17T03:33:02.554769+0000","last_unstale":"2024-09-17T03:33:05.426395+0000","last_undegraded":"2024-09-17T03:33:05.426395+0000","last_fullsized":"2024-09-17T03:33:05.426395+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:38.916564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215985+0000","last_change":"2024-09-17T03:33:03.570252+0000","last_active":"2024-09-17T03:33:05.215985+0000","last_peered":"2024-09-17T03:33:05.215985+0000","last_clean":"2024-09-17T03:33:05.215985+0000","last_became_active":"2024-09-17T03:33:03.569899+0000","last_became_peered":"2024-09-17T03:33:03.569899+0000","last_unstale":"2024-09-17T03:33:05.215985+0000","last_undegraded":"2024-09-17T03:33:05.215985+0000","last_fullsized":"2024-09-17T03:33:05.215985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:27.095812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539125+0000","last_change":"2024-09-17T03:33:02.551914+0000","last_active":"2024-09-17T03:33:04.539125+0000","last_peered":"2024-09-17T03:33:04.539125+0000","last_clean":"2024-09-17T03:33:04.539125+0000","last_became_active":"2024-09-17T03:33:02.551576+0000","last_became_peered":"2024-09-17T03:33:02.551576+0000","last_unstale":"2024-09-17T03:33:04.539125+0000","last_undegraded":"2024-09-17T03:33:04.539125+0000","last_fullsized":"2024-09-17T03:33:04.539125+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:46.988395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420593+0000","last_change":"2024-09-17T03:33:03.558995+0000","last_active":"2024-09-17T03:33:05.420593+0000","last_peered":"2024-09-17T03:33:05.420593+0000","last_clean":"2024-09-17T03:33:05.420593+0000","last_became_active":"2024-09-17T03:33:03.558645+0000","last_became_peered":"2024-09-17T03:33:03.558645+0000","last_unstale":"2024-09-17T03:33:05.420593+0000","last_undegraded":"2024-09-17T03:33:05.420593+0000","last_fullsized":"2024-09-17T03:33:05.420593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:28.537475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539936+0000","last_change":"2024-09-17T03:33:02.556533+0000","last_active":"2024-09-17T03:33:04.539936+0000","last_peered":"2024-09-17T03:33:04.539936+0000","last_clean":"2024-09-17T03:33:04.539936+0000","last_became_active":"2024-09-17T03:33:02.556134+0000","last_became_peered":"2024-09-17T03:33:02.556134+0000","last_unstale":"2024-09-17T03:33:04.539936+0000","last_undegraded":"2024-09-17T03:33:04.539936+0000","last_fullsized":"2024-09-17T03:33:04.539936+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:08.763261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216780+0000","last_change":"2024-09-17T03:33:03.575625+0000","last_active":"2024-09-17T03:33:05.216780+0000","last_peered":"2024-09-17T03:33:05.216780+0000","last_clean":"2024-09-17T03:33:05.216780+0000","last_became_active":"2024-09-17T03:33:03.575239+0000","last_became_peered":"2024-09-17T03:33:03.575239+0000","last_unstale":"2024-09-17T03:33:05.216780+0000","last_undegraded":"2024-09-17T03:33:05.216780+0000","last_fullsized":"2024-09-17T03:33:05.216780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:57.701386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541519+0000","last_change":"2024-09-17T03:33:02.547247+0000","last_active":"2024-09-17T03:33:04.541519+0000","last_peered":"2024-09-17T03:33:04.541519+0000","last_clean":"2024-09-17T03:33:04.541519+0000","last_became_active":"2024-09-17T03:33:02.546932+0000","last_became_peered":"2024-09-17T03:33:02.546932+0000","last_unstale":"2024-09-17T03:33:04.541519+0000","last_undegraded":"2024-09-17T03:33:04.541519+0000","last_fullsized":"2024-09-17T03:33:04.541519+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.962710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427462+0000","last_change":"2024-09-17T03:33:03.569370+0000","last_active":"2024-09-17T03:33:05.427462+0000","last_peered":"2024-09-17T03:33:05.427462+0000","last_clean":"2024-09-17T03:33:05.427462+0000","last_became_active":"2024-09-17T03:33:03.569034+0000","last_became_peered":"2024-09-17T03:33:03.569034+0000","last_unstale":"2024-09-17T03:33:05.427462+0000","last_undegraded":"2024-09-17T03:33:05.427462+0000","last_fullsized":"2024-09-17T03:33:05.427462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:36.003958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548375+0000","last_change":"2024-09-17T03:33:02.557519+0000","last_active":"2024-09-17T03:33:04.548375+0000","last_peered":"2024-09-17T03:33:04.548375+0000","last_clean":"2024-09-17T03:33:04.548375+0000","last_became_active":"2024-09-17T03:33:02.557143+0000","last_became_peered":"2024-09-17T03:33:02.557143+0000","last_unstale":"2024-09-17T03:33:04.548375+0000","last_undegraded":"2024-09-17T03:33:04.548375+0000","last_fullsized":"2024-09-17T03:33:04.548375+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.298939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548299+0000","last_change":"2024-09-17T03:33:03.567932+0000","last_active":"2024-09-17T03:33:04.548299+0000","last_peered":"2024-09-17T03:33:04.548299+0000","last_clean":"2024-09-17T03:33:04.548299+0000","last_became_active":"2024-09-17T03:33:03.567611+0000","last_became_peered":"2024-09-17T03:33:03.567611+0000","last_unstale":"2024-09-17T03:33:04.548299+0000","last_undegraded":"2024-09-17T03:33:04.548299+0000","last_fullsized":"2024-09-17T03:33:04.548299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:15.466755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216540+0000","last_change":"2024-09-17T03:33:02.552966+0000","last_active":"2024-09-17T03:33:05.216540+0000","last_peered":"2024-09-17T03:33:05.216540+0000","last_clean":"2024-09-17T03:33:05.216540+0000","last_became_active":"2024-09-17T03:33:02.552610+0000","last_became_peered":"2024-09-17T03:33:02.552610+0000","last_unstale":"2024-09-17T03:33:05.216540+0000","last_undegraded":"2024-09-17T03:33:05.216540+0000","last_fullsized":"2024-09-17T03:33:05.216540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:57.033583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541014+0000","last_change":"2024-09-17T03:33:03.551601+0000","last_active":"2024-09-17T03:33:04.541014+0000","last_peered":"2024-09-17T03:33:04.541014+0000","last_clean":"2024-09-17T03:33:04.541014+0000","last_became_active":"2024-09-17T03:33:03.551024+0000","last_became_peered":"2024-09-17T03:33:03.551024+0000","last_unstale":"2024-09-17T03:33:04.541014+0000","last_undegraded":"2024-09-17T03:33:04.541014+0000","last_fullsized":"2024-09-17T03:33:04.541014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.061549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.547749+0000","last_change":"2024-09-17T03:33:02.539385+0000","last_active":"2024-09-17T03:33:02.547749+0000","last_peered":"2024-09-17T03:33:02.547749+0000","last_clean":"2024-09-17T03:33:02.547749+0000","last_became_active":"2024-09-17T03:33:02.539072+0000","last_became_peered":"2024-09-17T03:33:02.539072+0000","last_unstale":"2024-09-17T03:33:02.547749+0000","last_undegraded":"2024-09-17T03:33:02.547749+0000","last_fullsized":"2024-09-17T03:33:02.547749+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:33.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539523+0000","last_change":"2024-09-17T03:33:03.581118+0000","last_active":"2024-09-17T03:33:04.539523+0000","last_peered":"2024-09-17T03:33:04.539523+0000","last_clean":"2024-09-17T03:33:04.539523+0000","last_became_active":"2024-09-17T03:33:03.580758+0000","last_became_peered":"2024-09-17T03:33:03.580758+0000","last_unstale":"2024-09-17T03:33:04.539523+0000","last_undegraded":"2024-09-17T03:33:04.539523+0000","last_fullsized":"2024-09-17T03:33:04.539523+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:53.643044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.555053+0000","last_change":"2024-09-17T03:33:02.553011+0000","last_active":"2024-09-17T03:33:02.555053+0000","last_peered":"2024-09-17T03:33:02.555053+0000","last_clean":"2024-09-17T03:33:02.555053+0000","last_became_active":"2024-09-17T03:33:02.552649+0000","last_became_peered":"2024-09-17T03:33:02.552649+0000","last_unstale":"2024-09-17T03:33:02.555053+0000","last_undegraded":"2024-09-17T03:33:02.555053+0000","last_fullsized":"2024-09-17T03:33:02.555053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:31.641522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215951+0000","last_change":"2024-09-17T03:33:03.570295+0000","last_active":"2024-09-17T03:33:05.215951+0000","last_peered":"2024-09-17T03:33:05.215951+0000","last_clean":"2024-09-17T03:33:05.215951+0000","last_became_active":"2024-09-17T03:33:03.569927+0000","last_became_peered":"2024-09-17T03:33:03.569927+0000","last_unstale":"2024-09-17T03:33:05.215951+0000","last_undegraded":"2024-09-17T03:33:05.215951+0000","last_fullsized":"2024-09-17T03:33:05.215951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:36.691869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215879+0000","last_change":"2024-09-17T03:33:02.553261+0000","last_active":"2024-09-17T03:33:05.215879+0000","last_peered":"2024-09-17T03:33:05.215879+0000","last_clean":"2024-09-17T03:33:05.215879+0000","last_became_active":"2024-09-17T03:33:02.551020+0000","last_became_peered":"2024-09-17T03:33:02.551020+0000","last_unstale":"2024-09-17T03:33:05.215879+0000","last_undegraded":"2024-09-17T03:33:05.215879+0000","last_fullsized":"2024-09-17T03:33:05.215879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:01.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.531873+0000","last_change":"2024-09-17T03:33:02.531873+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.531873+0000","last_undegraded":"2024-09-17T03:33:02.531873+0000","last_fullsized":"2024-09-17T03:33:02.531873+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539246+0000","last_change":"2024-09-17T03:33:02.553687+0000","last_active":"2024-09-17T03:33:04.539246+0000","last_peered":"2024-09-17T03:33:04.539246+0000","last_clean":"2024-09-17T03:33:04.539246+0000","last_became_active":"2024-09-17T03:33:02.552355+0000","last_became_peered":"2024-09-17T03:33:02.552355+0000","last_unstale":"2024-09-17T03:33:04.539246+0000","last_undegraded":"2024-09-17T03:33:04.539246+0000","last_fullsized":"2024-09-17T03:33:04.539246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:35.198181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427004+0000","last_change":"2024-09-17T03:33:03.558620+0000","last_active":"2024-09-17T03:33:05.427004+0000","last_peered":"2024-09-17T03:33:05.427004+0000","last_clean":"2024-09-17T03:33:05.427004+0000","last_became_active":"2024-09-17T03:33:03.558274+0000","last_became_peered":"2024-09-17T03:33:03.558274+0000","last_unstale":"2024-09-17T03:33:05.427004+0000","last_undegraded":"2024-09-17T03:33:05.427004+0000","last_fullsized":"2024-09-17T03:33:05.427004+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216404+0000","last_change":"2024-09-17T03:33:02.550762+0000","last_active":"2024-09-17T03:33:05.216404+0000","last_peered":"2024-09-17T03:33:05.216404+0000","last_clean":"2024-09-17T03:33:05.216404+0000","last_became_active":"2024-09-17T03:33:02.550405+0000","last_became_peered":"2024-09-17T03:33:02.550405+0000","last_unstale":"2024-09-17T03:33:05.216404+0000","last_undegraded":"2024-09-17T03:33:05.216404+0000","last_fullsized":"2024-09-17T03:33:05.216404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426813+0000","last_change":"2024-09-17T03:33:03.572043+0000","last_active":"2024-09-17T03:33:05.426813+0000","last_peered":"2024-09-17T03:33:05.426813+0000","last_clean":"2024-09-17T03:33:05.426813+0000","last_became_active":"2024-09-17T03:33:03.570999+0000","last_became_peered":"2024-09-17T03:33:03.570999+0000","last_unstale":"2024-09-17T03:33:05.426813+0000","last_undegraded":"2024-09-17T03:33:05.426813+0000","last_fullsized":"2024-09-17T03:33:05.426813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.832675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426583+0000","last_change":"2024-09-17T03:33:02.546791+0000","last_active":"2024-09-17T03:33:05.426583+0000","last_peered":"2024-09-17T03:33:05.426583+0000","last_clean":"2024-09-17T03:33:05.426583+0000","last_became_active":"2024-09-17T03:33:02.546444+0000","last_became_peered":"2024-09-17T03:33:02.546444+0000","last_unstale":"2024-09-17T03:33:05.426583+0000","last_undegraded":"2024-09-17T03:33:05.426583+0000","last_fullsized":"2024-09-17T03:33:05.426583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.093303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548398+0000","last_change":"2024-09-17T03:33:03.561978+0000","last_active":"2024-09-17T03:33:04.548398+0000","last_peered":"2024-09-17T03:33:04.548398+0000","last_clean":"2024-09-17T03:33:04.548398+0000","last_became_active":"2024-09-17T03:33:03.561642+0000","last_became_peered":"2024-09-17T03:33:03.561642+0000","last_unstale":"2024-09-17T03:33:04.548398+0000","last_undegraded":"2024-09-17T03:33:04.548398+0000","last_fullsized":"2024-09-17T03:33:04.548398+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419887+0000","last_change":"2024-09-17T03:33:02.540951+0000","last_active":"2024-09-17T03:33:05.419887+0000","last_peered":"2024-09-17T03:33:05.419887+0000","last_clean":"2024-09-17T03:33:05.419887+0000","last_became_active":"2024-09-17T03:33:02.540486+0000","last_became_peered":"2024-09-17T03:33:02.540486+0000","last_unstale":"2024-09-17T03:33:05.419887+0000","last_undegraded":"2024-09-17T03:33:05.419887+0000","last_fullsized":"2024-09-17T03:33:05.419887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:48.319608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530771+0000","last_change":"2024-09-17T03:33:02.530771+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530771+0000","last_undegraded":"2024-09-17T03:33:02.530771+0000","last_fullsized":"2024-09-17T03:33:02.530771+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419837+0000","last_change":"2024-09-17T03:33:02.550968+0000","last_active":"2024-09-17T03:33:05.419837+0000","last_peered":"2024-09-17T03:33:05.419837+0000","last_clean":"2024-09-17T03:33:05.419837+0000","last_became_active":"2024-09-17T03:33:02.550661+0000","last_became_peered":"2024-09-17T03:33:02.550661+0000","last_unstale":"2024-09-17T03:33:05.419837+0000","last_undegraded":"2024-09-17T03:33:05.419837+0000","last_fullsized":"2024-09-17T03:33:05.419837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:42.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530834+0000","last_change":"2024-09-17T03:33:02.530834+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530834+0000","last_undegraded":"2024-09-17T03:33:02.530834+0000","last_fullsized":"2024-09-17T03:33:02.530834+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426662+0000","last_change":"2024-09-17T03:33:02.556914+0000","last_active":"2024-09-17T03:33:05.426662+0000","last_peered":"2024-09-17T03:33:05.426662+0000","last_clean":"2024-09-17T03:33:05.426662+0000","last_became_active":"2024-09-17T03:33:02.556176+0000","last_became_peered":"2024-09-17T03:33:02.556176+0000","last_unstale":"2024-09-17T03:33:05.426662+0000","last_undegraded":"2024-09-17T03:33:05.426662+0000","last_fullsized":"2024-09-17T03:33:05.426662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:19.803397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530656+0000","last_change":"2024-09-17T03:33:02.530656+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530656+0000","last_undegraded":"2024-09-17T03:33:02.530656+0000","last_fullsized":"2024-09-17T03:33:02.530656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426418+0000","last_change":"2024-09-17T03:33:02.554769+0000","last_active":"2024-09-17T03:33:05.426418+0000","last_peered":"2024-09-17T03:33:05.426418+0000","last_clean":"2024-09-17T03:33:05.426418+0000","last_became_active":"2024-09-17T03:33:02.554148+0000","last_became_peered":"2024-09-17T03:33:02.554148+0000","last_unstale":"2024-09-17T03:33:05.426418+0000","last_undegraded":"2024-09-17T03:33:05.426418+0000","last_fullsized":"2024-09-17T03:33:05.426418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:47:37.685954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426425+0000","last_change":"2024-09-17T03:33:03.587346+0000","last_active":"2024-09-17T03:33:05.426425+0000","last_peered":"2024-09-17T03:33:05.426425+0000","last_clean":"2024-09-17T03:33:05.426425+0000","last_became_active":"2024-09-17T03:33:03.587021+0000","last_became_peered":"2024-09-17T03:33:03.587021+0000","last_unstale":"2024-09-17T03:33:05.426425+0000","last_undegraded":"2024-09-17T03:33:05.426425+0000","last_fullsized":"2024-09-17T03:33:05.426425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:16.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419857+0000","last_change":"2024-09-17T03:33:02.552916+0000","last_active":"2024-09-17T03:33:05.419857+0000","last_peered":"2024-09-17T03:33:05.419857+0000","last_clean":"2024-09-17T03:33:05.419857+0000","last_became_active":"2024-09-17T03:33:02.551707+0000","last_became_peered":"2024-09-17T03:33:02.551707+0000","last_unstale":"2024-09-17T03:33:05.419857+0000","last_undegraded":"2024-09-17T03:33:05.419857+0000","last_fullsized":"2024-09-17T03:33:05.419857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:28.925660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420744+0000","last_change":"2024-09-17T03:33:03.572163+0000","last_active":"2024-09-17T03:33:05.420744+0000","last_peered":"2024-09-17T03:33:05.420744+0000","last_clean":"2024-09-17T03:33:05.420744+0000","last_became_active":"2024-09-17T03:33:03.571635+0000","last_became_peered":"2024-09-17T03:33:03.571635+0000","last_unstale":"2024-09-17T03:33:05.420744+0000","last_undegraded":"2024-09-17T03:33:05.420744+0000","last_fullsized":"2024-09-17T03:33:05.420744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:54.558593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427248+0000","last_change":"2024-09-17T03:33:02.547235+0000","last_active":"2024-09-17T03:33:05.427248+0000","last_peered":"2024-09-17T03:33:05.427248+0000","last_clean":"2024-09-17T03:33:05.427248+0000","last_became_active":"2024-09-17T03:33:02.546057+0000","last_became_peered":"2024-09-17T03:33:02.546057+0000","last_unstale":"2024-09-17T03:33:05.427248+0000","last_undegraded":"2024-09-17T03:33:05.427248+0000","last_fullsized":"2024-09-17T03:33:05.427248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:57.628934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539866+0000","last_change":"2024-09-17T03:33:03.580070+0000","last_active":"2024-09-17T03:33:04.539866+0000","last_peered":"2024-09-17T03:33:04.539866+0000","last_clean":"2024-09-17T03:33:04.539866+0000","last_became_active":"2024-09-17T03:33:03.579536+0000","last_became_peered":"2024-09-17T03:33:03.579536+0000","last_unstale":"2024-09-17T03:33:04.539866+0000","last_undegraded":"2024-09-17T03:33:04.539866+0000","last_fullsized":"2024-09-17T03:33:04.539866+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:15.857981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420763+0000","last_change":"2024-09-17T03:33:02.549424+0000","last_active":"2024-09-17T03:33:05.420763+0000","last_peered":"2024-09-17T03:33:05.420763+0000","last_clean":"2024-09-17T03:33:05.420763+0000","last_became_active":"2024-09-17T03:33:02.548964+0000","last_became_peered":"2024-09-17T03:33:02.548964+0000","last_unstale":"2024-09-17T03:33:05.420763+0000","last_undegraded":"2024-09-17T03:33:05.420763+0000","last_fullsized":"2024-09-17T03:33:05.420763+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:18.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540218+0000","last_change":"2024-09-17T03:33:03.583513+0000","last_active":"2024-09-17T03:33:04.540218+0000","last_peered":"2024-09-17T03:33:04.540218+0000","last_clean":"2024-09-17T03:33:04.540218+0000","last_became_active":"2024-09-17T03:33:03.583186+0000","last_became_peered":"2024-09-17T03:33:03.583186+0000","last_unstale":"2024-09-17T03:33:04.540218+0000","last_undegraded":"2024-09-17T03:33:04.540218+0000","last_fullsized":"2024-09-17T03:33:04.540218+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:49.099577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420208+0000","last_change":"2024-09-17T03:33:02.550099+0000","last_active":"2024-09-17T03:33:05.420208+0000","last_peered":"2024-09-17T03:33:05.420208+0000","last_clean":"2024-09-17T03:33:05.420208+0000","last_became_active":"2024-09-17T03:33:02.549761+0000","last_became_peered":"2024-09-17T03:33:02.549761+0000","last_unstale":"2024-09-17T03:33:05.420208+0000","last_undegraded":"2024-09-17T03:33:05.420208+0000","last_fullsized":"2024-09-17T03:33:05.420208+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:58.843636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426573+0000","last_change":"2024-09-17T03:33:03.561146+0000","last_active":"2024-09-17T03:33:05.426573+0000","last_peered":"2024-09-17T03:33:05.426573+0000","last_clean":"2024-09-17T03:33:05.426573+0000","last_became_active":"2024-09-17T03:33:03.560840+0000","last_became_peered":"2024-09-17T03:33:03.560840+0000","last_unstale":"2024-09-17T03:33:05.426573+0000","last_undegraded":"2024-09-17T03:33:05.426573+0000","last_fullsized":"2024-09-17T03:33:05.426573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:39.264426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426977+0000","last_change":"2024-09-17T03:33:02.550579+0000","last_active":"2024-09-17T03:33:05.426977+0000","last_peered":"2024-09-17T03:33:05.426977+0000","last_clean":"2024-09-17T03:33:05.426977+0000","last_became_active":"2024-09-17T03:33:02.550238+0000","last_became_peered":"2024-09-17T03:33:02.550238+0000","last_unstale":"2024-09-17T03:33:05.426977+0000","last_undegraded":"2024-09-17T03:33:05.426977+0000","last_fullsized":"2024-09-17T03:33:05.426977+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:42.312117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.528893+0000","last_change":"2024-09-17T03:33:02.528893+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.528893+0000","last_undegraded":"2024-09-17T03:33:02.528893+0000","last_fullsized":"2024-09-17T03:33:02.528893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420094+0000","last_change":"2024-09-17T03:33:02.554041+0000","last_active":"2024-09-17T03:33:05.420094+0000","last_peered":"2024-09-17T03:33:05.420094+0000","last_clean":"2024-09-17T03:33:05.420094+0000","last_became_active":"2024-09-17T03:33:02.553699+0000","last_became_peered":"2024-09-17T03:33:02.553699+0000","last_unstale":"2024-09-17T03:33:05.420094+0000","last_undegraded":"2024-09-17T03:33:05.420094+0000","last_fullsized":"2024-09-17T03:33:05.420094+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216039+0000","last_change":"2024-09-17T03:33:03.563718+0000","last_active":"2024-09-17T03:33:05.216039+0000","last_peered":"2024-09-17T03:33:05.216039+0000","last_clean":"2024-09-17T03:33:05.216039+0000","last_became_active":"2024-09-17T03:33:03.563409+0000","last_became_peered":"2024-09-17T03:33:03.563409+0000","last_unstale":"2024-09-17T03:33:05.216039+0000","last_undegraded":"2024-09-17T03:33:05.216039+0000","last_fullsized":"2024-09-17T03:33:05.216039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:38.241017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426089+0000","last_change":"2024-09-17T03:33:02.556605+0000","last_active":"2024-09-17T03:33:05.426089+0000","last_peered":"2024-09-17T03:33:05.426089+0000","last_clean":"2024-09-17T03:33:05.426089+0000","last_became_active":"2024-09-17T03:33:02.556146+0000","last_became_peered":"2024-09-17T03:33:02.556146+0000","last_unstale":"2024-09-17T03:33:05.426089+0000","last_undegraded":"2024-09-17T03:33:05.426089+0000","last_fullsized":"2024-09-17T03:33:05.426089+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:36.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530028+0000","last_change":"2024-09-17T03:33:02.530028+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530028+0000","last_undegraded":"2024-09-17T03:33:02.530028+0000","last_fullsized":"2024-09-17T03:33:02.530028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540887+0000","last_change":"2024-09-17T03:33:02.547114+0000","last_active":"2024-09-17T03:33:04.540887+0000","last_peered":"2024-09-17T03:33:04.540887+0000","last_clean":"2024-09-17T03:33:04.540887+0000","last_became_active":"2024-09-17T03:33:02.546778+0000","last_became_peered":"2024-09-17T03:33:02.546778+0000","last_unstale":"2024-09-17T03:33:04.540887+0000","last_undegraded":"2024-09-17T03:33:04.540887+0000","last_fullsized":"2024-09-17T03:33:04.540887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:19.611348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426304+0000","last_change":"2024-09-17T03:33:03.561358+0000","last_active":"2024-09-17T03:33:05.426304+0000","last_peered":"2024-09-17T03:33:05.426304+0000","last_clean":"2024-09-17T03:33:05.426304+0000","last_became_active":"2024-09-17T03:33:03.561027+0000","last_became_peered":"2024-09-17T03:33:03.561027+0000","last_unstale":"2024-09-17T03:33:05.426304+0000","last_undegraded":"2024-09-17T03:33:05.426304+0000","last_fullsized":"2024-09-17T03:33:05.426304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:04.965218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.553304+0000","last_change":"2024-09-17T03:33:02.544930+0000","last_active":"2024-09-17T03:33:02.553304+0000","last_peered":"2024-09-17T03:33:02.553304+0000","last_clean":"2024-09-17T03:33:02.553304+0000","last_became_active":"2024-09-17T03:33:02.542640+0000","last_became_peered":"2024-09-17T03:33:02.542640+0000","last_unstale":"2024-09-17T03:33:02.553304+0000","last_undegraded":"2024-09-17T03:33:02.553304+0000","last_fullsized":"2024-09-17T03:33:02.553304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:12.140218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530977+0000","last_change":"2024-09-17T03:33:02.530977+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530977+0000","last_undegraded":"2024-09-17T03:33:02.530977+0000","last_fullsized":"2024-09-17T03:33:02.530977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547713+0000","last_change":"2024-09-17T03:33:02.558759+0000","last_active":"2024-09-17T03:33:04.547713+0000","last_peered":"2024-09-17T03:33:04.547713+0000","last_clean":"2024-09-17T03:33:04.547713+0000","last_became_active":"2024-09-17T03:33:02.558503+0000","last_became_peered":"2024-09-17T03:33:02.558503+0000","last_unstale":"2024-09-17T03:33:04.547713+0000","last_undegraded":"2024-09-17T03:33:04.547713+0000","last_fullsized":"2024-09-17T03:33:04.547713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547648+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:33:04.547648+0000","last_peered":"2024-09-17T03:33:04.547648+0000","last_clean":"2024-09-17T03:33:04.547648+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:33:04.547648+0000","last_undegraded":"2024-09-17T03:33:04.547648+0000","last_fullsized":"2024-09-17T03:33:04.547648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127344,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244496,"statfs":{"total":96636764160,"available":96506363904,"internally_reserved":0,"allocated":1855488,"data_stored":1132666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136484,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235356,"statfs":{"total":96636764160,"available":96497004544,"internally_reserved":0,"allocated":1261568,"data_stored":542838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144816,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227024,"statfs":{"total":96636764160,"available":96488472576,"internally_reserved":0,"allocated":1273856,"data_stored":543922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130924,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240916,"statfs":{"total":96636764160,"available":96502697984,"internally_reserved":0,"allocated":1269760,"data_stored":543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136452,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235388,"statfs":{"total":96636764160,"available":96497037312,"internally_reserved":0,"allocated":1216512,"data_stored":522836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119016,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252824,"statfs":{"total":96636764160,"available":96514891776,"internally_reserved":0,"allocated":1851392,"data_stored":1133136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82324,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289516,"statfs":{"total":96636764160,"available":96552464384,"internally_reserved":0,"allocated":1245184,"data_stored":535956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140640,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231200,"statfs":{"total":96636764160,"available":96492748800,"internally_reserved":0,"allocated":1253376,"data_stored":542186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:33:08.596 INFO:tasks.ceph.ceph_manager.ceph:PG 15.32 is not active+clean 2024-09-17T03:33:08.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.535651+0000', 'last_change': '2024-09-17T03:33:02.535651+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.535651+0000', 'last_undegraded': '2024-09-17T03:33:02.535651+0000', 'last_fullsized': '2024-09-17T03:33:02.535651+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.596 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2d is not active+clean 2024-09-17T03:33:08.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.535461+0000', 'last_change': '2024-09-17T03:33:02.535461+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.535461+0000', 'last_undegraded': '2024-09-17T03:33:02.535461+0000', 'last_fullsized': '2024-09-17T03:33:02.535461+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2b is not active+clean 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.534384+0000', 'last_change': '2024-09-17T03:33:02.534384+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.534384+0000', 'last_undegraded': '2024-09-17T03:33:02.534384+0000', 'last_fullsized': '2024-09-17T03:33:02.534384+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:PG 15.22 is not active+clean 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.533906+0000', 'last_change': '2024-09-17T03:33:02.533906+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.533906+0000', 'last_undegraded': '2024-09-17T03:33:02.533906+0000', 'last_fullsized': '2024-09-17T03:33:02.533906+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1e is not active+clean 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.534628+0000', 'last_change': '2024-09-17T03:33:02.534628+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.534628+0000', 'last_undegraded': '2024-09-17T03:33:02.534628+0000', 'last_fullsized': '2024-09-17T03:33:02.534628+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:PG 15.11 is not active+clean 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.531873+0000', 'last_change': '2024-09-17T03:33:02.531873+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.531873+0000', 'last_undegraded': '2024-09-17T03:33:02.531873+0000', 'last_fullsized': '2024-09-17T03:33:02.531873+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:PG 15.a is not active+clean 2024-09-17T03:33:08.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.530771+0000', 'last_change': '2024-09-17T03:33:02.530771+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.530771+0000', 'last_undegraded': '2024-09-17T03:33:02.530771+0000', 'last_fullsized': '2024-09-17T03:33:02.530771+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:PG 15.b is not active+clean 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.530834+0000', 'last_change': '2024-09-17T03:33:02.530834+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.530834+0000', 'last_undegraded': '2024-09-17T03:33:02.530834+0000', 'last_fullsized': '2024-09-17T03:33:02.530834+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:PG 15.8 is not active+clean 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.530656+0000', 'last_change': '2024-09-17T03:33:02.530656+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.530656+0000', 'last_undegraded': '2024-09-17T03:33:02.530656+0000', 'last_fullsized': '2024-09-17T03:33:02.530656+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2 is not active+clean 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.528893+0000', 'last_change': '2024-09-17T03:33:02.528893+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.528893+0000', 'last_undegraded': '2024-09-17T03:33:02.528893+0000', 'last_fullsized': '2024-09-17T03:33:02.528893+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:PG 15.0 is not active+clean 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.530028+0000', 'last_change': '2024-09-17T03:33:02.530028+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.530028+0000', 'last_undegraded': '2024-09-17T03:33:02.530028+0000', 'last_fullsized': '2024-09-17T03:33:02.530028+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:PG 15.e is not active+clean 2024-09-17T03:33:08.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:33:02.530977+0000', 'last_change': '2024-09-17T03:33:02.530977+0000', 'last_active': '2024-09-17T03:33:02.519387+0000', 'last_peered': '2024-09-17T03:33:02.519387+0000', 'last_clean': '2024-09-17T03:33:02.519387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:33:02.530977+0000', 'last_undegraded': '2024-09-17T03:33:02.530977+0000', 'last_fullsized': '2024-09-17T03:33:02.530977+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'last_clean_scrub_stamp': '2024-09-17T03:33:02.519387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:33:08.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 -- 172.21.15.46:0/250681267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a98003a10 msgr2=0x7f7a98003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/250681267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a98003a10 0x7f7a98003e80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c4a0 tx=0x7f7aa401cf50 comp rx=0 tx=0).stop 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab3400640 1 -- 172.21.15.46:0/250681267 reap_dead start 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 -- 172.21.15.46:0/250681267 shutdown_connections 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 -- 172.21.15.46:0/250681267 >> 172.21.15.46:0/250681267 conn(0x7f7ab40b2040 msgr2=0x7f7ab40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 -- 172.21.15.46:0/250681267 shutdown_connections 2024-09-17T03:33:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 -- 172.21.15.46:0/250681267 wait complete. 2024-09-17T03:33:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.796+0000 7f7ab9a00640 1 Processor -- start 2024-09-17T03:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- start start 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7a98003a10 0x7f7ab4150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab40af960 0x7f7ab40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 0x7f7ab40cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4166910 con 0x7f7ab40cd340 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4166790 con 0x7f7ab40af960 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab4166a90 con 0x7f7a98003a10 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 0x7f7ab40cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 0x7f7ab40cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57108/0 (socket says 172.21.15.46:57108) 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 -- 172.21.15.46:0/4265250894 learned_addr learned my addr 172.21.15.46:0/4265250894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab2a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7a98003a10 0x7f7ab4150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7aaa000640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab40af960 0x7f7ab40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 -- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7a98003a10 msgr2=0x7f7ab4150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7a98003a10 0x7f7ab4150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 -- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab40af960 msgr2=0x7f7ab40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab40af960 0x7f7ab40cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 -- 172.21.15.46:0/4265250894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab4155af0 con 0x7f7ab40cd340 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab3400640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 0x7f7ab40cd750 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4156170 tx=0x7f7aac011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac004b30 con 0x7f7ab40cd340 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7aac004cd0 con 0x7f7ab40cd340 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab4153db0 con 0x7f7ab40cd340 2024-09-17T03:33:08.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac003be0 con 0x7f7ab40cd340 2024-09-17T03:33:08.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab40cdd00 con 0x7f7ab40cd340 2024-09-17T03:33:08.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.800+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab40cdf60 con 0x7f7ab40cd340 2024-09-17T03:33:08.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.804+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f7aac004030 con 0x7f7ab40cd340 2024-09-17T03:33:08.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.804+0000 7f7aab400640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7a90074910 0x7f7a90076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:08.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.804+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f7aac093dc0 con 0x7f7ab40cd340 2024-09-17T03:33:08.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.804+0000 7f7ab2a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7a90074910 0x7f7a90076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:08.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.804+0000 7f7ab2a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7a90074910 0x7f7a90076e20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40aa000 tx=0x7f7aa4019d90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:08.808+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7aac01f060 con 0x7f7ab40cd340 2024-09-17T03:33:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.044+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7ab40b51f0 con 0x7f7ab40cd340 2024-09-17T03:33:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.044+0000 7f7aab400640 1 -- 172.21.15.46:0/4265250894 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1146 (secure 0 0 0) 0x7f7aac0608d0 con 0x7f7ab40cd340 2024-09-17T03:33:09.047 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:09.047 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":746,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1726543255,"num_in_osds":8,"osd_in_since":1726543245,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":85},{"state_name":"creating+peering","count":12}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592744,"bytes_used":1042432000,"bytes_avail":772051681280,"bytes_total":773094113280,"inactive_pgs_ratio":0.12371134012937546,"write_bytes_sec":1196,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":65,"btime":"2024-09-17T03:33:06:567020+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":247692}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-17T03:31:25.427659+0000","services":{}},"progress_events":{}} 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7a90074910 msgr2=0x7f7a90076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7a90074910 0x7f7a90076e20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40aa000 tx=0x7f7aa4019d90 comp rx=0 tx=0).stop 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 msgr2=0x7f7ab40cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 0x7f7ab40cd750 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4156170 tx=0x7f7aac011ef0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 shutdown_connections 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7a90074910 0x7f7a90076e20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ab40cd340 0x7f7ab40cd750 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab40af960 0x7f7ab40cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 --2- 172.21.15.46:0/4265250894 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7a98003a10 0x7f7ab4150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 >> 172.21.15.46:0/4265250894 conn(0x7f7ab40b2040 msgr2=0x7f7ab415a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 shutdown_connections 2024-09-17T03:33:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.048+0000 7f7ab9a00640 1 -- 172.21.15.46:0/4265250894 wait complete. 2024-09-17T03:33:09.069 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:33:09.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:33:09.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b2b800640 1 -- 172.21.15.46:0/795395899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c005160 msgr2=0x7f9b0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b2b800640 1 --2- 172.21.15.46:0/795395899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c005160 0x7f9b0c00b9d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009ee0 tx=0x7f9b1c01d630 comp rx=0 tx=0).stop 2024-09-17T03:33:09.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b29800640 1 -- 172.21.15.46:0/795395899 reap_dead start 2024-09-17T03:33:09.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b2b800640 1 -- 172.21.15.46:0/795395899 shutdown_connections 2024-09-17T03:33:09.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b2b800640 1 -- 172.21.15.46:0/795395899 >> 172.21.15.46:0/795395899 conn(0x7f9b240b2040 msgr2=0x7f9b240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:09.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b2b800640 1 -- 172.21.15.46:0/795395899 shutdown_connections 2024-09-17T03:33:09.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.228+0000 7f9b2b800640 1 -- 172.21.15.46:0/795395899 wait complete. 2024-09-17T03:33:09.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 Processor -- start 2024-09-17T03:33:09.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 -- start start 2024-09-17T03:33:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b0c005160 0x7f9b2414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 0x7f9b240abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b240ac410 0x7f9b240aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2415cf30 con 0x7f9b2414ecc0 2024-09-17T03:33:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2415cdb0 con 0x7f9b0c005160 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2415d0b0 con 0x7f9b240ac410 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 0x7f9b240abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 0x7f9b240abed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57132/0 (socket says 172.21.15.46:57132) 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 -- 172.21.15.46:0/1580666523 learned_addr learned my addr 172.21.15.46:0/1580666523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b28e00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b0c005160 0x7f9b2414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b29800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b240ac410 0x7f9b240aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 -- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b240ac410 msgr2=0x7f9b240aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b240ac410 0x7f9b240aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 -- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b0c005160 msgr2=0x7f9b2414e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b0c005160 0x7f9b2414e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 -- 172.21.15.46:0/1580666523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b240ac820 con 0x7f9b2414ecc0 2024-09-17T03:33:09.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b23e00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 0x7f9b240abed0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9b10004540 tx=0x7f9b100127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:09.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10010690 con 0x7f9b2414ecc0 2024-09-17T03:33:09.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b10010830 con 0x7f9b2414ecc0 2024-09-17T03:33:09.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10010c60 con 0x7f9b2414ecc0 2024-09-17T03:33:09.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241fa230 con 0x7f9b2414ecc0 2024-09-17T03:33:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.232+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241eb630 con 0x7f9b2414ecc0 2024-09-17T03:33:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.236+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b24072a10 con 0x7f9b2414ecc0 2024-09-17T03:33:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.236+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f9b10013450 con 0x7f9b2414ecc0 2024-09-17T03:33:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.236+0000 7f9b20c00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b04074910 0x7f9b04076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.236+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9b10096b60 con 0x7f9b2414ecc0 2024-09-17T03:33:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.236+0000 7f9b28e00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b04074910 0x7f9b04076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.240+0000 7f9b28e00640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b04074910 0x7f9b04076e20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9b24071cb0 tx=0x7f9b180091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.240+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9b24072a10 con 0x7f9b2414ecc0 2024-09-17T03:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.412+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b2406b540 con 0x7f9b04074910 2024-09-17T03:33:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.416+0000 7f9b20c00640 1 -- 172.21.15.46:0/1580666523 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251442 (secure 0 0 0) 0x7f9b18009c90 con 0x7f9b04074910 2024-09-17T03:33:09.420 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b04074910 msgr2=0x7f9b04076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b04074910 0x7f9b04076e20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9b24071cb0 tx=0x7f9b180091e0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 msgr2=0x7f9b240abed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 0x7f9b240abed0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9b10004540 tx=0x7f9b100127b0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 shutdown_connections 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9b04074910 0x7f9b04076e20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9b240ac410 0x7f9b240aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b2414ecc0 0x7f9b240abed0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 --2- 172.21.15.46:0/1580666523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b0c005160 0x7f9b2414e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 >> 172.21.15.46:0/1580666523 conn(0x7f9b240b2040 msgr2=0x7f9b24151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 shutdown_connections 2024-09-17T03:33:09.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:09.420+0000 7f9b2b800640 1 -- 172.21.15.46:0/1580666523 wait complete. 2024-09-17T03:33:09.442 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-17T03:33:07.511563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":143,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1018000,"kb_used_data":10964,"kb_used_omap":12,"kb_used_meta":1006835,"kb_avail":753956720,"statfs":{"total":773094113280,"available":772051681280,"internally_reserved":0,"allocated":11227136,"data_stored":5497360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1030999600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991341"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539642+0000","last_change":"2024-09-17T03:33:03.581362+0000","last_active":"2024-09-17T03:33:04.539642+0000","last_peered":"2024-09-17T03:33:04.539642+0000","last_clean":"2024-09-17T03:33:04.539642+0000","last_became_active":"2024-09-17T03:33:03.580475+0000","last_became_peered":"2024-09-17T03:33:03.580475+0000","last_unstale":"2024-09-17T03:33:04.539642+0000","last_undegraded":"2024-09-17T03:33:04.539642+0000","last_fullsized":"2024-09-17T03:33:04.539642+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:11.925403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420228+0000","last_change":"2024-09-17T03:33:03.572549+0000","last_active":"2024-09-17T03:33:05.420228+0000","last_peered":"2024-09-17T03:33:05.420228+0000","last_clean":"2024-09-17T03:33:05.420228+0000","last_became_active":"2024-09-17T03:33:03.572232+0000","last_became_peered":"2024-09-17T03:33:03.572232+0000","last_unstale":"2024-09-17T03:33:05.420228+0000","last_undegraded":"2024-09-17T03:33:05.420228+0000","last_fullsized":"2024-09-17T03:33:05.420228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:46.104711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539783+0000","last_change":"2024-09-17T03:33:03.580570+0000","last_active":"2024-09-17T03:33:04.539783+0000","last_peered":"2024-09-17T03:33:04.539783+0000","last_clean":"2024-09-17T03:33:04.539783+0000","last_became_active":"2024-09-17T03:33:03.580241+0000","last_became_peered":"2024-09-17T03:33:03.580241+0000","last_unstale":"2024-09-17T03:33:04.539783+0000","last_undegraded":"2024-09-17T03:33:04.539783+0000","last_fullsized":"2024-09-17T03:33:04.539783+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:36.303054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541593+0000","last_change":"2024-09-17T03:33:03.551187+0000","last_active":"2024-09-17T03:33:04.541593+0000","last_peered":"2024-09-17T03:33:04.541593+0000","last_clean":"2024-09-17T03:33:04.541593+0000","last_became_active":"2024-09-17T03:33:03.550854+0000","last_became_peered":"2024-09-17T03:33:03.550854+0000","last_unstale":"2024-09-17T03:33:04.541593+0000","last_undegraded":"2024-09-17T03:33:04.541593+0000","last_fullsized":"2024-09-17T03:33:04.541593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:32.888001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427161+0000","last_change":"2024-09-17T03:33:03.562221+0000","last_active":"2024-09-17T03:33:05.427161+0000","last_peered":"2024-09-17T03:33:05.427161+0000","last_clean":"2024-09-17T03:33:05.427161+0000","last_became_active":"2024-09-17T03:33:03.561816+0000","last_became_peered":"2024-09-17T03:33:03.561816+0000","last_unstale":"2024-09-17T03:33:05.427161+0000","last_undegraded":"2024-09-17T03:33:05.427161+0000","last_fullsized":"2024-09-17T03:33:05.427161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.528483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547618+0000","last_change":"2024-09-17T03:33:03.580482+0000","last_active":"2024-09-17T03:33:04.547618+0000","last_peered":"2024-09-17T03:33:04.547618+0000","last_clean":"2024-09-17T03:33:04.547618+0000","last_became_active":"2024-09-17T03:33:03.579513+0000","last_became_peered":"2024-09-17T03:33:03.579513+0000","last_unstale":"2024-09-17T03:33:04.547618+0000","last_undegraded":"2024-09-17T03:33:04.547618+0000","last_fullsized":"2024-09-17T03:33:04.547618+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:22.208855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547571+0000","last_change":"2024-09-17T03:33:03.576842+0000","last_active":"2024-09-17T03:33:04.547571+0000","last_peered":"2024-09-17T03:33:04.547571+0000","last_clean":"2024-09-17T03:33:04.547571+0000","last_became_active":"2024-09-17T03:33:03.576572+0000","last_became_peered":"2024-09-17T03:33:03.576572+0000","last_unstale":"2024-09-17T03:33:04.547571+0000","last_undegraded":"2024-09-17T03:33:04.547571+0000","last_fullsized":"2024-09-17T03:33:04.547571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420470+0000","last_change":"2024-09-17T03:33:03.564312+0000","last_active":"2024-09-17T03:33:05.420470+0000","last_peered":"2024-09-17T03:33:05.420470+0000","last_clean":"2024-09-17T03:33:05.420470+0000","last_became_active":"2024-09-17T03:33:03.563925+0000","last_became_peered":"2024-09-17T03:33:03.563925+0000","last_unstale":"2024-09-17T03:33:05.420470+0000","last_undegraded":"2024-09-17T03:33:05.420470+0000","last_fullsized":"2024-09-17T03:33:05.420470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:00.239782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216739+0000","last_change":"2024-09-17T03:33:03.556879+0000","last_active":"2024-09-17T03:33:05.216739+0000","last_peered":"2024-09-17T03:33:05.216739+0000","last_clean":"2024-09-17T03:33:05.216739+0000","last_became_active":"2024-09-17T03:33:03.556579+0000","last_became_peered":"2024-09-17T03:33:03.556579+0000","last_unstale":"2024-09-17T03:33:05.216739+0000","last_undegraded":"2024-09-17T03:33:05.216739+0000","last_fullsized":"2024-09-17T03:33:05.216739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:00.787049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426696+0000","last_change":"2024-09-17T03:33:03.590158+0000","last_active":"2024-09-17T03:33:05.426696+0000","last_peered":"2024-09-17T03:33:05.426696+0000","last_clean":"2024-09-17T03:33:05.426696+0000","last_became_active":"2024-09-17T03:33:03.589751+0000","last_became_peered":"2024-09-17T03:33:03.589751+0000","last_unstale":"2024-09-17T03:33:05.426696+0000","last_undegraded":"2024-09-17T03:33:05.426696+0000","last_fullsized":"2024-09-17T03:33:05.426696+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:08.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548057+0000","last_change":"2024-09-17T03:33:03.559214+0000","last_active":"2024-09-17T03:33:04.548057+0000","last_peered":"2024-09-17T03:33:04.548057+0000","last_clean":"2024-09-17T03:33:04.548057+0000","last_became_active":"2024-09-17T03:33:03.558880+0000","last_became_peered":"2024-09-17T03:33:03.558880+0000","last_unstale":"2024-09-17T03:33:04.548057+0000","last_undegraded":"2024-09-17T03:33:04.548057+0000","last_fullsized":"2024-09-17T03:33:04.548057+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:32.880704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216475+0000","last_change":"2024-09-17T03:33:03.595048+0000","last_active":"2024-09-17T03:33:05.216475+0000","last_peered":"2024-09-17T03:33:05.216475+0000","last_clean":"2024-09-17T03:33:05.216475+0000","last_became_active":"2024-09-17T03:33:03.594649+0000","last_became_peered":"2024-09-17T03:33:03.594649+0000","last_unstale":"2024-09-17T03:33:05.216475+0000","last_undegraded":"2024-09-17T03:33:05.216475+0000","last_fullsized":"2024-09-17T03:33:05.216475+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:26.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541461+0000","last_change":"2024-09-17T03:33:03.546430+0000","last_active":"2024-09-17T03:33:04.541461+0000","last_peered":"2024-09-17T03:33:04.541461+0000","last_clean":"2024-09-17T03:33:04.541461+0000","last_became_active":"2024-09-17T03:33:03.546006+0000","last_became_peered":"2024-09-17T03:33:03.546006+0000","last_unstale":"2024-09-17T03:33:04.541461+0000","last_undegraded":"2024-09-17T03:33:04.541461+0000","last_fullsized":"2024-09-17T03:33:04.541461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535651+0000","last_change":"2024-09-17T03:33:02.535651+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535651+0000","last_undegraded":"2024-09-17T03:33:02.535651+0000","last_fullsized":"2024-09-17T03:33:02.535651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540556+0000","last_change":"2024-09-17T03:33:03.580895+0000","last_active":"2024-09-17T03:33:04.540556+0000","last_peered":"2024-09-17T03:33:04.540556+0000","last_clean":"2024-09-17T03:33:04.540556+0000","last_became_active":"2024-09-17T03:33:03.580561+0000","last_became_peered":"2024-09-17T03:33:03.580561+0000","last_unstale":"2024-09-17T03:33:04.540556+0000","last_undegraded":"2024-09-17T03:33:04.540556+0000","last_fullsized":"2024-09-17T03:33:04.540556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:59.846145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547563+0000","last_change":"2024-09-17T03:33:03.573694+0000","last_active":"2024-09-17T03:33:04.547563+0000","last_peered":"2024-09-17T03:33:04.547563+0000","last_clean":"2024-09-17T03:33:04.547563+0000","last_became_active":"2024-09-17T03:33:03.573199+0000","last_became_peered":"2024-09-17T03:33:03.573199+0000","last_unstale":"2024-09-17T03:33:04.547563+0000","last_undegraded":"2024-09-17T03:33:04.547563+0000","last_fullsized":"2024-09-17T03:33:04.547563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:14.793531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420606+0000","last_change":"2024-09-17T03:33:03.570549+0000","last_active":"2024-09-17T03:33:05.420606+0000","last_peered":"2024-09-17T03:33:05.420606+0000","last_clean":"2024-09-17T03:33:05.420606+0000","last_became_active":"2024-09-17T03:33:03.570244+0000","last_became_peered":"2024-09-17T03:33:03.570244+0000","last_unstale":"2024-09-17T03:33:05.420606+0000","last_undegraded":"2024-09-17T03:33:05.420606+0000","last_fullsized":"2024-09-17T03:33:05.420606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:57.566540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541670+0000","last_change":"2024-09-17T03:33:03.558416+0000","last_active":"2024-09-17T03:33:04.541670+0000","last_peered":"2024-09-17T03:33:04.541670+0000","last_clean":"2024-09-17T03:33:04.541670+0000","last_became_active":"2024-09-17T03:33:03.558033+0000","last_became_peered":"2024-09-17T03:33:03.558033+0000","last_unstale":"2024-09-17T03:33:04.541670+0000","last_undegraded":"2024-09-17T03:33:04.541670+0000","last_fullsized":"2024-09-17T03:33:04.541670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:07.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.535461+0000","last_change":"2024-09-17T03:33:02.535461+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.535461+0000","last_undegraded":"2024-09-17T03:33:02.535461+0000","last_fullsized":"2024-09-17T03:33:02.535461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216563+0000","last_change":"2024-09-17T03:33:03.556063+0000","last_active":"2024-09-17T03:33:05.216563+0000","last_peered":"2024-09-17T03:33:05.216563+0000","last_clean":"2024-09-17T03:33:05.216563+0000","last_became_active":"2024-09-17T03:33:03.555571+0000","last_became_peered":"2024-09-17T03:33:03.555571+0000","last_unstale":"2024-09-17T03:33:05.216563+0000","last_undegraded":"2024-09-17T03:33:05.216563+0000","last_fullsized":"2024-09-17T03:33:05.216563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:58.787401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534384+0000","last_change":"2024-09-17T03:33:02.534384+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534384+0000","last_undegraded":"2024-09-17T03:33:02.534384+0000","last_fullsized":"2024-09-17T03:33:02.534384+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420061+0000","last_change":"2024-09-17T03:33:03.572171+0000","last_active":"2024-09-17T03:33:05.420061+0000","last_peered":"2024-09-17T03:33:05.420061+0000","last_clean":"2024-09-17T03:33:05.420061+0000","last_became_active":"2024-09-17T03:33:03.571829+0000","last_became_peered":"2024-09-17T03:33:03.571829+0000","last_unstale":"2024-09-17T03:33:05.420061+0000","last_undegraded":"2024-09-17T03:33:05.420061+0000","last_fullsized":"2024-09-17T03:33:05.420061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:38.888214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419929+0000","last_change":"2024-09-17T03:33:03.572010+0000","last_active":"2024-09-17T03:33:05.419929+0000","last_peered":"2024-09-17T03:33:05.419929+0000","last_clean":"2024-09-17T03:33:05.419929+0000","last_became_active":"2024-09-17T03:33:03.571696+0000","last_became_peered":"2024-09-17T03:33:03.571696+0000","last_unstale":"2024-09-17T03:33:05.419929+0000","last_undegraded":"2024-09-17T03:33:05.419929+0000","last_fullsized":"2024-09-17T03:33:05.419929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.483130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547565+0000","last_change":"2024-09-17T03:33:03.589869+0000","last_active":"2024-09-17T03:33:04.547565+0000","last_peered":"2024-09-17T03:33:04.547565+0000","last_clean":"2024-09-17T03:33:04.547565+0000","last_became_active":"2024-09-17T03:33:03.589493+0000","last_became_peered":"2024-09-17T03:33:03.589493+0000","last_unstale":"2024-09-17T03:33:04.547565+0000","last_undegraded":"2024-09-17T03:33:04.547565+0000","last_fullsized":"2024-09-17T03:33:04.547565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.449496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426760+0000","last_change":"2024-09-17T03:33:03.576923+0000","last_active":"2024-09-17T03:33:05.426760+0000","last_peered":"2024-09-17T03:33:05.426760+0000","last_clean":"2024-09-17T03:33:05.426760+0000","last_became_active":"2024-09-17T03:33:03.576548+0000","last_became_peered":"2024-09-17T03:33:03.576548+0000","last_unstale":"2024-09-17T03:33:05.426760+0000","last_undegraded":"2024-09-17T03:33:05.426760+0000","last_fullsized":"2024-09-17T03:33:05.426760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:08.997404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420450+0000","last_change":"2024-09-17T03:33:03.583037+0000","last_active":"2024-09-17T03:33:05.420450+0000","last_peered":"2024-09-17T03:33:05.420450+0000","last_clean":"2024-09-17T03:33:05.420450+0000","last_became_active":"2024-09-17T03:33:03.582714+0000","last_became_peered":"2024-09-17T03:33:03.582714+0000","last_unstale":"2024-09-17T03:33:05.420450+0000","last_undegraded":"2024-09-17T03:33:05.420450+0000","last_fullsized":"2024-09-17T03:33:05.420450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:08.781666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548160+0000","last_change":"2024-09-17T03:33:03.567964+0000","last_active":"2024-09-17T03:33:04.548160+0000","last_peered":"2024-09-17T03:33:04.548160+0000","last_clean":"2024-09-17T03:33:04.548160+0000","last_became_active":"2024-09-17T03:33:03.567632+0000","last_became_peered":"2024-09-17T03:33:03.567632+0000","last_unstale":"2024-09-17T03:33:04.548160+0000","last_undegraded":"2024-09-17T03:33:04.548160+0000","last_fullsized":"2024-09-17T03:33:04.548160+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:39.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426889+0000","last_change":"2024-09-17T03:33:03.559826+0000","last_active":"2024-09-17T03:33:05.426889+0000","last_peered":"2024-09-17T03:33:05.426889+0000","last_clean":"2024-09-17T03:33:05.426889+0000","last_became_active":"2024-09-17T03:33:03.559504+0000","last_became_peered":"2024-09-17T03:33:03.559504+0000","last_unstale":"2024-09-17T03:33:05.426889+0000","last_undegraded":"2024-09-17T03:33:05.426889+0000","last_fullsized":"2024-09-17T03:33:05.426889+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:54.833105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420363+0000","last_change":"2024-09-17T03:33:03.580138+0000","last_active":"2024-09-17T03:33:05.420363+0000","last_peered":"2024-09-17T03:33:05.420363+0000","last_clean":"2024-09-17T03:33:05.420363+0000","last_became_active":"2024-09-17T03:33:03.579782+0000","last_became_peered":"2024-09-17T03:33:03.579782+0000","last_unstale":"2024-09-17T03:33:05.420363+0000","last_undegraded":"2024-09-17T03:33:05.420363+0000","last_fullsized":"2024-09-17T03:33:05.420363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:07.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.533906+0000","last_change":"2024-09-17T03:33:02.533906+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.533906+0000","last_undegraded":"2024-09-17T03:33:02.533906+0000","last_fullsized":"2024-09-17T03:33:02.533906+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541093+0000","last_change":"2024-09-17T03:33:03.565826+0000","last_active":"2024-09-17T03:33:04.541093+0000","last_peered":"2024-09-17T03:33:04.541093+0000","last_clean":"2024-09-17T03:33:04.541093+0000","last_became_active":"2024-09-17T03:33:03.557053+0000","last_became_peered":"2024-09-17T03:33:03.557053+0000","last_unstale":"2024-09-17T03:33:04.541093+0000","last_undegraded":"2024-09-17T03:33:04.541093+0000","last_fullsized":"2024-09-17T03:33:04.541093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:54.693438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539233+0000","last_change":"2024-09-17T03:33:03.581059+0000","last_active":"2024-09-17T03:33:04.539233+0000","last_peered":"2024-09-17T03:33:04.539233+0000","last_clean":"2024-09-17T03:33:04.539233+0000","last_became_active":"2024-09-17T03:33:03.580721+0000","last_became_peered":"2024-09-17T03:33:03.580721+0000","last_unstale":"2024-09-17T03:33:04.539233+0000","last_undegraded":"2024-09-17T03:33:04.539233+0000","last_fullsized":"2024-09-17T03:33:04.539233+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:12.767168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.534628+0000","last_change":"2024-09-17T03:33:02.534628+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.534628+0000","last_undegraded":"2024-09-17T03:33:02.534628+0000","last_fullsized":"2024-09-17T03:33:02.534628+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420603+0000","last_change":"2024-09-17T03:33:02.543361+0000","last_active":"2024-09-17T03:33:05.420603+0000","last_peered":"2024-09-17T03:33:05.420603+0000","last_clean":"2024-09-17T03:33:05.420603+0000","last_became_active":"2024-09-17T03:33:02.543063+0000","last_became_peered":"2024-09-17T03:33:02.543063+0000","last_unstale":"2024-09-17T03:33:05.420603+0000","last_undegraded":"2024-09-17T03:33:05.420603+0000","last_fullsized":"2024-09-17T03:33:05.420603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:01.651709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541863+0000","last_change":"2024-09-17T03:33:03.556411+0000","last_active":"2024-09-17T03:33:04.541863+0000","last_peered":"2024-09-17T03:33:04.541863+0000","last_clean":"2024-09-17T03:33:04.541863+0000","last_became_active":"2024-09-17T03:33:03.555405+0000","last_became_peered":"2024-09-17T03:33:03.555405+0000","last_unstale":"2024-09-17T03:33:04.541863+0000","last_undegraded":"2024-09-17T03:33:04.541863+0000","last_fullsized":"2024-09-17T03:33:04.541863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:44.098643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547818+0000","last_change":"2024-09-17T03:33:02.563716+0000","last_active":"2024-09-17T03:33:04.547818+0000","last_peered":"2024-09-17T03:33:04.547818+0000","last_clean":"2024-09-17T03:33:04.547818+0000","last_became_active":"2024-09-17T03:33:02.563400+0000","last_became_peered":"2024-09-17T03:33:02.563400+0000","last_unstale":"2024-09-17T03:33:04.547818+0000","last_undegraded":"2024-09-17T03:33:04.547818+0000","last_fullsized":"2024-09-17T03:33:04.547818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540439+0000","last_change":"2024-09-17T03:33:03.587192+0000","last_active":"2024-09-17T03:33:04.540439+0000","last_peered":"2024-09-17T03:33:04.540439+0000","last_clean":"2024-09-17T03:33:04.540439+0000","last_became_active":"2024-09-17T03:33:03.586800+0000","last_became_peered":"2024-09-17T03:33:03.586800+0000","last_unstale":"2024-09-17T03:33:04.540439+0000","last_undegraded":"2024-09-17T03:33:04.540439+0000","last_fullsized":"2024-09-17T03:33:04.540439+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:04.696195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419965+0000","last_change":"2024-09-17T03:33:02.555846+0000","last_active":"2024-09-17T03:33:05.419965+0000","last_peered":"2024-09-17T03:33:05.419965+0000","last_clean":"2024-09-17T03:33:05.419965+0000","last_became_active":"2024-09-17T03:33:02.554535+0000","last_became_peered":"2024-09-17T03:33:02.554535+0000","last_unstale":"2024-09-17T03:33:05.419965+0000","last_undegraded":"2024-09-17T03:33:05.419965+0000","last_fullsized":"2024-09-17T03:33:05.419965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:20.329251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216814+0000","last_change":"2024-09-17T03:33:03.576310+0000","last_active":"2024-09-17T03:33:05.216814+0000","last_peered":"2024-09-17T03:33:05.216814+0000","last_clean":"2024-09-17T03:33:05.216814+0000","last_became_active":"2024-09-17T03:33:03.575941+0000","last_became_peered":"2024-09-17T03:33:03.575941+0000","last_unstale":"2024-09-17T03:33:05.216814+0000","last_undegraded":"2024-09-17T03:33:05.216814+0000","last_fullsized":"2024-09-17T03:33:05.216814+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:19.596987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419816+0000","last_change":"2024-09-17T03:33:02.539022+0000","last_active":"2024-09-17T03:33:05.419816+0000","last_peered":"2024-09-17T03:33:05.419816+0000","last_clean":"2024-09-17T03:33:05.419816+0000","last_became_active":"2024-09-17T03:33:02.538700+0000","last_became_peered":"2024-09-17T03:33:02.538700+0000","last_unstale":"2024-09-17T03:33:05.419816+0000","last_undegraded":"2024-09-17T03:33:05.419816+0000","last_fullsized":"2024-09-17T03:33:05.419816+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:31.500503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426743+0000","last_change":"2024-09-17T03:33:03.569261+0000","last_active":"2024-09-17T03:33:05.426743+0000","last_peered":"2024-09-17T03:33:05.426743+0000","last_clean":"2024-09-17T03:33:05.426743+0000","last_became_active":"2024-09-17T03:33:03.568877+0000","last_became_peered":"2024-09-17T03:33:03.568877+0000","last_unstale":"2024-09-17T03:33:05.426743+0000","last_undegraded":"2024-09-17T03:33:05.426743+0000","last_fullsized":"2024-09-17T03:33:05.426743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:04.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539368+0000","last_change":"2024-09-17T03:33:02.550110+0000","last_active":"2024-09-17T03:33:04.539368+0000","last_peered":"2024-09-17T03:33:04.539368+0000","last_clean":"2024-09-17T03:33:04.539368+0000","last_became_active":"2024-09-17T03:33:02.549795+0000","last_became_peered":"2024-09-17T03:33:02.549795+0000","last_unstale":"2024-09-17T03:33:04.539368+0000","last_undegraded":"2024-09-17T03:33:04.539368+0000","last_fullsized":"2024-09-17T03:33:04.539368+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:47.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547977+0000","last_change":"2024-09-17T03:33:03.564062+0000","last_active":"2024-09-17T03:33:04.547977+0000","last_peered":"2024-09-17T03:33:04.547977+0000","last_clean":"2024-09-17T03:33:04.547977+0000","last_became_active":"2024-09-17T03:33:03.563650+0000","last_became_peered":"2024-09-17T03:33:03.563650+0000","last_unstale":"2024-09-17T03:33:04.547977+0000","last_undegraded":"2024-09-17T03:33:04.547977+0000","last_fullsized":"2024-09-17T03:33:04.547977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:08.738111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419621+0000","last_change":"2024-09-17T03:33:02.549221+0000","last_active":"2024-09-17T03:33:05.419621+0000","last_peered":"2024-09-17T03:33:05.419621+0000","last_clean":"2024-09-17T03:33:05.419621+0000","last_became_active":"2024-09-17T03:33:02.548861+0000","last_became_peered":"2024-09-17T03:33:02.548861+0000","last_unstale":"2024-09-17T03:33:05.419621+0000","last_undegraded":"2024-09-17T03:33:05.419621+0000","last_fullsized":"2024-09-17T03:33:05.419621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540576+0000","last_change":"2024-09-17T03:33:03.581594+0000","last_active":"2024-09-17T03:33:04.540576+0000","last_peered":"2024-09-17T03:33:04.540576+0000","last_clean":"2024-09-17T03:33:04.540576+0000","last_became_active":"2024-09-17T03:33:03.581295+0000","last_became_peered":"2024-09-17T03:33:03.581295+0000","last_unstale":"2024-09-17T03:33:04.540576+0000","last_undegraded":"2024-09-17T03:33:04.540576+0000","last_fullsized":"2024-09-17T03:33:04.540576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:02.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426395+0000","last_change":"2024-09-17T03:33:02.555268+0000","last_active":"2024-09-17T03:33:05.426395+0000","last_peered":"2024-09-17T03:33:05.426395+0000","last_clean":"2024-09-17T03:33:05.426395+0000","last_became_active":"2024-09-17T03:33:02.554769+0000","last_became_peered":"2024-09-17T03:33:02.554769+0000","last_unstale":"2024-09-17T03:33:05.426395+0000","last_undegraded":"2024-09-17T03:33:05.426395+0000","last_fullsized":"2024-09-17T03:33:05.426395+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:38.916564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215985+0000","last_change":"2024-09-17T03:33:03.570252+0000","last_active":"2024-09-17T03:33:05.215985+0000","last_peered":"2024-09-17T03:33:05.215985+0000","last_clean":"2024-09-17T03:33:05.215985+0000","last_became_active":"2024-09-17T03:33:03.569899+0000","last_became_peered":"2024-09-17T03:33:03.569899+0000","last_unstale":"2024-09-17T03:33:05.215985+0000","last_undegraded":"2024-09-17T03:33:05.215985+0000","last_fullsized":"2024-09-17T03:33:05.215985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:27.095812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539125+0000","last_change":"2024-09-17T03:33:02.551914+0000","last_active":"2024-09-17T03:33:04.539125+0000","last_peered":"2024-09-17T03:33:04.539125+0000","last_clean":"2024-09-17T03:33:04.539125+0000","last_became_active":"2024-09-17T03:33:02.551576+0000","last_became_peered":"2024-09-17T03:33:02.551576+0000","last_unstale":"2024-09-17T03:33:04.539125+0000","last_undegraded":"2024-09-17T03:33:04.539125+0000","last_fullsized":"2024-09-17T03:33:04.539125+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:46.988395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420593+0000","last_change":"2024-09-17T03:33:03.558995+0000","last_active":"2024-09-17T03:33:05.420593+0000","last_peered":"2024-09-17T03:33:05.420593+0000","last_clean":"2024-09-17T03:33:05.420593+0000","last_became_active":"2024-09-17T03:33:03.558645+0000","last_became_peered":"2024-09-17T03:33:03.558645+0000","last_unstale":"2024-09-17T03:33:05.420593+0000","last_undegraded":"2024-09-17T03:33:05.420593+0000","last_fullsized":"2024-09-17T03:33:05.420593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:28.537475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539936+0000","last_change":"2024-09-17T03:33:02.556533+0000","last_active":"2024-09-17T03:33:04.539936+0000","last_peered":"2024-09-17T03:33:04.539936+0000","last_clean":"2024-09-17T03:33:04.539936+0000","last_became_active":"2024-09-17T03:33:02.556134+0000","last_became_peered":"2024-09-17T03:33:02.556134+0000","last_unstale":"2024-09-17T03:33:04.539936+0000","last_undegraded":"2024-09-17T03:33:04.539936+0000","last_fullsized":"2024-09-17T03:33:04.539936+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:08.763261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216780+0000","last_change":"2024-09-17T03:33:03.575625+0000","last_active":"2024-09-17T03:33:05.216780+0000","last_peered":"2024-09-17T03:33:05.216780+0000","last_clean":"2024-09-17T03:33:05.216780+0000","last_became_active":"2024-09-17T03:33:03.575239+0000","last_became_peered":"2024-09-17T03:33:03.575239+0000","last_unstale":"2024-09-17T03:33:05.216780+0000","last_undegraded":"2024-09-17T03:33:05.216780+0000","last_fullsized":"2024-09-17T03:33:05.216780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:57.701386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541519+0000","last_change":"2024-09-17T03:33:02.547247+0000","last_active":"2024-09-17T03:33:04.541519+0000","last_peered":"2024-09-17T03:33:04.541519+0000","last_clean":"2024-09-17T03:33:04.541519+0000","last_became_active":"2024-09-17T03:33:02.546932+0000","last_became_peered":"2024-09-17T03:33:02.546932+0000","last_unstale":"2024-09-17T03:33:04.541519+0000","last_undegraded":"2024-09-17T03:33:04.541519+0000","last_fullsized":"2024-09-17T03:33:04.541519+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.962710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427462+0000","last_change":"2024-09-17T03:33:03.569370+0000","last_active":"2024-09-17T03:33:05.427462+0000","last_peered":"2024-09-17T03:33:05.427462+0000","last_clean":"2024-09-17T03:33:05.427462+0000","last_became_active":"2024-09-17T03:33:03.569034+0000","last_became_peered":"2024-09-17T03:33:03.569034+0000","last_unstale":"2024-09-17T03:33:05.427462+0000","last_undegraded":"2024-09-17T03:33:05.427462+0000","last_fullsized":"2024-09-17T03:33:05.427462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:36.003958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548375+0000","last_change":"2024-09-17T03:33:02.557519+0000","last_active":"2024-09-17T03:33:04.548375+0000","last_peered":"2024-09-17T03:33:04.548375+0000","last_clean":"2024-09-17T03:33:04.548375+0000","last_became_active":"2024-09-17T03:33:02.557143+0000","last_became_peered":"2024-09-17T03:33:02.557143+0000","last_unstale":"2024-09-17T03:33:04.548375+0000","last_undegraded":"2024-09-17T03:33:04.548375+0000","last_fullsized":"2024-09-17T03:33:04.548375+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.298939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548299+0000","last_change":"2024-09-17T03:33:03.567932+0000","last_active":"2024-09-17T03:33:04.548299+0000","last_peered":"2024-09-17T03:33:04.548299+0000","last_clean":"2024-09-17T03:33:04.548299+0000","last_became_active":"2024-09-17T03:33:03.567611+0000","last_became_peered":"2024-09-17T03:33:03.567611+0000","last_unstale":"2024-09-17T03:33:04.548299+0000","last_undegraded":"2024-09-17T03:33:04.548299+0000","last_fullsized":"2024-09-17T03:33:04.548299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:15.466755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216540+0000","last_change":"2024-09-17T03:33:02.552966+0000","last_active":"2024-09-17T03:33:05.216540+0000","last_peered":"2024-09-17T03:33:05.216540+0000","last_clean":"2024-09-17T03:33:05.216540+0000","last_became_active":"2024-09-17T03:33:02.552610+0000","last_became_peered":"2024-09-17T03:33:02.552610+0000","last_unstale":"2024-09-17T03:33:05.216540+0000","last_undegraded":"2024-09-17T03:33:05.216540+0000","last_fullsized":"2024-09-17T03:33:05.216540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:57.033583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541014+0000","last_change":"2024-09-17T03:33:03.551601+0000","last_active":"2024-09-17T03:33:04.541014+0000","last_peered":"2024-09-17T03:33:04.541014+0000","last_clean":"2024-09-17T03:33:04.541014+0000","last_became_active":"2024-09-17T03:33:03.551024+0000","last_became_peered":"2024-09-17T03:33:03.551024+0000","last_unstale":"2024-09-17T03:33:04.541014+0000","last_undegraded":"2024-09-17T03:33:04.541014+0000","last_fullsized":"2024-09-17T03:33:04.541014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.061549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.547749+0000","last_change":"2024-09-17T03:33:02.539385+0000","last_active":"2024-09-17T03:33:02.547749+0000","last_peered":"2024-09-17T03:33:02.547749+0000","last_clean":"2024-09-17T03:33:02.547749+0000","last_became_active":"2024-09-17T03:33:02.539072+0000","last_became_peered":"2024-09-17T03:33:02.539072+0000","last_unstale":"2024-09-17T03:33:02.547749+0000","last_undegraded":"2024-09-17T03:33:02.547749+0000","last_fullsized":"2024-09-17T03:33:02.547749+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:33.440931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539523+0000","last_change":"2024-09-17T03:33:03.581118+0000","last_active":"2024-09-17T03:33:04.539523+0000","last_peered":"2024-09-17T03:33:04.539523+0000","last_clean":"2024-09-17T03:33:04.539523+0000","last_became_active":"2024-09-17T03:33:03.580758+0000","last_became_peered":"2024-09-17T03:33:03.580758+0000","last_unstale":"2024-09-17T03:33:04.539523+0000","last_undegraded":"2024-09-17T03:33:04.539523+0000","last_fullsized":"2024-09-17T03:33:04.539523+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:53.643044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.555053+0000","last_change":"2024-09-17T03:33:02.553011+0000","last_active":"2024-09-17T03:33:02.555053+0000","last_peered":"2024-09-17T03:33:02.555053+0000","last_clean":"2024-09-17T03:33:02.555053+0000","last_became_active":"2024-09-17T03:33:02.552649+0000","last_became_peered":"2024-09-17T03:33:02.552649+0000","last_unstale":"2024-09-17T03:33:02.555053+0000","last_undegraded":"2024-09-17T03:33:02.555053+0000","last_fullsized":"2024-09-17T03:33:02.555053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:31.641522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215951+0000","last_change":"2024-09-17T03:33:03.570295+0000","last_active":"2024-09-17T03:33:05.215951+0000","last_peered":"2024-09-17T03:33:05.215951+0000","last_clean":"2024-09-17T03:33:05.215951+0000","last_became_active":"2024-09-17T03:33:03.569927+0000","last_became_peered":"2024-09-17T03:33:03.569927+0000","last_unstale":"2024-09-17T03:33:05.215951+0000","last_undegraded":"2024-09-17T03:33:05.215951+0000","last_fullsized":"2024-09-17T03:33:05.215951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:36.691869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215879+0000","last_change":"2024-09-17T03:33:02.553261+0000","last_active":"2024-09-17T03:33:05.215879+0000","last_peered":"2024-09-17T03:33:05.215879+0000","last_clean":"2024-09-17T03:33:05.215879+0000","last_became_active":"2024-09-17T03:33:02.551020+0000","last_became_peered":"2024-09-17T03:33:02.551020+0000","last_unstale":"2024-09-17T03:33:05.215879+0000","last_undegraded":"2024-09-17T03:33:05.215879+0000","last_fullsized":"2024-09-17T03:33:05.215879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:01.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.531873+0000","last_change":"2024-09-17T03:33:02.531873+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.531873+0000","last_undegraded":"2024-09-17T03:33:02.531873+0000","last_fullsized":"2024-09-17T03:33:02.531873+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539246+0000","last_change":"2024-09-17T03:33:02.553687+0000","last_active":"2024-09-17T03:33:04.539246+0000","last_peered":"2024-09-17T03:33:04.539246+0000","last_clean":"2024-09-17T03:33:04.539246+0000","last_became_active":"2024-09-17T03:33:02.552355+0000","last_became_peered":"2024-09-17T03:33:02.552355+0000","last_unstale":"2024-09-17T03:33:04.539246+0000","last_undegraded":"2024-09-17T03:33:04.539246+0000","last_fullsized":"2024-09-17T03:33:04.539246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:35.198181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427004+0000","last_change":"2024-09-17T03:33:03.558620+0000","last_active":"2024-09-17T03:33:05.427004+0000","last_peered":"2024-09-17T03:33:05.427004+0000","last_clean":"2024-09-17T03:33:05.427004+0000","last_became_active":"2024-09-17T03:33:03.558274+0000","last_became_peered":"2024-09-17T03:33:03.558274+0000","last_unstale":"2024-09-17T03:33:05.427004+0000","last_undegraded":"2024-09-17T03:33:05.427004+0000","last_fullsized":"2024-09-17T03:33:05.427004+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216404+0000","last_change":"2024-09-17T03:33:02.550762+0000","last_active":"2024-09-17T03:33:05.216404+0000","last_peered":"2024-09-17T03:33:05.216404+0000","last_clean":"2024-09-17T03:33:05.216404+0000","last_became_active":"2024-09-17T03:33:02.550405+0000","last_became_peered":"2024-09-17T03:33:02.550405+0000","last_unstale":"2024-09-17T03:33:05.216404+0000","last_undegraded":"2024-09-17T03:33:05.216404+0000","last_fullsized":"2024-09-17T03:33:05.216404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426813+0000","last_change":"2024-09-17T03:33:03.572043+0000","last_active":"2024-09-17T03:33:05.426813+0000","last_peered":"2024-09-17T03:33:05.426813+0000","last_clean":"2024-09-17T03:33:05.426813+0000","last_became_active":"2024-09-17T03:33:03.570999+0000","last_became_peered":"2024-09-17T03:33:03.570999+0000","last_unstale":"2024-09-17T03:33:05.426813+0000","last_undegraded":"2024-09-17T03:33:05.426813+0000","last_fullsized":"2024-09-17T03:33:05.426813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.832675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426583+0000","last_change":"2024-09-17T03:33:02.546791+0000","last_active":"2024-09-17T03:33:05.426583+0000","last_peered":"2024-09-17T03:33:05.426583+0000","last_clean":"2024-09-17T03:33:05.426583+0000","last_became_active":"2024-09-17T03:33:02.546444+0000","last_became_peered":"2024-09-17T03:33:02.546444+0000","last_unstale":"2024-09-17T03:33:05.426583+0000","last_undegraded":"2024-09-17T03:33:05.426583+0000","last_fullsized":"2024-09-17T03:33:05.426583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.093303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548398+0000","last_change":"2024-09-17T03:33:03.561978+0000","last_active":"2024-09-17T03:33:04.548398+0000","last_peered":"2024-09-17T03:33:04.548398+0000","last_clean":"2024-09-17T03:33:04.548398+0000","last_became_active":"2024-09-17T03:33:03.561642+0000","last_became_peered":"2024-09-17T03:33:03.561642+0000","last_unstale":"2024-09-17T03:33:04.548398+0000","last_undegraded":"2024-09-17T03:33:04.548398+0000","last_fullsized":"2024-09-17T03:33:04.548398+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419887+0000","last_change":"2024-09-17T03:33:02.540951+0000","last_active":"2024-09-17T03:33:05.419887+0000","last_peered":"2024-09-17T03:33:05.419887+0000","last_clean":"2024-09-17T03:33:05.419887+0000","last_became_active":"2024-09-17T03:33:02.540486+0000","last_became_peered":"2024-09-17T03:33:02.540486+0000","last_unstale":"2024-09-17T03:33:05.419887+0000","last_undegraded":"2024-09-17T03:33:05.419887+0000","last_fullsized":"2024-09-17T03:33:05.419887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:48.319608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530771+0000","last_change":"2024-09-17T03:33:02.530771+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530771+0000","last_undegraded":"2024-09-17T03:33:02.530771+0000","last_fullsized":"2024-09-17T03:33:02.530771+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419837+0000","last_change":"2024-09-17T03:33:02.550968+0000","last_active":"2024-09-17T03:33:05.419837+0000","last_peered":"2024-09-17T03:33:05.419837+0000","last_clean":"2024-09-17T03:33:05.419837+0000","last_became_active":"2024-09-17T03:33:02.550661+0000","last_became_peered":"2024-09-17T03:33:02.550661+0000","last_unstale":"2024-09-17T03:33:05.419837+0000","last_undegraded":"2024-09-17T03:33:05.419837+0000","last_fullsized":"2024-09-17T03:33:05.419837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:42.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530834+0000","last_change":"2024-09-17T03:33:02.530834+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530834+0000","last_undegraded":"2024-09-17T03:33:02.530834+0000","last_fullsized":"2024-09-17T03:33:02.530834+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426662+0000","last_change":"2024-09-17T03:33:02.556914+0000","last_active":"2024-09-17T03:33:05.426662+0000","last_peered":"2024-09-17T03:33:05.426662+0000","last_clean":"2024-09-17T03:33:05.426662+0000","last_became_active":"2024-09-17T03:33:02.556176+0000","last_became_peered":"2024-09-17T03:33:02.556176+0000","last_unstale":"2024-09-17T03:33:05.426662+0000","last_undegraded":"2024-09-17T03:33:05.426662+0000","last_fullsized":"2024-09-17T03:33:05.426662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:19.803397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530656+0000","last_change":"2024-09-17T03:33:02.530656+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530656+0000","last_undegraded":"2024-09-17T03:33:02.530656+0000","last_fullsized":"2024-09-17T03:33:02.530656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426418+0000","last_change":"2024-09-17T03:33:02.554769+0000","last_active":"2024-09-17T03:33:05.426418+0000","last_peered":"2024-09-17T03:33:05.426418+0000","last_clean":"2024-09-17T03:33:05.426418+0000","last_became_active":"2024-09-17T03:33:02.554148+0000","last_became_peered":"2024-09-17T03:33:02.554148+0000","last_unstale":"2024-09-17T03:33:05.426418+0000","last_undegraded":"2024-09-17T03:33:05.426418+0000","last_fullsized":"2024-09-17T03:33:05.426418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:47:37.685954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426425+0000","last_change":"2024-09-17T03:33:03.587346+0000","last_active":"2024-09-17T03:33:05.426425+0000","last_peered":"2024-09-17T03:33:05.426425+0000","last_clean":"2024-09-17T03:33:05.426425+0000","last_became_active":"2024-09-17T03:33:03.587021+0000","last_became_peered":"2024-09-17T03:33:03.587021+0000","last_unstale":"2024-09-17T03:33:05.426425+0000","last_undegraded":"2024-09-17T03:33:05.426425+0000","last_fullsized":"2024-09-17T03:33:05.426425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:16.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419857+0000","last_change":"2024-09-17T03:33:02.552916+0000","last_active":"2024-09-17T03:33:05.419857+0000","last_peered":"2024-09-17T03:33:05.419857+0000","last_clean":"2024-09-17T03:33:05.419857+0000","last_became_active":"2024-09-17T03:33:02.551707+0000","last_became_peered":"2024-09-17T03:33:02.551707+0000","last_unstale":"2024-09-17T03:33:05.419857+0000","last_undegraded":"2024-09-17T03:33:05.419857+0000","last_fullsized":"2024-09-17T03:33:05.419857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:28.925660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420744+0000","last_change":"2024-09-17T03:33:03.572163+0000","last_active":"2024-09-17T03:33:05.420744+0000","last_peered":"2024-09-17T03:33:05.420744+0000","last_clean":"2024-09-17T03:33:05.420744+0000","last_became_active":"2024-09-17T03:33:03.571635+0000","last_became_peered":"2024-09-17T03:33:03.571635+0000","last_unstale":"2024-09-17T03:33:05.420744+0000","last_undegraded":"2024-09-17T03:33:05.420744+0000","last_fullsized":"2024-09-17T03:33:05.420744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:54.558593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427248+0000","last_change":"2024-09-17T03:33:02.547235+0000","last_active":"2024-09-17T03:33:05.427248+0000","last_peered":"2024-09-17T03:33:05.427248+0000","last_clean":"2024-09-17T03:33:05.427248+0000","last_became_active":"2024-09-17T03:33:02.546057+0000","last_became_peered":"2024-09-17T03:33:02.546057+0000","last_unstale":"2024-09-17T03:33:05.427248+0000","last_undegraded":"2024-09-17T03:33:05.427248+0000","last_fullsized":"2024-09-17T03:33:05.427248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:57.628934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539866+0000","last_change":"2024-09-17T03:33:03.580070+0000","last_active":"2024-09-17T03:33:04.539866+0000","last_peered":"2024-09-17T03:33:04.539866+0000","last_clean":"2024-09-17T03:33:04.539866+0000","last_became_active":"2024-09-17T03:33:03.579536+0000","last_became_peered":"2024-09-17T03:33:03.579536+0000","last_unstale":"2024-09-17T03:33:04.539866+0000","last_undegraded":"2024-09-17T03:33:04.539866+0000","last_fullsized":"2024-09-17T03:33:04.539866+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:15.857981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420763+0000","last_change":"2024-09-17T03:33:02.549424+0000","last_active":"2024-09-17T03:33:05.420763+0000","last_peered":"2024-09-17T03:33:05.420763+0000","last_clean":"2024-09-17T03:33:05.420763+0000","last_became_active":"2024-09-17T03:33:02.548964+0000","last_became_peered":"2024-09-17T03:33:02.548964+0000","last_unstale":"2024-09-17T03:33:05.420763+0000","last_undegraded":"2024-09-17T03:33:05.420763+0000","last_fullsized":"2024-09-17T03:33:05.420763+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:18.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540218+0000","last_change":"2024-09-17T03:33:03.583513+0000","last_active":"2024-09-17T03:33:04.540218+0000","last_peered":"2024-09-17T03:33:04.540218+0000","last_clean":"2024-09-17T03:33:04.540218+0000","last_became_active":"2024-09-17T03:33:03.583186+0000","last_became_peered":"2024-09-17T03:33:03.583186+0000","last_unstale":"2024-09-17T03:33:04.540218+0000","last_undegraded":"2024-09-17T03:33:04.540218+0000","last_fullsized":"2024-09-17T03:33:04.540218+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:49.099577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420208+0000","last_change":"2024-09-17T03:33:02.550099+0000","last_active":"2024-09-17T03:33:05.420208+0000","last_peered":"2024-09-17T03:33:05.420208+0000","last_clean":"2024-09-17T03:33:05.420208+0000","last_became_active":"2024-09-17T03:33:02.549761+0000","last_became_peered":"2024-09-17T03:33:02.549761+0000","last_unstale":"2024-09-17T03:33:05.420208+0000","last_undegraded":"2024-09-17T03:33:05.420208+0000","last_fullsized":"2024-09-17T03:33:05.420208+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:58.843636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426573+0000","last_change":"2024-09-17T03:33:03.561146+0000","last_active":"2024-09-17T03:33:05.426573+0000","last_peered":"2024-09-17T03:33:05.426573+0000","last_clean":"2024-09-17T03:33:05.426573+0000","last_became_active":"2024-09-17T03:33:03.560840+0000","last_became_peered":"2024-09-17T03:33:03.560840+0000","last_unstale":"2024-09-17T03:33:05.426573+0000","last_undegraded":"2024-09-17T03:33:05.426573+0000","last_fullsized":"2024-09-17T03:33:05.426573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:39.264426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426977+0000","last_change":"2024-09-17T03:33:02.550579+0000","last_active":"2024-09-17T03:33:05.426977+0000","last_peered":"2024-09-17T03:33:05.426977+0000","last_clean":"2024-09-17T03:33:05.426977+0000","last_became_active":"2024-09-17T03:33:02.550238+0000","last_became_peered":"2024-09-17T03:33:02.550238+0000","last_unstale":"2024-09-17T03:33:05.426977+0000","last_undegraded":"2024-09-17T03:33:05.426977+0000","last_fullsized":"2024-09-17T03:33:05.426977+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:42.312117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.528893+0000","last_change":"2024-09-17T03:33:02.528893+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.528893+0000","last_undegraded":"2024-09-17T03:33:02.528893+0000","last_fullsized":"2024-09-17T03:33:02.528893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420094+0000","last_change":"2024-09-17T03:33:02.554041+0000","last_active":"2024-09-17T03:33:05.420094+0000","last_peered":"2024-09-17T03:33:05.420094+0000","last_clean":"2024-09-17T03:33:05.420094+0000","last_became_active":"2024-09-17T03:33:02.553699+0000","last_became_peered":"2024-09-17T03:33:02.553699+0000","last_unstale":"2024-09-17T03:33:05.420094+0000","last_undegraded":"2024-09-17T03:33:05.420094+0000","last_fullsized":"2024-09-17T03:33:05.420094+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216039+0000","last_change":"2024-09-17T03:33:03.563718+0000","last_active":"2024-09-17T03:33:05.216039+0000","last_peered":"2024-09-17T03:33:05.216039+0000","last_clean":"2024-09-17T03:33:05.216039+0000","last_became_active":"2024-09-17T03:33:03.563409+0000","last_became_peered":"2024-09-17T03:33:03.563409+0000","last_unstale":"2024-09-17T03:33:05.216039+0000","last_undegraded":"2024-09-17T03:33:05.216039+0000","last_fullsized":"2024-09-17T03:33:05.216039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:38.241017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426089+0000","last_change":"2024-09-17T03:33:02.556605+0000","last_active":"2024-09-17T03:33:05.426089+0000","last_peered":"2024-09-17T03:33:05.426089+0000","last_clean":"2024-09-17T03:33:05.426089+0000","last_became_active":"2024-09-17T03:33:02.556146+0000","last_became_peered":"2024-09-17T03:33:02.556146+0000","last_unstale":"2024-09-17T03:33:05.426089+0000","last_undegraded":"2024-09-17T03:33:05.426089+0000","last_fullsized":"2024-09-17T03:33:05.426089+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:36.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530028+0000","last_change":"2024-09-17T03:33:02.530028+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530028+0000","last_undegraded":"2024-09-17T03:33:02.530028+0000","last_fullsized":"2024-09-17T03:33:02.530028+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540887+0000","last_change":"2024-09-17T03:33:02.547114+0000","last_active":"2024-09-17T03:33:04.540887+0000","last_peered":"2024-09-17T03:33:04.540887+0000","last_clean":"2024-09-17T03:33:04.540887+0000","last_became_active":"2024-09-17T03:33:02.546778+0000","last_became_peered":"2024-09-17T03:33:02.546778+0000","last_unstale":"2024-09-17T03:33:04.540887+0000","last_undegraded":"2024-09-17T03:33:04.540887+0000","last_fullsized":"2024-09-17T03:33:04.540887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:19.611348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426304+0000","last_change":"2024-09-17T03:33:03.561358+0000","last_active":"2024-09-17T03:33:05.426304+0000","last_peered":"2024-09-17T03:33:05.426304+0000","last_clean":"2024-09-17T03:33:05.426304+0000","last_became_active":"2024-09-17T03:33:03.561027+0000","last_became_peered":"2024-09-17T03:33:03.561027+0000","last_unstale":"2024-09-17T03:33:05.426304+0000","last_undegraded":"2024-09-17T03:33:05.426304+0000","last_fullsized":"2024-09-17T03:33:05.426304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:04.965218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T03:33:02.553304+0000","last_change":"2024-09-17T03:33:02.544930+0000","last_active":"2024-09-17T03:33:02.553304+0000","last_peered":"2024-09-17T03:33:02.553304+0000","last_clean":"2024-09-17T03:33:02.553304+0000","last_became_active":"2024-09-17T03:33:02.542640+0000","last_became_peered":"2024-09-17T03:33:02.542640+0000","last_unstale":"2024-09-17T03:33:02.553304+0000","last_undegraded":"2024-09-17T03:33:02.553304+0000","last_fullsized":"2024-09-17T03:33:02.553304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:12.140218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-09-17T03:33:02.530977+0000","last_change":"2024-09-17T03:33:02.530977+0000","last_active":"2024-09-17T03:33:02.519387+0000","last_peered":"2024-09-17T03:33:02.519387+0000","last_clean":"2024-09-17T03:33:02.519387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:33:02.530977+0000","last_undegraded":"2024-09-17T03:33:02.530977+0000","last_fullsized":"2024-09-17T03:33:02.530977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547713+0000","last_change":"2024-09-17T03:33:02.558759+0000","last_active":"2024-09-17T03:33:04.547713+0000","last_peered":"2024-09-17T03:33:04.547713+0000","last_clean":"2024-09-17T03:33:04.547713+0000","last_became_active":"2024-09-17T03:33:02.558503+0000","last_became_peered":"2024-09-17T03:33:02.558503+0000","last_unstale":"2024-09-17T03:33:04.547713+0000","last_undegraded":"2024-09-17T03:33:04.547713+0000","last_fullsized":"2024-09-17T03:33:04.547713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547648+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:33:04.547648+0000","last_peered":"2024-09-17T03:33:04.547648+0000","last_clean":"2024-09-17T03:33:04.547648+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:33:04.547648+0000","last_undegraded":"2024-09-17T03:33:04.547648+0000","last_fullsized":"2024-09-17T03:33:04.547648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127344,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244496,"statfs":{"total":96636764160,"available":96506363904,"internally_reserved":0,"allocated":1855488,"data_stored":1132666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136484,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235356,"statfs":{"total":96636764160,"available":96497004544,"internally_reserved":0,"allocated":1261568,"data_stored":542838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144816,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227024,"statfs":{"total":96636764160,"available":96488472576,"internally_reserved":0,"allocated":1273856,"data_stored":543922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130924,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240916,"statfs":{"total":96636764160,"available":96502697984,"internally_reserved":0,"allocated":1269760,"data_stored":543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136452,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235388,"statfs":{"total":96636764160,"available":96497037312,"internally_reserved":0,"allocated":1216512,"data_stored":522836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119016,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252824,"statfs":{"total":96636764160,"available":96514891776,"internally_reserved":0,"allocated":1851392,"data_stored":1133136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82324,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289516,"statfs":{"total":96636764160,"available":96552464384,"internally_reserved":0,"allocated":1245184,"data_stored":535956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140640,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231200,"statfs":{"total":96636764160,"available":96492748800,"internally_reserved":0,"allocated":1253376,"data_stored":542186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:33:12.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:33:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 -- 172.21.15.46:0/139708559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600cefa0 msgr2=0x7f09600cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 --2- 172.21.15.46:0/139708559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600cefa0 0x7f09600cf3b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0948009840 tx=0x7f094801c820 comp rx=0 tx=0).stop 2024-09-17T03:33:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 -- 172.21.15.46:0/139708559 shutdown_connections 2024-09-17T03:33:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 --2- 172.21.15.46:0/139708559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600cefa0 0x7f09600cf3b0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 -- 172.21.15.46:0/139708559 >> 172.21.15.46:0/139708559 conn(0x7f09600b2040 msgr2=0x7f09600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 -- 172.21.15.46:0/139708559 shutdown_connections 2024-09-17T03:33:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.619+0000 7f0965800640 1 -- 172.21.15.46:0/139708559 wait complete. 2024-09-17T03:33:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 Processor -- start 2024-09-17T03:33:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 -- start start 2024-09-17T03:33:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09600749a0 0x7f09600b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 0x7f09601fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09600cefa0 0x7f0960158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f096016b2e0 con 0x7f09600b3340 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f096016b160 con 0x7f09600749a0 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0965800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f096016b460 con 0x7f09600cefa0 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 0x7f09601fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 0x7f09601fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39858/0 (socket says 172.21.15.46:39858) 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 -- 172.21.15.46:0/3196524720 learned_addr learned my addr 172.21.15.46:0/3196524720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f095ea00640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09600749a0 0x7f09600b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f095f400640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09600cefa0 0x7f0960158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 -- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09600cefa0 msgr2=0x7f0960158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09600cefa0 0x7f0960158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 -- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09600749a0 msgr2=0x7f09600b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09600749a0 0x7f09600b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 -- 172.21.15.46:0/3196524720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09601fd180 con 0x7f09600b3340 2024-09-17T03:33:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.623+0000 7f0956000640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 0x7f09601fcc40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0948009790 tx=0x7f0948005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09480bb070 con 0x7f09600b3340 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0948004330 con 0x7f09600b3340 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09480b2040 con 0x7f09600b3340 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0960159650 con 0x7f09600b3340 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09600affe0 con 0x7f09600b3340 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09600cf3b0 con 0x7f09600b3340 2024-09-17T03:33:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f09480be070 con 0x7f09600b3340 2024-09-17T03:33:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0957400640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f093c074910 0x7f093c076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f094812c8a0 con 0x7f09600b3340 2024-09-17T03:33:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.627+0000 7f095ea00640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f093c074910 0x7f093c076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.631+0000 7f095ea00640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f093c074910 0x7f093c076e20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0950001cf0 tx=0x7f0950009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.631+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f09480f2db0 con 0x7f09600b3340 2024-09-17T03:33:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.803+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09600b14f0 con 0x7f093c074910 2024-09-17T03:33:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.811+0000 7f0957400640 1 -- 172.21.15.46:0/3196524720 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252424 (secure 0 0 0) 0x7f0950003980 con 0x7f093c074910 2024-09-17T03:33:12.814 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f093c074910 msgr2=0x7f093c076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f093c074910 0x7f093c076e20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0950001cf0 tx=0x7f0950009040 comp rx=0 tx=0).stop 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 msgr2=0x7f09601fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 0x7f09601fcc40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0948009790 tx=0x7f0948005d10 comp rx=0 tx=0).stop 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 shutdown_connections 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f093c074910 0x7f093c076e20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f09600cefa0 0x7f0960158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09600b3340 0x7f09601fcc40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 --2- 172.21.15.46:0/3196524720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09600749a0 0x7f09600b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 >> 172.21.15.46:0/3196524720 conn(0x7f09600b2040 msgr2=0x7f0960072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 shutdown_connections 2024-09-17T03:33:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:12.815+0000 7f0965800640 1 -- 172.21.15.46:0/3196524720 wait complete. 2024-09-17T03:33:12.836 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-17T03:33:11.512940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1018052,"kb_used_data":11016,"kb_used_omap":12,"kb_used_meta":1006835,"kb_avail":753956668,"statfs":{"total":773094113280,"available":772051628032,"internally_reserved":0,"allocated":11280384,"data_stored":5523044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1030999600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002471"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539642+0000","last_change":"2024-09-17T03:33:03.581362+0000","last_active":"2024-09-17T03:33:04.539642+0000","last_peered":"2024-09-17T03:33:04.539642+0000","last_clean":"2024-09-17T03:33:04.539642+0000","last_became_active":"2024-09-17T03:33:03.580475+0000","last_became_peered":"2024-09-17T03:33:03.580475+0000","last_unstale":"2024-09-17T03:33:04.539642+0000","last_undegraded":"2024-09-17T03:33:04.539642+0000","last_fullsized":"2024-09-17T03:33:04.539642+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:11.925403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420228+0000","last_change":"2024-09-17T03:33:03.572549+0000","last_active":"2024-09-17T03:33:05.420228+0000","last_peered":"2024-09-17T03:33:05.420228+0000","last_clean":"2024-09-17T03:33:05.420228+0000","last_became_active":"2024-09-17T03:33:03.572232+0000","last_became_peered":"2024-09-17T03:33:03.572232+0000","last_unstale":"2024-09-17T03:33:05.420228+0000","last_undegraded":"2024-09-17T03:33:05.420228+0000","last_fullsized":"2024-09-17T03:33:05.420228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:46.104711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539783+0000","last_change":"2024-09-17T03:33:03.580570+0000","last_active":"2024-09-17T03:33:04.539783+0000","last_peered":"2024-09-17T03:33:04.539783+0000","last_clean":"2024-09-17T03:33:04.539783+0000","last_became_active":"2024-09-17T03:33:03.580241+0000","last_became_peered":"2024-09-17T03:33:03.580241+0000","last_unstale":"2024-09-17T03:33:04.539783+0000","last_undegraded":"2024-09-17T03:33:04.539783+0000","last_fullsized":"2024-09-17T03:33:04.539783+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:36.303054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541593+0000","last_change":"2024-09-17T03:33:03.551187+0000","last_active":"2024-09-17T03:33:04.541593+0000","last_peered":"2024-09-17T03:33:04.541593+0000","last_clean":"2024-09-17T03:33:04.541593+0000","last_became_active":"2024-09-17T03:33:03.550854+0000","last_became_peered":"2024-09-17T03:33:03.550854+0000","last_unstale":"2024-09-17T03:33:04.541593+0000","last_undegraded":"2024-09-17T03:33:04.541593+0000","last_fullsized":"2024-09-17T03:33:04.541593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:32.888001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427161+0000","last_change":"2024-09-17T03:33:03.562221+0000","last_active":"2024-09-17T03:33:05.427161+0000","last_peered":"2024-09-17T03:33:05.427161+0000","last_clean":"2024-09-17T03:33:05.427161+0000","last_became_active":"2024-09-17T03:33:03.561816+0000","last_became_peered":"2024-09-17T03:33:03.561816+0000","last_unstale":"2024-09-17T03:33:05.427161+0000","last_undegraded":"2024-09-17T03:33:05.427161+0000","last_fullsized":"2024-09-17T03:33:05.427161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:52.528483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547618+0000","last_change":"2024-09-17T03:33:03.580482+0000","last_active":"2024-09-17T03:33:04.547618+0000","last_peered":"2024-09-17T03:33:04.547618+0000","last_clean":"2024-09-17T03:33:04.547618+0000","last_became_active":"2024-09-17T03:33:03.579513+0000","last_became_peered":"2024-09-17T03:33:03.579513+0000","last_unstale":"2024-09-17T03:33:04.547618+0000","last_undegraded":"2024-09-17T03:33:04.547618+0000","last_fullsized":"2024-09-17T03:33:04.547618+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:22.208855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547571+0000","last_change":"2024-09-17T03:33:03.576842+0000","last_active":"2024-09-17T03:33:04.547571+0000","last_peered":"2024-09-17T03:33:04.547571+0000","last_clean":"2024-09-17T03:33:04.547571+0000","last_became_active":"2024-09-17T03:33:03.576572+0000","last_became_peered":"2024-09-17T03:33:03.576572+0000","last_unstale":"2024-09-17T03:33:04.547571+0000","last_undegraded":"2024-09-17T03:33:04.547571+0000","last_fullsized":"2024-09-17T03:33:04.547571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420470+0000","last_change":"2024-09-17T03:33:03.564312+0000","last_active":"2024-09-17T03:33:05.420470+0000","last_peered":"2024-09-17T03:33:05.420470+0000","last_clean":"2024-09-17T03:33:05.420470+0000","last_became_active":"2024-09-17T03:33:03.563925+0000","last_became_peered":"2024-09-17T03:33:03.563925+0000","last_unstale":"2024-09-17T03:33:05.420470+0000","last_undegraded":"2024-09-17T03:33:05.420470+0000","last_fullsized":"2024-09-17T03:33:05.420470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:00.239782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216739+0000","last_change":"2024-09-17T03:33:03.556879+0000","last_active":"2024-09-17T03:33:05.216739+0000","last_peered":"2024-09-17T03:33:05.216739+0000","last_clean":"2024-09-17T03:33:05.216739+0000","last_became_active":"2024-09-17T03:33:03.556579+0000","last_became_peered":"2024-09-17T03:33:03.556579+0000","last_unstale":"2024-09-17T03:33:05.216739+0000","last_undegraded":"2024-09-17T03:33:05.216739+0000","last_fullsized":"2024-09-17T03:33:05.216739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:00.787049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426696+0000","last_change":"2024-09-17T03:33:03.590158+0000","last_active":"2024-09-17T03:33:05.426696+0000","last_peered":"2024-09-17T03:33:05.426696+0000","last_clean":"2024-09-17T03:33:05.426696+0000","last_became_active":"2024-09-17T03:33:03.589751+0000","last_became_peered":"2024-09-17T03:33:03.589751+0000","last_unstale":"2024-09-17T03:33:05.426696+0000","last_undegraded":"2024-09-17T03:33:05.426696+0000","last_fullsized":"2024-09-17T03:33:05.426696+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:08.526897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548057+0000","last_change":"2024-09-17T03:33:03.559214+0000","last_active":"2024-09-17T03:33:04.548057+0000","last_peered":"2024-09-17T03:33:04.548057+0000","last_clean":"2024-09-17T03:33:04.548057+0000","last_became_active":"2024-09-17T03:33:03.558880+0000","last_became_peered":"2024-09-17T03:33:03.558880+0000","last_unstale":"2024-09-17T03:33:04.548057+0000","last_undegraded":"2024-09-17T03:33:04.548057+0000","last_fullsized":"2024-09-17T03:33:04.548057+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:32.880704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216475+0000","last_change":"2024-09-17T03:33:03.595048+0000","last_active":"2024-09-17T03:33:05.216475+0000","last_peered":"2024-09-17T03:33:05.216475+0000","last_clean":"2024-09-17T03:33:05.216475+0000","last_became_active":"2024-09-17T03:33:03.594649+0000","last_became_peered":"2024-09-17T03:33:03.594649+0000","last_unstale":"2024-09-17T03:33:05.216475+0000","last_undegraded":"2024-09-17T03:33:05.216475+0000","last_fullsized":"2024-09-17T03:33:05.216475+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:26.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541461+0000","last_change":"2024-09-17T03:33:03.546430+0000","last_active":"2024-09-17T03:33:04.541461+0000","last_peered":"2024-09-17T03:33:04.541461+0000","last_clean":"2024-09-17T03:33:04.541461+0000","last_became_active":"2024-09-17T03:33:03.546006+0000","last_became_peered":"2024-09-17T03:33:03.546006+0000","last_unstale":"2024-09-17T03:33:04.541461+0000","last_undegraded":"2024-09-17T03:33:04.541461+0000","last_fullsized":"2024-09-17T03:33:04.541461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.191716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.031756+0000","last_change":"2024-09-17T03:33:03.557367+0000","last_active":"2024-09-17T03:33:05.031756+0000","last_peered":"2024-09-17T03:33:05.031756+0000","last_clean":"2024-09-17T03:33:05.031756+0000","last_became_active":"2024-09-17T03:33:03.556682+0000","last_became_peered":"2024-09-17T03:33:03.556682+0000","last_unstale":"2024-09-17T03:33:05.031756+0000","last_undegraded":"2024-09-17T03:33:05.031756+0000","last_fullsized":"2024-09-17T03:33:05.031756+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:22.673342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540556+0000","last_change":"2024-09-17T03:33:03.580895+0000","last_active":"2024-09-17T03:33:04.540556+0000","last_peered":"2024-09-17T03:33:04.540556+0000","last_clean":"2024-09-17T03:33:04.540556+0000","last_became_active":"2024-09-17T03:33:03.580561+0000","last_became_peered":"2024-09-17T03:33:03.580561+0000","last_unstale":"2024-09-17T03:33:04.540556+0000","last_undegraded":"2024-09-17T03:33:04.540556+0000","last_fullsized":"2024-09-17T03:33:04.540556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:59.846145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547563+0000","last_change":"2024-09-17T03:33:03.573694+0000","last_active":"2024-09-17T03:33:04.547563+0000","last_peered":"2024-09-17T03:33:04.547563+0000","last_clean":"2024-09-17T03:33:04.547563+0000","last_became_active":"2024-09-17T03:33:03.573199+0000","last_became_peered":"2024-09-17T03:33:03.573199+0000","last_unstale":"2024-09-17T03:33:04.547563+0000","last_undegraded":"2024-09-17T03:33:04.547563+0000","last_fullsized":"2024-09-17T03:33:04.547563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:14.793531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420606+0000","last_change":"2024-09-17T03:33:03.570549+0000","last_active":"2024-09-17T03:33:05.420606+0000","last_peered":"2024-09-17T03:33:05.420606+0000","last_clean":"2024-09-17T03:33:05.420606+0000","last_became_active":"2024-09-17T03:33:03.570244+0000","last_became_peered":"2024-09-17T03:33:03.570244+0000","last_unstale":"2024-09-17T03:33:05.420606+0000","last_undegraded":"2024-09-17T03:33:05.420606+0000","last_fullsized":"2024-09-17T03:33:05.420606+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:57.566540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541670+0000","last_change":"2024-09-17T03:33:03.558416+0000","last_active":"2024-09-17T03:33:04.541670+0000","last_peered":"2024-09-17T03:33:04.541670+0000","last_clean":"2024-09-17T03:33:04.541670+0000","last_became_active":"2024-09-17T03:33:03.558033+0000","last_became_peered":"2024-09-17T03:33:03.558033+0000","last_unstale":"2024-09-17T03:33:04.541670+0000","last_undegraded":"2024-09-17T03:33:04.541670+0000","last_fullsized":"2024-09-17T03:33:04.541670+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:07.895328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032501+0000","last_change":"2024-09-17T03:33:03.584346+0000","last_active":"2024-09-17T03:33:05.032501+0000","last_peered":"2024-09-17T03:33:05.032501+0000","last_clean":"2024-09-17T03:33:05.032501+0000","last_became_active":"2024-09-17T03:33:03.584051+0000","last_became_peered":"2024-09-17T03:33:03.584051+0000","last_unstale":"2024-09-17T03:33:05.032501+0000","last_undegraded":"2024-09-17T03:33:05.032501+0000","last_fullsized":"2024-09-17T03:33:05.032501+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:19.855756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216563+0000","last_change":"2024-09-17T03:33:03.556063+0000","last_active":"2024-09-17T03:33:05.216563+0000","last_peered":"2024-09-17T03:33:05.216563+0000","last_clean":"2024-09-17T03:33:05.216563+0000","last_became_active":"2024-09-17T03:33:03.555571+0000","last_became_peered":"2024-09-17T03:33:03.555571+0000","last_unstale":"2024-09-17T03:33:05.216563+0000","last_undegraded":"2024-09-17T03:33:05.216563+0000","last_fullsized":"2024-09-17T03:33:05.216563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:58.787401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032144+0000","last_change":"2024-09-17T03:33:03.571904+0000","last_active":"2024-09-17T03:33:05.032144+0000","last_peered":"2024-09-17T03:33:05.032144+0000","last_clean":"2024-09-17T03:33:05.032144+0000","last_became_active":"2024-09-17T03:33:03.571563+0000","last_became_peered":"2024-09-17T03:33:03.571563+0000","last_unstale":"2024-09-17T03:33:05.032144+0000","last_undegraded":"2024-09-17T03:33:05.032144+0000","last_fullsized":"2024-09-17T03:33:05.032144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:12.383302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420061+0000","last_change":"2024-09-17T03:33:03.572171+0000","last_active":"2024-09-17T03:33:05.420061+0000","last_peered":"2024-09-17T03:33:05.420061+0000","last_clean":"2024-09-17T03:33:05.420061+0000","last_became_active":"2024-09-17T03:33:03.571829+0000","last_became_peered":"2024-09-17T03:33:03.571829+0000","last_unstale":"2024-09-17T03:33:05.420061+0000","last_undegraded":"2024-09-17T03:33:05.420061+0000","last_fullsized":"2024-09-17T03:33:05.420061+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:38.888214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419929+0000","last_change":"2024-09-17T03:33:03.572010+0000","last_active":"2024-09-17T03:33:05.419929+0000","last_peered":"2024-09-17T03:33:05.419929+0000","last_clean":"2024-09-17T03:33:05.419929+0000","last_became_active":"2024-09-17T03:33:03.571696+0000","last_became_peered":"2024-09-17T03:33:03.571696+0000","last_unstale":"2024-09-17T03:33:05.419929+0000","last_undegraded":"2024-09-17T03:33:05.419929+0000","last_fullsized":"2024-09-17T03:33:05.419929+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.483130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547565+0000","last_change":"2024-09-17T03:33:03.589869+0000","last_active":"2024-09-17T03:33:04.547565+0000","last_peered":"2024-09-17T03:33:04.547565+0000","last_clean":"2024-09-17T03:33:04.547565+0000","last_became_active":"2024-09-17T03:33:03.589493+0000","last_became_peered":"2024-09-17T03:33:03.589493+0000","last_unstale":"2024-09-17T03:33:04.547565+0000","last_undegraded":"2024-09-17T03:33:04.547565+0000","last_fullsized":"2024-09-17T03:33:04.547565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.449496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426760+0000","last_change":"2024-09-17T03:33:03.576923+0000","last_active":"2024-09-17T03:33:05.426760+0000","last_peered":"2024-09-17T03:33:05.426760+0000","last_clean":"2024-09-17T03:33:05.426760+0000","last_became_active":"2024-09-17T03:33:03.576548+0000","last_became_peered":"2024-09-17T03:33:03.576548+0000","last_unstale":"2024-09-17T03:33:05.426760+0000","last_undegraded":"2024-09-17T03:33:05.426760+0000","last_fullsized":"2024-09-17T03:33:05.426760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:08.997404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420450+0000","last_change":"2024-09-17T03:33:03.583037+0000","last_active":"2024-09-17T03:33:05.420450+0000","last_peered":"2024-09-17T03:33:05.420450+0000","last_clean":"2024-09-17T03:33:05.420450+0000","last_became_active":"2024-09-17T03:33:03.582714+0000","last_became_peered":"2024-09-17T03:33:03.582714+0000","last_unstale":"2024-09-17T03:33:05.420450+0000","last_undegraded":"2024-09-17T03:33:05.420450+0000","last_fullsized":"2024-09-17T03:33:05.420450+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:08.781666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548160+0000","last_change":"2024-09-17T03:33:03.567964+0000","last_active":"2024-09-17T03:33:04.548160+0000","last_peered":"2024-09-17T03:33:04.548160+0000","last_clean":"2024-09-17T03:33:04.548160+0000","last_became_active":"2024-09-17T03:33:03.567632+0000","last_became_peered":"2024-09-17T03:33:03.567632+0000","last_unstale":"2024-09-17T03:33:04.548160+0000","last_undegraded":"2024-09-17T03:33:04.548160+0000","last_fullsized":"2024-09-17T03:33:04.548160+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:39.626714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426889+0000","last_change":"2024-09-17T03:33:03.559826+0000","last_active":"2024-09-17T03:33:05.426889+0000","last_peered":"2024-09-17T03:33:05.426889+0000","last_clean":"2024-09-17T03:33:05.426889+0000","last_became_active":"2024-09-17T03:33:03.559504+0000","last_became_peered":"2024-09-17T03:33:03.559504+0000","last_unstale":"2024-09-17T03:33:05.426889+0000","last_undegraded":"2024-09-17T03:33:05.426889+0000","last_fullsized":"2024-09-17T03:33:05.426889+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:54.833105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420363+0000","last_change":"2024-09-17T03:33:03.580138+0000","last_active":"2024-09-17T03:33:05.420363+0000","last_peered":"2024-09-17T03:33:05.420363+0000","last_clean":"2024-09-17T03:33:05.420363+0000","last_became_active":"2024-09-17T03:33:03.579782+0000","last_became_peered":"2024-09-17T03:33:03.579782+0000","last_unstale":"2024-09-17T03:33:05.420363+0000","last_undegraded":"2024-09-17T03:33:05.420363+0000","last_fullsized":"2024-09-17T03:33:05.420363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:07.780474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.031777+0000","last_change":"2024-09-17T03:33:03.585711+0000","last_active":"2024-09-17T03:33:05.031777+0000","last_peered":"2024-09-17T03:33:05.031777+0000","last_clean":"2024-09-17T03:33:05.031777+0000","last_became_active":"2024-09-17T03:33:03.585294+0000","last_became_peered":"2024-09-17T03:33:03.585294+0000","last_unstale":"2024-09-17T03:33:05.031777+0000","last_undegraded":"2024-09-17T03:33:05.031777+0000","last_fullsized":"2024-09-17T03:33:05.031777+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:26.419969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541093+0000","last_change":"2024-09-17T03:33:03.565826+0000","last_active":"2024-09-17T03:33:04.541093+0000","last_peered":"2024-09-17T03:33:04.541093+0000","last_clean":"2024-09-17T03:33:04.541093+0000","last_became_active":"2024-09-17T03:33:03.557053+0000","last_became_peered":"2024-09-17T03:33:03.557053+0000","last_unstale":"2024-09-17T03:33:04.541093+0000","last_undegraded":"2024-09-17T03:33:04.541093+0000","last_fullsized":"2024-09-17T03:33:04.541093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:54.693438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539233+0000","last_change":"2024-09-17T03:33:03.581059+0000","last_active":"2024-09-17T03:33:04.539233+0000","last_peered":"2024-09-17T03:33:04.539233+0000","last_clean":"2024-09-17T03:33:04.539233+0000","last_became_active":"2024-09-17T03:33:03.580721+0000","last_became_peered":"2024-09-17T03:33:03.580721+0000","last_unstale":"2024-09-17T03:33:04.539233+0000","last_undegraded":"2024-09-17T03:33:04.539233+0000","last_fullsized":"2024-09-17T03:33:04.539233+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:12.767168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032208+0000","last_change":"2024-09-17T03:33:03.555950+0000","last_active":"2024-09-17T03:33:05.032208+0000","last_peered":"2024-09-17T03:33:05.032208+0000","last_clean":"2024-09-17T03:33:05.032208+0000","last_became_active":"2024-09-17T03:33:03.555593+0000","last_became_peered":"2024-09-17T03:33:03.555593+0000","last_unstale":"2024-09-17T03:33:05.032208+0000","last_undegraded":"2024-09-17T03:33:05.032208+0000","last_fullsized":"2024-09-17T03:33:05.032208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:09.226584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420603+0000","last_change":"2024-09-17T03:33:02.543361+0000","last_active":"2024-09-17T03:33:05.420603+0000","last_peered":"2024-09-17T03:33:05.420603+0000","last_clean":"2024-09-17T03:33:05.420603+0000","last_became_active":"2024-09-17T03:33:02.543063+0000","last_became_peered":"2024-09-17T03:33:02.543063+0000","last_unstale":"2024-09-17T03:33:05.420603+0000","last_undegraded":"2024-09-17T03:33:05.420603+0000","last_fullsized":"2024-09-17T03:33:05.420603+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:01.651709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541863+0000","last_change":"2024-09-17T03:33:03.556411+0000","last_active":"2024-09-17T03:33:04.541863+0000","last_peered":"2024-09-17T03:33:04.541863+0000","last_clean":"2024-09-17T03:33:04.541863+0000","last_became_active":"2024-09-17T03:33:03.555405+0000","last_became_peered":"2024-09-17T03:33:03.555405+0000","last_unstale":"2024-09-17T03:33:04.541863+0000","last_undegraded":"2024-09-17T03:33:04.541863+0000","last_fullsized":"2024-09-17T03:33:04.541863+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:44.098643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547818+0000","last_change":"2024-09-17T03:33:02.563716+0000","last_active":"2024-09-17T03:33:04.547818+0000","last_peered":"2024-09-17T03:33:04.547818+0000","last_clean":"2024-09-17T03:33:04.547818+0000","last_became_active":"2024-09-17T03:33:02.563400+0000","last_became_peered":"2024-09-17T03:33:02.563400+0000","last_unstale":"2024-09-17T03:33:04.547818+0000","last_undegraded":"2024-09-17T03:33:04.547818+0000","last_fullsized":"2024-09-17T03:33:04.547818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540439+0000","last_change":"2024-09-17T03:33:03.587192+0000","last_active":"2024-09-17T03:33:04.540439+0000","last_peered":"2024-09-17T03:33:04.540439+0000","last_clean":"2024-09-17T03:33:04.540439+0000","last_became_active":"2024-09-17T03:33:03.586800+0000","last_became_peered":"2024-09-17T03:33:03.586800+0000","last_unstale":"2024-09-17T03:33:04.540439+0000","last_undegraded":"2024-09-17T03:33:04.540439+0000","last_fullsized":"2024-09-17T03:33:04.540439+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:04.696195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419965+0000","last_change":"2024-09-17T03:33:02.555846+0000","last_active":"2024-09-17T03:33:05.419965+0000","last_peered":"2024-09-17T03:33:05.419965+0000","last_clean":"2024-09-17T03:33:05.419965+0000","last_became_active":"2024-09-17T03:33:02.554535+0000","last_became_peered":"2024-09-17T03:33:02.554535+0000","last_unstale":"2024-09-17T03:33:05.419965+0000","last_undegraded":"2024-09-17T03:33:05.419965+0000","last_fullsized":"2024-09-17T03:33:05.419965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:20.329251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216814+0000","last_change":"2024-09-17T03:33:03.576310+0000","last_active":"2024-09-17T03:33:05.216814+0000","last_peered":"2024-09-17T03:33:05.216814+0000","last_clean":"2024-09-17T03:33:05.216814+0000","last_became_active":"2024-09-17T03:33:03.575941+0000","last_became_peered":"2024-09-17T03:33:03.575941+0000","last_unstale":"2024-09-17T03:33:05.216814+0000","last_undegraded":"2024-09-17T03:33:05.216814+0000","last_fullsized":"2024-09-17T03:33:05.216814+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:19.596987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419816+0000","last_change":"2024-09-17T03:33:02.539022+0000","last_active":"2024-09-17T03:33:05.419816+0000","last_peered":"2024-09-17T03:33:05.419816+0000","last_clean":"2024-09-17T03:33:05.419816+0000","last_became_active":"2024-09-17T03:33:02.538700+0000","last_became_peered":"2024-09-17T03:33:02.538700+0000","last_unstale":"2024-09-17T03:33:05.419816+0000","last_undegraded":"2024-09-17T03:33:05.419816+0000","last_fullsized":"2024-09-17T03:33:05.419816+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:31.500503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426743+0000","last_change":"2024-09-17T03:33:03.569261+0000","last_active":"2024-09-17T03:33:05.426743+0000","last_peered":"2024-09-17T03:33:05.426743+0000","last_clean":"2024-09-17T03:33:05.426743+0000","last_became_active":"2024-09-17T03:33:03.568877+0000","last_became_peered":"2024-09-17T03:33:03.568877+0000","last_unstale":"2024-09-17T03:33:05.426743+0000","last_undegraded":"2024-09-17T03:33:05.426743+0000","last_fullsized":"2024-09-17T03:33:05.426743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:04.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539368+0000","last_change":"2024-09-17T03:33:02.550110+0000","last_active":"2024-09-17T03:33:04.539368+0000","last_peered":"2024-09-17T03:33:04.539368+0000","last_clean":"2024-09-17T03:33:04.539368+0000","last_became_active":"2024-09-17T03:33:02.549795+0000","last_became_peered":"2024-09-17T03:33:02.549795+0000","last_unstale":"2024-09-17T03:33:04.539368+0000","last_undegraded":"2024-09-17T03:33:04.539368+0000","last_fullsized":"2024-09-17T03:33:04.539368+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:47.697183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547977+0000","last_change":"2024-09-17T03:33:03.564062+0000","last_active":"2024-09-17T03:33:04.547977+0000","last_peered":"2024-09-17T03:33:04.547977+0000","last_clean":"2024-09-17T03:33:04.547977+0000","last_became_active":"2024-09-17T03:33:03.563650+0000","last_became_peered":"2024-09-17T03:33:03.563650+0000","last_unstale":"2024-09-17T03:33:04.547977+0000","last_undegraded":"2024-09-17T03:33:04.547977+0000","last_fullsized":"2024-09-17T03:33:04.547977+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:08.738111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419621+0000","last_change":"2024-09-17T03:33:02.549221+0000","last_active":"2024-09-17T03:33:05.419621+0000","last_peered":"2024-09-17T03:33:05.419621+0000","last_clean":"2024-09-17T03:33:05.419621+0000","last_became_active":"2024-09-17T03:33:02.548861+0000","last_became_peered":"2024-09-17T03:33:02.548861+0000","last_unstale":"2024-09-17T03:33:05.419621+0000","last_undegraded":"2024-09-17T03:33:05.419621+0000","last_fullsized":"2024-09-17T03:33:05.419621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540576+0000","last_change":"2024-09-17T03:33:03.581594+0000","last_active":"2024-09-17T03:33:04.540576+0000","last_peered":"2024-09-17T03:33:04.540576+0000","last_clean":"2024-09-17T03:33:04.540576+0000","last_became_active":"2024-09-17T03:33:03.581295+0000","last_became_peered":"2024-09-17T03:33:03.581295+0000","last_unstale":"2024-09-17T03:33:04.540576+0000","last_undegraded":"2024-09-17T03:33:04.540576+0000","last_fullsized":"2024-09-17T03:33:04.540576+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:02.654217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426395+0000","last_change":"2024-09-17T03:33:02.555268+0000","last_active":"2024-09-17T03:33:05.426395+0000","last_peered":"2024-09-17T03:33:05.426395+0000","last_clean":"2024-09-17T03:33:05.426395+0000","last_became_active":"2024-09-17T03:33:02.554769+0000","last_became_peered":"2024-09-17T03:33:02.554769+0000","last_unstale":"2024-09-17T03:33:05.426395+0000","last_undegraded":"2024-09-17T03:33:05.426395+0000","last_fullsized":"2024-09-17T03:33:05.426395+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:38.916564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215985+0000","last_change":"2024-09-17T03:33:03.570252+0000","last_active":"2024-09-17T03:33:05.215985+0000","last_peered":"2024-09-17T03:33:05.215985+0000","last_clean":"2024-09-17T03:33:05.215985+0000","last_became_active":"2024-09-17T03:33:03.569899+0000","last_became_peered":"2024-09-17T03:33:03.569899+0000","last_unstale":"2024-09-17T03:33:05.215985+0000","last_undegraded":"2024-09-17T03:33:05.215985+0000","last_fullsized":"2024-09-17T03:33:05.215985+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:27.095812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539125+0000","last_change":"2024-09-17T03:33:02.551914+0000","last_active":"2024-09-17T03:33:04.539125+0000","last_peered":"2024-09-17T03:33:04.539125+0000","last_clean":"2024-09-17T03:33:04.539125+0000","last_became_active":"2024-09-17T03:33:02.551576+0000","last_became_peered":"2024-09-17T03:33:02.551576+0000","last_unstale":"2024-09-17T03:33:04.539125+0000","last_undegraded":"2024-09-17T03:33:04.539125+0000","last_fullsized":"2024-09-17T03:33:04.539125+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:46.988395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420593+0000","last_change":"2024-09-17T03:33:03.558995+0000","last_active":"2024-09-17T03:33:05.420593+0000","last_peered":"2024-09-17T03:33:05.420593+0000","last_clean":"2024-09-17T03:33:05.420593+0000","last_became_active":"2024-09-17T03:33:03.558645+0000","last_became_peered":"2024-09-17T03:33:03.558645+0000","last_unstale":"2024-09-17T03:33:05.420593+0000","last_undegraded":"2024-09-17T03:33:05.420593+0000","last_fullsized":"2024-09-17T03:33:05.420593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:28.537475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539936+0000","last_change":"2024-09-17T03:33:02.556533+0000","last_active":"2024-09-17T03:33:04.539936+0000","last_peered":"2024-09-17T03:33:04.539936+0000","last_clean":"2024-09-17T03:33:04.539936+0000","last_became_active":"2024-09-17T03:33:02.556134+0000","last_became_peered":"2024-09-17T03:33:02.556134+0000","last_unstale":"2024-09-17T03:33:04.539936+0000","last_undegraded":"2024-09-17T03:33:04.539936+0000","last_fullsized":"2024-09-17T03:33:04.539936+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:08.763261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216780+0000","last_change":"2024-09-17T03:33:03.575625+0000","last_active":"2024-09-17T03:33:05.216780+0000","last_peered":"2024-09-17T03:33:05.216780+0000","last_clean":"2024-09-17T03:33:05.216780+0000","last_became_active":"2024-09-17T03:33:03.575239+0000","last_became_peered":"2024-09-17T03:33:03.575239+0000","last_unstale":"2024-09-17T03:33:05.216780+0000","last_undegraded":"2024-09-17T03:33:05.216780+0000","last_fullsized":"2024-09-17T03:33:05.216780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:57.701386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541519+0000","last_change":"2024-09-17T03:33:02.547247+0000","last_active":"2024-09-17T03:33:04.541519+0000","last_peered":"2024-09-17T03:33:04.541519+0000","last_clean":"2024-09-17T03:33:04.541519+0000","last_became_active":"2024-09-17T03:33:02.546932+0000","last_became_peered":"2024-09-17T03:33:02.546932+0000","last_unstale":"2024-09-17T03:33:04.541519+0000","last_undegraded":"2024-09-17T03:33:04.541519+0000","last_fullsized":"2024-09-17T03:33:04.541519+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:03.962710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427462+0000","last_change":"2024-09-17T03:33:03.569370+0000","last_active":"2024-09-17T03:33:05.427462+0000","last_peered":"2024-09-17T03:33:05.427462+0000","last_clean":"2024-09-17T03:33:05.427462+0000","last_became_active":"2024-09-17T03:33:03.569034+0000","last_became_peered":"2024-09-17T03:33:03.569034+0000","last_unstale":"2024-09-17T03:33:05.427462+0000","last_undegraded":"2024-09-17T03:33:05.427462+0000","last_fullsized":"2024-09-17T03:33:05.427462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:36.003958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548375+0000","last_change":"2024-09-17T03:33:02.557519+0000","last_active":"2024-09-17T03:33:04.548375+0000","last_peered":"2024-09-17T03:33:04.548375+0000","last_clean":"2024-09-17T03:33:04.548375+0000","last_became_active":"2024-09-17T03:33:02.557143+0000","last_became_peered":"2024-09-17T03:33:02.557143+0000","last_unstale":"2024-09-17T03:33:04.548375+0000","last_undegraded":"2024-09-17T03:33:04.548375+0000","last_fullsized":"2024-09-17T03:33:04.548375+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.298939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548299+0000","last_change":"2024-09-17T03:33:03.567932+0000","last_active":"2024-09-17T03:33:04.548299+0000","last_peered":"2024-09-17T03:33:04.548299+0000","last_clean":"2024-09-17T03:33:04.548299+0000","last_became_active":"2024-09-17T03:33:03.567611+0000","last_became_peered":"2024-09-17T03:33:03.567611+0000","last_unstale":"2024-09-17T03:33:04.548299+0000","last_undegraded":"2024-09-17T03:33:04.548299+0000","last_fullsized":"2024-09-17T03:33:04.548299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:49:15.466755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216540+0000","last_change":"2024-09-17T03:33:02.552966+0000","last_active":"2024-09-17T03:33:05.216540+0000","last_peered":"2024-09-17T03:33:05.216540+0000","last_clean":"2024-09-17T03:33:05.216540+0000","last_became_active":"2024-09-17T03:33:02.552610+0000","last_became_peered":"2024-09-17T03:33:02.552610+0000","last_unstale":"2024-09-17T03:33:05.216540+0000","last_undegraded":"2024-09-17T03:33:05.216540+0000","last_fullsized":"2024-09-17T03:33:05.216540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:57.033583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.541014+0000","last_change":"2024-09-17T03:33:03.551601+0000","last_active":"2024-09-17T03:33:04.541014+0000","last_peered":"2024-09-17T03:33:04.541014+0000","last_clean":"2024-09-17T03:33:04.541014+0000","last_became_active":"2024-09-17T03:33:03.551024+0000","last_became_peered":"2024-09-17T03:33:03.551024+0000","last_unstale":"2024-09-17T03:33:04.541014+0000","last_undegraded":"2024-09-17T03:33:04.541014+0000","last_fullsized":"2024-09-17T03:33:04.541014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.061549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032315+0000","last_change":"2024-09-17T03:33:02.539385+0000","last_active":"2024-09-17T03:33:05.032315+0000","last_peered":"2024-09-17T03:33:05.032315+0000","last_clean":"2024-09-17T03:33:05.032315+0000","last_became_active":"2024-09-17T03:33:02.539072+0000","last_became_peered":"2024-09-17T03:33:02.539072+0000","last_unstale":"2024-09-17T03:33:05.032315+0000","last_undegraded":"2024-09-17T03:33:05.032315+0000","last_fullsized":"2024-09-17T03:33:05.032315+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:20.882893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539523+0000","last_change":"2024-09-17T03:33:03.581118+0000","last_active":"2024-09-17T03:33:04.539523+0000","last_peered":"2024-09-17T03:33:04.539523+0000","last_clean":"2024-09-17T03:33:04.539523+0000","last_became_active":"2024-09-17T03:33:03.580758+0000","last_became_peered":"2024-09-17T03:33:03.580758+0000","last_unstale":"2024-09-17T03:33:04.539523+0000","last_undegraded":"2024-09-17T03:33:04.539523+0000","last_fullsized":"2024-09-17T03:33:04.539523+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:53.643044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032300+0000","last_change":"2024-09-17T03:33:02.553011+0000","last_active":"2024-09-17T03:33:05.032300+0000","last_peered":"2024-09-17T03:33:05.032300+0000","last_clean":"2024-09-17T03:33:05.032300+0000","last_became_active":"2024-09-17T03:33:02.552649+0000","last_became_peered":"2024-09-17T03:33:02.552649+0000","last_unstale":"2024-09-17T03:33:05.032300+0000","last_undegraded":"2024-09-17T03:33:05.032300+0000","last_fullsized":"2024-09-17T03:33:05.032300+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:53.985445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215951+0000","last_change":"2024-09-17T03:33:03.570295+0000","last_active":"2024-09-17T03:33:05.215951+0000","last_peered":"2024-09-17T03:33:05.215951+0000","last_clean":"2024-09-17T03:33:05.215951+0000","last_became_active":"2024-09-17T03:33:03.569927+0000","last_became_peered":"2024-09-17T03:33:03.569927+0000","last_unstale":"2024-09-17T03:33:05.215951+0000","last_undegraded":"2024-09-17T03:33:05.215951+0000","last_fullsized":"2024-09-17T03:33:05.215951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:36.691869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.215879+0000","last_change":"2024-09-17T03:33:02.553261+0000","last_active":"2024-09-17T03:33:05.215879+0000","last_peered":"2024-09-17T03:33:05.215879+0000","last_clean":"2024-09-17T03:33:05.215879+0000","last_became_active":"2024-09-17T03:33:02.551020+0000","last_became_peered":"2024-09-17T03:33:02.551020+0000","last_unstale":"2024-09-17T03:33:05.215879+0000","last_undegraded":"2024-09-17T03:33:05.215879+0000","last_fullsized":"2024-09-17T03:33:05.215879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:01.322413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.031751+0000","last_change":"2024-09-17T03:33:03.559565+0000","last_active":"2024-09-17T03:33:05.031751+0000","last_peered":"2024-09-17T03:33:05.031751+0000","last_clean":"2024-09-17T03:33:05.031751+0000","last_became_active":"2024-09-17T03:33:03.559248+0000","last_became_peered":"2024-09-17T03:33:03.559248+0000","last_unstale":"2024-09-17T03:33:05.031751+0000","last_undegraded":"2024-09-17T03:33:05.031751+0000","last_fullsized":"2024-09-17T03:33:05.031751+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:45.826245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539246+0000","last_change":"2024-09-17T03:33:02.553687+0000","last_active":"2024-09-17T03:33:04.539246+0000","last_peered":"2024-09-17T03:33:04.539246+0000","last_clean":"2024-09-17T03:33:04.539246+0000","last_became_active":"2024-09-17T03:33:02.552355+0000","last_became_peered":"2024-09-17T03:33:02.552355+0000","last_unstale":"2024-09-17T03:33:04.539246+0000","last_undegraded":"2024-09-17T03:33:04.539246+0000","last_fullsized":"2024-09-17T03:33:04.539246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:35.198181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427004+0000","last_change":"2024-09-17T03:33:03.558620+0000","last_active":"2024-09-17T03:33:05.427004+0000","last_peered":"2024-09-17T03:33:05.427004+0000","last_clean":"2024-09-17T03:33:05.427004+0000","last_became_active":"2024-09-17T03:33:03.558274+0000","last_became_peered":"2024-09-17T03:33:03.558274+0000","last_unstale":"2024-09-17T03:33:05.427004+0000","last_undegraded":"2024-09-17T03:33:05.427004+0000","last_fullsized":"2024-09-17T03:33:05.427004+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216404+0000","last_change":"2024-09-17T03:33:02.550762+0000","last_active":"2024-09-17T03:33:05.216404+0000","last_peered":"2024-09-17T03:33:05.216404+0000","last_clean":"2024-09-17T03:33:05.216404+0000","last_became_active":"2024-09-17T03:33:02.550405+0000","last_became_peered":"2024-09-17T03:33:02.550405+0000","last_unstale":"2024-09-17T03:33:05.216404+0000","last_undegraded":"2024-09-17T03:33:05.216404+0000","last_fullsized":"2024-09-17T03:33:05.216404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426813+0000","last_change":"2024-09-17T03:33:03.572043+0000","last_active":"2024-09-17T03:33:05.426813+0000","last_peered":"2024-09-17T03:33:05.426813+0000","last_clean":"2024-09-17T03:33:05.426813+0000","last_became_active":"2024-09-17T03:33:03.570999+0000","last_became_peered":"2024-09-17T03:33:03.570999+0000","last_unstale":"2024-09-17T03:33:05.426813+0000","last_undegraded":"2024-09-17T03:33:05.426813+0000","last_fullsized":"2024-09-17T03:33:05.426813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:21.832675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426583+0000","last_change":"2024-09-17T03:33:02.546791+0000","last_active":"2024-09-17T03:33:05.426583+0000","last_peered":"2024-09-17T03:33:05.426583+0000","last_clean":"2024-09-17T03:33:05.426583+0000","last_became_active":"2024-09-17T03:33:02.546444+0000","last_became_peered":"2024-09-17T03:33:02.546444+0000","last_unstale":"2024-09-17T03:33:05.426583+0000","last_undegraded":"2024-09-17T03:33:05.426583+0000","last_fullsized":"2024-09-17T03:33:05.426583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:26.093303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.548398+0000","last_change":"2024-09-17T03:33:03.561978+0000","last_active":"2024-09-17T03:33:04.548398+0000","last_peered":"2024-09-17T03:33:04.548398+0000","last_clean":"2024-09-17T03:33:04.548398+0000","last_became_active":"2024-09-17T03:33:03.561642+0000","last_became_peered":"2024-09-17T03:33:03.561642+0000","last_unstale":"2024-09-17T03:33:04.548398+0000","last_undegraded":"2024-09-17T03:33:04.548398+0000","last_fullsized":"2024-09-17T03:33:04.548398+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:25.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419887+0000","last_change":"2024-09-17T03:33:02.540951+0000","last_active":"2024-09-17T03:33:05.419887+0000","last_peered":"2024-09-17T03:33:05.419887+0000","last_clean":"2024-09-17T03:33:05.419887+0000","last_became_active":"2024-09-17T03:33:02.540486+0000","last_became_peered":"2024-09-17T03:33:02.540486+0000","last_unstale":"2024-09-17T03:33:05.419887+0000","last_undegraded":"2024-09-17T03:33:05.419887+0000","last_fullsized":"2024-09-17T03:33:05.419887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:48.319608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032067+0000","last_change":"2024-09-17T03:33:03.547881+0000","last_active":"2024-09-17T03:33:05.032067+0000","last_peered":"2024-09-17T03:33:05.032067+0000","last_clean":"2024-09-17T03:33:05.032067+0000","last_became_active":"2024-09-17T03:33:03.547503+0000","last_became_peered":"2024-09-17T03:33:03.547503+0000","last_unstale":"2024-09-17T03:33:05.032067+0000","last_undegraded":"2024-09-17T03:33:05.032067+0000","last_fullsized":"2024-09-17T03:33:05.032067+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:01.178191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419837+0000","last_change":"2024-09-17T03:33:02.550968+0000","last_active":"2024-09-17T03:33:05.419837+0000","last_peered":"2024-09-17T03:33:05.419837+0000","last_clean":"2024-09-17T03:33:05.419837+0000","last_became_active":"2024-09-17T03:33:02.550661+0000","last_became_peered":"2024-09-17T03:33:02.550661+0000","last_unstale":"2024-09-17T03:33:05.419837+0000","last_undegraded":"2024-09-17T03:33:05.419837+0000","last_fullsized":"2024-09-17T03:33:05.419837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:42.597069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032019+0000","last_change":"2024-09-17T03:33:03.560952+0000","last_active":"2024-09-17T03:33:05.032019+0000","last_peered":"2024-09-17T03:33:05.032019+0000","last_clean":"2024-09-17T03:33:05.032019+0000","last_became_active":"2024-09-17T03:33:03.560700+0000","last_became_peered":"2024-09-17T03:33:03.560700+0000","last_unstale":"2024-09-17T03:33:05.032019+0000","last_undegraded":"2024-09-17T03:33:05.032019+0000","last_fullsized":"2024-09-17T03:33:05.032019+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:38.904274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426662+0000","last_change":"2024-09-17T03:33:02.556914+0000","last_active":"2024-09-17T03:33:05.426662+0000","last_peered":"2024-09-17T03:33:05.426662+0000","last_clean":"2024-09-17T03:33:05.426662+0000","last_became_active":"2024-09-17T03:33:02.556176+0000","last_became_peered":"2024-09-17T03:33:02.556176+0000","last_unstale":"2024-09-17T03:33:05.426662+0000","last_undegraded":"2024-09-17T03:33:05.426662+0000","last_fullsized":"2024-09-17T03:33:05.426662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:19.803397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032058+0000","last_change":"2024-09-17T03:33:03.544958+0000","last_active":"2024-09-17T03:33:05.032058+0000","last_peered":"2024-09-17T03:33:05.032058+0000","last_clean":"2024-09-17T03:33:05.032058+0000","last_became_active":"2024-09-17T03:33:03.544652+0000","last_became_peered":"2024-09-17T03:33:03.544652+0000","last_unstale":"2024-09-17T03:33:05.032058+0000","last_undegraded":"2024-09-17T03:33:05.032058+0000","last_fullsized":"2024-09-17T03:33:05.032058+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:16.945205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426418+0000","last_change":"2024-09-17T03:33:02.554769+0000","last_active":"2024-09-17T03:33:05.426418+0000","last_peered":"2024-09-17T03:33:05.426418+0000","last_clean":"2024-09-17T03:33:05.426418+0000","last_became_active":"2024-09-17T03:33:02.554148+0000","last_became_peered":"2024-09-17T03:33:02.554148+0000","last_unstale":"2024-09-17T03:33:05.426418+0000","last_undegraded":"2024-09-17T03:33:05.426418+0000","last_fullsized":"2024-09-17T03:33:05.426418+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:47:37.685954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426425+0000","last_change":"2024-09-17T03:33:03.587346+0000","last_active":"2024-09-17T03:33:05.426425+0000","last_peered":"2024-09-17T03:33:05.426425+0000","last_clean":"2024-09-17T03:33:05.426425+0000","last_became_active":"2024-09-17T03:33:03.587021+0000","last_became_peered":"2024-09-17T03:33:03.587021+0000","last_unstale":"2024-09-17T03:33:05.426425+0000","last_undegraded":"2024-09-17T03:33:05.426425+0000","last_fullsized":"2024-09-17T03:33:05.426425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:16.346122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.419857+0000","last_change":"2024-09-17T03:33:02.552916+0000","last_active":"2024-09-17T03:33:05.419857+0000","last_peered":"2024-09-17T03:33:05.419857+0000","last_clean":"2024-09-17T03:33:05.419857+0000","last_became_active":"2024-09-17T03:33:02.551707+0000","last_became_peered":"2024-09-17T03:33:02.551707+0000","last_unstale":"2024-09-17T03:33:05.419857+0000","last_undegraded":"2024-09-17T03:33:05.419857+0000","last_fullsized":"2024-09-17T03:33:05.419857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:28.925660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420744+0000","last_change":"2024-09-17T03:33:03.572163+0000","last_active":"2024-09-17T03:33:05.420744+0000","last_peered":"2024-09-17T03:33:05.420744+0000","last_clean":"2024-09-17T03:33:05.420744+0000","last_became_active":"2024-09-17T03:33:03.571635+0000","last_became_peered":"2024-09-17T03:33:03.571635+0000","last_unstale":"2024-09-17T03:33:05.420744+0000","last_undegraded":"2024-09-17T03:33:05.420744+0000","last_fullsized":"2024-09-17T03:33:05.420744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:54.558593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.427248+0000","last_change":"2024-09-17T03:33:02.547235+0000","last_active":"2024-09-17T03:33:05.427248+0000","last_peered":"2024-09-17T03:33:05.427248+0000","last_clean":"2024-09-17T03:33:05.427248+0000","last_became_active":"2024-09-17T03:33:02.546057+0000","last_became_peered":"2024-09-17T03:33:02.546057+0000","last_unstale":"2024-09-17T03:33:05.427248+0000","last_undegraded":"2024-09-17T03:33:05.427248+0000","last_fullsized":"2024-09-17T03:33:05.427248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:57.628934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.539866+0000","last_change":"2024-09-17T03:33:03.580070+0000","last_active":"2024-09-17T03:33:04.539866+0000","last_peered":"2024-09-17T03:33:04.539866+0000","last_clean":"2024-09-17T03:33:04.539866+0000","last_became_active":"2024-09-17T03:33:03.579536+0000","last_became_peered":"2024-09-17T03:33:03.579536+0000","last_unstale":"2024-09-17T03:33:04.539866+0000","last_undegraded":"2024-09-17T03:33:04.539866+0000","last_fullsized":"2024-09-17T03:33:04.539866+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:15.857981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420763+0000","last_change":"2024-09-17T03:33:02.549424+0000","last_active":"2024-09-17T03:33:05.420763+0000","last_peered":"2024-09-17T03:33:05.420763+0000","last_clean":"2024-09-17T03:33:05.420763+0000","last_became_active":"2024-09-17T03:33:02.548964+0000","last_became_peered":"2024-09-17T03:33:02.548964+0000","last_unstale":"2024-09-17T03:33:05.420763+0000","last_undegraded":"2024-09-17T03:33:05.420763+0000","last_fullsized":"2024-09-17T03:33:05.420763+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:18.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540218+0000","last_change":"2024-09-17T03:33:03.583513+0000","last_active":"2024-09-17T03:33:04.540218+0000","last_peered":"2024-09-17T03:33:04.540218+0000","last_clean":"2024-09-17T03:33:04.540218+0000","last_became_active":"2024-09-17T03:33:03.583186+0000","last_became_peered":"2024-09-17T03:33:03.583186+0000","last_unstale":"2024-09-17T03:33:04.540218+0000","last_undegraded":"2024-09-17T03:33:04.540218+0000","last_fullsized":"2024-09-17T03:33:04.540218+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:49.099577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420208+0000","last_change":"2024-09-17T03:33:02.550099+0000","last_active":"2024-09-17T03:33:05.420208+0000","last_peered":"2024-09-17T03:33:05.420208+0000","last_clean":"2024-09-17T03:33:05.420208+0000","last_became_active":"2024-09-17T03:33:02.549761+0000","last_became_peered":"2024-09-17T03:33:02.549761+0000","last_unstale":"2024-09-17T03:33:05.420208+0000","last_undegraded":"2024-09-17T03:33:05.420208+0000","last_fullsized":"2024-09-17T03:33:05.420208+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:58.843636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426573+0000","last_change":"2024-09-17T03:33:03.561146+0000","last_active":"2024-09-17T03:33:05.426573+0000","last_peered":"2024-09-17T03:33:05.426573+0000","last_clean":"2024-09-17T03:33:05.426573+0000","last_became_active":"2024-09-17T03:33:03.560840+0000","last_became_peered":"2024-09-17T03:33:03.560840+0000","last_unstale":"2024-09-17T03:33:05.426573+0000","last_undegraded":"2024-09-17T03:33:05.426573+0000","last_fullsized":"2024-09-17T03:33:05.426573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:39.264426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426977+0000","last_change":"2024-09-17T03:33:02.550579+0000","last_active":"2024-09-17T03:33:05.426977+0000","last_peered":"2024-09-17T03:33:05.426977+0000","last_clean":"2024-09-17T03:33:05.426977+0000","last_became_active":"2024-09-17T03:33:02.550238+0000","last_became_peered":"2024-09-17T03:33:02.550238+0000","last_unstale":"2024-09-17T03:33:05.426977+0000","last_undegraded":"2024-09-17T03:33:05.426977+0000","last_fullsized":"2024-09-17T03:33:05.426977+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:42.312117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032338+0000","last_change":"2024-09-17T03:33:03.557017+0000","last_active":"2024-09-17T03:33:05.032338+0000","last_peered":"2024-09-17T03:33:05.032338+0000","last_clean":"2024-09-17T03:33:05.032338+0000","last_became_active":"2024-09-17T03:33:03.556238+0000","last_became_peered":"2024-09-17T03:33:03.556238+0000","last_unstale":"2024-09-17T03:33:05.032338+0000","last_undegraded":"2024-09-17T03:33:05.032338+0000","last_fullsized":"2024-09-17T03:33:05.032338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:04.728186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.420094+0000","last_change":"2024-09-17T03:33:02.554041+0000","last_active":"2024-09-17T03:33:05.420094+0000","last_peered":"2024-09-17T03:33:05.420094+0000","last_clean":"2024-09-17T03:33:05.420094+0000","last_became_active":"2024-09-17T03:33:02.553699+0000","last_became_peered":"2024-09-17T03:33:02.553699+0000","last_unstale":"2024-09-17T03:33:05.420094+0000","last_undegraded":"2024-09-17T03:33:05.420094+0000","last_fullsized":"2024-09-17T03:33:05.420094+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:50.837733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.216039+0000","last_change":"2024-09-17T03:33:03.563718+0000","last_active":"2024-09-17T03:33:05.216039+0000","last_peered":"2024-09-17T03:33:05.216039+0000","last_clean":"2024-09-17T03:33:05.216039+0000","last_became_active":"2024-09-17T03:33:03.563409+0000","last_became_peered":"2024-09-17T03:33:03.563409+0000","last_unstale":"2024-09-17T03:33:05.216039+0000","last_undegraded":"2024-09-17T03:33:05.216039+0000","last_fullsized":"2024-09-17T03:33:05.216039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:38.241017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426089+0000","last_change":"2024-09-17T03:33:02.556605+0000","last_active":"2024-09-17T03:33:05.426089+0000","last_peered":"2024-09-17T03:33:05.426089+0000","last_clean":"2024-09-17T03:33:05.426089+0000","last_became_active":"2024-09-17T03:33:02.556146+0000","last_became_peered":"2024-09-17T03:33:02.556146+0000","last_unstale":"2024-09-17T03:33:05.426089+0000","last_undegraded":"2024-09-17T03:33:05.426089+0000","last_fullsized":"2024-09-17T03:33:05.426089+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:36.472203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.031751+0000","last_change":"2024-09-17T03:33:03.563354+0000","last_active":"2024-09-17T03:33:05.031751+0000","last_peered":"2024-09-17T03:33:05.031751+0000","last_clean":"2024-09-17T03:33:05.031751+0000","last_became_active":"2024-09-17T03:33:03.555768+0000","last_became_peered":"2024-09-17T03:33:03.555768+0000","last_unstale":"2024-09-17T03:33:05.031751+0000","last_undegraded":"2024-09-17T03:33:05.031751+0000","last_fullsized":"2024-09-17T03:33:05.031751+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:07.334439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.540887+0000","last_change":"2024-09-17T03:33:02.547114+0000","last_active":"2024-09-17T03:33:04.540887+0000","last_peered":"2024-09-17T03:33:04.540887+0000","last_clean":"2024-09-17T03:33:04.540887+0000","last_became_active":"2024-09-17T03:33:02.546778+0000","last_became_peered":"2024-09-17T03:33:02.546778+0000","last_unstale":"2024-09-17T03:33:04.540887+0000","last_undegraded":"2024-09-17T03:33:04.540887+0000","last_fullsized":"2024-09-17T03:33:04.540887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:19.611348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.426304+0000","last_change":"2024-09-17T03:33:03.561358+0000","last_active":"2024-09-17T03:33:05.426304+0000","last_peered":"2024-09-17T03:33:05.426304+0000","last_clean":"2024-09-17T03:33:05.426304+0000","last_became_active":"2024-09-17T03:33:03.561027+0000","last_became_peered":"2024-09-17T03:33:03.561027+0000","last_unstale":"2024-09-17T03:33:05.426304+0000","last_undegraded":"2024-09-17T03:33:05.426304+0000","last_fullsized":"2024-09-17T03:33:05.426304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:04.965218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032293+0000","last_change":"2024-09-17T03:33:02.544930+0000","last_active":"2024-09-17T03:33:05.032293+0000","last_peered":"2024-09-17T03:33:05.032293+0000","last_clean":"2024-09-17T03:33:05.032293+0000","last_became_active":"2024-09-17T03:33:02.542640+0000","last_became_peered":"2024-09-17T03:33:02.542640+0000","last_unstale":"2024-09-17T03:33:05.032293+0000","last_undegraded":"2024-09-17T03:33:05.032293+0000","last_fullsized":"2024-09-17T03:33:05.032293+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:48.516030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:05.032351+0000","last_change":"2024-09-17T03:33:03.564026+0000","last_active":"2024-09-17T03:33:05.032351+0000","last_peered":"2024-09-17T03:33:05.032351+0000","last_clean":"2024-09-17T03:33:05.032351+0000","last_became_active":"2024-09-17T03:33:03.563719+0000","last_became_peered":"2024-09-17T03:33:03.563719+0000","last_unstale":"2024-09-17T03:33:05.032351+0000","last_undegraded":"2024-09-17T03:33:05.032351+0000","last_fullsized":"2024-09-17T03:33:05.032351+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:02.519387+0000","last_clean_scrub_stamp":"2024-09-17T03:33:02.519387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:39.230810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547713+0000","last_change":"2024-09-17T03:33:02.558759+0000","last_active":"2024-09-17T03:33:04.547713+0000","last_peered":"2024-09-17T03:33:04.547713+0000","last_clean":"2024-09-17T03:33:04.547713+0000","last_became_active":"2024-09-17T03:33:02.558503+0000","last_became_peered":"2024-09-17T03:33:02.558503+0000","last_unstale":"2024-09-17T03:33:04.547713+0000","last_undegraded":"2024-09-17T03:33:04.547713+0000","last_fullsized":"2024-09-17T03:33:04.547713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:33:01.516878+0000","last_clean_scrub_stamp":"2024-09-17T03:33:01.516878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:02.359661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T03:33:04.547648+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:33:04.547648+0000","last_peered":"2024-09-17T03:33:04.547648+0000","last_clean":"2024-09-17T03:33:04.547648+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:33:04.547648+0000","last_undegraded":"2024-09-17T03:33:04.547648+0000","last_fullsized":"2024-09-17T03:33:04.547648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127344,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244496,"statfs":{"total":96636764160,"available":96506363904,"internally_reserved":0,"allocated":1855488,"data_stored":1132666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136484,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235356,"statfs":{"total":96636764160,"available":96497004544,"internally_reserved":0,"allocated":1261568,"data_stored":542838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509589,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144816,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227024,"statfs":{"total":96636764160,"available":96488472576,"internally_reserved":0,"allocated":1273856,"data_stored":543922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130924,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240916,"statfs":{"total":96636764160,"available":96502697984,"internally_reserved":0,"allocated":1269760,"data_stored":543820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136488,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235352,"statfs":{"total":96636764160,"available":96497000448,"internally_reserved":0,"allocated":1257472,"data_stored":542220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119016,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252824,"statfs":{"total":96636764160,"available":96514891776,"internally_reserved":0,"allocated":1851392,"data_stored":1133136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82340,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289500,"statfs":{"total":96636764160,"available":96552448000,"internally_reserved":0,"allocated":1257472,"data_stored":542256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140640,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231200,"statfs":{"total":96636764160,"available":96492748800,"internally_reserved":0,"allocated":1253376,"data_stored":542186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:33:12.840 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:33:12.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:33:13.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 -- 172.21.15.46:0/2813281773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d9006cdf0 msgr2=0x7f4d9006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 --2- 172.21.15.46:0/2813281773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d9006cdf0 0x7f4d9006d200 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009900 tx=0x7f4d7c01c850 comp rx=0 tx=0).stop 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 -- 172.21.15.46:0/2813281773 shutdown_connections 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 --2- 172.21.15.46:0/2813281773 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d9006fc90 0x7f4d900700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 --2- 172.21.15.46:0/2813281773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d9006cdf0 0x7f4d9006d200 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 -- 172.21.15.46:0/2813281773 >> 172.21.15.46:0/2813281773 conn(0x7f4d900b35f0 msgr2=0x7f4d900b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 -- 172.21.15.46:0/2813281773 shutdown_connections 2024-09-17T03:33:13.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.043+0000 7f4d97600640 1 -- 172.21.15.46:0/2813281773 wait complete. 2024-09-17T03:33:13.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 Processor -- start 2024-09-17T03:33:13.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 -- start start 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 0x7f4d901fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d9006fc90 0x7f4d90068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d90159130 0x7f4d90069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d90168df0 con 0x7f4d9006cdf0 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d90168c70 con 0x7f4d90159130 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4d90168f70 con 0x7f4d9006fc90 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 0x7f4d901fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 0x7f4d901fb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39874/0 (socket says 172.21.15.46:39874) 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 -- 172.21.15.46:0/2151496628 learned_addr learned my addr 172.21.15.46:0/2151496628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d95600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d90159130 0x7f4d90069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d8fe00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d9006fc90 0x7f4d90068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 -- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d9006fc90 msgr2=0x7f4d90068df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d9006fc90 0x7f4d90068df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 -- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d90159130 msgr2=0x7f4d90069330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d90159130 0x7f4d90069330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 -- 172.21.15.46:0/2151496628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d90166060 con 0x7f4d9006cdf0 2024-09-17T03:33:13.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d94c00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 0x7f4d901fb870 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c3c0 tx=0x7f4d8400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84017070 con 0x7f4d9006cdf0 2024-09-17T03:33:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d840053c0 con 0x7f4d9006cdf0 2024-09-17T03:33:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84003860 con 0x7f4d9006cdf0 2024-09-17T03:33:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.047+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9015a850 con 0x7f4d9006cdf0 2024-09-17T03:33:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.051+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9015ae10 con 0x7f4d9006cdf0 2024-09-17T03:33:13.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.051+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9006d200 con 0x7f4d9006cdf0 2024-09-17T03:33:13.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.051+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f4d8401a070 con 0x7f4d9006cdf0 2024-09-17T03:33:13.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.051+0000 7f4d8cc00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d70074910 0x7f4d70076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.051+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4d840920b0 con 0x7f4d9006cdf0 2024-09-17T03:33:13.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.051+0000 7f4d8fe00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d70074910 0x7f4d70076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.055+0000 7f4d8fe00640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d70074910 0x7f4d70076e20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0af000 tx=0x7f4d7c01b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:13.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.055+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4d8405e610 con 0x7f4d9006cdf0 2024-09-17T03:33:13.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.295+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4d900b3c50 con 0x7f4d9006cdf0 2024-09-17T03:33:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.295+0000 7f4d8cc00640 1 -- 172.21.15.46:0/2151496628 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v41) ==== 190+0+147 (secure 0 0 0) 0x7f4d84063040 con 0x7f4d9006cdf0 2024-09-17T03:33:13.299 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:33:13.299 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:33:13.299 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:33:13.299 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:33:13.299 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:33:13.299 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d70074910 msgr2=0x7f4d70076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d70074910 0x7f4d70076e20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0af000 tx=0x7f4d7c01b090 comp rx=0 tx=0).stop 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 msgr2=0x7f4d901fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 0x7f4d901fb870 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400c3c0 tx=0x7f4d8400c7c0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 shutdown_connections 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4d70074910 0x7f4d70076e20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d90159130 0x7f4d90069330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4d9006fc90 0x7f4d90068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 --2- 172.21.15.46:0/2151496628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d9006cdf0 0x7f4d901fb870 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 >> 172.21.15.46:0/2151496628 conn(0x7f4d900b35f0 msgr2=0x7f4d901647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:13.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 shutdown_connections 2024-09-17T03:33:13.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.299+0000 7f4d97600640 1 -- 172.21.15.46:0/2151496628 wait complete. 2024-09-17T03:33:13.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:33:13.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 -- 172.21.15.46:0/2973576053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b700cefa0 msgr2=0x7f8b700cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 --2- 172.21.15.46:0/2973576053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b700cefa0 0x7f8b700cf3b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009840 tx=0x7f8b5c01c820 comp rx=0 tx=0).stop 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 -- 172.21.15.46:0/2973576053 shutdown_connections 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 --2- 172.21.15.46:0/2973576053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b700cefa0 0x7f8b700cf3b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 --2- 172.21.15.46:0/2973576053 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b700749a0 0x7f8b70074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 -- 172.21.15.46:0/2973576053 >> 172.21.15.46:0/2973576053 conn(0x7f8b700b2040 msgr2=0x7f8b700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 -- 172.21.15.46:0/2973576053 shutdown_connections 2024-09-17T03:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 -- 172.21.15.46:0/2973576053 wait complete. 2024-09-17T03:33:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 Processor -- start 2024-09-17T03:33:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.479+0000 7f8b78200640 1 -- start start 2024-09-17T03:33:13.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b700749a0 0x7f8b7015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b700cefa0 0x7f8b70156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 0x7f8b700cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7016a060 con 0x7f8b70160330 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70169ee0 con 0x7f8b700cefa0 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7016a1e0 con 0x7f8b700749a0 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 0x7f8b700cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 0x7f8b700cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39884/0 (socket says 172.21.15.46:39884) 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 -- 172.21.15.46:0/139303553 learned_addr learned my addr 172.21.15.46:0/139303553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b74e00640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b700cefa0 0x7f8b70156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b75800640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b700749a0 0x7f8b7015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 -- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b700749a0 msgr2=0x7f8b7015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b700749a0 0x7f8b7015e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 -- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b700cefa0 msgr2=0x7f8b70156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b700cefa0 0x7f8b70156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 -- 172.21.15.46:0/139303553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7013bb50 con 0x7f8b70160330 2024-09-17T03:33:13.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b76200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 0x7f8b700cdde0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c760 tx=0x7f8b6800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68003db0 con 0x7f8b70160330 2024-09-17T03:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b6800ce10 con 0x7f8b70160330 2024-09-17T03:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7013be30 con 0x7f8b70160330 2024-09-17T03:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b680052d0 con 0x7f8b70160330 2024-09-17T03:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7013c3f0 con 0x7f8b70160330 2024-09-17T03:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.483+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b70074db0 con 0x7f8b70160330 2024-09-17T03:33:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.487+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f8b68005470 con 0x7f8b70160330 2024-09-17T03:33:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.487+0000 7f8b6d600640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8b50074910 0x7f8b50076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.487+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f8b68092e50 con 0x7f8b70160330 2024-09-17T03:33:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.487+0000 7f8b75800640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8b50074910 0x7f8b50076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.487+0000 7f8b75800640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8b50074910 0x7f8b50076e20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005fd0 tx=0x7f8b64005e40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.491+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b6805f8d0 con 0x7f8b70160330 2024-09-17T03:33:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b7015f600 con 0x7f8b70160330 2024-09-17T03:33:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b6d600640 1 -- 172.21.15.46:0/139303553 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5238 (secure 0 0 0) 0x7f8b6805f1d0 con 0x7f8b70160330 2024-09-17T03:33:13.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:13.704 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"btime":"2024-09-17T03:33:06:567020+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:33:02.961494+0000","modified":"2024-09-17T03:33:05.989948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":247692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_247692":{"gid":247692,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4088974288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4088974288},{"type":"v1","addr":"172.21.15.46:6836","nonce":4088974288}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":247692,"qdb_cluster":[247692]},"id":7}]} 2024-09-17T03:33:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 65 2024-09-17T03:33:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8b50074910 msgr2=0x7f8b50076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8b50074910 0x7f8b50076e20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005fd0 tx=0x7f8b64005e40 comp rx=0 tx=0).stop 2024-09-17T03:33:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 msgr2=0x7f8b700cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 0x7f8b700cdde0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c760 tx=0x7f8b6800cc20 comp rx=0 tx=0).stop 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 shutdown_connections 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8b50074910 0x7f8b50076e20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b70160330 0x7f8b700cdde0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b700cefa0 0x7f8b70156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 --2- 172.21.15.46:0/139303553 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8b700749a0 0x7f8b7015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 >> 172.21.15.46:0/139303553 conn(0x7f8b700b2040 msgr2=0x7f8b700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 shutdown_connections 2024-09-17T03:33:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:13.703+0000 7f8b78200640 1 -- 172.21.15.46:0/139303553 wait complete. 2024-09-17T03:33:14.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:33:02.961494+0000', 'modified': '2024-09-17T03:33:05.989948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 247692}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_247692': {'gid': 247692, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/4088974288', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6835', 'nonce': 4088974288}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 4088974288}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 247692, 'qdb_cluster': [247692]} 2024-09-17T03:33:14.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:33:14.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T03:33:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.291+0000 7fd7dfc00640 1 -- 172.21.15.46:0/3263962902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d815aa50 msgr2=0x7fd7d80cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.291+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/3263962902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d815aa50 0x7fd7d80cd260 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80ccc30 tx=0x7fd7c801c800 comp rx=0 tx=0).stop 2024-09-17T03:33:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 -- 172.21.15.46:0/3263962902 shutdown_connections 2024-09-17T03:33:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/3263962902 >> v2:172.21.15.99:3300/0 conn(0x7fd7d80cd7a0 0x7fd7d80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/3263962902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d815aa50 0x7fd7d80cd260 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 -- 172.21.15.46:0/3263962902 >> 172.21.15.46:0/3263962902 conn(0x7fd7d80b2040 msgr2=0x7fd7d80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 -- 172.21.15.46:0/3263962902 shutdown_connections 2024-09-17T03:33:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 -- 172.21.15.46:0/3263962902 wait complete. 2024-09-17T03:33:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 Processor -- start 2024-09-17T03:33:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.295+0000 7fd7dfc00640 1 -- start start 2024-09-17T03:33:14.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 0x7fd7d8149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7d815aa50 0x7fd7d8157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d815e4e0 0x7fd7d80738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8166310 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8166190 con 0x7fd7d815e4e0 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d8166490 con 0x7fd7d815aa50 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 0x7fd7d8149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 0x7fd7d8149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39892/0 (socket says 172.21.15.46:39892) 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 -- 172.21.15.46:0/4125616678 learned_addr learned my addr 172.21.15.46:0/4125616678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d815e4e0 0x7fd7d80738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7d815aa50 0x7fd7d8157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 -- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7d815aa50 msgr2=0x7fd7d8157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7d815aa50 0x7fd7d8157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 -- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d815e4e0 msgr2=0x7fd7d80738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d815e4e0 0x7fd7d80738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 -- 172.21.15.46:0/4125616678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d8153e60 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dd200640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 0x7fd7d8149f20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8098400 tx=0x7fd7c80a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7d5600640 1 -- 172.21.15.46:0/4125616678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80b2040 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7d5600640 1 -- 172.21.15.46:0/4125616678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c80bb070 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d8154140 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7d5600640 1 -- 172.21.15.46:0/4125616678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8003600 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.299+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81558f0 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.303+0000 7fd7d5600640 1 -- 172.21.15.46:0/4125616678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fd7c80037a0 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.303+0000 7fd7d5600640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7b4074910 0x7fd7b4076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.303+0000 7fd7d5600640 1 -- 172.21.15.46:0/4125616678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd7c812d460 con 0x7fd7d80cd7a0 2024-09-17T03:33:14.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.303+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7b4074910 0x7fd7b4076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 Processor -- start 2024-09-17T03:33:14.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7b4074910 0x7fd7b4076e20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80b25a0 tx=0x7fd7c000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- start start 2024-09-17T03:33:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a52be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 0x55ec89a53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec89927410 con 0x55ec898eaca0 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ec89864b40 con 0x55ec89511460 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55ec89853f70 con 0x55ec89901950 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 0x55ec89a53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 0x55ec89a53120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39902/0 (socket says 172.21.15.46:39902) 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3474928430 learned_addr learned my addr 172.21.15.46:0/3474928430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7dd200640 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a52be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 msgr2=0x55ec89a52be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a52be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 msgr2=0x55ec89a526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3474928430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8981b5b0 con 0x55ec898eaca0 2024-09-17T03:33:14.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7dd200640 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:14.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 0x55ec89a53120 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0001e10 tx=0x7fd7d000f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ff737f09f2195f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7cd600640 1 -- 172.21.15.46:0/3474928430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0001f00 con 0x55ec898eaca0 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7cd600640 1 -- 172.21.15.46:0/3474928430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7d0018070 con 0x55ec898eaca0 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7cd600640 1 -- 172.21.15.46:0/3474928430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0003be0 con 0x55ec898eaca0 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3474928430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 msgr2=0x55ec89a53120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 0x55ec89a53120 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0001e10 tx=0x7fd7d000f5a0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3474928430 shutdown_connections 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec898eaca0 0x55ec89a53120 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a52be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3474928430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.307+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3474928430 >> 172.21.15.46:0/3474928430 conn(0x55ec899d7d90 msgr2=0x55ec89a07690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3474928430 shutdown_connections 2024-09-17T03:33:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3474928430 wait complete. 2024-09-17T03:33:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 -- start start 2024-09-17T03:33:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 Processor -- start 2024-09-17T03:33:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89511460 0x55ec89a59920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a59e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec897ee5f0 con 0x55ec89901950 2024-09-17T03:33:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ec89857d80 con 0x55ec898eaca0 2024-09-17T03:33:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55ec897fa520 con 0x55ec89511460 2024-09-17T03:33:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7e73ef000 10 client.0 _ll_register_callbacks cb 0x55ec897fe830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:33:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a5a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39908/0 (socket says 172.21.15.46:39908) 2024-09-17T03:33:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3003852342 learned_addr learned my addr 172.21.15.46:0/3003852342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a59e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7dd200640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89511460 0x55ec89a59920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89511460 msgr2=0x55ec89a59920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89511460 0x55ec89a59920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 msgr2=0x55ec89a59e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a59e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 -- 172.21.15.46:0/3003852342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec897f2190 con 0x55ec89901950 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a5a3a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00012f0 tx=0x7fd7d0006eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fe8fbb7f25da48 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.311+0000 7fd7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 1 -- 172.21.15.46:0/3003852342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00050d0 con 0x55ec89901950 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 1 -- 172.21.15.46:0/3003852342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd7d0005270 con 0x55ec89901950 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ec897ec110 con 0x55ec89901950 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 1 -- 172.21.15.46:0/3003852342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fd7d002d050 con 0x55ec89901950 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 10 client.247839 fetch_fsmap learned FSMap version 65 2024-09-17T03:33:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55ec898d8f10 con 0x55ec89901950 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 1 -- 172.21.15.46:0/3003852342 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ec897ec110 con 0x55ec89901950 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 1 -- 172.21.15.46:0/3003852342 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) ==== 3502+0+0 (secure 0 0 0) 0x7fd7d00054b0 con 0x55ec89901950 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 10 client.247839 fetch_fsmap finished waiting for FSMap version 65 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 10 client.247839 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 20 client.247839 populate_metadata read hostname 'smithi046' 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894f67d0 0x55ec89a63d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 4 client.247839 mds_command: new command op to 247692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 --> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec89866a90 con 0x55ec894f67d0 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7dd200640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894f67d0 0x55ec89a63d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7dd200640 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894f67d0 0x55ec89a63d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad1b1d6eea0c716 server_cookie=9903c8062a758de1 in_seq=0 out_seq=0 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 10 client.247839 ms_handle_connect on v2:172.21.15.46:6835/4088974288 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 1 -- 172.21.15.46:0/3003852342 <== mds.0 v2:172.21.15.46:6835/4088974288 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec89866a90 con 0x55ec894f67d0 2024-09-17T03:33:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 10 client.247839 handle_command_reply: tid=0 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 client.247839 shutdown 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 msgr2=0x55ec89a5a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a5a3a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00012f0 tx=0x7fd7d0006eb0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 shutdown_connections 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894f67d0 0x55ec89a63d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a5a3a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a59e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7c6000640 0 client.247839 ms_handle_reset on v2:172.21.15.46:6835/4088974288 2024-09-17T03:33:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/3003852342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89511460 0x55ec89a59920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 >> 172.21.15.46:0/3003852342 conn(0x55ec899d7d90 msgr2=0x55ec89a0e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 shutdown_connections 2024-09-17T03:33:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 1 -- 172.21.15.46:0/3003852342 wait complete. 2024-09-17T03:33:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.315+0000 7fd7e73ef000 20 client.247839 trim_cache size 0 max 16384 2024-09-17T03:33:14.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 Processor -- start 2024-09-17T03:33:14.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 -- start start 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a06cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec898eaca0 0x55ec89a07210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a07750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8205900 con 0x55ec89901950 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8204900 con 0x55ec89511460 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d8152fb0 con 0x55ec898eaca0 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.335+0000 7fd7ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a07750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a07750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39918/0 (socket says 172.21.15.46:39918) 2024-09-17T03:33:14.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 -- 172.21.15.46:0/2988345048 learned_addr learned my addr 172.21.15.46:0/2988345048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7dd200640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a06cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec898eaca0 0x55ec89a07210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 -- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec898eaca0 msgr2=0x55ec89a07210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec898eaca0 0x55ec89a07210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 -- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 msgr2=0x55ec89a06cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a06cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 -- 172.21.15.46:0/2988345048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec898afdb0 con 0x55ec89901950 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7dd200640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a06cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec898eaca0 0x55ec89a07210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a07750 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000fbf0 tx=0x7fd7d0006b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d27852d86941d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ccc00640 1 -- 172.21.15.46:0/2988345048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00052c0 con 0x55ec89901950 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ccc00640 1 -- 172.21.15.46:0/2988345048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7d0036070 con 0x55ec89901950 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7ccc00640 1 -- 172.21.15.46:0/2988345048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00071a0 con 0x55ec89901950 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2988345048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 msgr2=0x55ec89a07750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a07750 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000fbf0 tx=0x7fd7d0006b40 comp rx=0 tx=0).stop 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2988345048 shutdown_connections 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89901950 0x55ec89a07750 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec898eaca0 0x55ec89a07210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2988345048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec89511460 0x55ec89a06cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2988345048 >> 172.21.15.46:0/2988345048 conn(0x55ec899d7d90 msgr2=0x55ec89a50620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2988345048 shutdown_connections 2024-09-17T03:33:14.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2988345048 wait complete. 2024-09-17T03:33:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 -- start start 2024-09-17T03:33:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.339+0000 7fd7e73ef000 1 Processor -- start 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 0x55ec89a09db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a0a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a84220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec89863910 con 0x55ec89511460 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ec8985c3b0 con 0x55ec898eaca0 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55ec89913750 con 0x55ec89901950 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 10 client.0 _ll_register_callbacks cb 0x7fd7d0001d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:33:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 0x55ec89a09db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 0x55ec89a09db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39930/0 (socket says 172.21.15.46:39930) 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 -- 172.21.15.46:0/2674685567 learned_addr learned my addr 172.21.15.46:0/2674685567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7ddc00640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a84220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a0a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 -- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 msgr2=0x55ec89a84220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a84220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 -- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 msgr2=0x55ec89a0a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:33:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a0a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 -- 172.21.15.46:0/2674685567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8981aab0 con 0x55ec89511460 2024-09-17T03:33:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a0a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7dd200640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 0x55ec89a09db0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80a55f0 tx=0x7fd7c80f6030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cb51b4cd95f6c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:33:14.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 1 -- 172.21.15.46:0/2674685567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80fe350 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 1 -- 172.21.15.46:0/2674685567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd7c8102d70 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d0036070 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 1 -- 172.21.15.46:0/2674685567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fd7c8103200 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 10 client.247845 fetch_fsmap learned FSMap version 65 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55ec897f9070 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 1 -- 172.21.15.46:0/2674685567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7d0036070 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 1 -- 172.21.15.46:0/2674685567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) ==== 3502+0+0 (secure 0 0 0) 0x7fd7c80b5070 con 0x55ec89511460 2024-09-17T03:33:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 10 client.247845 fetch_fsmap finished waiting for FSMap version 65 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 10 client.247845 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 20 client.247845 populate_metadata read hostname 'smithi046' 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894e4830 0x55ec89a84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 4 client.247845 mds_command: new command op to 247692 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 --> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] -- command(tid 0: {"prefix": "status"}) -- 0x55ec89862540 con 0x55ec894e4830 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894e4830 0x55ec89a84b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 10 client.247845 ms_handle_connect on v2:172.21.15.46:6835/4088974288 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7cfe00640 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894e4830 0x55ec89a84b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5c0f867a8f0f8a server_cookie=715fb0873d001a35 in_seq=0 out_seq=0 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 1 -- 172.21.15.46:0/2674685567 <== mds.0 v2:172.21.15.46:6835/4088974288 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55ec89862540 con 0x55ec894e4830 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 10 client.247845 handle_command_reply: tid=0 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 client.247845 shutdown 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 msgr2=0x55ec89a09db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 0x55ec89a09db0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80a55f0 tx=0x7fd7c80f6030 comp rx=0 tx=0).stop 2024-09-17T03:33:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 shutdown_connections 2024-09-17T03:33:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:6835/4088974288,v1:172.21.15.46:6836/4088974288] conn(0x55ec894e4830 0x55ec89a84b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55ec89901950 0x55ec89a84220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ec898eaca0 0x55ec89a0a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 --2- 172.21.15.46:0/2674685567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec89511460 0x55ec89a09db0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7c6000640 0 client.247845 ms_handle_reset on v2:172.21.15.46:6835/4088974288 2024-09-17T03:33:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.343+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 >> 172.21.15.46:0/2674685567 conn(0x55ec899d7d90 msgr2=0x55ec89a636a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 shutdown_connections 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7e73ef000 1 -- 172.21.15.46:0/2674685567 wait complete. 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7e73ef000 20 client.247845 trim_cache size 0 max 16384 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:33:14.355 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 247692, 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.814187523999999, 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 65, 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 102, 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 102, 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.154998094 2024-09-17T03:33:14.356 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7b4074910 msgr2=0x7fd7b4076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7b4074910 0x7fd7b4076e20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80b25a0 tx=0x7fd7c000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 msgr2=0x7fd7d8149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 0x7fd7d8149f20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8098400 tx=0x7fd7c80a5760 comp rx=0 tx=0).stop 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 shutdown_connections 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd7b4074910 0x7fd7b4076e20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d815e4e0 0x7fd7d80738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7d815aa50 0x7fd7d8157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 --2- 172.21.15.46:0/4125616678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7d80cd7a0 0x7fd7d8149f20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 >> 172.21.15.46:0/4125616678 conn(0x7fd7d80b2040 msgr2=0x7fd7d8159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 shutdown_connections 2024-09-17T03:33:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.347+0000 7fd7dfc00640 1 -- 172.21.15.46:0/4125616678 wait complete. 2024-09-17T03:33:14.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:33:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 -- 172.21.15.46:0/2817908020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 msgr2=0x7fd0000afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2817908020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0000afc30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd0000b2d20 tx=0x7fcff001c800 comp rx=0 tx=0).stop 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 -- 172.21.15.46:0/2817908020 shutdown_connections 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2817908020 >> v2:172.21.15.99:3300/0 conn(0x7fd000162410 0x7fd000162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2817908020 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0000ae8e0 0x7fd0000aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2817908020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0000afc30 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 -- 172.21.15.46:0/2817908020 >> 172.21.15.46:0/2817908020 conn(0x7fd00006d3f0 msgr2=0x7fd0000b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 -- 172.21.15.46:0/2817908020 shutdown_connections 2024-09-17T03:33:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.527+0000 7fcfffe00640 1 -- 172.21.15.46:0/2817908020 wait complete. 2024-09-17T03:33:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 Processor -- start 2024-09-17T03:33:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 -- start start 2024-09-17T03:33:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0001516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0000ae8e0 0x7fd0001fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd000162410 0x7fd0001fc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd00006f5c0 con 0x7fd0000702e0 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd00006f440 con 0x7fd0000ae8e0 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfffe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd00006f740 con 0x7fd000162410 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcffea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0001516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcffea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0001516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39954/0 (socket says 172.21.15.46:39954) 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcffea00640 1 -- 172.21.15.46:0/2889754603 learned_addr learned my addr 172.21.15.46:0/2889754603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcff6000640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0000ae8e0 0x7fd0001fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcfff400640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd000162410 0x7fd0001fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcffea00640 1 -- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd000162410 msgr2=0x7fd0001fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.531+0000 7fcffea00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd000162410 0x7fd0001fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcffea00640 1 -- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0000ae8e0 msgr2=0x7fd0001fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcffea00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0000ae8e0 0x7fd0001fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcffea00640 1 -- 172.21.15.46:0/2889754603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd000152e10 con 0x7fd0000702e0 2024-09-17T03:33:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcffea00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0001516e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fcff0000c10 tx=0x7fcff00a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00ba020 con 0x7fd0000702e0 2024-09-17T03:33:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcff0005030 con 0x7fd0000702e0 2024-09-17T03:33:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00b2040 con 0x7fd0000702e0 2024-09-17T03:33:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0001530f0 con 0x7fd0000702e0 2024-09-17T03:33:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0001536b0 con 0x7fd0000702e0 2024-09-17T03:33:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000152260 con 0x7fd0000702e0 2024-09-17T03:33:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fcff0005480 con 0x7fd0000702e0 2024-09-17T03:33:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.535+0000 7fcff7e00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfdc074940 0x7fcfdc076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.539+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fcff012cf80 con 0x7fd0000702e0 2024-09-17T03:33:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.539+0000 7fcff6000640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfdc074940 0x7fcfdc076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.539+0000 7fcff6000640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfdc074940 0x7fcfdc076e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd0001559c0 tx=0x7fcfe800b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:14.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.539+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd000152260 con 0x7fd0000702e0 2024-09-17T03:33:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd0000aecf0 con 0x7fd0000702e0 2024-09-17T03:33:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcff7e00640 1 -- 172.21.15.46:0/2889754603 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7fcff00164b0 con 0x7fd0000702e0 2024-09-17T03:33:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:33:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfdc074940 msgr2=0x7fcfdc076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfdc074940 0x7fcfdc076e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd0001559c0 tx=0x7fcfe800b260 comp rx=0 tx=0).stop 2024-09-17T03:33:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 msgr2=0x7fd0001516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0001516e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fcff0000c10 tx=0x7fcff00a57b0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 shutdown_connections 2024-09-17T03:33:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcfdc074940 0x7fcfdc076e50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd000162410 0x7fd0001fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0000ae8e0 0x7fd0001fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 --2- 172.21.15.46:0/2889754603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0000702e0 0x7fd0001516e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 >> 172.21.15.46:0/2889754603 conn(0x7fd00006d3f0 msgr2=0x7fd00015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 shutdown_connections 2024-09-17T03:33:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:14.755+0000 7fcfffe00640 1 -- 172.21.15.46:0/2889754603 wait complete. 2024-09-17T03:33:14.778 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:33:14.778 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:33:14.778 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:33:14.778 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:33:14.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:33:14.779 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:33:14.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:33:14.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:33:14.779 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:33:14.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:14.779 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:33:14.789 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:33:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:33:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:33:14.789 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:33:14.790 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41694sec preferred_lft 41694sec 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.791 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:33:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:33:14.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:14.793 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:33:14.853 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:33:14.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:14.855 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:33:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:33:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:33:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:33:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:33:14.909 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:33:14.909 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:33:14.909 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:33:14.909 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:33:14.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:33:14.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:33:14.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:33:14.909 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:33:15.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:33:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:33:15.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:15.052 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:33:15.061 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:33:15.062 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:33:15.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:33:15.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:15.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:33:15.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:33:15.127 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:33:15.183 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:33:15.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:33:15.183 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:33:15.234 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:33:15.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:15.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:33:15.344 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:33:15.344 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:33:15.359 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:33:15.359 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:33:15.359 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:33:15.359 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:33:15.359 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:33:15.363 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:33:15.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:15.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:33:15.384 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:33:15.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:33:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:33:15.492 INFO:teuthology.orchestra.run.smithi046.stdout:client246848 2024-09-17T03:33:15.492 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:33:15.492 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:33:15.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:33:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 -- 172.21.15.46:0/3417244731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 msgr2=0x7fe0d8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 --2- 172.21.15.46:0/3417244731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 0x7fe0d8003ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009b60 tx=0x7fe0e401d270 comp rx=0 tx=0).stop 2024-09-17T03:33:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 -- 172.21.15.46:0/3417244731 shutdown_connections 2024-09-17T03:33:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 --2- 172.21.15.46:0/3417244731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 0x7fe0d8003ed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 -- 172.21.15.46:0/3417244731 >> 172.21.15.46:0/3417244731 conn(0x7fe0f00708c0 msgr2=0x7fe0f0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 -- 172.21.15.46:0/3417244731 shutdown_connections 2024-09-17T03:33:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 -- 172.21.15.46:0/3417244731 wait complete. 2024-09-17T03:33:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 Processor -- start 2024-09-17T03:33:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.711+0000 7fe0f7800640 1 -- start start 2024-09-17T03:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 0x7fe0f00b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 0x7fe0f01fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0f01478b0 0x7fe0f0147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f015d590 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f015d410 con 0x7fe0f01478b0 2024-09-17T03:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f015d710 con 0x7fe0d8003a60 2024-09-17T03:33:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 0x7fe0f01fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 0x7fe0f01fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39984/0 (socket says 172.21.15.46:39984) 2024-09-17T03:33:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 -- 172.21.15.46:0/4169751989 learned_addr learned my addr 172.21.15.46:0/4169751989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f4e00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 0x7fe0f00b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f5800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0f01478b0 0x7fe0f0147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 -- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 msgr2=0x7fe0f00b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 0x7fe0f00b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 -- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0f01478b0 msgr2=0x7fe0f0147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0f01478b0 0x7fe0f0147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 -- 172.21.15.46:0/4169751989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01fd470 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0efe00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 0x7fe0f01fcc00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003e80 tx=0x7fe0dc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc015070 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0dc011840 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc001d50 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f0148200 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01f9960 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.715+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f01f93f0 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.719+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fe0dc018070 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.719+0000 7fe0ecc00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0d0074910 0x7fe0d0076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.719+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe0dc093a90 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.719+0000 7fe0f4e00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0d0074910 0x7fe0d0076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.719+0000 7fe0f4e00640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0d0074910 0x7fe0d0076e20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40ae000 tx=0x7fe0e401a010 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.723+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe0dc05fec0 con 0x7fe0f01fc7f0 2024-09-17T03:33:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:15.935+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe0f0071ce0 con 0x7fe0f01fc7f0 2024-09-17T03:33:16.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.607+0000 7fe0ecc00640 1 -- 172.21.15.46:0/4169751989 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v66) ==== 136+0+0 (secure 0 0 0) 0x7fe0dc0648f0 con 0x7fe0f01fc7f0 2024-09-17T03:33:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.607+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0d0074910 msgr2=0x7fe0d0076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.607+0000 7fe0f7800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0d0074910 0x7fe0d0076e20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40ae000 tx=0x7fe0e401a010 comp rx=0 tx=0).stop 2024-09-17T03:33:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.607+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 msgr2=0x7fe0f01fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.607+0000 7fe0f7800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 0x7fe0f01fcc00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003e80 tx=0x7fe0dc001640 comp rx=0 tx=0).stop 2024-09-17T03:33:16.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 shutdown_connections 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0d0074910 0x7fe0d0076e20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0f01478b0 0x7fe0f0147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f01fc7f0 0x7fe0f01fcc00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 --2- 172.21.15.46:0/4169751989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0d8003a60 0x7fe0f00b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 >> 172.21.15.46:0/4169751989 conn(0x7fe0f00708c0 msgr2=0x7fe0f0155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 shutdown_connections 2024-09-17T03:33:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.611+0000 7fe0f7800640 1 -- 172.21.15.46:0/4169751989 wait complete. 2024-09-17T03:33:16.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:33:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 -- 172.21.15.46:0/2770981370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad841527b0 msgr2=0x7fad8415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 --2- 172.21.15.46:0/2770981370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad841527b0 0x7fad8415bb60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fad84074b20 tx=0x7fad6c01c760 comp rx=0 tx=0).stop 2024-09-17T03:33:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 -- 172.21.15.46:0/2770981370 shutdown_connections 2024-09-17T03:33:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 --2- 172.21.15.46:0/2770981370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad841527b0 0x7fad8415bb60 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 -- 172.21.15.46:0/2770981370 >> 172.21.15.46:0/2770981370 conn(0x7fad840b2040 msgr2=0x7fad840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 -- 172.21.15.46:0/2770981370 shutdown_connections 2024-09-17T03:33:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 -- 172.21.15.46:0/2770981370 wait complete. 2024-09-17T03:33:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 Processor -- start 2024-09-17T03:33:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.795+0000 7fad89a00640 1 -- start start 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad840b4de0 0x7fad841561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 0x7fad841f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad841527b0 0x7fad8414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad84169bd0 con 0x7fad840b5c70 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fad84169a50 con 0x7fad840b4de0 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fad84169d50 con 0x7fad841527b0 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 0x7fad841f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 0x7fad841f98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39994/0 (socket says 172.21.15.46:39994) 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 -- 172.21.15.46:0/3834545567 learned_addr learned my addr 172.21.15.46:0/3834545567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad82a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad840b4de0 0x7fad841561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad83400640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad841527b0 0x7fad8414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 -- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad841527b0 msgr2=0x7fad8414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad841527b0 0x7fad8414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 -- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad840b4de0 msgr2=0x7fad841561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad840b4de0 0x7fad841561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 -- 172.21.15.46:0/3834545567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad840ced90 con 0x7fad840b5c70 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad83400640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad841527b0 0x7fad8414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7be00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 0x7fad841f98d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fad6c01cc40 tx=0x7fad6c0a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0b2040 con 0x7fad840b5c70 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6c0bb070 con 0x7fad840b5c70 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003600 con 0x7fad840b5c70 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad840cccd0 con 0x7fad840b5c70 2024-09-17T03:33:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad840cd290 con 0x7fad840b5c70 2024-09-17T03:33:16.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.799+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8415bb60 con 0x7fad840b5c70 2024-09-17T03:33:16.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.803+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fad6c0037a0 con 0x7fad840b5c70 2024-09-17T03:33:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.803+0000 7fad7aa00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad60074910 0x7fad60076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.803+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fad6c12d5d0 con 0x7fad840b5c70 2024-09-17T03:33:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.803+0000 7fad82a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad60074910 0x7fad60076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.803+0000 7fad82a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad60074910 0x7fad60076e20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad7400b810 tx=0x7fad74008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:16.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:16.807+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad6c016410 con 0x7fad840b5c70 2024-09-17T03:33:17.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.003+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fad840ccb20 con 0x7fad60074910 2024-09-17T03:33:17.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.003+0000 7fad7aa00640 1 -- 172.21.15.46:0/3834545567 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fad840ccb20 con 0x7fad60074910 2024-09-17T03:33:17.006 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:33:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:33:17.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:33:17.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:17.007 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:33:17.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad60074910 msgr2=0x7fad60076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad60074910 0x7fad60076e20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad7400b810 tx=0x7fad74008040 comp rx=0 tx=0).stop 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 msgr2=0x7fad841f98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 0x7fad841f98d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fad6c01cc40 tx=0x7fad6c0a56a0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 shutdown_connections 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fad60074910 0x7fad60076e20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fad841527b0 0x7fad8414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad840b5c70 0x7fad841f98d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 --2- 172.21.15.46:0/3834545567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad840b4de0 0x7fad841561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 >> 172.21.15.46:0/3834545567 conn(0x7fad840b2040 msgr2=0x7fad840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 shutdown_connections 2024-09-17T03:33:17.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.007+0000 7fad89a00640 1 -- 172.21.15.46:0/3834545567 wait complete. 2024-09-17T03:33:17.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 -- 172.21.15.46:0/3592964112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d00b4de0 msgr2=0x7f31d00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 --2- 172.21.15.46:0/3592964112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d00b4de0 0x7f31d00b51f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009c80 tx=0x7f31c801cce0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 -- 172.21.15.46:0/3592964112 shutdown_connections 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 --2- 172.21.15.46:0/3592964112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d00b4de0 0x7f31d00b51f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 --2- 172.21.15.46:0/3592964112 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d00b5c70 0x7f31d015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 -- 172.21.15.46:0/3592964112 >> 172.21.15.46:0/3592964112 conn(0x7f31d00b2040 msgr2=0x7f31d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 -- 172.21.15.46:0/3592964112 shutdown_connections 2024-09-17T03:33:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 -- 172.21.15.46:0/3592964112 wait complete. 2024-09-17T03:33:17.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 Processor -- start 2024-09-17T03:33:17.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.191+0000 7f31d6000640 1 -- start start 2024-09-17T03:33:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d00b4de0 0x7f31d0141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 0x7f31d00cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d01527b0 0x7f31d00cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0168ab0 con 0x7f31d00b5c70 2024-09-17T03:33:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0168930 con 0x7f31d01527b0 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f31d0168c30 con 0x7f31d00b4de0 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 0x7f31d00cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 0x7f31d00cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40006/0 (socket says 172.21.15.46:40006) 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 -- 172.21.15.46:0/2992016599 learned_addr learned my addr 172.21.15.46:0/2992016599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31cf400640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d00b4de0 0x7f31d0141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31cfe00640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d01527b0 0x7f31d00cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 -- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d00b4de0 msgr2=0x7f31d0141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d00b4de0 0x7f31d0141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 -- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d01527b0 msgr2=0x7f31d00cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d01527b0 0x7f31d00cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 -- 172.21.15.46:0/2992016599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d01586c0 con 0x7f31d00b5c70 2024-09-17T03:33:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31cfe00640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d01527b0 0x7f31d00cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:33:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c6a00640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 0x7f31d00cd360 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f31b800c3c0 tx=0x7f31b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8017070 con 0x7f31d00b5c70 2024-09-17T03:33:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31b80053c0 con 0x7f31d00b5c70 2024-09-17T03:33:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d01589a0 con 0x7f31d00b5c70 2024-09-17T03:33:17.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8003840 con 0x7f31d00b5c70 2024-09-17T03:33:17.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d00cce40 con 0x7f31d00b5c70 2024-09-17T03:33:17.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.195+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d00cc950 con 0x7f31d00b5c70 2024-09-17T03:33:17.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.199+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f31b801a070 con 0x7f31d00b5c70 2024-09-17T03:33:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.199+0000 7f31c7e00640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31ac074940 0x7f31ac076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.199+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f31b80925c0 con 0x7f31d00b5c70 2024-09-17T03:33:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.199+0000 7f31cf400640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31ac074940 0x7f31ac076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.199+0000 7f31cf400640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31ac074940 0x7f31ac076e50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009b40 tx=0x7f31c00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:17.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.203+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31b8021020 con 0x7f31d00b5c70 2024-09-17T03:33:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f31d00b51f0 con 0x7f31d00b5c70 2024-09-17T03:33:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31c7e00640 1 -- 172.21.15.46:0/2992016599 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f31d00b5730 con 0x7f31d00b5c70 2024-09-17T03:33:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31ac074940 msgr2=0x7f31ac076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31ac074940 0x7f31ac076e50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009b40 tx=0x7f31c00091e0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 msgr2=0x7f31d00cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 0x7f31d00cd360 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f31b800c3c0 tx=0x7f31b800c880 comp rx=0 tx=0).stop 2024-09-17T03:33:17.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 shutdown_connections 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f31ac074940 0x7f31ac076e50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31d01527b0 0x7f31d00cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d00b5c70 0x7f31d00cd360 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 --2- 172.21.15.46:0/2992016599 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31d00b4de0 0x7f31d0141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 >> 172.21.15.46:0/2992016599 conn(0x7f31d00b2040 msgr2=0x7f31d0154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 shutdown_connections 2024-09-17T03:33:17.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.379+0000 7f31d6000640 1 -- 172.21.15.46:0/2992016599 wait complete. 2024-09-17T03:33:17.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 -- 172.21.15.46:0/2850802787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 msgr2=0x7f078815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 --2- 172.21.15.46:0/2850802787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f078815bb60 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0788070710 tx=0x7f077801c890 comp rx=0 tx=0).stop 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 -- 172.21.15.46:0/2850802787 shutdown_connections 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 --2- 172.21.15.46:0/2850802787 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07880b4de0 0x7f07880b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 --2- 172.21.15.46:0/2850802787 >> v2:172.21.15.99:3300/0 conn(0x7f07881527b0 0x7f0788074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 --2- 172.21.15.46:0/2850802787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f078815bb60 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 -- 172.21.15.46:0/2850802787 >> 172.21.15.46:0/2850802787 conn(0x7f07880b2040 msgr2=0x7f07880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 -- 172.21.15.46:0/2850802787 shutdown_connections 2024-09-17T03:33:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.563+0000 7f078d600640 1 -- 172.21.15.46:0/2850802787 wait complete. 2024-09-17T03:33:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 Processor -- start 2024-09-17T03:33:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 -- start start 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07880b4de0 0x7f0788150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f07880cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07881527b0 0x7f07880aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f078815f6b0 con 0x7f07880b5c70 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f078815f530 con 0x7f07881527b0 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f078815f830 con 0x7f07880b4de0 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f07880cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f07880cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40026/0 (socket says 172.21.15.46:40026) 2024-09-17T03:33:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 -- 172.21.15.46:0/227276214 learned_addr learned my addr 172.21.15.46:0/227276214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f0786a00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07880b4de0 0x7f0788150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f0787400640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07881527b0 0x7f07880aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 -- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07880b4de0 msgr2=0x7f0788150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07880b4de0 0x7f0788150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 -- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07881527b0 msgr2=0x7f07880aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07881527b0 0x7f07880aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 -- 172.21.15.46:0/227276214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07880d00b0 con 0x7f07880b5c70 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077fe00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f07880cdaa0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f07700058d0 tx=0x7f077000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770017070 con 0x7f07880b5c70 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0770009d20 con 0x7f07880b5c70 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.567+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0788152090 con 0x7f07880b5c70 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700106c0 con 0x7f07880b5c70 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07881512d0 con 0x7f07880b5c70 2024-09-17T03:33:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078815fdd0 con 0x7f07880b5c70 2024-09-17T03:33:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f077001a070 con 0x7f07880b5c70 2024-09-17T03:33:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f077ea00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0764074910 0x7f0764076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f0770093140 con 0x7f07880b5c70 2024-09-17T03:33:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.571+0000 7f0786a00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0764074910 0x7f0764076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.575+0000 7f0786a00640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0764074910 0x7f0764076e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f07780af000 tx=0x7f07780a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.575+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07880d00b0 con 0x7f07880b5c70 2024-09-17T03:33:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f07880b51f0 con 0x7f07880b5c70 2024-09-17T03:33:17.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f077ea00640 1 -- 172.21.15.46:0/227276214 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v49) ==== 116+0+2 (secure 0 0 0) 0x7f077005f570 con 0x7f07880b5c70 2024-09-17T03:33:17.755 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0764074910 msgr2=0x7f0764076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0764074910 0x7f0764076e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f07780af000 tx=0x7f07780a7040 comp rx=0 tx=0).stop 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 msgr2=0x7f07880cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f07880cdaa0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f07700058d0 tx=0x7f077000dde0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 shutdown_connections 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0764074910 0x7f0764076e20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07881527b0 0x7f07880aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880b5c70 0x7f07880cdaa0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 --2- 172.21.15.46:0/227276214 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07880b4de0 0x7f0788150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 >> 172.21.15.46:0/227276214 conn(0x7f07880b2040 msgr2=0x7f078815a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 shutdown_connections 2024-09-17T03:33:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.755+0000 7f078d600640 1 -- 172.21.15.46:0/227276214 wait complete. 2024-09-17T03:33:17.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:33:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 -- 172.21.15.46:0/695432186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 msgr2=0x7f865c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 --2- 172.21.15.46:0/695432186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c0b51f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f865c071cb0 tx=0x7f865401cae0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 -- 172.21.15.46:0/695432186 shutdown_connections 2024-09-17T03:33:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 --2- 172.21.15.46:0/695432186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c0b51f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 --2- 172.21.15.46:0/695432186 >> v2:172.21.15.99:3300/0 conn(0x7f865c0b5c70 0x7f865c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 -- 172.21.15.46:0/695432186 >> 172.21.15.46:0/695432186 conn(0x7f865c0b2040 msgr2=0x7f865c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 -- 172.21.15.46:0/695432186 shutdown_connections 2024-09-17T03:33:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 -- 172.21.15.46:0/695432186 wait complete. 2024-09-17T03:33:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 Processor -- start 2024-09-17T03:33:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.939+0000 7f8661000640 1 -- start start 2024-09-17T03:33:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c1f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f865c0b5c70 0x7f865c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c1527b0 0x7f865c14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f865c168950 con 0x7f865c0b4de0 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1687d0 con 0x7f865c1527b0 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f865c168ad0 con 0x7f865c0b5c70 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c1f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c1f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40044/0 (socket says 172.21.15.46:40044) 2024-09-17T03:33:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 -- 172.21.15.46:0/979500873 learned_addr learned my addr 172.21.15.46:0/979500873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f865a800640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c1527b0 0x7f865c14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8653e00640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f865c0b5c70 0x7f865c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 -- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f865c0b5c70 msgr2=0x7f865c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f865c0b5c70 0x7f865c158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 -- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c1527b0 msgr2=0x7f865c14bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c1527b0 0x7f865c14bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 -- 172.21.15.46:0/979500873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c157cc0 con 0x7f865c0b4de0 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f865a800640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c1527b0 0x7f865c14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8659e00640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c1f66d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ca20 tx=0x7f864c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c007c70 con 0x7f865c0b4de0 2024-09-17T03:33:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f864c00f450 con 0x7f865c0b4de0 2024-09-17T03:33:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c157fa0 con 0x7f865c0b4de0 2024-09-17T03:33:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c00f8a0 con 0x7f865c0b4de0 2024-09-17T03:33:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c158560 con 0x7f865c0b4de0 2024-09-17T03:33:17.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.943+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c0b51f0 con 0x7f865c0b4de0 2024-09-17T03:33:17.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.947+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f864c00fa40 con 0x7f865c0b4de0 2024-09-17T03:33:17.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.947+0000 7f8652000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8638074910 0x7f8638076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:17.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.947+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f864c0937b0 con 0x7f865c0b4de0 2024-09-17T03:33:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.947+0000 7f8653e00640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8638074910 0x7f8638076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.947+0000 7f8653e00640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8638074910 0x7f8638076e20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f865c15b3e0 tx=0x7f8644008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:17.951+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f864c014030 con 0x7f865c0b4de0 2024-09-17T03:33:18.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.123+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f865c0cfd10 con 0x7f865c0b4de0 2024-09-17T03:33:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.127+0000 7f8652000640 1 -- 172.21.15.46:0/979500873 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v50)=0 v50) ==== 131+0+0 (secure 0 0 0) 0x7f864c0602b0 con 0x7f865c0b4de0 2024-09-17T03:33:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8638074910 msgr2=0x7f8638076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8638074910 0x7f8638076e20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f865c15b3e0 tx=0x7f8644008040 comp rx=0 tx=0).stop 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 msgr2=0x7f865c1f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c1f66d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ca20 tx=0x7f864c00cee0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 shutdown_connections 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8638074910 0x7f8638076e20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c1527b0 0x7f865c14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f865c0b5c70 0x7f865c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 --2- 172.21.15.46:0/979500873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f865c0b4de0 0x7f865c1f66d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 >> 172.21.15.46:0/979500873 conn(0x7f865c0b2040 msgr2=0x7f865c15a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 shutdown_connections 2024-09-17T03:33:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.131+0000 7f8661000640 1 -- 172.21.15.46:0/979500873 wait complete. 2024-09-17T03:33:18.152 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.151+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.152 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.151+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.152 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.151+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.153 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.151+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.153 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.151+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:33:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 -- 172.21.15.46:0/3621559571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c15c5d0 msgr2=0x7f0b8c073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 --2- 172.21.15.46:0/3621559571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c15c5d0 0x7f0b8c073dc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009900 tx=0x7f0b7c01c8b0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 -- 172.21.15.46:0/3621559571 shutdown_connections 2024-09-17T03:33:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 --2- 172.21.15.46:0/3621559571 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c074300 0x7f0b8c15f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 --2- 172.21.15.46:0/3621559571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c15c5d0 0x7f0b8c073dc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 -- 172.21.15.46:0/3621559571 >> 172.21.15.46:0/3621559571 conn(0x7f0b8c0b2c40 msgr2=0x7f0b8c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 -- 172.21.15.46:0/3621559571 shutdown_connections 2024-09-17T03:33:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.319+0000 7f0b92200640 1 -- 172.21.15.46:0/3621559571 wait complete. 2024-09-17T03:33:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 Processor -- start 2024-09-17T03:33:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 -- start start 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 0x7f0b8c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8c15f6b0 0x7f0b8c1f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c06f980 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c0b3d50 con 0x7f0b8c074300 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c06fb00 con 0x7f0b8c15c5d0 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b83e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8c15f6b0 0x7f0b8c1f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 0x7f0b8c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b83e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c14ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:41568/0 (socket says 172.21.15.46:41568) 2024-09-17T03:33:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b83e00640 1 -- 172.21.15.46:0/140861759 learned_addr learned my addr 172.21.15.46:0/140861759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 0x7f0b8c14f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:33878/0 (socket says 172.21.15.46:33878) 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 -- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c15c5d0 msgr2=0x7f0b8c14ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c14ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 -- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 msgr2=0x7f0b8c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 0x7f0b8c14f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 -- 172.21.15.46:0/140861759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c14ff50 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8b400640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 0x7f0b8c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b8be00640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8c15f6b0 0x7f0b8c1f8a40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c700 tx=0x7f0b8400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84003b80 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b84003d20 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c14e050 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b840052d0 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.323+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c14e610 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.327+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c14fcc0 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.327+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7f0b84005470 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.327+0000 7f0b83400640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b68074910 0x7f0b68076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.327+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f0b84093620 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.327+0000 7f0b8b400640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b68074910 0x7f0b68076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.327+0000 7f0b8b400640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b68074910 0x7f0b68076e20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0af000 tx=0x7f0b7c01b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.331+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0b8c14ff50 con 0x7f0b8c15f6b0 2024-09-17T03:33:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.539+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f0b8c06b540 con 0x7f0b68074910 2024-09-17T03:33:18.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.547+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.547+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.547+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.547+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:18.547+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.583+0000 7f0b83400640 1 -- 172.21.15.46:0/140861759 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b7c0b3040 con 0x7f0b68074910 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b68074910 msgr2=0x7f0b68076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b68074910 0x7f0b68076e20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0af000 tx=0x7f0b7c01b0f0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8c15f6b0 msgr2=0x7f0b8c1f8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8c15f6b0 0x7f0b8c1f8a40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c700 tx=0x7f0b8400cbc0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 shutdown_connections 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0b68074910 0x7f0b68076e20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b8c15f6b0 0x7f0b8c1f8a40 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0b8c15c5d0 0x7f0b8c14ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 --2- 172.21.15.46:0/140861759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c074300 0x7f0b8c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 >> 172.21.15.46:0/140861759 conn(0x7f0b8c0b2c40 msgr2=0x7f0b8c155680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:18.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 shutdown_connections 2024-09-17T03:33:18.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.587+0000 7f0b92200640 1 -- 172.21.15.46:0/140861759 wait complete. 2024-09-17T03:33:18.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:33:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 -- 172.21.15.46:0/2198105180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4740d1250 msgr2=0x7fd474068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 --2- 172.21.15.46:0/2198105180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4740d1250 0x7fd474068df0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009840 tx=0x7fd45c01c820 comp rx=0 tx=0).stop 2024-09-17T03:33:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 -- 172.21.15.46:0/2198105180 shutdown_connections 2024-09-17T03:33:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 --2- 172.21.15.46:0/2198105180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4740d1250 0x7fd474068df0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 -- 172.21.15.46:0/2198105180 >> 172.21.15.46:0/2198105180 conn(0x7fd4740b35f0 msgr2=0x7fd4740b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 -- 172.21.15.46:0/2198105180 shutdown_connections 2024-09-17T03:33:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 -- 172.21.15.46:0/2198105180 wait complete. 2024-09-17T03:33:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 Processor -- start 2024-09-17T03:33:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.771+0000 7fd47a000640 1 -- start start 2024-09-17T03:33:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 0x7fd47414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4740d0e40 0x7fd474157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474071c40 0x7fd474158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd47416b4c0 con 0x7fd474069330 2024-09-17T03:33:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd47416b340 con 0x7fd474071c40 2024-09-17T03:33:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd47416b640 con 0x7fd4740d0e40 2024-09-17T03:33:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 0x7fd47414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 0x7fd47414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40086/0 (socket says 172.21.15.46:40086) 2024-09-17T03:33:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 -- 172.21.15.46:0/2711974713 learned_addr learned my addr 172.21.15.46:0/2711974713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd46be00640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4740d0e40 0x7fd474157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473e00640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474071c40 0x7fd474158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 -- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4740d0e40 msgr2=0x7fd474157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4740d0e40 0x7fd474157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 -- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474071c40 msgr2=0x7fd474158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474071c40 0x7fd474158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 -- 172.21.15.46:0/2711974713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd474152150 con 0x7fd474069330 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd473400640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 0x7fd47414f900 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c730 tx=0x7fd46400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464017070 con 0x7fd474069330 2024-09-17T03:33:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd464006990 con 0x7fd474069330 2024-09-17T03:33:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd474157480 con 0x7fd474069330 2024-09-17T03:33:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464006e20 con 0x7fd474069330 2024-09-17T03:33:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd474154a30 con 0x7fd474069330 2024-09-17T03:33:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.775+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474154c90 con 0x7fd474069330 2024-09-17T03:33:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.779+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) ==== 99325+0+0 (secure 0 0 0) 0x7fd46401a070 con 0x7fd474069330 2024-09-17T03:33:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.779+0000 7fd46b400640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd450074910 0x7fd450076e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.779+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd4640927d0 con 0x7fd474069330 2024-09-17T03:33:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.779+0000 7fd46be00640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd450074910 0x7fd450076e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:18.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.779+0000 7fd46be00640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd450074910 0x7fd450076e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0af000 tx=0x7fd45c01b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.783+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd464021020 con 0x7fd474069330 2024-09-17T03:33:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.983+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd4741502f0 con 0x7fd450074910 2024-09-17T03:33:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd46b400640 1 -- 172.21.15.46:0/2711974713 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd4741502f0 con 0x7fd450074910 2024-09-17T03:33:18.992 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd450074910 msgr2=0x7fd450076e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd450074910 0x7fd450076e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0af000 tx=0x7fd45c01b060 comp rx=0 tx=0).stop 2024-09-17T03:33:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 msgr2=0x7fd47414f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 0x7fd47414f900 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c730 tx=0x7fd46400cbf0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 shutdown_connections 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd450074910 0x7fd450076e20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474071c40 0x7fd474158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd4740d0e40 0x7fd474157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 --2- 172.21.15.46:0/2711974713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd474069330 0x7fd47414f900 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 >> 172.21.15.46:0/2711974713 conn(0x7fd4740b35f0 msgr2=0x7fd47406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 shutdown_connections 2024-09-17T03:33:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:18.991+0000 7fd47a000640 1 -- 172.21.15.46:0/2711974713 wait complete. 2024-09-17T03:33:19.014 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:19.014 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:33:19.014 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:33:19.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:19.024 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:33:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:33:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:33:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.22034 s, 131 MB/s 2024-09-17T03:33:27.304 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:33:27.304 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:33:27.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:27.313 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:33:30.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:33:30.495 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:33:35.882 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:33:35.882 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:33:35.883 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.51749 s, 126 MB/s 2024-09-17T03:33:35.884 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:33:35.884 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:33:35.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:35.893 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:33:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:33:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:33:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.51743 s, 126 MB/s 2024-09-17T03:33:44.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 -- 172.21.15.46:0/3662986009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58004d50 msgr2=0x7f1d58005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 --2- 172.21.15.46:0/3662986009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d58004d50 0x7f1d58005180 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c001e60 tx=0x7f1d6c01cbd0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d72800640 1 -- 172.21.15.46:0/3662986009 reap_dead start 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 -- 172.21.15.46:0/3662986009 shutdown_connections 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 -- 172.21.15.46:0/3662986009 >> 172.21.15.46:0/3662986009 conn(0x7f1d740b2040 msgr2=0x7f1d740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 -- 172.21.15.46:0/3662986009 shutdown_connections 2024-09-17T03:33:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 -- 172.21.15.46:0/3662986009 wait complete. 2024-09-17T03:33:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.627+0000 7f1d78e00640 1 Processor -- start 2024-09-17T03:33:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- start start 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58004d50 0x7f1d74154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d74151cd0 0x7f1d74151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 0x7f1d740aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74167bc0 con 0x7f1d740ae910 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74167a40 con 0x7f1d58004d50 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1d74167d40 con 0x7f1d74151cd0 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 0x7f1d740aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 0x7f1d740aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37590/0 (socket says 172.21.15.46:37590) 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 -- 172.21.15.46:0/557331958 learned_addr learned my addr 172.21.15.46:0/557331958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d71e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58004d50 0x7f1d74154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d6be00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d74151cd0 0x7f1d74151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 -- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d74151cd0 msgr2=0x7f1d74151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d74151cd0 0x7f1d74151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 -- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58004d50 msgr2=0x7f1d74154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58004d50 0x7f1d74154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 -- 172.21.15.46:0/557331958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d740af260 con 0x7f1d740ae910 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d71e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58004d50 0x7f1d74154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d72800640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 0x7f1d740aed20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c01ca00 tx=0x7f1d6c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c001ab0 con 0x7f1d740ae910 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d6c0a59c0 con 0x7f1d740ae910 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c006c50 con 0x7f1d740ae910 2024-09-17T03:33:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d74153320 con 0x7f1d740ae910 2024-09-17T03:33:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d74153600 con 0x7f1d740ae910 2024-09-17T03:33:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.631+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d740b51f0 con 0x7f1d740ae910 2024-09-17T03:33:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.635+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 41) ==== 99427+0+0 (secure 0 0 0) 0x7f1d6c006090 con 0x7f1d740ae910 2024-09-17T03:33:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.635+0000 7f1d6a000640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d50074a30 0x7f1d50076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.635+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f1d6c12c7a0 con 0x7f1d740ae910 2024-09-17T03:33:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.635+0000 7f1d71e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d50074a30 0x7f1d50076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.635+0000 7f1d71e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d50074a30 0x7f1d50076f40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009c60 tx=0x7f1d640091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.639+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1d740af260 con 0x7f1d740ae910 2024-09-17T03:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.843+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f1d7406b540 con 0x7f1d50074a30 2024-09-17T03:33:44.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d6a000640 1 -- 172.21.15.46:0/557331958 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1d640098b0 con 0x7f1d50074a30 2024-09-17T03:33:44.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d50074a30 msgr2=0x7f1d50076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:44.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d50074a30 0x7f1d50076f40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009c60 tx=0x7f1d640091e0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 msgr2=0x7f1d740aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:44.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 0x7f1d740aed20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c01ca00 tx=0x7f1d6c004f90 comp rx=0 tx=0).stop 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 shutdown_connections 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1d50074a30 0x7f1d50076f40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d740ae910 0x7f1d740aed20 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1d74151cd0 0x7f1d74151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 --2- 172.21.15.46:0/557331958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d58004d50 0x7f1d74154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 >> 172.21.15.46:0/557331958 conn(0x7f1d740b2040 msgr2=0x7f1d7415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 shutdown_connections 2024-09-17T03:33:44.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:44.859+0000 7f1d78e00640 1 -- 172.21.15.46:0/557331958 wait complete. 2024-09-17T03:33:44.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:33:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.043+0000 7fa49fa00640 1 -- 172.21.15.46:0/252982785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980749a0 msgr2=0x7fa498074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.043+0000 7fa49fa00640 1 --2- 172.21.15.46:0/252982785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980749a0 0x7fa498074db0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa488009840 tx=0x7fa48801c820 comp rx=0 tx=0).stop 2024-09-17T03:33:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- 172.21.15.46:0/252982785 shutdown_connections 2024-09-17T03:33:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 --2- 172.21.15.46:0/252982785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980749a0 0x7fa498074db0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- 172.21.15.46:0/252982785 >> 172.21.15.46:0/252982785 conn(0x7fa4980b2040 msgr2=0x7fa4980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- 172.21.15.46:0/252982785 shutdown_connections 2024-09-17T03:33:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- 172.21.15.46:0/252982785 wait complete. 2024-09-17T03:33:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 Processor -- start 2024-09-17T03:33:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- start start 2024-09-17T03:33:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4980749a0 0x7fa49813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 0x7fa498159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4980cefa0 0x7fa4980b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa49816b350 con 0x7fa4980b3340 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa49816b1d0 con 0x7fa4980749a0 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.047+0000 7fa49fa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa49816b4d0 con 0x7fa4980cefa0 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 0x7fa498159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 0x7fa498159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37610/0 (socket says 172.21.15.46:37610) 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 -- 172.21.15.46:0/2558184124 learned_addr learned my addr 172.21.15.46:0/2558184124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa49d000640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4980749a0 0x7fa49813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa49da00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4980cefa0 0x7fa4980b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 -- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4980cefa0 msgr2=0x7fa4980b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4980cefa0 0x7fa4980b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 -- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4980749a0 msgr2=0x7fa49813c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4980749a0 0x7fa49813c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 -- 172.21.15.46:0/2558184124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa498159cc0 con 0x7fa4980b3340 2024-09-17T03:33:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa48fe00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 0x7fa498159780 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c430 tx=0x7fa48000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa4980b3340 2024-09-17T03:33:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4800053e0 con 0x7fa4980b3340 2024-09-17T03:33:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480003860 con 0x7fa4980b3340 2024-09-17T03:33:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49813f740 con 0x7fa4980b3340 2024-09-17T03:33:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49813e200 con 0x7fa4980b3340 2024-09-17T03:33:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49813e3d0 con 0x7fa4980b3340 2024-09-17T03:33:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.051+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 41) ==== 99427+0+0 (secure 0 0 0) 0x7fa48001a070 con 0x7fa4980b3340 2024-09-17T03:33:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.055+0000 7fa495600640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa474074a30 0x7fa474076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.055+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa480092330 con 0x7fa4980b3340 2024-09-17T03:33:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.055+0000 7fa49d000640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa474074a30 0x7fa474076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.055+0000 7fa49d000640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa474074a30 0x7fa474076f40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa4880af000 tx=0x7fa48801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.059+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa480098e20 con 0x7fa4980b3340 2024-09-17T03:33:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.263+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa49806b540 con 0x7fa474074a30 2024-09-17T03:33:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.267+0000 7fa495600640 1 -- 172.21.15.46:0/2558184124 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa488004040 con 0x7fa474074a30 2024-09-17T03:33:45.270 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa474074a30 msgr2=0x7fa474076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa474074a30 0x7fa474076f40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa4880af000 tx=0x7fa48801b060 comp rx=0 tx=0).stop 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 msgr2=0x7fa498159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 0x7fa498159780 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c430 tx=0x7fa48000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 shutdown_connections 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa474074a30 0x7fa474076f40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4980cefa0 0x7fa4980b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4980b3340 0x7fa498159780 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 --2- 172.21.15.46:0/2558184124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4980749a0 0x7fa49813c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 >> 172.21.15.46:0/2558184124 conn(0x7fa4980b2040 msgr2=0x7fa49815aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 shutdown_connections 2024-09-17T03:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:45.271+0000 7fa49fa00640 1 -- 172.21.15.46:0/2558184124 wait complete. 2024-09-17T03:33:45.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:45.292 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:45.292 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:33:45.302 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:45.302 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:33:45.302 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:45.303 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:33:47.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:47.304 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:47.304 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:33:47.316 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:47.316 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:33:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:47.317 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:33:49.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:33:49.318 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:49.318 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:33:49.329 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/ea255d40-3846-4b2a-b236-14fbc08d6ca8 2024-09-17T03:33:49.329 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T03:33:49.329 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:33:49.330 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T03:33:49.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-17T03:33:49.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 -- 172.21.15.46:0/3474156647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4900b5c70 msgr2=0x7fa49015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 --2- 172.21.15.46:0/3474156647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4900b5c70 0x7fa49015bb60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa490072e30 tx=0x7fa48401c800 comp rx=0 tx=0).stop 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 -- 172.21.15.46:0/3474156647 shutdown_connections 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 --2- 172.21.15.46:0/3474156647 >> v2:172.21.15.99:3300/0 conn(0x7fa4900b4de0 0x7fa4900b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 --2- 172.21.15.46:0/3474156647 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4901527b0 0x7fa490074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 --2- 172.21.15.46:0/3474156647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4900b5c70 0x7fa49015bb60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 -- 172.21.15.46:0/3474156647 >> 172.21.15.46:0/3474156647 conn(0x7fa4900b2040 msgr2=0x7fa4900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 -- 172.21.15.46:0/3474156647 shutdown_connections 2024-09-17T03:33:49.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.547+0000 7fa497800640 1 -- 172.21.15.46:0/3474156647 wait complete. 2024-09-17T03:33:49.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 Processor -- start 2024-09-17T03:33:49.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 -- start start 2024-09-17T03:33:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4900b4de0 0x7fa4901561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4900b5c70 0x7fa4901510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 0x7fa4900cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa49015f700 con 0x7fa4901527b0 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa49015f580 con 0x7fa4900b5c70 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa49015f880 con 0x7fa4900b4de0 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 0x7fa4900cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 0x7fa4900cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37628/0 (socket says 172.21.15.46:37628) 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 -- 172.21.15.46:0/1218930486 learned_addr learned my addr 172.21.15.46:0/1218930486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa494e00640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4900b4de0 0x7fa4901561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa48fe00640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4900b5c70 0x7fa4901510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 -- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4900b4de0 msgr2=0x7fa4901561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4900b4de0 0x7fa4901561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 -- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4900b5c70 msgr2=0x7fa4901510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4900b5c70 0x7fa4901510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 -- 172.21.15.46:0/1218930486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4900ccb80 con 0x7fa4901527b0 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa495800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 0x7fa4900cfa00 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa48800e9c0 tx=0x7fa48800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488004d30 con 0x7fa4901527b0 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa48800cb20 con 0x7fa4901527b0 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4900cce20 con 0x7fa4901527b0 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488005020 con 0x7fa4901527b0 2024-09-17T03:33:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.551+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4900cf4e0 con 0x7fa4901527b0 2024-09-17T03:33:49.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.555+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900af670 con 0x7fa4901527b0 2024-09-17T03:33:49.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.555+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 41) ==== 99427+0+0 (secure 0 0 0) 0x7fa4880051c0 con 0x7fa4901527b0 2024-09-17T03:33:49.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.555+0000 7fa48cc00640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa470074a30 0x7fa470076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.555+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa4880937b0 con 0x7fa4901527b0 2024-09-17T03:33:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.555+0000 7fa494e00640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa470074a30 0x7fa470076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.555+0000 7fa494e00640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa470074a30 0x7fa470076f40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa4840af000 tx=0x7fa4840a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:49.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.559+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa488016030 con 0x7fa4901527b0 2024-09-17T03:33:49.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.767+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa49006b540 con 0x7fa470074a30 2024-09-17T03:33:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.827+0000 7fa48cc00640 1 -- 172.21.15.46:0/1218930486 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4840b3040 con 0x7fa470074a30 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa470074a30 msgr2=0x7fa470076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa470074a30 0x7fa470076f40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa4840af000 tx=0x7fa4840a7040 comp rx=0 tx=0).stop 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 msgr2=0x7fa4900cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 0x7fa4900cfa00 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa48800e9c0 tx=0x7fa48800ee80 comp rx=0 tx=0).stop 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 shutdown_connections 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa470074a30 0x7fa470076f40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4901527b0 0x7fa4900cfa00 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4900b5c70 0x7fa4901510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 --2- 172.21.15.46:0/1218930486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4900b4de0 0x7fa4901561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 >> 172.21.15.46:0/1218930486 conn(0x7fa4900b2040 msgr2=0x7fa49015ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 shutdown_connections 2024-09-17T03:33:49.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fa497800640 1 -- 172.21.15.46:0/1218930486 wait complete. 2024-09-17T03:33:49.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.833 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.834 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.834 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.831+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.853 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.851+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.853 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.851+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.853 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.851+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.854 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.851+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.854 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:49.851+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:49.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 -- 172.21.15.46:0/2244231573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43640b5c70 msgr2=0x7f436415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 --2- 172.21.15.46:0/2244231573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43640b5c70 0x7f436415bb60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4354009840 tx=0x7f435401c760 comp rx=0 tx=0).stop 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 -- 172.21.15.46:0/2244231573 shutdown_connections 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 --2- 172.21.15.46:0/2244231573 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43641527b0 0x7f4364074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 --2- 172.21.15.46:0/2244231573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43640b5c70 0x7f436415bb60 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 -- 172.21.15.46:0/2244231573 >> 172.21.15.46:0/2244231573 conn(0x7f43640b2040 msgr2=0x7f43640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:50.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 -- 172.21.15.46:0/2244231573 shutdown_connections 2024-09-17T03:33:50.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 -- 172.21.15.46:0/2244231573 wait complete. 2024-09-17T03:33:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 Processor -- start 2024-09-17T03:33:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.023+0000 7f436a400640 1 -- start start 2024-09-17T03:33:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43640b4de0 0x7f4364158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43640b5c70 0x7f43641539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 0x7f4364150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4364168ca0 con 0x7f43641527b0 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4364168b20 con 0x7f43640b4de0 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4364168e20 con 0x7f43640b5c70 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f4363e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 0x7f4364150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f4363e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 0x7f4364150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37674/0 (socket says 172.21.15.46:37674) 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f4363e00640 1 -- 172.21.15.46:0/3237748168 learned_addr learned my addr 172.21.15.46:0/3237748168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43640b5c70 0x7f43641539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f4363400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43640b4de0 0x7f4364158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 -- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43640b4de0 msgr2=0x7f4364158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43640b4de0 0x7f4364158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 -- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 msgr2=0x7f4364150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 0x7f4364150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 -- 172.21.15.46:0/3237748168 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641f8ac0 con 0x7f43640b5c70 2024-09-17T03:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f4363e00640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 0x7f4364150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435be00640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43640b5c70 0x7f43641539f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e6c0 tx=0x7f434c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c004220 con 0x7f43640b5c70 2024-09-17T03:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f434c00c8b0 con 0x7f43640b5c70 2024-09-17T03:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c00cd00 con 0x7f43640b5c70 2024-09-17T03:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43641593b0 con 0x7f43640b5c70 2024-09-17T03:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43641f9990 con 0x7f43640b5c70 2024-09-17T03:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.027+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436415bb60 con 0x7f43640b5c70 2024-09-17T03:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.031+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 41) ==== 99427+0+0 (secure 0 0 0) 0x7f434c0388a0 con 0x7f43640b5c70 2024-09-17T03:33:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.031+0000 7f435b400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4340074a30 0x7f4340076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.031+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f434c093690 con 0x7f43640b5c70 2024-09-17T03:33:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.031+0000 7f4363400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4340074a30 0x7f4340076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.031+0000 7f4363400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4340074a30 0x7f4340076f40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f43540af000 tx=0x7f435401afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.035+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f434c05fb40 con 0x7f43640b5c70 2024-09-17T03:33:50.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.243+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f436415bfa0 con 0x7f4340074a30 2024-09-17T03:33:50.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:50.831+0000 7fbdf3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:50.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:50.831+0000 7fbdf3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:50.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:50.831+0000 7fbdf3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:50.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:50.831+0000 7fbdf3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:50.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:33:50.831+0000 7fbdf3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:33:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.867+0000 7f435b400640 1 -- 172.21.15.46:0/3237748168 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f436415bfa0 con 0x7f4340074a30 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "in-progress", 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_report": { 2024-09-17T03:33:50.869 INFO:teuthology.orchestra.run.smithi046.stdout: "percentage cloned": "0.0%", 2024-09-17T03:33:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "amount cloned": "184/3.0G", 2024-09-17T03:33:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: "files cloned": "3/4" 2024-09-17T03:33:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:50.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:50.870 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4340074a30 msgr2=0x7f4340076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4340074a30 0x7f4340076f40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f43540af000 tx=0x7f435401afa0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43640b5c70 msgr2=0x7f43641539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43640b5c70 0x7f43641539f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e6c0 tx=0x7f434c00eb80 comp rx=0 tx=0).stop 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 shutdown_connections 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4340074a30 0x7f4340076f40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43641527b0 0x7f4364150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f43640b5c70 0x7f43641539f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 --2- 172.21.15.46:0/3237748168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43640b4de0 0x7f4364158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 >> 172.21.15.46:0/3237748168 conn(0x7f43640b2040 msgr2=0x7f43640b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 shutdown_connections 2024-09-17T03:33:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:50.871+0000 7f436a400640 1 -- 172.21.15.46:0/3237748168 wait complete. 2024-09-17T03:33:50.905 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T03:33:50.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:33:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 -- 172.21.15.46:0/433692423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 msgr2=0x7f33000cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 --2- 172.21.15.46:0/433692423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33000cefb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b3f0 tx=0x7f32ec01ce10 comp rx=0 tx=0).stop 2024-09-17T03:33:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 -- 172.21.15.46:0/433692423 shutdown_connections 2024-09-17T03:33:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 --2- 172.21.15.46:0/433692423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33000cefb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 --2- 172.21.15.46:0/433692423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f330015ef10 0x7f330015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 -- 172.21.15.46:0/433692423 >> 172.21.15.46:0/433692423 conn(0x7f330006e1d0 msgr2=0x7f330006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 -- 172.21.15.46:0/433692423 shutdown_connections 2024-09-17T03:33:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.239+0000 7f3306200640 1 -- 172.21.15.46:0/433692423 wait complete. 2024-09-17T03:33:51.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.243+0000 7f3306200640 1 Processor -- start 2024-09-17T03:33:51.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.243+0000 7f3306200640 1 -- start start 2024-09-17T03:33:51.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f3306200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 0x7f33001572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:51.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f3306200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33001f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f3306200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330015ef10 0x7f33001f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f3306200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3300168230 con 0x7f330015ef10 2024-09-17T03:33:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f3306200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33001680b0 con 0x7f33000ceba0 2024-09-17T03:33:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f3306200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f33001683b0 con 0x7f33000cd260 2024-09-17T03:33:51.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 0x7f33001572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:51.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 0x7f33001572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:44706/0 (socket says 172.21.15.46:44706) 2024-09-17T03:33:51.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 -- 172.21.15.46:0/3180053297 learned_addr learned my addr 172.21.15.46:0/3180053297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:51.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32fea00640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33001f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 -- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 msgr2=0x7f33001f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33001f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 -- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330015ef10 msgr2=0x7f33001f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330015ef10 0x7f33001f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 -- 172.21.15.46:0/3180053297 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33001ee7e0 con 0x7f33000cd260 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32fea00640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33001f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32ff400640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 0x7f33001572a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c890 tx=0x7f32f000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0004220 con 0x7f33000cd260 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32f00079b0 con 0x7f33000cd260 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.247+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f00044b0 con 0x7f33000cd260 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.251+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3300200a10 con 0x7f33000cd260 2024-09-17T03:33:51.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.251+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3300200fd0 con 0x7f33000cd260 2024-09-17T03:33:51.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.255+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33001ee9b0 con 0x7f33000cd260 2024-09-17T03:33:51.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.255+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 41) ==== 99427+0+0 (secure 0 0 0) 0x7f32f0005020 con 0x7f33000cd260 2024-09-17T03:33:51.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.259+0000 7f32f7400640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32e0085a40 0x7f32e0087f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:51.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.259+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f32f0093440 con 0x7f33000cd260 2024-09-17T03:33:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.263+0000 7f32fea00640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32e0085a40 0x7f32e0087f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.263+0000 7f32fea00640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32e0085a40 0x7f32e0087f50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0af000 tx=0x7f32ec01b6b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.271+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32f0014030 con 0x7f33000cd260 2024-09-17T03:33:51.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.659+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f330006b540 con 0x7f32e0085a40 2024-09-17T03:33:51.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.727+0000 7f32f7400640 1 -- 172.21.15.46:0/3180053297 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f32ec009290 con 0x7f32e0085a40 2024-09-17T03:33:51.729 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:51.729 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "in-progress", 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_report": { 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "percentage cloned": "0.0%", 2024-09-17T03:33:51.730 INFO:teuthology.orchestra.run.smithi046.stdout: "amount cloned": "184/3.0G", 2024-09-17T03:33:51.731 INFO:teuthology.orchestra.run.smithi046.stdout: "files cloned": "4/4" 2024-09-17T03:33:51.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:51.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:51.731 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32e0085a40 msgr2=0x7f32e0087f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32e0085a40 0x7f32e0087f50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0af000 tx=0x7f32ec01b6b0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 msgr2=0x7f33001572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 0x7f33001572a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c890 tx=0x7f32f000cd50 comp rx=0 tx=0).stop 2024-09-17T03:33:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 shutdown_connections 2024-09-17T03:33:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f32e0085a40 0x7f32e0087f50 secure :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0af000 tx=0x7f32ec01b6b0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f330015ef10 0x7f33001f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33000ceba0 0x7f33001f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 --2- 172.21.15.46:0/3180053297 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33000cd260 0x7f33001572a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.731+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 >> 172.21.15.46:0/3180053297 conn(0x7f330006e1d0 msgr2=0x7f33001f08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.735+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 shutdown_connections 2024-09-17T03:33:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:51.735+0000 7f3306200640 1 -- 172.21.15.46:0/3180053297 wait complete. 2024-09-17T03:33:51.798 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-17T03:33:52.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 -- 172.21.15.46:0/1287088554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c1557e0 msgr2=0x7f985c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 --2- 172.21.15.46:0/1287088554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c1557e0 0x7f985c0ae8e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9848009900 tx=0x7f984801c760 comp rx=0 tx=0).stop 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 -- 172.21.15.46:0/1287088554 shutdown_connections 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 --2- 172.21.15.46:0/1287088554 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c0aee20 0x7f985c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 --2- 172.21.15.46:0/1287088554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c1557e0 0x7f985c0ae8e0 secure :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9848009900 tx=0x7f984801c760 comp rx=0 tx=0).stop 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 -- 172.21.15.46:0/1287088554 >> 172.21.15.46:0/1287088554 conn(0x7f985c06e1d0 msgr2=0x7f985c06e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:53.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.075+0000 7f9863e00640 1 -- 172.21.15.46:0/1287088554 shutdown_connections 2024-09-17T03:33:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.079+0000 7f9863e00640 1 -- 172.21.15.46:0/1287088554 wait complete. 2024-09-17T03:33:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.079+0000 7f9863e00640 1 Processor -- start 2024-09-17T03:33:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.079+0000 7f9863e00640 1 -- start start 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9863e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c0aee20 0x7f985c14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9863e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c15f8f0 0x7f985c144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9863e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f985c1451b0 0x7f985c14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9863e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0ccc00 con 0x7f985c0aee20 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9863e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0cca80 con 0x7f985c1451b0 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9863e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f985c0ccd80 con 0x7f985c15f8f0 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9861400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c0aee20 0x7f985c14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c15f8f0 0x7f985c144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9861400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c0aee20 0x7f985c14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48864/0 (socket says 172.21.15.46:48864) 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9861400640 1 -- 172.21.15.46:0/4257721776 learned_addr learned my addr 172.21.15.46:0/4257721776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9861e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f985c1451b0 0x7f985c14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 -- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f985c1451b0 msgr2=0x7f985c14c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f985c1451b0 0x7f985c14c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 -- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c0aee20 msgr2=0x7f985c14c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c0aee20 0x7f985c14c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 -- 172.21.15.46:0/4257721776 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c144fb0 con 0x7f985c15f8f0 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.083+0000 7f9860a00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c15f8f0 0x7f985c144a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98480ac000 tx=0x7f98480a56f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98480b2040 con 0x7f985c15f8f0 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9848004230 con 0x7f985c15f8f0 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848003600 con 0x7f985c15f8f0 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985c14b1b0 con 0x7f985c15f8f0 2024-09-17T03:33:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f985c14b770 con 0x7f985c15f8f0 2024-09-17T03:33:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985c14af20 con 0x7f985c15f8f0 2024-09-17T03:33:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.087+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f98480bf020 con 0x7f985c15f8f0 2024-09-17T03:33:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.091+0000 7f9859600640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f983c074be0 0x7f983c0770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.091+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f984812d060 con 0x7f985c15f8f0 2024-09-17T03:33:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.091+0000 7f9861400640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f983c074be0 0x7f983c0770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.095+0000 7f9861400640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f983c074be0 0x7f983c0770f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98500099a0 tx=0x7f98500090e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.095+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f985c144fb0 con 0x7f985c15f8f0 2024-09-17T03:33:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.563+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f985c06b540 con 0x7f983c074be0 2024-09-17T03:33:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.651+0000 7f9859600640 1 -- 172.21.15.46:0/4257721776 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f98500095f0 con 0x7f983c074be0 2024-09-17T03:33:53.654 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:33:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "in-progress", 2024-09-17T03:33:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:33:53.654 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_report": { 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "percentage cloned": "40.104%", 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "amount cloned": "1.2G/3.0G", 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: "files cloned": "5/4" 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:53.655 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.655+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f983c074be0 msgr2=0x7f983c0770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.655+0000 7f9863e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f983c074be0 0x7f983c0770f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98500099a0 tx=0x7f98500090e0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.655+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c15f8f0 msgr2=0x7f985c144a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.655+0000 7f9863e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c15f8f0 0x7f985c144a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98480ac000 tx=0x7f98480a56f0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 shutdown_connections 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f983c074be0 0x7f983c0770f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f985c1451b0 0x7f985c14c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f985c15f8f0 0x7f985c144a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 --2- 172.21.15.46:0/4257721776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f985c0aee20 0x7f985c14c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 >> 172.21.15.46:0/4257721776 conn(0x7f985c06e1d0 msgr2=0x7f985c0732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 shutdown_connections 2024-09-17T03:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:53.659+0000 7f9863e00640 1 -- 172.21.15.46:0/4257721776 wait complete. 2024-09-17T03:33:53.714 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 40.104% 2024-09-17T03:33:54.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.035+0000 7f240a000640 1 -- 172.21.15.46:0/3037723638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24040d1140 msgr2=0x7f24040b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.035+0000 7f240a000640 1 --2- 172.21.15.46:0/3037723638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24040d1140 0x7f24040b0730 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24040ced20 tx=0x7f23f801c830 comp rx=0 tx=0).stop 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.039+0000 7f240a000640 1 -- 172.21.15.46:0/3037723638 shutdown_connections 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.039+0000 7f240a000640 1 --2- 172.21.15.46:0/3037723638 >> v2:172.21.15.46:3300/0 conn(0x7f240415f5c0 0x7f240415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.039+0000 7f240a000640 1 --2- 172.21.15.46:0/3037723638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24040d1140 0x7f24040b0730 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.039+0000 7f240a000640 1 -- 172.21.15.46:0/3037723638 >> 172.21.15.46:0/3037723638 conn(0x7f24040b41f0 msgr2=0x7f24040b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.039+0000 7f240a000640 1 -- 172.21.15.46:0/3037723638 shutdown_connections 2024-09-17T03:33:55.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.039+0000 7f240a000640 1 -- 172.21.15.46:0/3037723638 wait complete. 2024-09-17T03:33:55.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.047+0000 7f240a000640 1 Processor -- start 2024-09-17T03:33:55.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.047+0000 7f240a000640 1 -- start start 2024-09-17T03:33:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.051+0000 7f240a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406dbc0 0x7f240409aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.051+0000 7f240a000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24040d1140 0x7f240415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.051+0000 7f240a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240415f5c0 0x7f240415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.051+0000 7f240a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2404069180 con 0x7f240406dbc0 2024-09-17T03:33:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.051+0000 7f240a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2404069000 con 0x7f240415f5c0 2024-09-17T03:33:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.051+0000 7f240a000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2404069300 con 0x7f24040d1140 2024-09-17T03:33:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2409600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240415f5c0 0x7f240415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2408c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406dbc0 0x7f240409aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24040d1140 0x7f240415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2409600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240415f5c0 0x7f240415aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:41540/0 (socket says 172.21.15.46:41540) 2024-09-17T03:33:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2409600640 1 -- 172.21.15.46:0/4168740847 learned_addr learned my addr 172.21.15.46:0/4168740847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 -- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240415f5c0 msgr2=0x7f240415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240415f5c0 0x7f240415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 -- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406dbc0 msgr2=0x7f240409aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406dbc0 0x7f240409aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 -- 172.21.15.46:0/4168740847 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2404076300 con 0x7f24040d1140 2024-09-17T03:33:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2408c00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406dbc0 0x7f240409aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:33:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2403e00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24040d1140 0x7f240415a4e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c3c0 tx=0x7f23f000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0017070 con 0x7f24040d1140 2024-09-17T03:33:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f00053e0 con 0x7f24040d1140 2024-09-17T03:33:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0003860 con 0x7f24040d1140 2024-09-17T03:33:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2404155750 con 0x7f24040d1140 2024-09-17T03:33:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.055+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041553b0 con 0x7f24040d1140 2024-09-17T03:33:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.059+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24040b0730 con 0x7f24040d1140 2024-09-17T03:33:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.063+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f23f001a070 con 0x7f24040d1140 2024-09-17T03:33:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.067+0000 7f2400c00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23e4074c10 0x7f23e4077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.067+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f23f0092630 con 0x7f24040d1140 2024-09-17T03:33:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.071+0000 7f2408c00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23e4074c10 0x7f23e4077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.071+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23f0021020 con 0x7f24040d1140 2024-09-17T03:33:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.071+0000 7f2408c00640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23e4074c10 0x7f23e4077120 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f23f80af000 tx=0x7f23f801b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:55.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.511+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f240406b540 con 0x7f23e4074c10 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f2400c00640 1 -- 172.21.15.46:0/4168740847 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f23f80b3040 con 0x7f23e4074c10 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "in-progress", 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:33:55.540 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_report": { 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: "percentage cloned": "66.667%", 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: "amount cloned": "2.0G/3.0G", 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: "files cloned": "5/4" 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:55.541 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:55.542 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23e4074c10 msgr2=0x7f23e4077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23e4074c10 0x7f23e4077120 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f23f80af000 tx=0x7f23f801b070 comp rx=0 tx=0).stop 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24040d1140 msgr2=0x7f240415a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24040d1140 0x7f240415a4e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c3c0 tx=0x7f23f000c880 comp rx=0 tx=0).stop 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 shutdown_connections 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23e4074c10 0x7f23e4077120 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f240415f5c0 0x7f240415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24040d1140 0x7f240415a4e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 --2- 172.21.15.46:0/4168740847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f240406dbc0 0x7f240409aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 >> 172.21.15.46:0/4168740847 conn(0x7f24040b41f0 msgr2=0x7f2404070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 shutdown_connections 2024-09-17T03:33:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:55.539+0000 7f240a000640 1 -- 172.21.15.46:0/4168740847 wait complete. 2024-09-17T03:33:55.584 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 66.667% 2024-09-17T03:33:56.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:33:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609f200640 1 -- 172.21.15.46:0/2734777770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f607c0041d0 msgr2=0x7f607c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609f200640 1 --2- 172.21.15.46:0/2734777770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f607c0041d0 0x7f607c004810 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6080006fa0 tx=0x7f608001cc20 comp rx=0 tx=0).stop 2024-09-17T03:33:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609d200640 1 -- 172.21.15.46:0/2734777770 reap_dead start 2024-09-17T03:33:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609f200640 1 -- 172.21.15.46:0/2734777770 shutdown_connections 2024-09-17T03:33:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609f200640 1 -- 172.21.15.46:0/2734777770 >> 172.21.15.46:0/2734777770 conn(0x7f60980b2040 msgr2=0x7f60980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609f200640 1 -- 172.21.15.46:0/2734777770 shutdown_connections 2024-09-17T03:33:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.747+0000 7f609f200640 1 -- 172.21.15.46:0/2734777770 wait complete. 2024-09-17T03:33:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 Processor -- start 2024-09-17T03:33:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 -- start start 2024-09-17T03:33:56.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f607c0041d0 0x7f609814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 0x7f60981f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60981f9100 0x7f60981f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6098074db0 con 0x7f609815a310 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60980b4280 con 0x7f60981f9100 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6098074f30 con 0x7f607c0041d0 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 0x7f60981f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 0x7f60981f8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48884/0 (socket says 172.21.15.46:48884) 2024-09-17T03:33:56.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 -- 172.21.15.46:0/1023514876 learned_addr learned my addr 172.21.15.46:0/1023514876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609d200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60981f9100 0x7f60981f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:56.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f6097e00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f607c0041d0 0x7f609814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 -- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f607c0041d0 msgr2=0x7f609814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f607c0041d0 0x7f609814f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 -- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60981f9100 msgr2=0x7f60981f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60981f9100 0x7f60981f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 -- 172.21.15.46:0/1023514876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609815a890 con 0x7f609815a310 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609d200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60981f9100 0x7f60981f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f608fe00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 0x7f60981f8bc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f608001b320 tx=0x7f6080001140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800a5b90 con 0x7f609815a310 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60800a5d30 con 0x7f609815a310 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609814fa10 con 0x7f609815a310 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800063a0 con 0x7f609815a310 2024-09-17T03:33:56.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.751+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609815ae30 con 0x7f609815a310 2024-09-17T03:33:56.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.755+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60980b3750 con 0x7f609815a310 2024-09-17T03:33:56.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.755+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f6080003600 con 0x7f609815a310 2024-09-17T03:33:56.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.755+0000 7f6094c00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6074074be0 0x7f60740770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.755+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f608012c170 con 0x7f609815a310 2024-09-17T03:33:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.755+0000 7f6097e00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6074074be0 0x7f60740770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.755+0000 7f6097e00640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6074074be0 0x7f60740770f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6088009b00 tx=0x7f60880091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.759+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6080016c90 con 0x7f609815a310 2024-09-17T03:33:56.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:56.971+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f609806b540 con 0x7f6074074be0 2024-09-17T03:33:59.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.207+0000 7f6094c00640 1 -- 172.21.15.46:0/1023514876 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f6088012b70 con 0x7f6074074be0 2024-09-17T03:33:59.210 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:33:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "complete" 2024-09-17T03:33:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:59.211 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6074074be0 msgr2=0x7f60740770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6074074be0 0x7f60740770f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6088009b00 tx=0x7f60880091e0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 msgr2=0x7f60981f8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 0x7f60981f8bc0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f608001b320 tx=0x7f6080001140 comp rx=0 tx=0).stop 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 shutdown_connections 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6074074be0 0x7f60740770f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60981f9100 0x7f60981f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f609815a310 0x7f60981f8bc0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 --2- 172.21.15.46:0/1023514876 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f607c0041d0 0x7f609814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 >> 172.21.15.46:0/1023514876 conn(0x7f60980b2040 msgr2=0x7f60980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 shutdown_connections 2024-09-17T03:33:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.211+0000 7f609f200640 1 -- 172.21.15.46:0/1023514876 wait complete. 2024-09-17T03:33:59.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:33:59.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb87200640 1 -- 172.21.15.46:0/2041512932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 msgr2=0x7feb640051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb87200640 1 --2- 172.21.15.46:0/2041512932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb640051a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7feb78009e00 tx=0x7feb7801d150 comp rx=0 tx=0).stop 2024-09-17T03:33:59.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb85200640 1 -- 172.21.15.46:0/2041512932 reap_dead start 2024-09-17T03:33:59.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb87200640 1 -- 172.21.15.46:0/2041512932 shutdown_connections 2024-09-17T03:33:59.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb87200640 1 -- 172.21.15.46:0/2041512932 >> 172.21.15.46:0/2041512932 conn(0x7feb800b2040 msgr2=0x7feb800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:59.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb87200640 1 -- 172.21.15.46:0/2041512932 shutdown_connections 2024-09-17T03:33:59.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.403+0000 7feb87200640 1 -- 172.21.15.46:0/2041512932 wait complete. 2024-09-17T03:33:59.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 Processor -- start 2024-09-17T03:33:59.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 -- start start 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb801e5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb80142730 0x7feb80142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb801e7e70 0x7feb801e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb8015cf30 con 0x7feb64004d50 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb8015cdb0 con 0x7feb801e7e70 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feb8015d0b0 con 0x7feb80142730 2024-09-17T03:33:59.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb801e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb801e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48906/0 (socket says 172.21.15.46:48906) 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 -- 172.21.15.46:0/891435781 learned_addr learned my addr 172.21.15.46:0/891435781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7f400640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb80142730 0x7feb80142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb85200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb801e7e70 0x7feb801e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 -- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb80142730 msgr2=0x7feb80142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb80142730 0x7feb80142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 -- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb801e7e70 msgr2=0x7feb801e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb801e7e70 0x7feb801e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 -- 172.21.15.46:0/891435781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb801e87c0 con 0x7feb64004d50 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb85200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb801e7e70 0x7feb801e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:33:59.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb7fe00640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb801e5680 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7feb70010a70 tx=0x7feb70010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000ecb0 con 0x7feb64004d50 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb7000ee50 con 0x7feb64004d50 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb700116a0 con 0x7feb64004d50 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb801f95e0 con 0x7feb64004d50 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.407+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801f9ba0 con 0x7feb64004d50 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.411+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb800768c0 con 0x7feb64004d50 2024-09-17T03:33:59.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.411+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7feb70011840 con 0x7feb64004d50 2024-09-17T03:33:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.411+0000 7feb77e00640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb60074be0 0x7feb600770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.411+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7feb70093f50 con 0x7feb64004d50 2024-09-17T03:33:59.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.411+0000 7feb7f400640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb60074be0 0x7feb600770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.411+0000 7feb7f400640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb60074be0 0x7feb600770f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feb6c006fd0 tx=0x7feb6c001d70 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:59.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.415+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb801e87c0 con 0x7feb64004d50 2024-09-17T03:33:59.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.631+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7feb801f9390 con 0x7feb60074be0 2024-09-17T03:33:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.635+0000 7feb77e00640 1 -- 172.21.15.46:0/891435781 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7feb801f9390 con 0x7feb60074be0 2024-09-17T03:33:59.637 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:33:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:33:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "complete" 2024-09-17T03:33:59.637 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:33:59.637 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:33:59.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb60074be0 msgr2=0x7feb600770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb60074be0 0x7feb600770f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feb6c006fd0 tx=0x7feb6c001d70 comp rx=0 tx=0).stop 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 msgr2=0x7feb801e5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb801e5680 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7feb70010a70 tx=0x7feb70010f30 comp rx=0 tx=0).stop 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 shutdown_connections 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb60074be0 0x7feb600770f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb801e7e70 0x7feb801e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb80142730 0x7feb80142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 --2- 172.21.15.46:0/891435781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb64004d50 0x7feb801e5680 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 >> 172.21.15.46:0/891435781 conn(0x7feb800b2040 msgr2=0x7feb80151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 shutdown_connections 2024-09-17T03:33:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.639+0000 7feb87200640 1 -- 172.21.15.46:0/891435781 wait complete. 2024-09-17T03:33:59.660 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-17T03:33:59.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 -- 172.21.15.46:0/3126528193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae48004810 msgr2=0x7fae48004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 --2- 172.21.15.46:0/3126528193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae48004810 0x7fae48004c20 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fae50006f90 tx=0x7fae5001cee0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae62800640 1 -- 172.21.15.46:0/3126528193 reap_dead start 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 -- 172.21.15.46:0/3126528193 shutdown_connections 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 -- 172.21.15.46:0/3126528193 >> 172.21.15.46:0/3126528193 conn(0x7fae640b2040 msgr2=0x7fae640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 -- 172.21.15.46:0/3126528193 shutdown_connections 2024-09-17T03:33:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 -- 172.21.15.46:0/3126528193 wait complete. 2024-09-17T03:33:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.819+0000 7fae68c00640 1 Processor -- start 2024-09-17T03:33:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- start start 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae48004810 0x7fae6415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 0x7fae6415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae6413f510 0x7fae6415f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae64165c40 con 0x7fae640ca0d0 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae64165ac0 con 0x7fae6413f510 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fae64165dc0 con 0x7fae48004810 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 0x7fae6415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 0x7fae6415ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48932/0 (socket says 172.21.15.46:48932) 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 -- 172.21.15.46:0/3342249420 learned_addr learned my addr 172.21.15.46:0/3342249420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61e00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae48004810 0x7fae6415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae62800640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae6413f510 0x7fae6415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 -- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae48004810 msgr2=0x7fae6415a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae48004810 0x7fae6415a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 -- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae6413f510 msgr2=0x7fae6415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae6413f510 0x7fae6415f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 -- 172.21.15.46:0/3342249420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64163c80 con 0x7fae640ca0d0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae61400640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 0x7fae6415ef30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fae50006f90 tx=0x7fae500011d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50006440 con 0x7fae640ca0d0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae50003600 con 0x7fae640ca0d0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae64161fa0 con 0x7fae640ca0d0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50003a50 con 0x7fae640ca0d0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6415a690 con 0x7fae640ca0d0 2024-09-17T03:33:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.823+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae64074db0 con 0x7fae640ca0d0 2024-09-17T03:33:59.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.827+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7fae50003bf0 con 0x7fae640ca0d0 2024-09-17T03:33:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.827+0000 7fae5a000640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074be0 0x7fae440770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:33:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.827+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fae5012dd60 con 0x7fae640ca0d0 2024-09-17T03:33:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.827+0000 7fae61e00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074be0 0x7fae440770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:33:59.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.827+0000 7fae61e00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074be0 0x7fae440770f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fae54000c10 tx=0x7fae5400ab70 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:33:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:33:59.831+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fae500c0060 con 0x7fae640ca0d0 2024-09-17T03:34:00.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.027+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fae6406b540 con 0x7fae44074be0 2024-09-17T03:34:00.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.027+0000 7fae5a000640 1 -- 172.21.15.46:0/3342249420 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fae54013450 con 0x7fae44074be0 2024-09-17T03:34:00.030 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:34:00.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:00.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:34:00.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:00.031 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074be0 msgr2=0x7fae440770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074be0 0x7fae440770f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fae54000c10 tx=0x7fae5400ab70 comp rx=0 tx=0).stop 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 msgr2=0x7fae6415ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 0x7fae6415ef30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fae50006f90 tx=0x7fae500011d0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 shutdown_connections 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fae44074be0 0x7fae440770f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae6413f510 0x7fae6415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae640ca0d0 0x7fae6415ef30 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 --2- 172.21.15.46:0/3342249420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fae48004810 0x7fae6415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 >> 172.21.15.46:0/3342249420 conn(0x7fae640b2040 msgr2=0x7fae6415bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 shutdown_connections 2024-09-17T03:34:00.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.031+0000 7fae68c00640 1 -- 172.21.15.46:0/3342249420 wait complete. 2024-09-17T03:34:00.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:34:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 -- 172.21.15.46:0/1776381447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 msgr2=0x7fa2d80b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 --2- 172.21.15.46:0/1776381447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80b21d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009870 tx=0x7fa2c001c760 comp rx=0 tx=0).stop 2024-09-17T03:34:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 -- 172.21.15.46:0/1776381447 shutdown_connections 2024-09-17T03:34:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 --2- 172.21.15.46:0/1776381447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80b21d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 -- 172.21.15.46:0/1776381447 >> 172.21.15.46:0/1776381447 conn(0x7fa2d806cdf0 msgr2=0x7fa2d806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 -- 172.21.15.46:0/1776381447 shutdown_connections 2024-09-17T03:34:00.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.211+0000 7fa2dd200640 1 -- 172.21.15.46:0/1776381447 wait complete. 2024-09-17T03:34:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 Processor -- start 2024-09-17T03:34:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 -- start start 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2d80b3130 0x7fa2d80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 0x7fa2d80d0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8169ba0 con 0x7fa2d8155a00 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8169a20 con 0x7fa2d80b1dc0 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2dd200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d8169d20 con 0x7fa2d80b3130 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 0x7fa2d80d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2cea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2d80b3130 0x7fa2d80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.215+0000 7fa2d7400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 0x7fa2d80d0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48946/0 (socket says 172.21.15.46:48946) 2024-09-17T03:34:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 -- 172.21.15.46:0/2293689514 learned_addr learned my addr 172.21.15.46:0/2293689514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:00.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2cea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2d80b3130 0x7fa2d80d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:57530/0 (socket says 172.21.15.46:57530) 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 -- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2d80b3130 msgr2=0x7fa2d80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2d80b3130 0x7fa2d80d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 -- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 msgr2=0x7fa2d80ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 -- 172.21.15.46:0/2293689514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d80764d0 con 0x7fa2d8155a00 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7400640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2d7e00640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 0x7fa2d80d0620 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000cad0 tx=0x7fa2d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0016020 con 0x7fa2d8155a00 2024-09-17T03:34:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d000ab40 con 0x7fa2d8155a00 2024-09-17T03:34:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d8150930 con 0x7fa2d8155a00 2024-09-17T03:34:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000ce10 con 0x7fa2d8155a00 2024-09-17T03:34:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81fc780 con 0x7fa2d8155a00 2024-09-17T03:34:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d80afba0 con 0x7fa2d8155a00 2024-09-17T03:34:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.219+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7fa2d0019020 con 0x7fa2d8155a00 2024-09-17T03:34:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.223+0000 7fa2cfe00640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa2b4074be0 0x7fa2b40770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.223+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa2d0092f40 con 0x7fa2d8155a00 2024-09-17T03:34:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.223+0000 7fa2d7400640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa2b4074be0 0x7fa2b40770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.223+0000 7fa2d7400640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa2b4074be0 0x7fa2b40770f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c10 tx=0x7fa2c8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.227+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa2d0020030 con 0x7fa2d8155a00 2024-09-17T03:34:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.427+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2d806b540 con 0x7fa2b4074be0 2024-09-17T03:34:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.431+0000 7fa2cfe00640 1 -- 172.21.15.46:0/2293689514 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fa2c8001ce0 con 0x7fa2b4074be0 2024-09-17T03:34:00.433 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1clone1" 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:00.434 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa2b4074be0 msgr2=0x7fa2b40770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa2b4074be0 0x7fa2b40770f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c10 tx=0x7fa2c8009040 comp rx=0 tx=0).stop 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 msgr2=0x7fa2d80d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 0x7fa2d80d0620 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000cad0 tx=0x7fa2d000cbc0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 shutdown_connections 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa2b4074be0 0x7fa2b40770f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2d8155a00 0x7fa2d80d0620 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa2d80b3130 0x7fa2d80d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 --2- 172.21.15.46:0/2293689514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d80b1dc0 0x7fa2d80ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 >> 172.21.15.46:0/2293689514 conn(0x7fa2d806cdf0 msgr2=0x7fa2d815cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 shutdown_connections 2024-09-17T03:34:00.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.435+0000 7fa2dd200640 1 -- 172.21.15.46:0/2293689514 wait complete. 2024-09-17T03:34:00.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:34:00.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:34:00.577 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:34:00.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 -- 172.21.15.46:0/1773315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976415aa50 msgr2=0x7f97640cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 --2- 172.21.15.46:0/1773315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976415aa50 0x7f97640cd260 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f975800a950 tx=0x7f975801db70 comp rx=0 tx=0).stop 2024-09-17T03:34:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 -- 172.21.15.46:0/1773315247 shutdown_connections 2024-09-17T03:34:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 --2- 172.21.15.46:0/1773315247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976415aa50 0x7f97640cd260 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 -- 172.21.15.46:0/1773315247 >> 172.21.15.46:0/1773315247 conn(0x7f97640b2040 msgr2=0x7f97640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 -- 172.21.15.46:0/1773315247 shutdown_connections 2024-09-17T03:34:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.619+0000 7f976b600640 1 -- 172.21.15.46:0/1773315247 wait complete. 2024-09-17T03:34:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 Processor -- start 2024-09-17T03:34:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 -- start start 2024-09-17T03:34:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 0x7f97641fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97640cdbb0 0x7f9764150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f976415aa50 0x7f976415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97641690a0 con 0x7f97640cd7a0 2024-09-17T03:34:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97640b5f50 con 0x7f97640cdbb0 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9764169220 con 0x7f976415aa50 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 0x7f97641fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 0x7f97641fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48954/0 (socket says 172.21.15.46:48954) 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 -- 172.21.15.46:0/1637138526 learned_addr learned my addr 172.21.15.46:0/1637138526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9763e00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97640cdbb0 0x7f9764150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9769600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f976415aa50 0x7f976415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 -- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f976415aa50 msgr2=0x7f976415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f976415aa50 0x7f976415c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 -- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97640cdbb0 msgr2=0x7f9764150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97640cdbb0 0x7f9764150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 -- 172.21.15.46:0/1637138526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9764150ff0 con 0x7f97640cd7a0 2024-09-17T03:34:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9763e00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97640cdbb0 0x7f9764150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:34:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9768c00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 0x7f97641fccf0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f975800a8a0 tx=0x7f9758005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580bd070 con 0x7f97640cd7a0 2024-09-17T03:34:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97580053e0 con 0x7f97640cd7a0 2024-09-17T03:34:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.623+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97640cc0b0 con 0x7f97640cd7a0 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580b3040 con 0x7f97640cd7a0 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641f9350 con 0x7f97640cd7a0 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764155210 con 0x7f97640cd7a0 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f97580046d0 con 0x7f97640cd7a0 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f9760c00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9744074bd0 0x7f97440770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f975812d390 con 0x7f97640cd7a0 2024-09-17T03:34:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.627+0000 7f9763e00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9744074bd0 0x7f97440770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.631+0000 7f9763e00640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9744074bd0 0x7f97440770e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97500046c0 tx=0x7f9750007cd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.631+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9764150ff0 con 0x7f97640cd7a0 2024-09-17T03:34:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.835+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f976406b540 con 0x7f9744074bd0 2024-09-17T03:34:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f9760c00640 1 -- 172.21.15.46:0/1637138526 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9750001cb0 con 0x7f9744074bd0 2024-09-17T03:34:00.849 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:00.849 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:34:00.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:00.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:34:00.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:00.849 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9744074bd0 msgr2=0x7f97440770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9744074bd0 0x7f97440770e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97500046c0 tx=0x7f9750007cd0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 msgr2=0x7f97641fccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 0x7f97641fccf0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f975800a8a0 tx=0x7f9758005db0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 shutdown_connections 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9744074bd0 0x7f97440770e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f976415aa50 0x7f976415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97640cdbb0 0x7f9764150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 --2- 172.21.15.46:0/1637138526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97640cd7a0 0x7f97641fccf0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 >> 172.21.15.46:0/1637138526 conn(0x7f97640b2040 msgr2=0x7f976406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.847+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 shutdown_connections 2024-09-17T03:34:00.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:00.851+0000 7f976b600640 1 -- 172.21.15.46:0/1637138526 wait complete. 2024-09-17T03:34:00.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 -- 172.21.15.46:0/3537372552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe548160060 msgr2=0x7fe548074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 --2- 172.21.15.46:0/3537372552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe548160060 0x7fe548074240 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe530009840 tx=0x7fe53001c790 comp rx=0 tx=0).stop 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 -- 172.21.15.46:0/3537372552 shutdown_connections 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 --2- 172.21.15.46:0/3537372552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe548160060 0x7fe548074240 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 --2- 172.21.15.46:0/3537372552 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe5481536f0 0x7fe54815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 -- 172.21.15.46:0/3537372552 >> 172.21.15.46:0/3537372552 conn(0x7fe5480b4560 msgr2=0x7fe5480d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 -- 172.21.15.46:0/3537372552 shutdown_connections 2024-09-17T03:34:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 -- 172.21.15.46:0/3537372552 wait complete. 2024-09-17T03:34:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 Processor -- start 2024-09-17T03:34:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.079+0000 7fe54e000640 1 -- start start 2024-09-17T03:34:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 0x7fe5480d2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe548160060 0x7fe54814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe548160470 0x7fe5481423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe54816a440 con 0x7fe5481536f0 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe54816a2c0 con 0x7fe548160060 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe54816a5c0 con 0x7fe548160470 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 0x7fe5480d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 0x7fe5480d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48964/0 (socket says 172.21.15.46:48964) 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 -- 172.21.15.46:0/3315171658 learned_addr learned my addr 172.21.15.46:0/3315171658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe53fe00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe548160060 0x7fe54814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54d600640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe548160470 0x7fe5481423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 -- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe548160470 msgr2=0x7fe5481423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe548160470 0x7fe5481423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 -- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe548160060 msgr2=0x7fe54814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe548160060 0x7fe54814a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 -- 172.21.15.46:0/3315171658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54814dcb0 con 0x7fe5481536f0 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe53fe00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe548160060 0x7fe54814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54cc00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 0x7fe5480d2210 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe53800e7a0 tx=0x7fe53800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53800ccc0 con 0x7fe5481536f0 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe538010430 con 0x7fe5481536f0 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5481499d0 con 0x7fe5481536f0 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538010860 con 0x7fe5481536f0 2024-09-17T03:34:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe548149f90 con 0x7fe5481536f0 2024-09-17T03:34:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.083+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54815d480 con 0x7fe5481536f0 2024-09-17T03:34:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.087+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7fe538010a00 con 0x7fe5481536f0 2024-09-17T03:34:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.087+0000 7fe545600640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe524074c10 0x7fe524077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.087+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe538094310 con 0x7fe5481536f0 2024-09-17T03:34:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.087+0000 7fe53fe00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe524074c10 0x7fe524077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.087+0000 7fe53fe00640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe524074c10 0x7fe524077120 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe5300ae000 tx=0x7fe53001afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.091+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe538060e10 con 0x7fe5481536f0 2024-09-17T03:34:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.295+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe54806b540 con 0x7fe524074c10 2024-09-17T03:34:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.323+0000 7fe545600640 1 -- 172.21.15.46:0/3315171658 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5300051c0 con 0x7fe524074c10 2024-09-17T03:34:01.326 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe524074c10 msgr2=0x7fe524077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe524074c10 0x7fe524077120 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe5300ae000 tx=0x7fe53001afd0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 msgr2=0x7fe5480d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 0x7fe5480d2210 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe53800e7a0 tx=0x7fe53800ec60 comp rx=0 tx=0).stop 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 shutdown_connections 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe524074c10 0x7fe524077120 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe548160470 0x7fe5481423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe548160060 0x7fe54814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 --2- 172.21.15.46:0/3315171658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5481536f0 0x7fe5480d2210 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 >> 172.21.15.46:0/3315171658 conn(0x7fe5480b4560 msgr2=0x7fe54815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:01.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 shutdown_connections 2024-09-17T03:34:01.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.327+0000 7fe54e000640 1 -- 172.21.15.46:0/3315171658 wait complete. 2024-09-17T03:34:01.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 -- 172.21.15.46:0/1199047666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 msgr2=0x7ff59815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 --2- 172.21.15.46:0/1199047666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff59815bb60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff598072e30 tx=0x7ff58801c800 comp rx=0 tx=0).stop 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 -- 172.21.15.46:0/1199047666 shutdown_connections 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 --2- 172.21.15.46:0/1199047666 >> v2:172.21.15.99:3300/0 conn(0x7ff5980b4de0 0x7ff5980b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 --2- 172.21.15.46:0/1199047666 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5981527b0 0x7ff598074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 --2- 172.21.15.46:0/1199047666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff59815bb60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 -- 172.21.15.46:0/1199047666 >> 172.21.15.46:0/1199047666 conn(0x7ff5980b2040 msgr2=0x7ff5980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 -- 172.21.15.46:0/1199047666 shutdown_connections 2024-09-17T03:34:01.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 -- 172.21.15.46:0/1199047666 wait complete. 2024-09-17T03:34:01.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 Processor -- start 2024-09-17T03:34:01.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.511+0000 7ff59ec00640 1 -- start start 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5980b4de0 0x7ff5981449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff5980ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5981527b0 0x7ff5980ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff59815f660 con 0x7ff5980b5c70 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff59815f4e0 con 0x7ff5981527b0 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff59815f7e0 con 0x7ff5980b4de0 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff5980ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff5980ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48984/0 (socket says 172.21.15.46:48984) 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 -- 172.21.15.46:0/3711531692 learned_addr learned my addr 172.21.15.46:0/3711531692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59cc00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5981527b0 0x7ff5980ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff597e00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5980b4de0 0x7ff5981449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 -- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5980b4de0 msgr2=0x7ff5981449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5980b4de0 0x7ff5981449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 -- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5981527b0 msgr2=0x7ff5980ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5981527b0 0x7ff5980ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 -- 172.21.15.46:0/3711531692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff598152190 con 0x7ff5980b5c70 2024-09-17T03:34:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff58fe00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff5980ac190 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff58000d6a0 tx=0x7ff58000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580017070 con 0x7ff5980b5c70 2024-09-17T03:34:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff580005310 con 0x7ff5980b5c70 2024-09-17T03:34:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5980aab60 con 0x7ff5980b5c70 2024-09-17T03:34:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580009820 con 0x7ff5980b5c70 2024-09-17T03:34:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5980ab120 con 0x7ff5980b5c70 2024-09-17T03:34:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.515+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5980aa8d0 con 0x7ff5980b5c70 2024-09-17T03:34:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.519+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7ff5800099c0 con 0x7ff5980b5c70 2024-09-17T03:34:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.519+0000 7ff594c00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff574074be0 0x7ff5740770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.519+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7ff580093350 con 0x7ff5980b5c70 2024-09-17T03:34:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.519+0000 7ff597e00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff574074be0 0x7ff5740770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.519+0000 7ff597e00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff574074be0 0x7ff5740770f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff5880af000 tx=0x7ff58801b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.523+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff58005f990 con 0x7ff5980b5c70 2024-09-17T03:34:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.723+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff59806b540 con 0x7ff574074be0 2024-09-17T03:34:01.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff594c00640 1 -- 172.21.15.46:0/3711531692 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff5880b3040 con 0x7ff574074be0 2024-09-17T03:34:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff574074be0 msgr2=0x7ff5740770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff574074be0 0x7ff5740770f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff5880af000 tx=0x7ff58801b040 comp rx=0 tx=0).stop 2024-09-17T03:34:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 msgr2=0x7ff5980ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff5980ac190 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff58000d6a0 tx=0x7ff58000db60 comp rx=0 tx=0).stop 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 shutdown_connections 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff574074be0 0x7ff5740770f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5981527b0 0x7ff5980ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5980b5c70 0x7ff5980ac190 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 --2- 172.21.15.46:0/3711531692 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5980b4de0 0x7ff5981449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 >> 172.21.15.46:0/3711531692 conn(0x7ff5980b2040 msgr2=0x7ff59815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 shutdown_connections 2024-09-17T03:34:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.743+0000 7ff59ec00640 1 -- 172.21.15.46:0/3711531692 wait complete. 2024-09-17T03:34:01.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:01.759+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:01.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:01.759+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:01.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:01.759+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:01.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:01.759+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:01.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:01.759+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:01.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T03:34:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f5094c00640 1 -- 172.21.15.46:0/323928003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5074004b00 msgr2=0x7f5074004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f5094c00640 1 --2- 172.21.15.46:0/323928003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5074004b00 0x7f5074004f30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f508000c2e0 tx=0x7f508001cf80 comp rx=0 tx=0).stop 2024-09-17T03:34:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f508e800640 1 -- 172.21.15.46:0/323928003 reap_dead start 2024-09-17T03:34:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f5094c00640 1 -- 172.21.15.46:0/323928003 shutdown_connections 2024-09-17T03:34:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f5094c00640 1 -- 172.21.15.46:0/323928003 >> 172.21.15.46:0/323928003 conn(0x7f50900b2040 msgr2=0x7f50900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f5094c00640 1 -- 172.21.15.46:0/323928003 shutdown_connections 2024-09-17T03:34:01.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.939+0000 7f5094c00640 1 -- 172.21.15.46:0/323928003 wait complete. 2024-09-17T03:34:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 Processor -- start 2024-09-17T03:34:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 -- start start 2024-09-17T03:34:01.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074004b00 0x7f50900cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 0x7f5090155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5090155ed0 0x7f50901562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f509015e440 con 0x7f50900ce3c0 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f509015e2c0 con 0x7f5074004b00 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.943+0000 7f5094c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5090166a50 con 0x7f5090155ed0 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 0x7f5090155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f508de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074004b00 0x7f50900cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 0x7f5090155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49016/0 (socket says 172.21.15.46:49016) 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 -- 172.21.15.46:0/685484026 learned_addr learned my addr 172.21.15.46:0/685484026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f508e800640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5090155ed0 0x7f50901562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 -- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5090155ed0 msgr2=0x7f50901562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5090155ed0 0x7f50901562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 -- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074004b00 msgr2=0x7f50900cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074004b00 0x7f50900cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 -- 172.21.15.46:0/685484026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50901573f0 con 0x7f50900ce3c0 2024-09-17T03:34:01.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5087e00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 0x7f5090155990 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5078006660 tx=0x7f5078001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078011640 con 0x7f50900ce3c0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5078016020 con 0x7f50900ce3c0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50900cd0c0 con 0x7f50900ce3c0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078011a80 con 0x7f50900ce3c0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50900cd680 con 0x7f50900ce3c0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5090157160 con 0x7f50900ce3c0 2024-09-17T03:34:01.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.947+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f5078011c20 con 0x7f50900ce3c0 2024-09-17T03:34:01.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.951+0000 7f5086000640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f506c074be0 0x7f506c0770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:01.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.951+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f50780942c0 con 0x7f50900ce3c0 2024-09-17T03:34:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.951+0000 7f508de00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f506c074be0 0x7f506c0770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:01.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.951+0000 7f508de00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f506c074be0 0x7f506c0770f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f50800ab000 tx=0x7f5080019dc0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:01.955+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5078020020 con 0x7f50900ce3c0 2024-09-17T03:34:02.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.159+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f509006b540 con 0x7f506c074be0 2024-09-17T03:34:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.175+0000 7f5086000640 1 -- 172.21.15.46:0/685484026 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50800a5a80 con 0x7f506c074be0 2024-09-17T03:34:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f506c074be0 msgr2=0x7f506c0770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f506c074be0 0x7f506c0770f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f50800ab000 tx=0x7f5080019dc0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 msgr2=0x7f5090155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 0x7f5090155990 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5078006660 tx=0x7f5078001720 comp rx=0 tx=0).stop 2024-09-17T03:34:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 shutdown_connections 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f506c074be0 0x7f506c0770f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5090155ed0 0x7f50901562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50900ce3c0 0x7f5090155990 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 --2- 172.21.15.46:0/685484026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074004b00 0x7f50900cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 >> 172.21.15.46:0/685484026 conn(0x7f50900b2040 msgr2=0x7f509013dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 shutdown_connections 2024-09-17T03:34:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.179+0000 7f5094c00640 1 -- 172.21.15.46:0/685484026 wait complete. 2024-09-17T03:34:02.193 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:02.191+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:02.194 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:02.191+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:02.194 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:02.191+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:02.194 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:02.191+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:02.194 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:02.191+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:02.209 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:34:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:34:02.498 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:34:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:34:02.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:34:02.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 -- 172.21.15.46:0/1041786534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b54f0 msgr2=0x7fac1815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 --2- 172.21.15.46:0/1041786534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b54f0 0x7fac1815bb60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fac10009bc0 tx=0x7fac1001cae0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 -- 172.21.15.46:0/1041786534 shutdown_connections 2024-09-17T03:34:02.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 --2- 172.21.15.46:0/1041786534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b54f0 0x7fac1815bb60 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 -- 172.21.15.46:0/1041786534 >> 172.21.15.46:0/1041786534 conn(0x7fac180b2040 msgr2=0x7fac180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 -- 172.21.15.46:0/1041786534 shutdown_connections 2024-09-17T03:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 -- 172.21.15.46:0/1041786534 wait complete. 2024-09-17T03:34:02.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 Processor -- start 2024-09-17T03:34:02.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.667+0000 7fac1ea00640 1 -- start start 2024-09-17T03:34:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 0x7fac1814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac181527b0 0x7fac180ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac181507f0 0x7fac180aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac18169bd0 con 0x7fac180b5c70 2024-09-17T03:34:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac18169a50 con 0x7fac181527b0 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fac18169d50 con 0x7fac181507f0 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 0x7fac1814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 0x7fac1814ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38466/0 (socket says 172.21.15.46:38466) 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 -- 172.21.15.46:0/269431279 learned_addr learned my addr 172.21.15.46:0/269431279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ca00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac181507f0 0x7fac180aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac0fe00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac181527b0 0x7fac180ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 -- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac181507f0 msgr2=0x7fac180aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac181507f0 0x7fac180aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 -- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac181527b0 msgr2=0x7fac180ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac181527b0 0x7fac180ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 -- 172.21.15.46:0/269431279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac18152c60 con 0x7fac180b5c70 2024-09-17T03:34:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac17e00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 0x7fac1814ff60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fac08001ec0 tx=0x7fac0800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:02.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08017070 con 0x7fac180b5c70 2024-09-17T03:34:02.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac08003600 con 0x7fac180b5c70 2024-09-17T03:34:02.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac18152f40 con 0x7fac180b5c70 2024-09-17T03:34:02.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08003a50 con 0x7fac180b5c70 2024-09-17T03:34:02.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac18153500 con 0x7fac180b5c70 2024-09-17T03:34:02.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.671+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1815bb60 con 0x7fac180b5c70 2024-09-17T03:34:02.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.675+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7fac0801a070 con 0x7fac180b5c70 2024-09-17T03:34:02.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.675+0000 7fac14c00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fabf4074be0 0x7fabf40770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:02.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.675+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fac08092480 con 0x7fac180b5c70 2024-09-17T03:34:02.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.675+0000 7fac0fe00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fabf4074be0 0x7fabf40770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:02.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.675+0000 7fac0fe00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fabf4074be0 0x7fabf40770f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fac180cc900 tx=0x7fac00008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:02.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.679+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac0805ef00 con 0x7fac180b5c70 2024-09-17T03:34:02.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fac181670f0 con 0x7fac180b5c70 2024-09-17T03:34:02.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac14c00640 1 -- 172.21.15.46:0/269431279 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v50)=0 v50) ==== 131+0+0 (secure 0 0 0) 0x7fac0805e800 con 0x7fac180b5c70 2024-09-17T03:34:02.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fabf4074be0 msgr2=0x7fabf40770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fabf4074be0 0x7fabf40770f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fac180cc900 tx=0x7fac00008040 comp rx=0 tx=0).stop 2024-09-17T03:34:02.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 msgr2=0x7fac1814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:02.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 0x7fac1814ff60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fac08001ec0 tx=0x7fac0800c890 comp rx=0 tx=0).stop 2024-09-17T03:34:02.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 shutdown_connections 2024-09-17T03:34:02.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fabf4074be0 0x7fabf40770f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fac181507f0 0x7fac180aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac181527b0 0x7fac180ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 --2- 172.21.15.46:0/269431279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac180b5c70 0x7fac1814ff60 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:02.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 >> 172.21.15.46:0/269431279 conn(0x7fac180b2040 msgr2=0x7fac180b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:02.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 shutdown_connections 2024-09-17T03:34:02.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:02.855+0000 7fac1ea00640 1 -- 172.21.15.46:0/269431279 wait complete. 2024-09-17T03:34:02.878 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:34:02.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:34:02.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:34:03.080 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:34:03.080 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:34:03.080 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:03.080 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:03.089 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:03.090 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:03.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:34:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 -- 172.21.15.46:0/3622404853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 msgr2=0x7f23f406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 --2- 172.21.15.46:0/3622404853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f406d200 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009c80 tx=0x7f23f001cd30 comp rx=0 tx=0).stop 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 -- 172.21.15.46:0/3622404853 shutdown_connections 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 --2- 172.21.15.46:0/3622404853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f406d200 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 --2- 172.21.15.46:0/3622404853 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23f4159130 0x7f23f415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 -- 172.21.15.46:0/3622404853 >> 172.21.15.46:0/3622404853 conn(0x7f23f40b35f0 msgr2=0x7f23f40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 -- 172.21.15.46:0/3622404853 shutdown_connections 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 -- 172.21.15.46:0/3622404853 wait complete. 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 Processor -- start 2024-09-17T03:34:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.351+0000 7f23fc800640 1 -- start start 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23f406cdf0 0x7f23f4163520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23f4159130 0x7f23f415e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4168e90 con 0x7f23f40703a0 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4168d10 con 0x7f23f4159130 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f23f4169010 con 0x7f23f406cdf0 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38488/0 (socket says 172.21.15.46:38488) 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 -- 172.21.15.46:0/2133680857 learned_addr learned my addr 172.21.15.46:0/2133680857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23f406cdf0 0x7f23f4163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fa800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23f4159130 0x7f23f415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 -- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23f406cdf0 msgr2=0x7f23f4163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23f406cdf0 0x7f23f4163520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 -- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23f4159130 msgr2=0x7f23f415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23f4159130 0x7f23f415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 -- 172.21.15.46:0/2133680857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f41fc4b0 con 0x7f23f40703a0 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23f9400640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f4151a40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c370 tx=0x7f23e400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4017070 con 0x7f23f40703a0 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23e4003600 con 0x7f23f40703a0 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4003a30 con 0x7f23f40703a0 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f415b900 con 0x7f23f40703a0 2024-09-17T03:34:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f40cefc0 con 0x7f23f40703a0 2024-09-17T03:34:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.355+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f40cea50 con 0x7f23f40703a0 2024-09-17T03:34:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.359+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f23e401a070 con 0x7f23f40703a0 2024-09-17T03:34:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.359+0000 7f23ee000640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23d8074be0 0x7f23d80770f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.359+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f23e40921e0 con 0x7f23f40703a0 2024-09-17T03:34:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.359+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23d8074be0 0x7f23d80770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.359+0000 7f23f9e00640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23d8074be0 0x7f23d80770f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009b40 tx=0x7f23e80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.363+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23f415b900 con 0x7f23f40703a0 2024-09-17T03:34:03.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.547+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23f406d200 con 0x7f23f40703a0 2024-09-17T03:34:03.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.547+0000 7f23ee000640 1 -- 172.21.15.46:0/2133680857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f23f415c0d0 con 0x7f23f40703a0 2024-09-17T03:34:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:34:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23d8074be0 msgr2=0x7f23d80770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23d8074be0 0x7f23d80770f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009b40 tx=0x7f23e80091e0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 msgr2=0x7f23f4151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f4151a40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c370 tx=0x7f23e400c830 comp rx=0 tx=0).stop 2024-09-17T03:34:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 shutdown_connections 2024-09-17T03:34:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f23d8074be0 0x7f23d80770f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23f4159130 0x7f23f415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f40703a0 0x7f23f4151a40 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 --2- 172.21.15.46:0/2133680857 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23f406cdf0 0x7f23f4163520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 >> 172.21.15.46:0/2133680857 conn(0x7f23f40b35f0 msgr2=0x7f23f40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 shutdown_connections 2024-09-17T03:34:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:03.551+0000 7f23fc800640 1 -- 172.21.15.46:0/2133680857 wait complete. 2024-09-17T03:34:03.574 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:34:03.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:03.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:34:03.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:34:03.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:03.650 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:34:03.672 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:34:03.673 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:34:03.673 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:03.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:34:03.684 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:34:03.684 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:03.684 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:03.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:34:03.730 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:34:03.730 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:34:03.730 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:03.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:34:03.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 -- 172.21.15.46:0/756170387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e8c0 msgr2=0x7f45c815ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 --2- 172.21.15.46:0/756170387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e8c0 0x7f45c815ecd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f45c815cd60 tx=0x7f45b801c800 comp rx=0 tx=0).stop 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 -- 172.21.15.46:0/756170387 shutdown_connections 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 --2- 172.21.15.46:0/756170387 >> v2:172.21.15.99:3300/0 conn(0x7f45c815e290 0x7f45c815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 --2- 172.21.15.46:0/756170387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e8c0 0x7f45c815ecd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 -- 172.21.15.46:0/756170387 >> 172.21.15.46:0/756170387 conn(0x7f45c80b2040 msgr2=0x7f45c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 -- 172.21.15.46:0/756170387 shutdown_connections 2024-09-17T03:34:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 -- 172.21.15.46:0/756170387 wait complete. 2024-09-17T03:34:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.055+0000 7f45cdc00640 1 Processor -- start 2024-09-17T03:34:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- start start 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45c815b2d0 0x7f45c81f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 0x7f45c8143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c815e8c0 0x7f45c8143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8168230 con 0x7f45c815e290 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81680b0 con 0x7f45c815e8c0 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f45c81683b0 con 0x7f45c815b2d0 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 0x7f45c8143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 0x7f45c8143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38510/0 (socket says 172.21.15.46:38510) 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 -- 172.21.15.46:0/925661435 learned_addr learned my addr 172.21.15.46:0/925661435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c7400640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c815e8c0 0x7f45c8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6a00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45c815b2d0 0x7f45c81f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 -- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45c815b2d0 msgr2=0x7f45c81f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45c815b2d0 0x7f45c81f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 -- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c815e8c0 msgr2=0x7f45c8143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c815e8c0 0x7f45c8143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 -- 172.21.15.46:0/925661435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c81f9e40 con 0x7f45c815e290 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45c6000640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 0x7f45c8143150 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f45b400c430 tx=0x7f45b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4011790 con 0x7f45c815e290 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45b4017070 con 0x7f45c815e290 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c81420f0 con 0x7f45c815e290 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4011cc0 con 0x7f45c815e290 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c8142670 con 0x7f45c815e290 2024-09-17T03:34:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.059+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c815fd40 con 0x7f45c815e290 2024-09-17T03:34:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.063+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) ==== 99580+0+0 (secure 0 0 0) 0x7f45b401c460 con 0x7f45c815e290 2024-09-17T03:34:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.063+0000 7f45bea00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f45a8074c10 0x7f45a8077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.063+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f45b4092c80 con 0x7f45c815e290 2024-09-17T03:34:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.063+0000 7f45c6a00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f45a8074c10 0x7f45a8077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.063+0000 7f45c6a00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f45a8074c10 0x7f45a8077120 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f45b80af000 tx=0x7f45b801b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:04.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.067+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f45b4021020 con 0x7f45c815e290 2024-09-17T03:34:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.275+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45c815ecd0 con 0x7f45c815e290 2024-09-17T03:34:04.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.275+0000 7f45bea00640 1 -- 172.21.15.46:0/925661435 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5238 (secure 0 0 0) 0x7f45b405f700 con 0x7f45c815e290 2024-09-17T03:34:04.279 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:04.279 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"btime":"2024-09-17T03:33:16:602820+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:33:02.961494+0000","modified":"2024-09-17T03:33:05.989948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":247692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_247692":{"gid":247692,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4088974288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4088974288},{"type":"v1","addr":"172.21.15.46:6836","nonce":4088974288}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":247692,"qdb_cluster":[247692]},"id":7}]} 2024-09-17T03:34:04.279 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f45a8074c10 msgr2=0x7f45a8077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f45a8074c10 0x7f45a8077120 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f45b80af000 tx=0x7f45b801b040 comp rx=0 tx=0).stop 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 msgr2=0x7f45c8143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 0x7f45c8143150 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f45b400c430 tx=0x7f45b400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 shutdown_connections 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f45a8074c10 0x7f45a8077120 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c815e8c0 0x7f45c8143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c815e290 0x7f45c8143150 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 --2- 172.21.15.46:0/925661435 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45c815b2d0 0x7f45c81f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 >> 172.21.15.46:0/925661435 conn(0x7f45c80b2040 msgr2=0x7f45c815a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:04.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 shutdown_connections 2024-09-17T03:34:04.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.279+0000 7f45cdc00640 1 -- 172.21.15.46:0/925661435 wait complete. 2024-09-17T03:34:04.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 -- 172.21.15.46:0/2870005467 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 msgr2=0x7fc840068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 --2- 172.21.15.46:0/2870005467 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 0x7fc840068df0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc830009bf0 tx=0x7fc83001cbb0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 -- 172.21.15.46:0/2870005467 shutdown_connections 2024-09-17T03:34:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 --2- 172.21.15.46:0/2870005467 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 0x7fc840068df0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 -- 172.21.15.46:0/2870005467 >> 172.21.15.46:0/2870005467 conn(0x7fc8400b35f0 msgr2=0x7fc8400b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 -- 172.21.15.46:0/2870005467 shutdown_connections 2024-09-17T03:34:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.467+0000 7fc847000640 1 -- 172.21.15.46:0/2870005467 wait complete. 2024-09-17T03:34:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 Processor -- start 2024-09-17T03:34:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 -- start start 2024-09-17T03:34:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc840069330 0x7fc840154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 0x7fc840166c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 0x7fc840157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc84016ae90 con 0x7fc840160af0 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc84016ad10 con 0x7fc840069330 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc847000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc84016b010 con 0x7fc84015c7b0 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc845000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 0x7fc840157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc845000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 0x7fc840157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38512/0 (socket says 172.21.15.46:38512) 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.471+0000 7fc845000640 1 -- 172.21.15.46:0/654562782 learned_addr learned my addr 172.21.15.46:0/654562782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc83fe00640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc840069330 0x7fc840154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc83f400640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 0x7fc840166c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc845000640 1 -- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 msgr2=0x7fc840166c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc845000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 0x7fc840166c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc845000640 1 -- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc840069330 msgr2=0x7fc840154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc845000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc840069330 0x7fc840154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc845000640 1 -- 172.21.15.46:0/654562782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc840155830 con 0x7fc840160af0 2024-09-17T03:34:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc83fe00640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc840069330 0x7fc840154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc845000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 0x7fc840157e90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c3f0 tx=0x7fc83800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838017070 con 0x7fc840160af0 2024-09-17T03:34:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8380053e0 con 0x7fc840160af0 2024-09-17T03:34:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838003860 con 0x7fc840160af0 2024-09-17T03:34:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8401fc1a0 con 0x7fc840160af0 2024-09-17T03:34:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8401fc740 con 0x7fc840160af0 2024-09-17T03:34:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc840068df0 con 0x7fc840160af0 2024-09-17T03:34:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.475+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fc83801a070 con 0x7fc840160af0 2024-09-17T03:34:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.479+0000 7fc837e00640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc820074d00 0x7fc820077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.479+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7fc838092410 con 0x7fc840160af0 2024-09-17T03:34:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.479+0000 7fc83fe00640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc820074d00 0x7fc820077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.479+0000 7fc83fe00640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc820074d00 0x7fc820077210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc8300af000 tx=0x7fc83001b3f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.483+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc840069740 con 0x7fc840160af0 2024-09-17T03:34:04.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.783+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8400b31b0 con 0x7fc840160af0 2024-09-17T03:34:04.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc837e00640 1 -- 172.21.15.46:0/654562782 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5238 (secure 0 0 0) 0x7fc83805ef30 con 0x7fc840160af0 2024-09-17T03:34:04.787 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:04.787 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"btime":"2024-09-17T03:33:16:602820+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:33:02.961494+0000","modified":"2024-09-17T03:33:05.989948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":247692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_247692":{"gid":247692,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4088974288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4088974288},{"type":"v1","addr":"172.21.15.46:6836","nonce":4088974288}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":247692,"qdb_cluster":[247692]},"id":7}]} 2024-09-17T03:34:04.788 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc820074d00 msgr2=0x7fc820077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc820074d00 0x7fc820077210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc8300af000 tx=0x7fc83001b3f0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 msgr2=0x7fc840157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 0x7fc840157e90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c3f0 tx=0x7fc83800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 shutdown_connections 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc820074d00 0x7fc820077210 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc840160af0 0x7fc840157e90 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc84015c7b0 0x7fc840166c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 --2- 172.21.15.46:0/654562782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc840069330 0x7fc840154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 >> 172.21.15.46:0/654562782 conn(0x7fc8400b35f0 msgr2=0x7fc840073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:04.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 shutdown_connections 2024-09-17T03:34:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:04.787+0000 7fc847000640 1 -- 172.21.15.46:0/654562782 wait complete. 2024-09-17T03:34:04.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:34:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.067+0000 7f4df9c00640 1 -- 172.21.15.46:0/4282985088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df406cdf0 msgr2=0x7f4df406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.067+0000 7f4df9c00640 1 --2- 172.21.15.46:0/4282985088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df406cdf0 0x7f4df406d200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009870 tx=0x7f4ddc01c6f0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.067+0000 7f4df9c00640 1 -- 172.21.15.46:0/4282985088 shutdown_connections 2024-09-17T03:34:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.067+0000 7f4df9c00640 1 --2- 172.21.15.46:0/4282985088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df406cdf0 0x7f4df406d200 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.067+0000 7f4df9c00640 1 -- 172.21.15.46:0/4282985088 >> 172.21.15.46:0/4282985088 conn(0x7f4df40b35f0 msgr2=0x7f4df40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 -- 172.21.15.46:0/4282985088 shutdown_connections 2024-09-17T03:34:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 -- 172.21.15.46:0/4282985088 wait complete. 2024-09-17T03:34:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 Processor -- start 2024-09-17T03:34:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 -- start start 2024-09-17T03:34:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 0x7f4df41f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df406fc90 0x7f4df41528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df4159130 0x7f4df40ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4df416a580 con 0x7f4df406cdf0 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4df416a400 con 0x7f4df406fc90 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.071+0000 7f4df9c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4df416a700 con 0x7f4df4159130 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 0x7f4df41f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 0x7f4df41f9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38518/0 (socket says 172.21.15.46:38518) 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 -- 172.21.15.46:0/2234849539 learned_addr learned my addr 172.21.15.46:0/2234849539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4debe00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df406fc90 0x7f4df41528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df3400640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df4159130 0x7f4df40ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 -- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df4159130 msgr2=0x7f4df40ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df4159130 0x7f4df40ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 -- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df406fc90 msgr2=0x7f4df41528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df406fc90 0x7f4df41528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 -- 172.21.15.46:0/2234849539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df4153790 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df2a00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 0x7f4df41f9c90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c6e0 tx=0x7f4de400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4016020 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4de400a930 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41fd240 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de400ad80 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41f9490 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df406d200 con 0x7f4df406cdf0 2024-09-17T03:34:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.075+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f4de4019020 con 0x7f4df406cdf0 2024-09-17T03:34:05.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.079+0000 7f4deaa00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4dd0074d00 0x7f4dd0077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.079+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f4de4093280 con 0x7f4df406cdf0 2024-09-17T03:34:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.079+0000 7f4debe00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4dd0074d00 0x7f4dd0077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.079+0000 7f4debe00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4dd0074d00 0x7f4dd0077210 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0af000 tx=0x7f4ddc01af30 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.083+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4de4020020 con 0x7f4df406cdf0 2024-09-17T03:34:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.259+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4df406b0d0 con 0x7f4df406cdf0 2024-09-17T03:34:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4deaa00640 1 -- 172.21.15.46:0/2234849539 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16754 (secure 0 0 0) 0x7f4de405f630 con 0x7f4df406cdf0 2024-09-17T03:34:05.263 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:05.263 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:34:04.346128+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:33:00.702058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T03:33:01.968680+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:34:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4dd0074d00 msgr2=0x7f4dd0077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4dd0074d00 0x7f4dd0077210 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0af000 tx=0x7f4ddc01af30 comp rx=0 tx=0).stop 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 msgr2=0x7f4df41f9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 0x7f4df41f9c90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c6e0 tx=0x7f4de400cba0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 shutdown_connections 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4dd0074d00 0x7f4dd0077210 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4df4159130 0x7f4df40ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df406fc90 0x7f4df41528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 --2- 172.21.15.46:0/2234849539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4df406cdf0 0x7f4df41f9c90 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 >> 172.21.15.46:0/2234849539 conn(0x7f4df40b35f0 msgr2=0x7f4df406d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 shutdown_connections 2024-09-17T03:34:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.263+0000 7f4df9c00640 1 -- 172.21.15.46:0/2234849539 wait complete. 2024-09-17T03:34:05.287 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:34:05.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 -- 172.21.15.46:0/2691081761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415e8c0 msgr2=0x7ff5c415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 --2- 172.21.15.46:0/2691081761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415e8c0 0x7ff5c415ecd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009840 tx=0x7ff5b401c740 comp rx=0 tx=0).stop 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 -- 172.21.15.46:0/2691081761 shutdown_connections 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 --2- 172.21.15.46:0/2691081761 >> v2:172.21.15.99:3300/0 conn(0x7ff5c415b2d0 0x7ff5c415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 --2- 172.21.15.46:0/2691081761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415e8c0 0x7ff5c415ecd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 -- 172.21.15.46:0/2691081761 >> 172.21.15.46:0/2691081761 conn(0x7ff5c40b2040 msgr2=0x7ff5c40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 -- 172.21.15.46:0/2691081761 shutdown_connections 2024-09-17T03:34:05.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.443+0000 7ff5ca200640 1 -- 172.21.15.46:0/2691081761 wait complete. 2024-09-17T03:34:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 Processor -- start 2024-09-17T03:34:05.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 -- start start 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415b2d0 0x7ff5c4139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 0x7ff5c4139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c415e8c0 0x7ff5c41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40b5530 con 0x7ff5c415bc20 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40b53b0 con 0x7ff5c415e8c0 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5ca200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c415d990 con 0x7ff5c415b2d0 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 0x7ff5c4139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 0x7ff5c4139cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38530/0 (socket says 172.21.15.46:38530) 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5c2a00640 1 -- 172.21.15.46:0/3258869110 learned_addr learned my addr 172.21.15.46:0/3258869110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5c3400640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415b2d0 0x7ff5c4139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.447+0000 7ff5c3e00640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c415e8c0 0x7ff5c41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5c2a00640 1 -- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415b2d0 msgr2=0x7ff5c4139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5c2a00640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415b2d0 0x7ff5c4139770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5c2a00640 1 -- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c415e8c0 msgr2=0x7ff5c41530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5c2a00640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c415e8c0 0x7ff5c41530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5c2a00640 1 -- 172.21.15.46:0/3258869110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c4143820 con 0x7ff5c415bc20 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5c2a00640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 0x7ff5c4139cb0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e7a0 tx=0x7ff5b000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000ccc0 con 0x7ff5c415bc20 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5b000ce60 con 0x7ff5c415bc20 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c4143b00 con 0x7ff5c415bc20 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0004280 con 0x7ff5c415bc20 2024-09-17T03:34:05.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41440c0 con 0x7ff5c415bc20 2024-09-17T03:34:05.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c4143590 con 0x7ff5c415bc20 2024-09-17T03:34:05.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7ff5b0003600 con 0x7ff5c415bc20 2024-09-17T03:34:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.451+0000 7ff5bb400640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff5a4074d00 0x7ff5a4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.455+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7ff5b00938c0 con 0x7ff5c415bc20 2024-09-17T03:34:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.455+0000 7ff5c3400640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff5a4074d00 0x7ff5a4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.455+0000 7ff5c3400640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff5a4074d00 0x7ff5a4077210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40ae000 tx=0x7ff5b401af80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:05.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.459+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff5b005fe20 con 0x7ff5c415bc20 2024-09-17T03:34:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c415b6e0 con 0x7ff5c415bc20 2024-09-17T03:34:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5bb400640 1 -- 172.21.15.46:0/3258869110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5238 (secure 0 0 0) 0x7ff5b0064850 con 0x7ff5c415bc20 2024-09-17T03:34:05.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:05.669 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"btime":"2024-09-17T03:33:16:602820+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:33:02.961494+0000","modified":"2024-09-17T03:33:05.989948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":247692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_247692":{"gid":247692,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4088974288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4088974288},{"type":"v1","addr":"172.21.15.46:6836","nonce":4088974288}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":247692,"qdb_cluster":[247692]},"id":7}]} 2024-09-17T03:34:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff5a4074d00 msgr2=0x7ff5a4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff5a4074d00 0x7ff5a4077210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40ae000 tx=0x7ff5b401af80 comp rx=0 tx=0).stop 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 msgr2=0x7ff5c4139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 0x7ff5c4139cb0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e7a0 tx=0x7ff5b000ec60 comp rx=0 tx=0).stop 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 shutdown_connections 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff5a4074d00 0x7ff5a4077210 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c415e8c0 0x7ff5c41530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5c415bc20 0x7ff5c4139cb0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 --2- 172.21.15.46:0/3258869110 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff5c415b2d0 0x7ff5c4139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 >> 172.21.15.46:0/3258869110 conn(0x7ff5c40b2040 msgr2=0x7ff5c413d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:05.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 shutdown_connections 2024-09-17T03:34:05.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.667+0000 7ff5ca200640 1 -- 172.21.15.46:0/3258869110 wait complete. 2024-09-17T03:34:05.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 -- 172.21.15.46:0/1725327236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0157030 msgr2=0x7f3bc00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1725327236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0157030 0x7f3bc00b5a80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00b6a50 tx=0x7f3bb001c830 comp rx=0 tx=0).stop 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 -- 172.21.15.46:0/1725327236 shutdown_connections 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1725327236 >> v2:172.21.15.99:3300/0 conn(0x7f3bc0074b20 0x7f3bc015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1725327236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0157030 0x7f3bc00b5a80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 -- 172.21.15.46:0/1725327236 >> 172.21.15.46:0/1725327236 conn(0x7f3bc00708c0 msgr2=0x7f3bc0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 -- 172.21.15.46:0/1725327236 shutdown_connections 2024-09-17T03:34:05.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.847+0000 7f3bc7000640 1 -- 172.21.15.46:0/1725327236 wait complete. 2024-09-17T03:34:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 Processor -- start 2024-09-17T03:34:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 -- start start 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc0157030 0x7f3bc013bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc0160d50 0x7f3bc014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 0x7f3bc0138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc016a740 con 0x7f3bc0138500 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc016a5c0 con 0x7f3bc0160d50 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc7000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc016a8c0 con 0x7f3bc0157030 2024-09-17T03:34:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 0x7f3bc0138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc0157030 0x7f3bc013bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 0x7f3bc0138910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38548/0 (socket says 172.21.15.46:38548) 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bc5000640 1 -- 172.21.15.46:0/1156281411 learned_addr learned my addr 172.21.15.46:0/1156281411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.851+0000 7f3bb7400640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc0160d50 0x7f3bc014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc5000640 1 -- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc0157030 msgr2=0x7f3bc013bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc5000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc0157030 0x7f3bc013bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc5000640 1 -- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc0160d50 msgr2=0x7f3bc014d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc5000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc0160d50 0x7f3bc014d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:05.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc5000640 1 -- 172.21.15.46:0/1156281411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc0151470 con 0x7f3bc0138500 2024-09-17T03:34:05.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc5000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 0x7f3bc0138910 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800e9a0 tx=0x7f3bb800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:05.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8004d30 con 0x7f3bc0138500 2024-09-17T03:34:05.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bb800cb20 con 0x7f3bc0138500 2024-09-17T03:34:05.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc014e2c0 con 0x7f3bc0138500 2024-09-17T03:34:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8005020 con 0x7f3bc0138500 2024-09-17T03:34:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc013b160 con 0x7f3bc0138500 2024-09-17T03:34:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc014e030 con 0x7f3bc0138500 2024-09-17T03:34:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.855+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f3bb80051c0 con 0x7f3bc0138500 2024-09-17T03:34:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.859+0000 7f3bbcc00640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3b9c074d00 0x7f3b9c077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.859+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f3bb8093990 con 0x7f3bc0138500 2024-09-17T03:34:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.859+0000 7f3bbfe00640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3b9c074d00 0x7f3b9c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:05.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.859+0000 7f3bbfe00640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3b9c074d00 0x7f3b9c077210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00af000 tx=0x7f3bb001b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:05.863+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3bb8016030 con 0x7f3bc0138500 2024-09-17T03:34:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bc015cf90 con 0x7f3bc0138500 2024-09-17T03:34:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bbcc00640 1 -- 172.21.15.46:0/1156281411 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5238 (secure 0 0 0) 0x7f3bb805fe50 con 0x7f3bc0138500 2024-09-17T03:34:06.077 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:06.077 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"btime":"2024-09-17T03:33:16:602820+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:33:02.961494+0000","modified":"2024-09-17T03:33:05.989948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":247692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_247692":{"gid":247692,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/4088974288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":4088974288},{"type":"v1","addr":"172.21.15.46:6836","nonce":4088974288}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":247692,"qdb_cluster":[247692]},"id":7}]} 2024-09-17T03:34:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3b9c074d00 msgr2=0x7f3b9c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3b9c074d00 0x7f3b9c077210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00af000 tx=0x7f3bb001b070 comp rx=0 tx=0).stop 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 msgr2=0x7f3bc0138910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 0x7f3bc0138910 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800e9a0 tx=0x7f3bb800ee60 comp rx=0 tx=0).stop 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 shutdown_connections 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3b9c074d00 0x7f3b9c077210 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bc0138500 0x7f3bc0138910 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bc0160d50 0x7f3bc014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 --2- 172.21.15.46:0/1156281411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3bc0157030 0x7f3bc013bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 >> 172.21.15.46:0/1156281411 conn(0x7f3bc00708c0 msgr2=0x7f3bc0151fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 shutdown_connections 2024-09-17T03:34:06.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.075+0000 7f3bc7000640 1 -- 172.21.15.46:0/1156281411 wait complete. 2024-09-17T03:34:06.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe74ca00640 1 -- 172.21.15.46:0/4188906775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c004d50 msgr2=0x7fe72c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe74ca00640 1 --2- 172.21.15.46:0/4188906775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c004d50 0x7fe72c005160 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe7480cfcd0 tx=0x7fe74001d5d0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe746800640 1 -- 172.21.15.46:0/4188906775 reap_dead start 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe74ca00640 1 -- 172.21.15.46:0/4188906775 shutdown_connections 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe74ca00640 1 -- 172.21.15.46:0/4188906775 >> 172.21.15.46:0/4188906775 conn(0x7fe7480b2040 msgr2=0x7fe7480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe74ca00640 1 -- 172.21.15.46:0/4188906775 shutdown_connections 2024-09-17T03:34:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.255+0000 7fe74ca00640 1 -- 172.21.15.46:0/4188906775 wait complete. 2024-09-17T03:34:06.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 Processor -- start 2024-09-17T03:34:06.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 -- start start 2024-09-17T03:34:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe72c004d50 0x7fe748144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 0x7fe748152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7481403c0 0x7fe7481407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7480cf8c0 con 0x7fe748145c50 2024-09-17T03:34:06.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7480cf740 con 0x7fe7481403c0 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe7480cfa40 con 0x7fe72c004d50 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 0x7fe748152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 0x7fe748152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38566/0 (socket says 172.21.15.46:38566) 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 -- 172.21.15.46:0/695763192 learned_addr learned my addr 172.21.15.46:0/695763192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe745e00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe72c004d50 0x7fe748144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe746800640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7481403c0 0x7fe7481407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 -- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe72c004d50 msgr2=0x7fe748144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe72c004d50 0x7fe748144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 -- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7481403c0 msgr2=0x7fe7481407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7481403c0 0x7fe7481407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 -- 172.21.15.46:0/695763192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe748140d10 con 0x7fe748145c50 2024-09-17T03:34:06.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73fe00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 0x7fe748152df0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe730011ab0 tx=0x7fe730011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:06.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73001e070 con 0x7fe748145c50 2024-09-17T03:34:06.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe73000fc70 con 0x7fe748145c50 2024-09-17T03:34:06.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe748153330 con 0x7fe748145c50 2024-09-17T03:34:06.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730019670 con 0x7fe748145c50 2024-09-17T03:34:06.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.259+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7481437f0 con 0x7fe748145c50 2024-09-17T03:34:06.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.263+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7480b4070 con 0x7fe748145c50 2024-09-17T03:34:06.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.263+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fe730021070 con 0x7fe748145c50 2024-09-17T03:34:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.263+0000 7fe73e000640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe724074d00 0x7fe724077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.263+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7fe730093a50 con 0x7fe748145c50 2024-09-17T03:34:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.263+0000 7fe745e00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe724074d00 0x7fe724077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.267+0000 7fe745e00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe724074d00 0x7fe724077210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe738001870 tx=0x7fe738001120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.267+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe748140d10 con 0x7fe748145c50 2024-09-17T03:34:06.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.447+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe748143580 con 0x7fe748145c50 2024-09-17T03:34:06.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe73e000640 1 -- 172.21.15.46:0/695763192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16754 (secure 0 0 0) 0x7fe73000fe60 con 0x7fe748145c50 2024-09-17T03:34:06.451 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:06.451 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:34:04.346128+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:33:00.702058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T03:33:01.968680+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe724074d00 msgr2=0x7fe724077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe724074d00 0x7fe724077210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe738001870 tx=0x7fe738001120 comp rx=0 tx=0).stop 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 msgr2=0x7fe748152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 0x7fe748152df0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe730011ab0 tx=0x7fe730011f70 comp rx=0 tx=0).stop 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 shutdown_connections 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe724074d00 0x7fe724077210 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7481403c0 0x7fe7481407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe748145c50 0x7fe748152df0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 --2- 172.21.15.46:0/695763192 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe72c004d50 0x7fe748144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 >> 172.21.15.46:0/695763192 conn(0x7fe7480b2040 msgr2=0x7fe7480b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:06.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 shutdown_connections 2024-09-17T03:34:06.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.451+0000 7fe74ca00640 1 -- 172.21.15.46:0/695763192 wait complete. 2024-09-17T03:34:06.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:34:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 -- 172.21.15.46:0/1142727814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff4163380 msgr2=0x7f0ff41657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1142727814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff4163380 0x7f0ff41657a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009bc0 tx=0x7f0fec01cac0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 -- 172.21.15.46:0/1142727814 shutdown_connections 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1142727814 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff4163380 0x7f0ff41657a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1142727814 >> v2:172.21.15.99:3300/0 conn(0x7f0ff4160a20 0x7f0ff4162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 -- 172.21.15.46:0/1142727814 >> 172.21.15.46:0/1142727814 conn(0x7f0ff40b2040 msgr2=0x7f0ff40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 -- 172.21.15.46:0/1142727814 shutdown_connections 2024-09-17T03:34:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.635+0000 7f0ff9600640 1 -- 172.21.15.46:0/1142727814 wait complete. 2024-09-17T03:34:06.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 Processor -- start 2024-09-17T03:34:06.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 -- start start 2024-09-17T03:34:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff415e8c0 0x7f0ff413d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff4160a20 0x7f0ff4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 0x7f0ff40cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff4168cb0 con 0x7f0ff4163380 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff4168b30 con 0x7f0ff4160a20 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff9600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff4168e30 con 0x7f0ff415e8c0 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 0x7f0ff40cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 0x7f0ff40cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38582/0 (socket says 172.21.15.46:38582) 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 -- 172.21.15.46:0/1170516178 learned_addr learned my addr 172.21.15.46:0/1170516178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff415e8c0 0x7f0ff413d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0fea000640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff4160a20 0x7f0ff4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 -- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff415e8c0 msgr2=0x7f0ff413d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff415e8c0 0x7f0ff413d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 -- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff4160a20 msgr2=0x7f0ff4159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff4160a20 0x7f0ff4159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.639+0000 7f0ff3400640 1 -- 172.21.15.46:0/1170516178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff413da10 con 0x7f0ff4163380 2024-09-17T03:34:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0ff3400640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 0x7f0ff40cddb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0fec01b170 tx=0x7f0fec003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0bb070 con 0x7f0ff4163380 2024-09-17T03:34:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff41566c0 con 0x7f0ff4163380 2024-09-17T03:34:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fec004d40 con 0x7f0ff4163380 2024-09-17T03:34:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0b2040 con 0x7f0ff4163380 2024-09-17T03:34:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff415a300 con 0x7f0ff4163380 2024-09-17T03:34:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff415ecd0 con 0x7f0ff4163380 2024-09-17T03:34:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f0fec0a5b10 con 0x7f0ff4163380 2024-09-17T03:34:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0feb400640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd0074d00 0x7f0fd0077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f0fec12cc80 con 0x7f0ff4163380 2024-09-17T03:34:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.643+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd0074d00 0x7f0fd0077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.647+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd0074d00 0x7f0fd0077210 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007990 tx=0x7f0fe4001da0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:06.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.647+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0fec0166b0 con 0x7f0ff4163380 2024-09-17T03:34:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.859+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0ff4160510 con 0x7f0ff4163380 2024-09-17T03:34:06.860 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:34:06.859+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:34:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.859+0000 7f0feb400640 1 -- 172.21.15.46:0/1170516178 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7f0fec0f9030 con 0x7f0ff4163380 2024-09-17T03:34:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:34:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd0074d00 msgr2=0x7f0fd0077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd0074d00 0x7f0fd0077210 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4007990 tx=0x7f0fe4001da0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 msgr2=0x7f0ff40cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 0x7f0ff40cddb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0fec01b170 tx=0x7f0fec003f60 comp rx=0 tx=0).stop 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 shutdown_connections 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd0074d00 0x7f0fd0077210 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff4163380 0x7f0ff40cddb0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff4160a20 0x7f0ff4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 --2- 172.21.15.46:0/1170516178 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0ff415e8c0 0x7f0ff413d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 >> 172.21.15.46:0/1170516178 conn(0x7f0ff40b2040 msgr2=0x7f0ff415f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 shutdown_connections 2024-09-17T03:34:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:06.867+0000 7f0ff9600640 1 -- 172.21.15.46:0/1170516178 wait complete. 2024-09-17T03:34:06.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:34:07.009 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:34:07.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 -- 172.21.15.46:0/2021237905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b1dc0 msgr2=0x7ff1100b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 --2- 172.21.15.46:0/2021237905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b1dc0 0x7ff1100b21d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff10001c760 comp rx=0 tx=0).stop 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 -- 172.21.15.46:0/2021237905 shutdown_connections 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 --2- 172.21.15.46:0/2021237905 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1100b3130 0x7ff1100b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 --2- 172.21.15.46:0/2021237905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b1dc0 0x7ff1100b21d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 -- 172.21.15.46:0/2021237905 >> 172.21.15.46:0/2021237905 conn(0x7ff11006cdf0 msgr2=0x7ff11006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 -- 172.21.15.46:0/2021237905 shutdown_connections 2024-09-17T03:34:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.055+0000 7ff117200640 1 -- 172.21.15.46:0/2021237905 wait complete. 2024-09-17T03:34:07.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 Processor -- start 2024-09-17T03:34:07.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 -- start start 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 0x7ff1101f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b3130 0x7ff1101fc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff110155a00 0x7ff110151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff110168da0 con 0x7ff1100b1dc0 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff110168c20 con 0x7ff1100b3130 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff110168f20 con 0x7ff110155a00 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 0x7ff1101f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 0x7ff1101f8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38598/0 (socket says 172.21.15.46:38598) 2024-09-17T03:34:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 -- 172.21.15.46:0/2423822741 learned_addr learned my addr 172.21.15.46:0/2423822741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115400640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b3130 0x7ff1101fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff116800640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff110155a00 0x7ff110151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 -- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff110155a00 msgr2=0x7ff110151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff110155a00 0x7ff110151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 -- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b3130 msgr2=0x7ff1101fc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b3130 0x7ff1101fc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 -- 172.21.15.46:0/2423822741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff110155230 con 0x7ff1100b1dc0 2024-09-17T03:34:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff115e00640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 0x7ff1101f8730 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c760 tx=0x7ff10400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104016020 con 0x7ff1100b1dc0 2024-09-17T03:34:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff10400a950 con 0x7ff1100b1dc0 2024-09-17T03:34:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1101fd100 con 0x7ff1100b1dc0 2024-09-17T03:34:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400ada0 con 0x7ff1100b1dc0 2024-09-17T03:34:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.059+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff110139910 con 0x7ff1100b1dc0 2024-09-17T03:34:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.063+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1100ce4a0 con 0x7ff1100b1dc0 2024-09-17T03:34:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.063+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7ff104019020 con 0x7ff1100b1dc0 2024-09-17T03:34:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.063+0000 7ff10a000640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f4074d00 0x7ff0f4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.063+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5823+0+0 (secure 0 0 0) 0x7ff104092e80 con 0x7ff1100b1dc0 2024-09-17T03:34:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.063+0000 7ff115400640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f4074d00 0x7ff0f4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.063+0000 7ff115400640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f4074d00 0x7ff0f4077210 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff1000af000 tx=0x7ff10001afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.067+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff1100ce4a0 con 0x7ff1100b1dc0 2024-09-17T03:34:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.279+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff1100b21d0 con 0x7ff1100b1dc0 2024-09-17T03:34:07.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.863+0000 7ff10a000640 1 -- 172.21.15.46:0/2423822741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) ==== 104+0+0 (secure 0 0 0) 0x7ff104020020 con 0x7ff1100b1dc0 2024-09-17T03:34:07.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f4074d00 msgr2=0x7ff0f4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:07.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f4074d00 0x7ff0f4077210 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff1000af000 tx=0x7ff10001afa0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 msgr2=0x7ff1101f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 0x7ff1101f8730 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c760 tx=0x7ff10400cc20 comp rx=0 tx=0).stop 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 shutdown_connections 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff0f4074d00 0x7ff0f4077210 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff110155a00 0x7ff110151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1100b3130 0x7ff1101fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 --2- 172.21.15.46:0/2423822741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1100b1dc0 0x7ff1101f8730 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 >> 172.21.15.46:0/2423822741 conn(0x7ff11006cdf0 msgr2=0x7ff110154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 shutdown_connections 2024-09-17T03:34:07.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:07.867+0000 7ff117200640 1 -- 172.21.15.46:0/2423822741 wait complete. 2024-09-17T03:34:07.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:34:08.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 -- 172.21.15.46:0/4226242548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd374005360 msgr2=0x7fd37400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 --2- 172.21.15.46:0/4226242548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd374005360 0x7fd37400b9d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd390074950 tx=0x7fd38801f660 comp rx=0 tx=0).stop 2024-09-17T03:34:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 -- 172.21.15.46:0/4226242548 shutdown_connections 2024-09-17T03:34:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 --2- 172.21.15.46:0/4226242548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd374005360 0x7fd37400b9d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 -- 172.21.15.46:0/4226242548 >> 172.21.15.46:0/4226242548 conn(0x7fd3900b2040 msgr2=0x7fd3900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 -- 172.21.15.46:0/4226242548 shutdown_connections 2024-09-17T03:34:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 -- 172.21.15.46:0/4226242548 wait complete. 2024-09-17T03:34:08.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 Processor -- start 2024-09-17T03:34:08.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.055+0000 7fd396a00640 1 -- start start 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd374005360 0x7fd39013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 0x7fd390140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901405a0 0x7fd3901f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900751c0 con 0x7fd390151c10 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd390075040 con 0x7fd3901405a0 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd390075340 con 0x7fd374005360 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 0x7fd390140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 0x7fd390140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38624/0 (socket says 172.21.15.46:38624) 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 -- 172.21.15.46:0/3022478385 learned_addr learned my addr 172.21.15.46:0/3022478385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd38fe00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd374005360 0x7fd39013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd394a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901405a0 0x7fd3901f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 -- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd374005360 msgr2=0x7fd39013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd374005360 0x7fd39013fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 -- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901405a0 msgr2=0x7fd3901f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901405a0 0x7fd3901f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 -- 172.21.15.46:0/3022478385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3900cd740 con 0x7fd390151c10 2024-09-17T03:34:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd387400640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 0x7fd390140060 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd378003ee0 tx=0x7fd378001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378015070 con 0x7fd390151c10 2024-09-17T03:34:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd378011840 con 0x7fd390151c10 2024-09-17T03:34:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3900cda20 con 0x7fd390151c10 2024-09-17T03:34:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378011c70 con 0x7fd390151c10 2024-09-17T03:34:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3900cdfe0 con 0x7fd390151c10 2024-09-17T03:34:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.059+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900b53b0 con 0x7fd390151c10 2024-09-17T03:34:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.063+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fd378018070 con 0x7fd390151c10 2024-09-17T03:34:08.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.063+0000 7fd38cc00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd36c074d00 0x7fd36c077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:08.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.063+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5823+0+0 (secure 0 0 0) 0x7fd378093d50 con 0x7fd390151c10 2024-09-17T03:34:08.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.063+0000 7fd38fe00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd36c074d00 0x7fd36c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:08.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.063+0000 7fd38fe00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd36c074d00 0x7fd36c077210 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd380009fb0 tx=0x7fd380009500 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.067+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd378060180 con 0x7fd390151c10 2024-09-17T03:34:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.243+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd3900cc870 con 0x7fd390151c10 2024-09-17T03:34:08.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.867+0000 7fd38cc00640 1 -- 172.21.15.46:0/3022478385 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7fd378064bb0 con 0x7fd390151c10 2024-09-17T03:34:08.869 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:34:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd36c074d00 msgr2=0x7fd36c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd36c074d00 0x7fd36c077210 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd380009fb0 tx=0x7fd380009500 comp rx=0 tx=0).stop 2024-09-17T03:34:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 msgr2=0x7fd390140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 0x7fd390140060 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd378003ee0 tx=0x7fd378001640 comp rx=0 tx=0).stop 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 shutdown_connections 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd36c074d00 0x7fd36c077210 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3901405a0 0x7fd3901f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd390151c10 0x7fd390140060 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 --2- 172.21.15.46:0/3022478385 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd374005360 0x7fd39013fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 >> 172.21.15.46:0/3022478385 conn(0x7fd3900b2040 msgr2=0x7fd3900b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:08.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 shutdown_connections 2024-09-17T03:34:08.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:08.871+0000 7fd396a00640 1 -- 172.21.15.46:0/3022478385 wait complete. 2024-09-17T03:34:08.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:34:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 -- 172.21.15.46:0/3819648558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cf1b0 msgr2=0x7f8fc00cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 --2- 172.21.15.46:0/3819648558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cf1b0 0x7f8fc00cf5c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009bc0 tx=0x7f8fb801cba0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 -- 172.21.15.46:0/3819648558 shutdown_connections 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 --2- 172.21.15.46:0/3819648558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cf1b0 0x7f8fc00cf5c0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 --2- 172.21.15.46:0/3819648558 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fc00cd2e0 0x7f8fc0070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 -- 172.21.15.46:0/3819648558 >> 172.21.15.46:0/3819648558 conn(0x7f8fc00b2c40 msgr2=0x7f8fc00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 -- 172.21.15.46:0/3819648558 shutdown_connections 2024-09-17T03:34:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 -- 172.21.15.46:0/3819648558 wait complete. 2024-09-17T03:34:09.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 Processor -- start 2024-09-17T03:34:09.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.067+0000 7f8fc7200640 1 -- start start 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 0x7f8fc0155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc00cf1b0 0x7f8fc013e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fc01602a0 0x7f8fc0165730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0169e60 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0169ce0 con 0x7f8fc00cf1b0 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc0169fe0 con 0x7f8fc01602a0 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 0x7f8fc0155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 0x7f8fc0155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38652/0 (socket says 172.21.15.46:38652) 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 -- 172.21.15.46:0/680672372 learned_addr learned my addr 172.21.15.46:0/680672372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc5200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fc01602a0 0x7f8fc0165730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fb7e00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc00cf1b0 0x7f8fc013e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 -- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fc01602a0 msgr2=0x7f8fc0165730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fc01602a0 0x7f8fc0165730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 -- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc00cf1b0 msgr2=0x7f8fc013e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:09.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc00cf1b0 0x7f8fc013e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 -- 172.21.15.46:0/680672372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc0155ff0 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbfe00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 0x7f8fc0155ab0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000d980 tx=0x7f8fb000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017070 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fb0009d20 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc01fc2e0 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0004280 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01fc880 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.071+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc00cf5c0 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.075+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f8fb0005020 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.075+0000 7f8fbcc00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f9c074d00 0x7f8f9c077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.075+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5385+0+0 (secure 0 0 0) 0x7f8fb00932e0 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.075+0000 7f8fb7e00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f9c074d00 0x7f8f9c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:09.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.075+0000 7f8fb7e00640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f9c074d00 0x7f8f9c077210 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0157230 tx=0x7f8fa8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.079+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8fb0020030 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.255+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8fc006b0b0 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.867+0000 7f8fbcc00640 1 -- 172.21.15.46:0/680672372 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f8fb005fd60 con 0x7f8fc00cd2e0 2024-09-17T03:34:09.869 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:34:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f9c074d00 msgr2=0x7f8f9c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f9c074d00 0x7f8f9c077210 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0157230 tx=0x7f8fa8009040 comp rx=0 tx=0).stop 2024-09-17T03:34:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 msgr2=0x7f8fc0155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 0x7f8fc0155ab0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000d980 tx=0x7f8fb000de40 comp rx=0 tx=0).stop 2024-09-17T03:34:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 shutdown_connections 2024-09-17T03:34:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f9c074d00 0x7f8f9c077210 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8fc01602a0 0x7f8fc0165730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc00cf1b0 0x7f8fc013e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 --2- 172.21.15.46:0/680672372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fc00cd2e0 0x7f8fc0155ab0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 >> 172.21.15.46:0/680672372 conn(0x7f8fc00b2c40 msgr2=0x7f8fc0160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 shutdown_connections 2024-09-17T03:34:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:09.871+0000 7f8fc7200640 1 -- 172.21.15.46:0/680672372 wait complete. 2024-09-17T03:34:09.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 -- 172.21.15.46:0/4136063609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d801536f0 msgr2=0x7f5d8015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 --2- 172.21.15.46:0/4136063609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d801536f0 0x7f5d8015d480 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009900 tx=0x7f5d7001c850 comp rx=0 tx=0).stop 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 -- 172.21.15.46:0/4136063609 shutdown_connections 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 --2- 172.21.15.46:0/4136063609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d801536f0 0x7f5d8015d480 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 -- 172.21.15.46:0/4136063609 >> 172.21.15.46:0/4136063609 conn(0x7f5d800b4560 msgr2=0x7f5d800d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 -- 172.21.15.46:0/4136063609 shutdown_connections 2024-09-17T03:34:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.063+0000 7f5d85a00640 1 -- 172.21.15.46:0/4136063609 wait complete. 2024-09-17T03:34:10.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 Processor -- start 2024-09-17T03:34:10.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 -- start start 2024-09-17T03:34:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d801536f0 0x7f5d8014b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d80160060 0x7f5d801517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 0x7f5d80151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8016b650 con 0x7f5d8014fcf0 2024-09-17T03:34:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8016b4d0 con 0x7f5d80160060 2024-09-17T03:34:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8016b7d0 con 0x7f5d801536f0 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 0x7f5d80151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 0x7f5d80151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38668/0 (socket says 172.21.15.46:38668) 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 -- 172.21.15.46:0/1783126681 learned_addr learned my addr 172.21.15.46:0/1783126681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d7fe00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d801536f0 0x7f5d8014b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d77e00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d80160060 0x7f5d801517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 -- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d801536f0 msgr2=0x7f5d8014b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d801536f0 0x7f5d8014b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 -- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d80160060 msgr2=0x7f5d801517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d80160060 0x7f5d801517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 -- 172.21.15.46:0/1783126681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d80150290 con 0x7f5d8014fcf0 2024-09-17T03:34:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.067+0000 7f5d85000640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 0x7f5d80151d20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b970 tx=0x7f5d7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78019070 con 0x7f5d8014fcf0 2024-09-17T03:34:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d7800cc30 con 0x7f5d8014fcf0 2024-09-17T03:34:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d80150570 con 0x7f5d8014fcf0 2024-09-17T03:34:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800ce60 con 0x7f5d8014fcf0 2024-09-17T03:34:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d80150b30 con 0x7f5d8014fcf0 2024-09-17T03:34:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8015d480 con 0x7f5d8014fcf0 2024-09-17T03:34:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f5d7801c070 con 0x7f5d8014fcf0 2024-09-17T03:34:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d5c074d30 0x7f5d5c077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f5d78092c40 con 0x7f5d8014fcf0 2024-09-17T03:34:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.071+0000 7f5d7fe00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d5c074d30 0x7f5d5c077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.075+0000 7f5d7fe00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d5c074d30 0x7f5d5c077240 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d700af000 tx=0x7f5d7001b090 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.075+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5d7805f100 con 0x7f5d8014fcf0 2024-09-17T03:34:10.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.255+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f5d80072410 con 0x7f5d8014fcf0 2024-09-17T03:34:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1783126681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v51) ==== 71+0+0 (secure 0 0 0) 0x7f5d78066e20 con 0x7f5d8014fcf0 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d5c074d30 msgr2=0x7f5d5c077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d5c074d30 0x7f5d5c077240 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d700af000 tx=0x7f5d7001b090 comp rx=0 tx=0).stop 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 msgr2=0x7f5d80151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 0x7f5d80151d20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800b970 tx=0x7f5d7800be30 comp rx=0 tx=0).stop 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 shutdown_connections 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d5c074d30 0x7f5d5c077240 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8014fcf0 0x7f5d80151d20 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d80160060 0x7f5d801517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 --2- 172.21.15.46:0/1783126681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d801536f0 0x7f5d8014b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 >> 172.21.15.46:0/1783126681 conn(0x7f5d800b4560 msgr2=0x7f5d800b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 shutdown_connections 2024-09-17T03:34:10.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.259+0000 7f5d85a00640 1 -- 172.21.15.46:0/1783126681 wait complete. 2024-09-17T03:34:10.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-17T03:34:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 -- 172.21.15.46:0/1872693118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44500cc860 msgr2=0x7f445015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 --2- 172.21.15.46:0/1872693118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44500cc860 0x7f445015e890 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4438009870 tx=0x7f443801c790 comp rx=0 tx=0).stop 2024-09-17T03:34:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 -- 172.21.15.46:0/1872693118 shutdown_connections 2024-09-17T03:34:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 --2- 172.21.15.46:0/1872693118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44500cc860 0x7f445015e890 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 --2- 172.21.15.46:0/1872693118 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f445013e790 0x7f44500cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 -- 172.21.15.46:0/1872693118 >> 172.21.15.46:0/1872693118 conn(0x7f44500b2040 msgr2=0x7f44500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 -- 172.21.15.46:0/1872693118 shutdown_connections 2024-09-17T03:34:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 -- 172.21.15.46:0/1872693118 wait complete. 2024-09-17T03:34:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 Processor -- start 2024-09-17T03:34:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.443+0000 7f4456000640 1 -- start start 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44500cc860 0x7f44501fc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445013e790 0x7f44501fc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 0x7f44501536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4450140920 con 0x7f445015edd0 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44501407a0 con 0x7f445013e790 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4450140aa0 con 0x7f44500cc860 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 0x7f44501536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44500cc860 0x7f44501fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 0x7f44501536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38690/0 (socket says 172.21.15.46:38690) 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 -- 172.21.15.46:0/998003404 learned_addr learned my addr 172.21.15.46:0/998003404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4447e00640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445013e790 0x7f44501fc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 -- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44500cc860 msgr2=0x7f44501fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44500cc860 0x7f44501fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 -- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445013e790 msgr2=0x7f44501fc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445013e790 0x7f44501fc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 -- 172.21.15.46:0/998003404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445015a4f0 con 0x7f445015edd0 2024-09-17T03:34:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f444fe00640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 0x7f44501536f0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f444800c730 tx=0x7f444800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448003b80 con 0x7f445015edd0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4448003d40 con 0x7f445015edd0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480052d0 con 0x7f445015edd0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445015a790 con 0x7f445015edd0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445015ad50 con 0x7f445015edd0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.447+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4450158600 con 0x7f445015edd0 2024-09-17T03:34:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.451+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f4448005680 con 0x7f445015edd0 2024-09-17T03:34:10.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.451+0000 7f4447400640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c074d30 0x7f442c077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:10.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.451+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f4448092e70 con 0x7f445015edd0 2024-09-17T03:34:10.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.451+0000 7f444f400640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c074d30 0x7f442c077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:10.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.451+0000 7f444f400640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c074d30 0x7f442c077240 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4440004630 tx=0x7f444000b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:10.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.455+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f445015a4f0 con 0x7f445015edd0 2024-09-17T03:34:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.691+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f4450151c40 con 0x7f445015edd0 2024-09-17T03:34:10.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.867+0000 7f4447400640 1 -- 172.21.15.46:0/998003404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v607) ==== 178+0+0 (secure 0 0 0) 0x7f445015f1e0 con 0x7f445015edd0 2024-09-17T03:34:10.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c074d30 msgr2=0x7f442c077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c074d30 0x7f442c077240 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4440004630 tx=0x7f444000b2e0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 msgr2=0x7f44501536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 0x7f44501536f0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f444800c730 tx=0x7f444800cbf0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 shutdown_connections 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f442c074d30 0x7f442c077240 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f445015edd0 0x7f44501536f0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445013e790 0x7f44501fc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 --2- 172.21.15.46:0/998003404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f44500cc860 0x7f44501fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 >> 172.21.15.46:0/998003404 conn(0x7f44500b2040 msgr2=0x7f445015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 shutdown_connections 2024-09-17T03:34:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:10.871+0000 7f4456000640 1 -- 172.21.15.46:0/998003404 wait complete. 2024-09-17T03:34:10.894 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-17T03:34:10.894 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-17T03:34:10.894 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:34:10.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:34:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 -- 172.21.15.46:0/3852457408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 msgr2=0x7f394c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 --2- 172.21.15.46:0/3852457408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f394c003ef0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3958009b40 tx=0x7f395801d220 comp rx=0 tx=0).stop 2024-09-17T03:34:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 -- 172.21.15.46:0/3852457408 shutdown_connections 2024-09-17T03:34:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 --2- 172.21.15.46:0/3852457408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f394c003ef0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 -- 172.21.15.46:0/3852457408 >> 172.21.15.46:0/3852457408 conn(0x7f39680b2040 msgr2=0x7f39680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 -- 172.21.15.46:0/3852457408 shutdown_connections 2024-09-17T03:34:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.051+0000 7f396e400640 1 -- 172.21.15.46:0/3852457408 wait complete. 2024-09-17T03:34:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 Processor -- start 2024-09-17T03:34:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 -- start start 2024-09-17T03:34:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f39681fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39680af6d0 0x7f3968158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39681588a0 0x7f3968148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39680b5580 con 0x7f394c003a60 2024-09-17T03:34:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39680b5400 con 0x7f39681588a0 2024-09-17T03:34:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f396e400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f39680b5700 con 0x7f39680af6d0 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f39681fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f39681fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38700/0 (socket says 172.21.15.46:38700) 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 -- 172.21.15.46:0/1954004724 learned_addr learned my addr 172.21.15.46:0/1954004724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f395fe00640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39680af6d0 0x7f3968158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967e00640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39681588a0 0x7f3968148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 -- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39680af6d0 msgr2=0x7f3968158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39680af6d0 0x7f3968158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 -- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39681588a0 msgr2=0x7f3968148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39681588a0 0x7f3968148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 -- 172.21.15.46:0/1954004724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3968158cb0 con 0x7f394c003a60 2024-09-17T03:34:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f3967400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f39681fbac0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f395801d050 tx=0x7f3958005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.055+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580bb030 con 0x7f394c003a60 2024-09-17T03:34:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39580046d0 con 0x7f394c003a60 2024-09-17T03:34:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39681f8cf0 con 0x7f394c003a60 2024-09-17T03:34:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958004b20 con 0x7f394c003a60 2024-09-17T03:34:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39681f9290 con 0x7f394c003a60 2024-09-17T03:34:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396815f500 con 0x7f394c003a60 2024-09-17T03:34:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f39580be030 con 0x7f394c003a60 2024-09-17T03:34:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f395f400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3944074d00 0x7f3944077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f395812e560 con 0x7f394c003a60 2024-09-17T03:34:11.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.059+0000 7f395fe00640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3944074d00 0x7f3944077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.063+0000 7f395fe00640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3944074d00 0x7f3944077210 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3950001870 tx=0x7f3950001ce0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.063+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3968158cb0 con 0x7f394c003a60 2024-09-17T03:34:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.243+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39680b1e00 con 0x7f394c003a60 2024-09-17T03:34:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.243+0000 7f395f400640 1 -- 172.21.15.46:0/1954004724 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+13050 (secure 0 0 0) 0x7f3958016f60 con 0x7f394c003a60 2024-09-17T03:34:11.246 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:11.246 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:34:09.869797+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6836/4088974288":"2024-09-18T03:34:06.863604+0000","172.21.15.46:6835/4088974288":"2024-09-18T03:34:06.863604+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3944074d00 msgr2=0x7f3944077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3944074d00 0x7f3944077210 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3950001870 tx=0x7f3950001ce0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 msgr2=0x7f39681fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f39681fbac0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f395801d050 tx=0x7f3958005850 comp rx=0 tx=0).stop 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 shutdown_connections 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3944074d00 0x7f3944077210 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39681588a0 0x7f3968148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f39680af6d0 0x7f3968158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 --2- 172.21.15.46:0/1954004724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394c003a60 0x7f39681fbac0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 >> 172.21.15.46:0/1954004724 conn(0x7f39680b2040 msgr2=0x7f39680b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 shutdown_connections 2024-09-17T03:34:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.247+0000 7f396e400640 1 -- 172.21.15.46:0/1954004724 wait complete. 2024-09-17T03:34:11.268 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:34:11.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 -- 172.21.15.46:0/1385978633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 msgr2=0x7fcd2c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 --2- 172.21.15.46:0/1385978633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c15bb60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0cca90 tx=0x7fcd1801c740 comp rx=0 tx=0).stop 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 -- 172.21.15.46:0/1385978633 shutdown_connections 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 --2- 172.21.15.46:0/1385978633 >> v2:172.21.15.99:3300/0 conn(0x7fcd2c0b4de0 0x7fcd2c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 --2- 172.21.15.46:0/1385978633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c15bb60 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 -- 172.21.15.46:0/1385978633 >> 172.21.15.46:0/1385978633 conn(0x7fcd2c0b2040 msgr2=0x7fcd2c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 -- 172.21.15.46:0/1385978633 shutdown_connections 2024-09-17T03:34:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 -- 172.21.15.46:0/1385978633 wait complete. 2024-09-17T03:34:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 Processor -- start 2024-09-17T03:34:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.431+0000 7fcd34200640 1 -- start start 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd2c0b4de0 0x7fcd2c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd2c0b5c70 0x7fcd2c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c168920 con 0x7fcd2c1527b0 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c1687a0 con 0x7fcd2c0b5c70 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2c168aa0 con 0x7fcd2c0b4de0 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd31800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd2c0b4de0 0x7fcd2c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38710/0 (socket says 172.21.15.46:38710) 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd30e00640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd2c0b5c70 0x7fcd2c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 -- 172.21.15.46:0/449504587 learned_addr learned my addr 172.21.15.46:0/449504587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 -- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd2c0b4de0 msgr2=0x7fcd2c14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd2c0b4de0 0x7fcd2c14f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 -- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd2c0b5c70 msgr2=0x7fcd2c0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd2c0b5c70 0x7fcd2c0cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 -- 172.21.15.46:0/449504587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c1f9d30 con 0x7fcd2c1527b0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd32200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c159720 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400e940 tx=0x7fcd2400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004d30 con 0x7fcd2c1527b0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd2400cac0 con 0x7fcd2c1527b0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c1fa010 con 0x7fcd2c1527b0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24005020 con 0x7fcd2c1527b0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1fa5d0 con 0x7fcd2c1527b0 2024-09-17T03:34:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.435+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c0b51f0 con 0x7fcd2c1527b0 2024-09-17T03:34:11.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.439+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fcd240051c0 con 0x7fcd2c1527b0 2024-09-17T03:34:11.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.439+0000 7fcd29600640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd0c074d00 0x7fcd0c077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.439+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fcd24093900 con 0x7fcd2c1527b0 2024-09-17T03:34:11.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.439+0000 7fcd31800640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd0c074d00 0x7fcd0c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.439+0000 7fcd31800640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd0c074d00 0x7fcd0c077210 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd20009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.443+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd24016030 con 0x7fcd2c1527b0 2024-09-17T03:34:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.619+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd2c15c2a0 con 0x7fcd2c1527b0 2024-09-17T03:34:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd29600640 1 -- 172.21.15.46:0/449504587 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7fcd2405fcb0 con 0x7fcd2c1527b0 2024-09-17T03:34:11.623 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:34:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd0c074d00 msgr2=0x7fcd0c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd0c074d00 0x7fcd0c077210 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007fd0 tx=0x7fcd20009040 comp rx=0 tx=0).stop 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 msgr2=0x7fcd2c159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c159720 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400e940 tx=0x7fcd2400ee00 comp rx=0 tx=0).stop 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 shutdown_connections 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcd0c074d00 0x7fcd0c077210 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd2c1527b0 0x7fcd2c159720 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd2c0b5c70 0x7fcd2c0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 --2- 172.21.15.46:0/449504587 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcd2c0b4de0 0x7fcd2c14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 >> 172.21.15.46:0/449504587 conn(0x7fcd2c0b2040 msgr2=0x7fcd2c150620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 shutdown_connections 2024-09-17T03:34:11.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.623+0000 7fcd34200640 1 -- 172.21.15.46:0/449504587 wait complete. 2024-09-17T03:34:11.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-17T03:34:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 -- 172.21.15.46:0/1734361781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410159130 msgr2=0x7f741015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 --2- 172.21.15.46:0/1734361781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410159130 0x7f741015bb90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7400009900 tx=0x7f740001c850 comp rx=0 tx=0).stop 2024-09-17T03:34:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 -- 172.21.15.46:0/1734361781 shutdown_connections 2024-09-17T03:34:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 --2- 172.21.15.46:0/1734361781 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f741006cdf0 0x7f741006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 --2- 172.21.15.46:0/1734361781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410159130 0x7f741015bb90 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 -- 172.21.15.46:0/1734361781 >> 172.21.15.46:0/1734361781 conn(0x7f74100b35f0 msgr2=0x7f74100b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 -- 172.21.15.46:0/1734361781 shutdown_connections 2024-09-17T03:34:11.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.803+0000 7f7416c00640 1 -- 172.21.15.46:0/1734361781 wait complete. 2024-09-17T03:34:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 Processor -- start 2024-09-17T03:34:11.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 -- start start 2024-09-17T03:34:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 0x7f741013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f741006fc90 0x7f741015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7410159130 0x7f741015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7410168da0 con 0x7f741006cdf0 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7410168c20 con 0x7f7410159130 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7410168f20 con 0x7f741006fc90 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 0x7f741013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 0x7f741013fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38734/0 (socket says 172.21.15.46:38734) 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 -- 172.21.15.46:0/1600113872 learned_addr learned my addr 172.21.15.46:0/1600113872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740f400640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f741006fc90 0x7f741015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7414c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7410159130 0x7f741015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 -- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f741006fc90 msgr2=0x7f741015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f741006fc90 0x7f741015a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 -- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7410159130 msgr2=0x7f741015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:11.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7410159130 0x7f741015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 -- 172.21.15.46:0/1600113872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7410151820 con 0x7f741006cdf0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f740fe00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 0x7f741013fd70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7400009850 tx=0x7f7400005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000b2040 con 0x7f741006cdf0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7400004330 con 0x7f741006cdf0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400005280 con 0x7f741006cdf0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.807+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7410151b00 con 0x7f741006cdf0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.811+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101520c0 con 0x7f741006cdf0 2024-09-17T03:34:11.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.811+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741006d200 con 0x7f741006cdf0 2024-09-17T03:34:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.811+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f7400005420 con 0x7f741006cdf0 2024-09-17T03:34:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.811+0000 7f7407e00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73f0074d00 0x7f73f0077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.815+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f74000be070 con 0x7f741006cdf0 2024-09-17T03:34:11.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.815+0000 7f740f400640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73f0074d00 0x7f73f0077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:11.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.815+0000 7f740f400640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73f0074d00 0x7f73f0077210 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7410150ff0 tx=0x7f73fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:11.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:11.815+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74000164d0 con 0x7f741006cdf0 2024-09-17T03:34:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.059+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f7410157630 con 0x7f741006cdf0 2024-09-17T03:34:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.875+0000 7f7407e00640 1 -- 172.21.15.46:0/1600113872 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v609) ==== 188+0+0 (secure 0 0 0) 0x7f74000f98d0 con 0x7f741006cdf0 2024-09-17T03:34:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73f0074d00 msgr2=0x7f73f0077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73f0074d00 0x7f73f0077210 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7410150ff0 tx=0x7f73fc00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:34:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 msgr2=0x7f741013fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 0x7f741013fd70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7400009850 tx=0x7f7400005e70 comp rx=0 tx=0).stop 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 shutdown_connections 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f73f0074d00 0x7f73f0077210 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7410159130 0x7f741015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f741006fc90 0x7f741015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 --2- 172.21.15.46:0/1600113872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f741006cdf0 0x7f741013fd70 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 >> 172.21.15.46:0/1600113872 conn(0x7f74100b35f0 msgr2=0x7f74101582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 shutdown_connections 2024-09-17T03:34:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:12.879+0000 7f7416c00640 1 -- 172.21.15.46:0/1600113872 wait complete. 2024-09-17T03:34:12.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:34:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 -- 172.21.15.46:0/2269383181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa488074c30 msgr2=0x7fa488075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 --2- 172.21.15.46:0/2269383181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa488074c30 0x7fa488075040 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa488075760 tx=0x7fa47001c740 comp rx=0 tx=0).stop 2024-09-17T03:34:13.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 -- 172.21.15.46:0/2269383181 shutdown_connections 2024-09-17T03:34:13.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 --2- 172.21.15.46:0/2269383181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa488074c30 0x7fa488075040 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 --2- 172.21.15.46:0/2269383181 >> v2:172.21.15.99:3300/0 conn(0x7fa4880746e0 0x7fa48815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 -- 172.21.15.46:0/2269383181 >> 172.21.15.46:0/2269383181 conn(0x7fa4880b41f0 msgr2=0x7fa4880b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:13.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 -- 172.21.15.46:0/2269383181 shutdown_connections 2024-09-17T03:34:13.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.067+0000 7fa48dc00640 1 -- 172.21.15.46:0/2269383181 wait complete. 2024-09-17T03:34:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 Processor -- start 2024-09-17T03:34:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 -- start start 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4880746e0 0x7fa48806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa488074c30 0x7fa4880995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 0x7fa48809a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa48816a100 con 0x7fa4880d1190 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa488169f80 con 0x7fa488074c30 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48dc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa48816a280 con 0x7fa4880746e0 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 0x7fa48809a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 0x7fa48809a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58096/0 (socket says 172.21.15.46:58096) 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 -- 172.21.15.46:0/3247573964 learned_addr learned my addr 172.21.15.46:0/3247573964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa487e00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4880746e0 0x7fa48806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa47fe00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa488074c30 0x7fa4880995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 -- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4880746e0 msgr2=0x7fa48806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4880746e0 0x7fa48806fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 -- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa488074c30 msgr2=0x7fa4880995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa488074c30 0x7fa4880995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa48d200640 1 -- 172.21.15.46:0/3247573964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4880af5f0 con 0x7fa4880d1190 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa487e00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4880746e0 0x7fa48806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.071+0000 7fa47fe00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa488074c30 0x7fa4880995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa48d200640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 0x7fa48809a870 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c700 tx=0x7fa48000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:13.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa4880d1190 2024-09-17T03:34:13.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa48000ce10 con 0x7fa4880d1190 2024-09-17T03:34:13.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48814d1e0 con 0x7fa4880d1190 2024-09-17T03:34:13.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4800052d0 con 0x7fa4880d1190 2024-09-17T03:34:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa488099d90 con 0x7fa4880d1190 2024-09-17T03:34:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48815c920 con 0x7fa4880d1190 2024-09-17T03:34:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fa48001a070 con 0x7fa4880d1190 2024-09-17T03:34:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa484c00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074d00 0x7fa464077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.075+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fa4800926d0 con 0x7fa4880d1190 2024-09-17T03:34:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.079+0000 7fa487e00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074d00 0x7fa464077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.079+0000 7fa487e00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074d00 0x7fa464077210 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa478005fd0 tx=0x7fa478005f00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.083+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa48005f1f0 con 0x7fa4880d1190 2024-09-17T03:34:13.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.255+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa4880ae1f0 con 0x7fa4880d1190 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa484c00640 1 -- 172.21.15.46:0/3247573964 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v51) ==== 59+0+2294 (secure 0 0 0) 0x7fa48005fe20 con 0x7fa4880d1190 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:34:13.259 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:34:13.260 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:34:13.261 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:34:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074d00 msgr2=0x7fa464077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074d00 0x7fa464077210 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa478005fd0 tx=0x7fa478005f00 comp rx=0 tx=0).stop 2024-09-17T03:34:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 msgr2=0x7fa48809a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 0x7fa48809a870 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c700 tx=0x7fa48000cbc0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 shutdown_connections 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa464074d00 0x7fa464077210 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4880d1190 0x7fa48809a870 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa488074c30 0x7fa4880995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 --2- 172.21.15.46:0/3247573964 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa4880746e0 0x7fa48806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 >> 172.21.15.46:0/3247573964 conn(0x7fa4880b41f0 msgr2=0x7fa488070260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 shutdown_connections 2024-09-17T03:34:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.259+0000 7fa48dc00640 1 -- 172.21.15.46:0/3247573964 wait complete. 2024-09-17T03:34:13.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:34:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 -- 172.21.15.46:0/3659520496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b4de0 msgr2=0x7fde580b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 --2- 172.21.15.46:0/3659520496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b4de0 0x7fde580b51f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fde58071cb0 tx=0x7fde5001cae0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 -- 172.21.15.46:0/3659520496 shutdown_connections 2024-09-17T03:34:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 --2- 172.21.15.46:0/3659520496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b4de0 0x7fde580b51f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 --2- 172.21.15.46:0/3659520496 >> v2:172.21.15.99:3300/0 conn(0x7fde580b5c70 0x7fde5815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 -- 172.21.15.46:0/3659520496 >> 172.21.15.46:0/3659520496 conn(0x7fde580b2040 msgr2=0x7fde580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 -- 172.21.15.46:0/3659520496 shutdown_connections 2024-09-17T03:34:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 -- 172.21.15.46:0/3659520496 wait complete. 2024-09-17T03:34:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 Processor -- start 2024-09-17T03:34:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.443+0000 7fde5fa00640 1 -- start start 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde580b4de0 0x7fde5814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 0x7fde58154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde581527b0 0x7fde58158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde581688b0 con 0x7fde580b5c70 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fde58168730 con 0x7fde581527b0 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fde58168a30 con 0x7fde580b4de0 2024-09-17T03:34:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 0x7fde58154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 0x7fde58154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58114/0 (socket says 172.21.15.46:58114) 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 -- 172.21.15.46:0/2512922120 learned_addr learned my addr 172.21.15.46:0/2512922120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5d000640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde580b4de0 0x7fde5814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5da00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde581527b0 0x7fde58158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 -- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde580b4de0 msgr2=0x7fde5814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde580b4de0 0x7fde5814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 -- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde581527b0 msgr2=0x7fde58158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde581527b0 0x7fde58158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 -- 172.21.15.46:0/2512922120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5814caa0 con 0x7fde580b5c70 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5da00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde581527b0 0x7fde58158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde57e00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 0x7fde58154610 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fde4400b410 tx=0x7fde4400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44015030 con 0x7fde580b5c70 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde44004020 con 0x7fde580b5c70 2024-09-17T03:34:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde440044b0 con 0x7fde580b5c70 2024-09-17T03:34:13.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde58153570 con 0x7fde580b5c70 2024-09-17T03:34:13.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde58153b30 con 0x7fde580b5c70 2024-09-17T03:34:13.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.447+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde580b51f0 con 0x7fde580b5c70 2024-09-17T03:34:13.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.451+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fde44018070 con 0x7fde580b5c70 2024-09-17T03:34:13.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.451+0000 7fde54c00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fde38074d00 0x7fde38077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:13.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.451+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fde44092b20 con 0x7fde580b5c70 2024-09-17T03:34:13.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.451+0000 7fde5d000640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fde38074d00 0x7fde38077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:13.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.451+0000 7fde5d000640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fde38074d00 0x7fde38077210 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00bb50 tx=0x7fde4c00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:13.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.455+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fde4405f6d0 con 0x7fde580b5c70 2024-09-17T03:34:13.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.631+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fde58151800 con 0x7fde580b5c70 2024-09-17T03:34:13.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde54c00640 1 -- 172.21.15.46:0/2512922120 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7fde4405efd0 con 0x7fde580b5c70 2024-09-17T03:34:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fde38074d00 msgr2=0x7fde38077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fde38074d00 0x7fde38077210 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00bb50 tx=0x7fde4c00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 msgr2=0x7fde58154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 0x7fde58154610 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fde4400b410 tx=0x7fde4400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 shutdown_connections 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fde38074d00 0x7fde38077210 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde581527b0 0x7fde58158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde580b5c70 0x7fde58154610 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 --2- 172.21.15.46:0/2512922120 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fde580b4de0 0x7fde5814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 >> 172.21.15.46:0/2512922120 conn(0x7fde580b2040 msgr2=0x7fde58151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 shutdown_connections 2024-09-17T03:34:13.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:13.635+0000 7fde5fa00640 1 -- 172.21.15.46:0/2512922120 wait complete. 2024-09-17T03:34:13.658 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:34:13.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:34:13.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:34:13.852 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:34:13.852 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:13.852 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:13.861 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:34:13.861 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:34:13.861 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:13.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:34:13.871 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:13.871 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:13.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:34:13.915 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:34:13.915 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:13.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:34:13.924 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:34:13.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:14.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 -- 172.21.15.46:0/305489194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 msgr2=0x7f33540b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 --2- 172.21.15.46:0/305489194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33540b21d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f333c009930 tx=0x7f333c01c820 comp rx=0 tx=0).stop 2024-09-17T03:34:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 -- 172.21.15.46:0/305489194 shutdown_connections 2024-09-17T03:34:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 --2- 172.21.15.46:0/305489194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33540b21d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 -- 172.21.15.46:0/305489194 >> 172.21.15.46:0/305489194 conn(0x7f335406cdf0 msgr2=0x7f335406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 -- 172.21.15.46:0/305489194 shutdown_connections 2024-09-17T03:34:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.119+0000 7f3359800640 1 -- 172.21.15.46:0/305489194 wait complete. 2024-09-17T03:34:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 Processor -- start 2024-09-17T03:34:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 -- start start 2024-09-17T03:34:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33541534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33540b3130 0x7f3354153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3354155a00 0x7f33541f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3354169ba0 con 0x7f33540b1dc0 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3354169a20 con 0x7f33540b3130 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3359800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3354169d20 con 0x7f3354155a00 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33541534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33541534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58138/0 (socket says 172.21.15.46:58138) 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 -- 172.21.15.46:0/174900525 learned_addr learned my addr 172.21.15.46:0/174900525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f334b400640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33540b3130 0x7f3354153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3358e00640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3354155a00 0x7f33541f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 -- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3354155a00 msgr2=0x7f33541f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3354155a00 0x7f33541f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 -- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33540b3130 msgr2=0x7f3354153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33540b3130 0x7f3354153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f3353e00640 1 -- 172.21.15.46:0/174900525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33540cda10 con 0x7f33540b1dc0 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.123+0000 7f334b400640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33540b3130 0x7f3354153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3353e00640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33541534e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f334400c420 tx=0x7f334400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344017070 con 0x7f33540b1dc0 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33440053c0 con 0x7f33540b1dc0 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344003860 con 0x7f33540b1dc0 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33540cdcf0 con 0x7f33540b1dc0 2024-09-17T03:34:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33540ce2b0 con 0x7f33540b1dc0 2024-09-17T03:34:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33540b21d0 con 0x7f33540b1dc0 2024-09-17T03:34:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.127+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f334401a070 con 0x7f33540b1dc0 2024-09-17T03:34:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.131+0000 7f3350c00640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3330074d30 0x7f3330077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.131+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f33440924d0 con 0x7f33540b1dc0 2024-09-17T03:34:14.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.131+0000 7f334b400640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3330074d30 0x7f3330077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.131+0000 7f334b400640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3330074d30 0x7f3330077240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f333c0af000 tx=0x7f333c01b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.135+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33540b2710 con 0x7f33540b1dc0 2024-09-17T03:34:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33540afa90 con 0x7f33540b1dc0 2024-09-17T03:34:14.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3350c00640 1 -- 172.21.15.46:0/174900525 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3913 (secure 0 0 0) 0x7f334405ef50 con 0x7f33540b1dc0 2024-09-17T03:34:14.345 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:14.345 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":68,"btime":"2024-09-17T03:34:07:865822+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":246716,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/294248966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":294248966},{"type":"v1","addr":"172.21.15.99:6834","nonce":294248966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":246719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3525075248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3525075248},{"type":"v1","addr":"172.21.15.99:6835","nonce":3525075248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":247444,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2661976193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2661976193},{"type":"v1","addr":"172.21.15.46:6837","nonce":2661976193}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":248091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/689687030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":689687030},{"type":"v1","addr":"172.21.15.46:6836","nonce":689687030}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[]} 2024-09-17T03:34:14.345 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 68 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3330074d30 msgr2=0x7f3330077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3330074d30 0x7f3330077240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f333c0af000 tx=0x7f333c01b060 comp rx=0 tx=0).stop 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 msgr2=0x7f33541534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33541534e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f334400c420 tx=0x7f334400c8e0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 shutdown_connections 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3330074d30 0x7f3330077240 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3354155a00 0x7f33541f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33540b3130 0x7f3354153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 --2- 172.21.15.46:0/174900525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33540b1dc0 0x7f33541534e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 >> 172.21.15.46:0/174900525 conn(0x7f335406cdf0 msgr2=0x7f33540cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 shutdown_connections 2024-09-17T03:34:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.343+0000 7f3359800640 1 -- 172.21.15.46:0/174900525 wait complete. 2024-09-17T03:34:14.369 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:34:14.369 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:34:14.369 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:34:14.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:34:14.370 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:34:14.370 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:34:14.370 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:34:14.370 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:34:14.370 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:34:14.370 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:34:14.371 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:34:14.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:34:14.371 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:34:14.371 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:34:14.371 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:34:14.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:34:14.382 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:34:14.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:34:14.386 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:34:14.386 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:34:14.387 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:34:14.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:34:14.388 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:34:14.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:34:14.436 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:34:14.436 DEBUG:teuthology.parallel:result is None 2024-09-17T03:34:14.437 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:34:14.437 DEBUG:teuthology.parallel:result is None 2024-09-17T03:34:14.438 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:34:14.438 DEBUG:teuthology.parallel:result is None 2024-09-17T03:34:14.439 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:34:14.439 DEBUG:teuthology.parallel:result is None 2024-09-17T03:34:14.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:34:14.539 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:34:14.542 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:34:14.542 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:34:14.543 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:34:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.603+0000 7ff1cbe00640 1 -- 172.21.15.46:0/2375440632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c40d29c0 msgr2=0x7ff1c40d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.603+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/2375440632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c40d29c0 0x7ff1c40d2dd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009900 tx=0x7ff1b001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.603+0000 7ff1cbe00640 1 -- 172.21.15.46:0/2375440632 shutdown_connections 2024-09-17T03:34:14.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.603+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/2375440632 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1c415f180 0x7ff1c415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.603+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/2375440632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c40d29c0 0x7ff1c40d2dd0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.603+0000 7ff1cbe00640 1 -- 172.21.15.46:0/2375440632 >> 172.21.15.46:0/2375440632 conn(0x7ff1c40708c0 msgr2=0x7ff1c4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 -- 172.21.15.46:0/2375440632 shutdown_connections 2024-09-17T03:34:14.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 -- 172.21.15.46:0/2375440632 wait complete. 2024-09-17T03:34:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 Processor -- start 2024-09-17T03:34:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 -- start start 2024-09-17T03:34:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1c40d29c0 0x7ff1c414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1c4155ad0 0x7ff1c40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 0x7ff1c40b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c416c4a0 con 0x7ff1c415f180 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c416c320 con 0x7ff1c4155ad0 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.607+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c416c620 con 0x7ff1c40d29c0 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 0x7ff1c40b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 0x7ff1c40b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58188/0 (socket says 172.21.15.46:58188) 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 -- 172.21.15.46:0/1162819741 learned_addr learned my addr 172.21.15.46:0/1162819741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c8a00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1c4155ad0 0x7ff1c40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9400640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1c40d29c0 0x7ff1c414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 -- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1c40d29c0 msgr2=0x7ff1c414af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1c40d29c0 0x7ff1c414af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 -- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1c4155ad0 msgr2=0x7ff1c40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1c4155ad0 0x7ff1c40b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 -- 172.21.15.46:0/1162819741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c415e940 con 0x7ff1c415f180 2024-09-17T03:34:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c9e00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 0x7ff1c40b3e70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00daf0 tx=0x7ff1bc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc017070 con 0x7ff1c415f180 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1bc004b30 con 0x7ff1c415f180 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc003980 con 0x7ff1c415f180 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c41565c0 con 0x7ff1c415f180 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41562e0 con 0x7ff1c415f180 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c415e6b0 con 0x7ff1c415f180 2024-09-17T03:34:14.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.611+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7ff1bc003b20 con 0x7ff1c415f180 2024-09-17T03:34:14.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.615+0000 7ff1c1600640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1a4074d30 0x7ff1a4077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:14.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.615+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7ff1bc093700 con 0x7ff1c415f180 2024-09-17T03:34:14.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.615+0000 7ff1c9400640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1a4074d30 0x7ff1a4077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:14.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.615+0000 7ff1c9400640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1a4074d30 0x7ff1a4077240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8007fd0 tx=0x7ff1b8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:14.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.619+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff1bc020020 con 0x7ff1c415f180 2024-09-17T03:34:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.807+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff1c40d2dd0 con 0x7ff1c415f180 2024-09-17T03:34:14.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.899+0000 7ff1c1600640 1 -- 172.21.15.46:0/1162819741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) ==== 97+0+0 (secure 0 0 0) 0x7ff1bc05fbc0 con 0x7ff1c415f180 2024-09-17T03:34:14.902 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:34:14.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1a4074d30 msgr2=0x7ff1a4077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1a4074d30 0x7ff1a4077240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8007fd0 tx=0x7ff1b8009040 comp rx=0 tx=0).stop 2024-09-17T03:34:14.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 msgr2=0x7ff1c40b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:14.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 0x7ff1c40b3e70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00daf0 tx=0x7ff1bc009f60 comp rx=0 tx=0).stop 2024-09-17T03:34:14.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 shutdown_connections 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff1a4074d30 0x7ff1a4077240 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c415f180 0x7ff1c40b3e70 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1c4155ad0 0x7ff1c40b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 --2- 172.21.15.46:0/1162819741 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff1c40d29c0 0x7ff1c414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 >> 172.21.15.46:0/1162819741 conn(0x7ff1c40708c0 msgr2=0x7ff1c4074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 shutdown_connections 2024-09-17T03:34:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:14.903+0000 7ff1cbe00640 1 -- 172.21.15.46:0/1162819741 wait complete. 2024-09-17T03:34:14.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 -- 172.21.15.46:0/2400227816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0749a0 msgr2=0x7fbd7c074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 --2- 172.21.15.46:0/2400227816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c074db0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0cfc90 tx=0x7fbd7001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 -- 172.21.15.46:0/2400227816 shutdown_connections 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 --2- 172.21.15.46:0/2400227816 >> v2:172.21.15.99:3300/0 conn(0x7fbd7c0b3340 0x7fbd7c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 --2- 172.21.15.46:0/2400227816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c074db0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 -- 172.21.15.46:0/2400227816 >> 172.21.15.46:0/2400227816 conn(0x7fbd7c0b2040 msgr2=0x7fbd7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 -- 172.21.15.46:0/2400227816 shutdown_connections 2024-09-17T03:34:15.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 -- 172.21.15.46:0/2400227816 wait complete. 2024-09-17T03:34:15.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 Processor -- start 2024-09-17T03:34:15.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.087+0000 7fbd83c00640 1 -- start start 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 0x7fbd7c13d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd7c0cefa0 0x7fbd7c13ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c169d20 con 0x7fbd7c0b3340 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c169ba0 con 0x7fbd7c0749a0 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c169ea0 con 0x7fbd7c0cefa0 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 0x7fbd7c13d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 0x7fbd7c13d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58212/0 (socket says 172.21.15.46:58212) 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 -- 172.21.15.46:0/392768457 learned_addr learned my addr 172.21.15.46:0/392768457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd81200640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd81c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd7c0cefa0 0x7fbd7c13ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 -- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd7c0cefa0 msgr2=0x7fbd7c13ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd7c0cefa0 0x7fbd7c13ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 -- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd7c0749a0 msgr2=0x7fbd7c145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 -- 172.21.15.46:0/392768457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c145ed0 con 0x7fbd7c0b3340 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd81200640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd7be00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 0x7fbd7c13d8d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800b3e0 tx=0x7fbd6800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68015020 con 0x7fbd7c0b3340 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd68011920 con 0x7fbd7c0b3340 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c13de10 con 0x7fbd7c0b3340 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011d70 con 0x7fbd7c0b3340 2024-09-17T03:34:15.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c1f9290 con 0x7fbd7c0b3340 2024-09-17T03:34:15.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.091+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7c1460a0 con 0x7fbd7c0b3340 2024-09-17T03:34:15.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.095+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fbd68018020 con 0x7fbd7c0b3340 2024-09-17T03:34:15.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.095+0000 7fbd78c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd5c074d00 0x7fbd5c077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.095+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7fbd68092bf0 con 0x7fbd7c0b3340 2024-09-17T03:34:15.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.095+0000 7fbd81200640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd5c074d00 0x7fbd5c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.095+0000 7fbd81200640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd5c074d00 0x7fbd5c077210 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbd700af000 tx=0x7fbd7001b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:15.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.099+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd6801f030 con 0x7fbd7c0b3340 2024-09-17T03:34:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.335+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbd7c074db0 con 0x7fbd7c0b3340 2024-09-17T03:34:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.335+0000 7fbd78c00640 1 -- 172.21.15.46:0/392768457 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v41) ==== 80+0+5304 (secure 0 0 0) 0x7fbd6805f780 con 0x7fbd7c0b3340 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:34:15.339 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:34:15.340 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:34:15.342 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:34:15.344 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:34:15.346 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:34:15.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:34:15.348 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:34:15.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.350 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:34:15.351 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:34:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd5c074d00 msgr2=0x7fbd5c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd5c074d00 0x7fbd5c077210 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbd700af000 tx=0x7fbd7001b100 comp rx=0 tx=0).stop 2024-09-17T03:34:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 msgr2=0x7fbd7c13d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 0x7fbd7c13d8d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800b3e0 tx=0x7fbd6800b8a0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 shutdown_connections 2024-09-17T03:34:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbd5c074d00 0x7fbd5c077210 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbd7c0cefa0 0x7fbd7c13ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd7c0b3340 0x7fbd7c13d8d0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 --2- 172.21.15.46:0/392768457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd7c0749a0 0x7fbd7c145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 >> 172.21.15.46:0/392768457 conn(0x7fbd7c0b2040 msgr2=0x7fbd7c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 shutdown_connections 2024-09-17T03:34:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.339+0000 7fbd83c00640 1 -- 172.21.15.46:0/392768457 wait complete. 2024-09-17T03:34:15.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 -- 172.21.15.46:0/1608333372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf641527b0 msgr2=0x7faf64074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 --2- 172.21.15.46:0/1608333372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf641527b0 0x7faf64074e30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7faf64071cb0 tx=0x7faf4c01c740 comp rx=0 tx=0).stop 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 -- 172.21.15.46:0/1608333372 shutdown_connections 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 --2- 172.21.15.46:0/1608333372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf641527b0 0x7faf64074e30 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 --2- 172.21.15.46:0/1608333372 >> v2:172.21.15.99:3300/0 conn(0x7faf640b5c70 0x7faf6415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 -- 172.21.15.46:0/1608333372 >> 172.21.15.46:0/1608333372 conn(0x7faf640b2040 msgr2=0x7faf640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 -- 172.21.15.46:0/1608333372 shutdown_connections 2024-09-17T03:34:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 -- 172.21.15.46:0/1608333372 wait complete. 2024-09-17T03:34:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 Processor -- start 2024-09-17T03:34:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.519+0000 7faf6a200640 1 -- start start 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 0x7faf641fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf640b5c70 0x7faf641fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf641527b0 0x7faf641fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf64168b20 con 0x7faf640b4de0 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf641689a0 con 0x7faf641527b0 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faf64168ca0 con 0x7faf640b5c70 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 0x7faf641fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 0x7faf641fb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58222/0 (socket says 172.21.15.46:58222) 2024-09-17T03:34:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 -- 172.21.15.46:0/458385331 learned_addr learned my addr 172.21.15.46:0/458385331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63e00640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf641527b0 0x7faf641fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf5aa00640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf640b5c70 0x7faf641fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 -- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf640b5c70 msgr2=0x7faf641fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf640b5c70 0x7faf641fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 -- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf641527b0 msgr2=0x7faf641fbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf641527b0 0x7faf641fbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 -- 172.21.15.46:0/458385331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf640cd3f0 con 0x7faf640b4de0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63e00640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf641527b0 0x7faf641fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf63400640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 0x7faf641fb240 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7faf5400d780 tx=0x7faf5400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54017070 con 0x7faf640b4de0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf5400de90 con 0x7faf640b4de0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004020 con 0x7faf640b4de0 2024-09-17T03:34:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf64154ed0 con 0x7faf640b4de0 2024-09-17T03:34:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf64155490 con 0x7faf640b4de0 2024-09-17T03:34:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.523+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6415bb60 con 0x7faf640b4de0 2024-09-17T03:34:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.527+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7faf540041c0 con 0x7faf640b4de0 2024-09-17T03:34:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.527+0000 7faf5be00640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf40074d30 0x7faf40077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.527+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7faf540934e0 con 0x7faf640b4de0 2024-09-17T03:34:15.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.527+0000 7faf5aa00640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf40074d30 0x7faf40077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.527+0000 7faf5aa00640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf40074d30 0x7faf40077240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0af000 tx=0x7faf4c01af80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:15.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.531+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf54020020 con 0x7faf640b4de0 2024-09-17T03:34:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.707+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faf640b0b10 con 0x7faf640b4de0 2024-09-17T03:34:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.707+0000 7faf5be00640 1 -- 172.21.15.46:0/458385331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) ==== 84+0+65 (secure 0 0 0) 0x7faf5405fff0 con 0x7faf640b4de0 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:15.711 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf40074d30 msgr2=0x7faf40077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf40074d30 0x7faf40077240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0af000 tx=0x7faf4c01af80 comp rx=0 tx=0).stop 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 msgr2=0x7faf641fb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 0x7faf641fb240 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7faf5400d780 tx=0x7faf5400dc40 comp rx=0 tx=0).stop 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 shutdown_connections 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf40074d30 0x7faf40077240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf641527b0 0x7faf641fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf640b5c70 0x7faf641fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 --2- 172.21.15.46:0/458385331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf640b4de0 0x7faf641fb240 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 >> 172.21.15.46:0/458385331 conn(0x7faf640b2040 msgr2=0x7faf64151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 shutdown_connections 2024-09-17T03:34:15.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.711+0000 7faf6a200640 1 -- 172.21.15.46:0/458385331 wait complete. 2024-09-17T03:34:15.733 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:34:15.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 -- 172.21.15.46:0/3449406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b4de0 msgr2=0x7f3cf00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/3449406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b4de0 0x7f3cf00b51f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009bc0 tx=0x7f3ce801cae0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 -- 172.21.15.46:0/3449406549 shutdown_connections 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/3449406549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b4de0 0x7f3cf00b51f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/3449406549 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3cf00b5c70 0x7f3cf015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 -- 172.21.15.46:0/3449406549 >> 172.21.15.46:0/3449406549 conn(0x7f3cf00b2040 msgr2=0x7f3cf00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 -- 172.21.15.46:0/3449406549 shutdown_connections 2024-09-17T03:34:15.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.895+0000 7f3cf5a00640 1 -- 172.21.15.46:0/3449406549 wait complete. 2024-09-17T03:34:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 Processor -- start 2024-09-17T03:34:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 -- start start 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf00b4de0 0x7f3cf013d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 0x7f3cf01408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3cf01527b0 0x7f3cf01fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0168a00 con 0x7f3cf00b5c70 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0168880 con 0x7f3cf00b4de0 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cf5a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf0168b80 con 0x7f3cf01527b0 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 0x7f3cf01408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 0x7f3cf01408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58230/0 (socket says 172.21.15.46:58230) 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cee000640 1 -- 172.21.15.46:0/703498518 learned_addr learned my addr 172.21.15.46:0/703498518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3cef400640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3cf01527b0 0x7f3cf01fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.899+0000 7f3ceea00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf00b4de0 0x7f3cf013d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cee000640 1 -- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3cf01527b0 msgr2=0x7f3cf01fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cee000640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3cf01527b0 0x7f3cf01fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cee000640 1 -- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf00b4de0 msgr2=0x7f3cf013d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cee000640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf00b4de0 0x7f3cf013d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cee000640 1 -- 172.21.15.46:0/703498518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf00b1dc0 con 0x7f3cf00b5c70 2024-09-17T03:34:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cee000640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 0x7f3cf01408a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c370 tx=0x7f3cdc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc017070 con 0x7f3cf00b5c70 2024-09-17T03:34:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf01fc710 con 0x7f3cf00b5c70 2024-09-17T03:34:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf01400a0 con 0x7f3cf00b5c70 2024-09-17T03:34:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cdc003600 con 0x7f3cf00b5c70 2024-09-17T03:34:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.903+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc003a70 con 0x7f3cf00b5c70 2024-09-17T03:34:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.907+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc80052a0 con 0x7f3cf00b5c70 2024-09-17T03:34:15.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.907+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f3cdc01a070 con 0x7f3cf00b5c70 2024-09-17T03:34:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.911+0000 7f3ce6a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cd0074d00 0x7f3cd0077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.911+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7f3cdc092510 con 0x7f3cf00b5c70 2024-09-17T03:34:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.911+0000 7f3ceea00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cd0074d00 0x7f3cd0077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.911+0000 7f3ceea00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cd0074d00 0x7f3cd0077210 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009b40 tx=0x7f3ce00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:15.915+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3cf00b5730 con 0x7f3cf00b5c70 2024-09-17T03:34:16.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.095+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3cc8005590 con 0x7f3cf00b5c70 2024-09-17T03:34:16.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.923+0000 7f3ce6a00640 1 -- 172.21.15.46:0/703498518 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) ==== 138+0+0 (secure 0 0 0) 0x7f3cdc05e940 con 0x7f3cf00b5c70 2024-09-17T03:34:16.925 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cd0074d00 msgr2=0x7f3cd0077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cd0074d00 0x7f3cd0077210 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009b40 tx=0x7f3ce00091e0 comp rx=0 tx=0).stop 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 msgr2=0x7f3cf01408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 0x7f3cf01408a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c370 tx=0x7f3cdc00c830 comp rx=0 tx=0).stop 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 shutdown_connections 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3cd0074d00 0x7f3cd0077210 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3cf01527b0 0x7f3cf01fbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf00b5c70 0x7f3cf01408a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 --2- 172.21.15.46:0/703498518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf00b4de0 0x7f3cf013d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 >> 172.21.15.46:0/703498518 conn(0x7f3cf00b2040 msgr2=0x7f3cf0151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:16.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 shutdown_connections 2024-09-17T03:34:16.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:16.927+0000 7f3cf5a00640 1 -- 172.21.15.46:0/703498518 wait complete. 2024-09-17T03:34:16.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 -- 172.21.15.46:0/2136671588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 msgr2=0x7f33f00cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 --2- 172.21.15.46:0/2136671588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 0x7f33f00cf3b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009900 tx=0x7f33dc01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 -- 172.21.15.46:0/2136671588 shutdown_connections 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 --2- 172.21.15.46:0/2136671588 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 0x7f33f00cf3b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 --2- 172.21.15.46:0/2136671588 >> v2:172.21.15.99:3300/0 conn(0x7f33f00749a0 0x7f33f0074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 -- 172.21.15.46:0/2136671588 >> 172.21.15.46:0/2136671588 conn(0x7f33f00b2040 msgr2=0x7f33f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 -- 172.21.15.46:0/2136671588 shutdown_connections 2024-09-17T03:34:17.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 -- 172.21.15.46:0/2136671588 wait complete. 2024-09-17T03:34:17.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 Processor -- start 2024-09-17T03:34:17.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.131+0000 7f33f7800640 1 -- start start 2024-09-17T03:34:17.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 0x7f33f015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 0x7f33f013a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33f0160330 0x7f33f00af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33f016a060 con 0x7f33f00749a0 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0169ee0 con 0x7f33f0160330 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f33f016a1e0 con 0x7f33f00cefa0 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 0x7f33f015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 0x7f33f015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58240/0 (socket says 172.21.15.46:58240) 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 -- 172.21.15.46:0/2356302349 learned_addr learned my addr 172.21.15.46:0/2356302349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f5800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33f0160330 0x7f33f00af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33efe00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 0x7f33f013a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 -- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 msgr2=0x7f33f013a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 0x7f33f013a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 -- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33f0160330 msgr2=0x7f33f00af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:17.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33f0160330 0x7f33f00af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 -- 172.21.15.46:0/2356302349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00b0ee0 con 0x7f33f00749a0 2024-09-17T03:34:17.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f5800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33f0160330 0x7f33f00af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:17.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f4e00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 0x7f33f015a300 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e7a0 tx=0x7f33e400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:17.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e400cc60 con 0x7f33f00749a0 2024-09-17T03:34:17.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f01452b0 con 0x7f33f00749a0 2024-09-17T03:34:17.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33e4010430 con 0x7f33f00749a0 2024-09-17T03:34:17.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4010860 con 0x7f33f00749a0 2024-09-17T03:34:17.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f0145870 con 0x7f33f00749a0 2024-09-17T03:34:17.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.135+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f0145030 con 0x7f33f00749a0 2024-09-17T03:34:17.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.139+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f33e4010a00 con 0x7f33f00749a0 2024-09-17T03:34:17.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.139+0000 7f33ecc00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33d0074d00 0x7f33d0077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:17.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.139+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5255+0+0 (secure 0 0 0) 0x7f33e4094220 con 0x7f33f00749a0 2024-09-17T03:34:17.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.139+0000 7f33efe00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33d0074d00 0x7f33d0077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:17.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.139+0000 7f33efe00640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33d0074d00 0x7f33d0077210 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0af000 tx=0x7f33dc01b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.143+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33f0145030 con 0x7f33f00749a0 2024-09-17T03:34:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.319+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f33f0074db0 con 0x7f33f00749a0 2024-09-17T03:34:17.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.931+0000 7f33ecc00640 1 -- 172.21.15.46:0/2356302349 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) ==== 170+0+0 (secure 0 0 0) 0x7f33e4060760 con 0x7f33f00749a0 2024-09-17T03:34:17.932 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33d0074d00 msgr2=0x7f33d0077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33d0074d00 0x7f33d0077210 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0af000 tx=0x7f33dc01b120 comp rx=0 tx=0).stop 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 msgr2=0x7f33f015a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 0x7f33f015a300 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e7a0 tx=0x7f33e400ec60 comp rx=0 tx=0).stop 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 shutdown_connections 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f33d0074d00 0x7f33d0077210 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33f0160330 0x7f33f00af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f33f00cefa0 0x7f33f013a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 --2- 172.21.15.46:0/2356302349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33f00749a0 0x7f33f015a300 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 >> 172.21.15.46:0/2356302349 conn(0x7f33f00b2040 msgr2=0x7f33f013d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 shutdown_connections 2024-09-17T03:34:17.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:17.935+0000 7f33f7800640 1 -- 172.21.15.46:0/2356302349 wait complete. 2024-09-17T03:34:17.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 -- 172.21.15.46:0/270730625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 msgr2=0x7fbeac15e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/270730625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac15e8c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009840 tx=0x7fbe9c01c820 comp rx=0 tx=0).stop 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 -- 172.21.15.46:0/270730625 shutdown_connections 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/270730625 >> v2:172.21.15.99:3300/0 conn(0x7fbeac15ee00 0x7fbeac15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/270730625 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac15e8c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 -- 172.21.15.46:0/270730625 >> 172.21.15.46:0/270730625 conn(0x7fbeac0b2040 msgr2=0x7fbeac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 -- 172.21.15.46:0/270730625 shutdown_connections 2024-09-17T03:34:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.135+0000 7fbeb0c00640 1 -- 172.21.15.46:0/270730625 wait complete. 2024-09-17T03:34:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 Processor -- start 2024-09-17T03:34:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 -- start start 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 0x7fbeac0abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeac15ee00 0x7fbeac0ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac168090 con 0x7fbeac15b7b0 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac167f10 con 0x7fbeac15ee00 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeb0c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac168210 con 0x7fbeac0cf900 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 0x7fbeac0abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 0x7fbeac0abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58248/0 (socket says 172.21.15.46:58248) 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbea3e00640 1 -- 172.21.15.46:0/1283241288 learned_addr learned my addr 172.21.15.46:0/1283241288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbea9e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.139+0000 7fbeaa800640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeac15ee00 0x7fbeac0ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea3e00640 1 -- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 msgr2=0x7fbeac142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea3e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea3e00640 1 -- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeac15ee00 msgr2=0x7fbeac0ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea3e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeac15ee00 0x7fbeac0ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea3e00640 1 -- 172.21.15.46:0/1283241288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac1f7930 con 0x7fbeac15b7b0 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea9e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea3e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 0x7fbeac0abbe0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400ea60 tx=0x7fbe9400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400cbc0 con 0x7fbeac15b7b0 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe9400ce50 con 0x7fbeac15b7b0 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94005280 con 0x7fbeac15b7b0 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac1f7c10 con 0x7fbeac15b7b0 2024-09-17T03:34:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac0ab6c0 con 0x7fbeac15b7b0 2024-09-17T03:34:18.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac15e8c0 con 0x7fbeac15b7b0 2024-09-17T03:34:18.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.143+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fbe94005420 con 0x7fbeac15b7b0 2024-09-17T03:34:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.147+0000 7fbea2000640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe88074d00 0x7fbe88077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.147+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5637+0+0 (secure 0 0 0) 0x7fbe94093cc0 con 0x7fbeac15b7b0 2024-09-17T03:34:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.147+0000 7fbea9e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe88074d00 0x7fbe88077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.147+0000 7fbea9e00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe88074d00 0x7fbe88077210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0af000 tx=0x7fbe9c01b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.151+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe94060170 con 0x7fbeac15b7b0 2024-09-17T03:34:18.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.359+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbeac1f76c0 con 0x7fbeac15b7b0 2024-09-17T03:34:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.931+0000 7fbea2000640 1 -- 172.21.15.46:0/1283241288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v71) ==== 174+0+0 (secure 0 0 0) 0x7fbe94064ba0 con 0x7fbeac15b7b0 2024-09-17T03:34:18.935 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 16 and data pool 17 2024-09-17T03:34:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe88074d00 msgr2=0x7fbe88077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe88074d00 0x7fbe88077210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0af000 tx=0x7fbe9c01b060 comp rx=0 tx=0).stop 2024-09-17T03:34:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 msgr2=0x7fbeac0abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 0x7fbeac0abbe0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400ea60 tx=0x7fbe9400ef20 comp rx=0 tx=0).stop 2024-09-17T03:34:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 shutdown_connections 2024-09-17T03:34:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbe88074d00 0x7fbe88077210 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeac15ee00 0x7fbeac0ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbeac15b7b0 0x7fbeac0abbe0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 --2- 172.21.15.46:0/1283241288 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbeac0cf900 0x7fbeac142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.939+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 >> 172.21.15.46:0/1283241288 conn(0x7fbeac0b2040 msgr2=0x7fbeac15bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.943+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 shutdown_connections 2024-09-17T03:34:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:18.943+0000 7fbeb0c00640 1 -- 172.21.15.46:0/1283241288 wait complete. 2024-09-17T03:34:18.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 -- 172.21.15.46:0/389386849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 msgr2=0x7f3e38074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/389386849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e38074db0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3e380726f0 tx=0x7f3e2801c8c0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 -- 172.21.15.46:0/389386849 shutdown_connections 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/389386849 >> v2:172.21.15.99:3300/0 conn(0x7f3e380cefa0 0x7f3e380cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/389386849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e38074db0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 -- 172.21.15.46:0/389386849 >> 172.21.15.46:0/389386849 conn(0x7f3e380b2040 msgr2=0x7f3e380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 -- 172.21.15.46:0/389386849 shutdown_connections 2024-09-17T03:34:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 -- 172.21.15.46:0/389386849 wait complete. 2024-09-17T03:34:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 Processor -- start 2024-09-17T03:34:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.155+0000 7f3e3ee00640 1 -- start start 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e381faad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e380cefa0 0x7f3e38158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e38160330 0x7f3e3815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3816a060 con 0x7f3e380749a0 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38169ee0 con 0x7f3e380cefa0 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3816a1e0 con 0x7f3e38160330 2024-09-17T03:34:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e381faad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e381faad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58270/0 (socket says 172.21.15.46:58270) 2024-09-17T03:34:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 -- 172.21.15.46:0/2593980938 learned_addr learned my addr 172.21.15.46:0/2593980938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 -- 172.21.15.46:0/2593980938 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e38160330 msgr2=0x7f3e3815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e38160330 0x7f3e3815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 -- 172.21.15.46:0/2593980938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e380cefa0 msgr2=0x7f3e38158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e380cefa0 0x7f3e38158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 -- 172.21.15.46:0/2593980938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3815a4a0 con 0x7f3e380749a0 2024-09-17T03:34:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e37e00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e381faad0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3e2801af70 tx=0x7f3e28005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280b2040 con 0x7f3e380749a0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e280038f0 con 0x7f3e380749a0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28003d40 con 0x7f3e380749a0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e38159400 con 0x7f3e380749a0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381599c0 con 0x7f3e380749a0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.159+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e380b27d0 con 0x7f3e380749a0 2024-09-17T03:34:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.163+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f3e280bf060 con 0x7f3e380749a0 2024-09-17T03:34:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.163+0000 7f3e34c00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e14074d00 0x7f3e14077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.163+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7f3e2812d180 con 0x7f3e380749a0 2024-09-17T03:34:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.163+0000 7f3e2fe00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e14074d00 0x7f3e14077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.163+0000 7f3e2fe00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e14074d00 0x7f3e14077210 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3e3815e550 tx=0x7f3e20005be0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.167+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e280164b0 con 0x7f3e380749a0 2024-09-17T03:34:19.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.343+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e380b24f0 con 0x7f3e380749a0 2024-09-17T03:34:19.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.343+0000 7f3e34c00640 1 -- 172.21.15.46:0/2593980938 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16708 (secure 0 0 0) 0x7f3e280f9c90 con 0x7f3e380749a0 2024-09-17T03:34:19.346 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:19.346 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":113,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:34:18.364837+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:34:16.113350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T03:34:17.334380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:34:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e14074d00 msgr2=0x7f3e14077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e14074d00 0x7f3e14077210 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3e3815e550 tx=0x7f3e20005be0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 msgr2=0x7f3e381faad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e381faad0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3e2801af70 tx=0x7f3e28005790 comp rx=0 tx=0).stop 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 shutdown_connections 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e14074d00 0x7f3e14077210 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e38160330 0x7f3e3815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e380cefa0 0x7f3e38158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 --2- 172.21.15.46:0/2593980938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e380749a0 0x7f3e381faad0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 >> 172.21.15.46:0/2593980938 conn(0x7f3e380b2040 msgr2=0x7f3e3813f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 shutdown_connections 2024-09-17T03:34:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.347+0000 7f3e3ee00640 1 -- 172.21.15.46:0/2593980938 wait complete. 2024-09-17T03:34:19.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:34:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 -- 172.21.15.46:0/1668214886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b4de0 msgr2=0x7ff2a40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/1668214886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b4de0 0x7ff2a40b51f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4070710 tx=0x7ff29c01cac0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 -- 172.21.15.46:0/1668214886 shutdown_connections 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/1668214886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b4de0 0x7ff2a40b51f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/1668214886 >> v2:172.21.15.99:3300/0 conn(0x7ff2a41527b0 0x7ff2a415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 -- 172.21.15.46:0/1668214886 >> 172.21.15.46:0/1668214886 conn(0x7ff2a40b2040 msgr2=0x7ff2a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 -- 172.21.15.46:0/1668214886 shutdown_connections 2024-09-17T03:34:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.527+0000 7ff2aaa00640 1 -- 172.21.15.46:0/1668214886 wait complete. 2024-09-17T03:34:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 Processor -- start 2024-09-17T03:34:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 -- start start 2024-09-17T03:34:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2a40b4de0 0x7ff2a414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 0x7ff2a40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a41527b0 0x7ff2a4153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a41688c0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4168740 con 0x7ff2a41527b0 2024-09-17T03:34:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a4168a40 con 0x7ff2a40b4de0 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 0x7ff2a40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 0x7ff2a40cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58294/0 (socket says 172.21.15.46:58294) 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 -- 172.21.15.46:0/4113225723 learned_addr learned my addr 172.21.15.46:0/4113225723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2a3e00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2a40b4de0 0x7ff2a414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2a8a00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a41527b0 0x7ff2a4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 -- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2a40b4de0 msgr2=0x7ff2a414ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2a40b4de0 0x7ff2a414ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 -- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a41527b0 msgr2=0x7ff2a4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a41527b0 0x7ff2a4153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 -- 172.21.15.46:0/4113225723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a414c2a0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2a3e00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2a40b4de0 0x7ff2a414ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff29be00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 0x7ff2a40cd740 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00e7c0 tx=0x7ff28c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c00ccc0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff28c00ce80 con 0x7ff2a40b5c70 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c005280 con 0x7ff2a40b5c70 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a414c580 con 0x7ff2a40b5c70 2024-09-17T03:34:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.531+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a4152e10 con 0x7ff2a40b5c70 2024-09-17T03:34:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.535+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a40b51f0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.535+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7ff28c005420 con 0x7ff2a40b5c70 2024-09-17T03:34:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.535+0000 7ff2a0c00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff280074d00 0x7ff280077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.535+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7ff28c093ea0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.535+0000 7ff2a3e00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff280074d00 0x7ff280077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.539+0000 7ff2a3e00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff280074d00 0x7ff280077210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff294000f70 tx=0x7ff294009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.539+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff28c0609a0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.719+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2a414c030 con 0x7ff2a40b5c70 2024-09-17T03:34:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2a0c00640 1 -- 172.21.15.46:0/4113225723 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16708 (secure 0 0 0) 0x7ff28c0602a0 con 0x7ff2a40b5c70 2024-09-17T03:34:19.724 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:19.725 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":113,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:34:18.364837+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:34:16.113350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T03:34:17.334380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:34:19.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff280074d00 msgr2=0x7ff280077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff280074d00 0x7ff280077210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff294000f70 tx=0x7ff294009040 comp rx=0 tx=0).stop 2024-09-17T03:34:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 msgr2=0x7ff2a40cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 0x7ff2a40cd740 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00e7c0 tx=0x7ff28c00ec80 comp rx=0 tx=0).stop 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 shutdown_connections 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff280074d00 0x7ff280077210 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2a41527b0 0x7ff2a4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2a40b5c70 0x7ff2a40cd740 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 --2- 172.21.15.46:0/4113225723 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2a40b4de0 0x7ff2a414ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 >> 172.21.15.46:0/4113225723 conn(0x7ff2a40b2040 msgr2=0x7ff2a40b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 shutdown_connections 2024-09-17T03:34:19.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.723+0000 7ff2aaa00640 1 -- 172.21.15.46:0/4113225723 wait complete. 2024-09-17T03:34:19.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 -- 172.21.15.46:0/1858159454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b40b5a80 msgr2=0x7f02b4074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 --2- 172.21.15.46:0/1858159454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b40b5a80 0x7f02b4074410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f029c009840 tx=0x7f029c01c6b0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 -- 172.21.15.46:0/1858159454 shutdown_connections 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 --2- 172.21.15.46:0/1858159454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b40b5a80 0x7f02b4074410 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 -- 172.21.15.46:0/1858159454 >> 172.21.15.46:0/1858159454 conn(0x7f02b40708c0 msgr2=0x7f02b4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 -- 172.21.15.46:0/1858159454 shutdown_connections 2024-09-17T03:34:19.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 -- 172.21.15.46:0/1858159454 wait complete. 2024-09-17T03:34:19.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 Processor -- start 2024-09-17T03:34:19.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.907+0000 7f02b9c00640 1 -- start start 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02b4074950 0x7f02b414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 0x7f02b4141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b4157030 0x7f02b40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02b416b640 con 0x7f02b40b5a80 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f02b416b4c0 con 0x7f02b4157030 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f02b416b7c0 con 0x7f02b4074950 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 0x7f02b4141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 0x7f02b4141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58308/0 (socket says 172.21.15.46:58308) 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 -- 172.21.15.46:0/3545951547 learned_addr learned my addr 172.21.15.46:0/3545951547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b3400640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b4157030 0x7f02b40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b2a00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02b4074950 0x7f02b414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 -- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02b4074950 msgr2=0x7f02b414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02b4074950 0x7f02b414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 -- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b4157030 msgr2=0x7f02b40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b4157030 0x7f02b40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 -- 172.21.15.46:0/3545951547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b41faeb0 con 0x7f02b40b5a80 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b2a00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02b4074950 0x7f02b414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b3400640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b4157030 0x7f02b40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02abe00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 0x7f02b4141590 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f029c009790 tx=0x7f029c0a96a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c0ad070 con 0x7f02b40b5a80 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f029c0a9c40 con 0x7f02b40b5a80 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b41fb190 con 0x7f02b40b5a80 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c003860 con 0x7f02b40b5a80 2024-09-17T03:34:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b414e600 con 0x7f02b40b5a80 2024-09-17T03:34:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.911+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b4074410 con 0x7f02b40b5a80 2024-09-17T03:34:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.915+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f029c0b0070 con 0x7f02b40b5a80 2024-09-17T03:34:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.915+0000 7f02aaa00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0290074d30 0x7f0290077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.915+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7f029c12d100 con 0x7f02b40b5a80 2024-09-17T03:34:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.915+0000 7f02b2a00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0290074d30 0x7f0290077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.915+0000 7f02b2a00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0290074d30 0x7f0290077240 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f02a40010f0 tx=0x7f02a4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:19.919+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f029c0b7070 con 0x7f02b40b5a80 2024-09-17T03:34:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.135+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f02b415d480 con 0x7f02b40b5a80 2024-09-17T03:34:20.914 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-09-17T03:34:20.911+0000 7f2b06200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:34:20.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.963+0000 7f02aaa00640 1 -- 172.21.15.46:0/3545951547 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) ==== 118+0+0 (secure 0 0 0) 0x7f029c016330 con 0x7f02b40b5a80 2024-09-17T03:34:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.963+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0290074d30 msgr2=0x7f0290077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.963+0000 7f02b9c00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0290074d30 0x7f0290077240 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f02a40010f0 tx=0x7f02a4009040 comp rx=0 tx=0).stop 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.963+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 msgr2=0x7f02b4141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.963+0000 7f02b9c00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 0x7f02b4141590 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f029c009790 tx=0x7f029c0a96a0 comp rx=0 tx=0).stop 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 shutdown_connections 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0290074d30 0x7f0290077240 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02b4157030 0x7f02b40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b40b5a80 0x7f02b4141590 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 --2- 172.21.15.46:0/3545951547 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f02b4074950 0x7f02b414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 >> 172.21.15.46:0/3545951547 conn(0x7f02b40708c0 msgr2=0x7f02b40d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 shutdown_connections 2024-09-17T03:34:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:20.967+0000 7f02b9c00640 1 -- 172.21.15.46:0/3545951547 wait complete. 2024-09-17T03:34:20.993 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:34:20.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:34:21.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 -- 172.21.15.46:0/2684373679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4157030 msgr2=0x7f25d40b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 --2- 172.21.15.46:0/2684373679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4157030 0x7f25d40b5a80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009840 tx=0x7f25c801c820 comp rx=0 tx=0).stop 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 -- 172.21.15.46:0/2684373679 shutdown_connections 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 --2- 172.21.15.46:0/2684373679 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25d4074b20 0x7f25d415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 --2- 172.21.15.46:0/2684373679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4157030 0x7f25d40b5a80 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 -- 172.21.15.46:0/2684373679 >> 172.21.15.46:0/2684373679 conn(0x7f25d40708c0 msgr2=0x7f25d4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 -- 172.21.15.46:0/2684373679 shutdown_connections 2024-09-17T03:34:21.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 -- 172.21.15.46:0/2684373679 wait complete. 2024-09-17T03:34:21.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 Processor -- start 2024-09-17T03:34:21.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.163+0000 7f25dc400640 1 -- start start 2024-09-17T03:34:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25d4157030 0x7f25d4141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 0x7f25d40b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d414f4b0 0x7f25d40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25d416a740 con 0x7f25d4160d50 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25d416a5c0 con 0x7f25d414f4b0 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f25d416a8c0 con 0x7f25d4157030 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 0x7f25d40b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 0x7f25d40b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58328/0 (socket says 172.21.15.46:58328) 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 -- 172.21.15.46:0/165864455 learned_addr learned my addr 172.21.15.46:0/165864455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9a00640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25d4157030 0x7f25d4141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25da400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d414f4b0 0x7f25d40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 -- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25d4157030 msgr2=0x7f25d4141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25d4157030 0x7f25d4141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 -- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d414f4b0 msgr2=0x7f25d40b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d414f4b0 0x7f25d40b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 -- 172.21.15.46:0/165864455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d40b2110 con 0x7f25d4160d50 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d9000640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 0x7f25d40b35e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c430 tx=0x7f25c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0011790 con 0x7f25d4160d50 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c0017070 con 0x7f25d4160d50 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0011ce0 con 0x7f25d4160d50 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d40b23f0 con 0x7f25d4160d50 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d40b29b0 con 0x7f25d4160d50 2024-09-17T03:34:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d40b1e80 con 0x7f25d4160d50 2024-09-17T03:34:21.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.167+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f25c001c460 con 0x7f25d4160d50 2024-09-17T03:34:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.171+0000 7f25d1600640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f25b4074d00 0x7f25b4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.171+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f25c0092eb0 con 0x7f25d4160d50 2024-09-17T03:34:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.171+0000 7f25d9a00640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f25b4074d00 0x7f25b4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:21.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.171+0000 7f25d9a00640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f25b4074d00 0x7f25b4077210 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f25c80af000 tx=0x7f25c801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.175+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f25c005fa80 con 0x7f25d4160d50 2024-09-17T03:34:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.427+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f25d40d2260 con 0x7f25d4160d50 2024-09-17T03:34:21.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.967+0000 7f25d1600640 1 -- 172.21.15.46:0/165864455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) ==== 122+0+0 (secure 0 0 0) 0x7f25c005f380 con 0x7f25d4160d50 2024-09-17T03:34:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f25b4074d00 msgr2=0x7f25b4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f25b4074d00 0x7f25b4077210 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f25c80af000 tx=0x7f25c801b060 comp rx=0 tx=0).stop 2024-09-17T03:34:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 msgr2=0x7f25d40b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 0x7f25d40b35e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c430 tx=0x7f25c000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 shutdown_connections 2024-09-17T03:34:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f25b4074d00 0x7f25b4077210 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d414f4b0 0x7f25d40b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25d4160d50 0x7f25d40b35e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 --2- 172.21.15.46:0/165864455 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f25d4157030 0x7f25d4141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 >> 172.21.15.46:0/165864455 conn(0x7f25d40708c0 msgr2=0x7f25d415eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 shutdown_connections 2024-09-17T03:34:21.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:21.971+0000 7f25dc400640 1 -- 172.21.15.46:0/165864455 wait complete. 2024-09-17T03:34:21.999 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:34:21.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 -- 172.21.15.46:0/4009901791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07280b54f0 msgr2=0x7f0728155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 --2- 172.21.15.46:0/4009901791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07280b54f0 0x7f0728155b50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f0714009900 tx=0x7f071401c8e0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 -- 172.21.15.46:0/4009901791 shutdown_connections 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 --2- 172.21.15.46:0/4009901791 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0728073a90 0x7f0728073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 --2- 172.21.15.46:0/4009901791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07280b54f0 0x7f0728155b50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 -- 172.21.15.46:0/4009901791 >> 172.21.15.46:0/4009901791 conn(0x7f07280b2040 msgr2=0x7f07280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 -- 172.21.15.46:0/4009901791 shutdown_connections 2024-09-17T03:34:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.159+0000 7f0730400640 1 -- 172.21.15.46:0/4009901791 wait complete. 2024-09-17T03:34:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 Processor -- start 2024-09-17T03:34:22.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 -- start start 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 0x7f07281fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728073a90 0x7f072813e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f072815a1c0 0x7f072815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0728168ee0 con 0x7f0728072100 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0728168d60 con 0x7f0728073a90 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f0730400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0728169060 con 0x7f072815a1c0 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f072da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 0x7f07281fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f072da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 0x7f07281fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58350/0 (socket says 172.21.15.46:58350) 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f072da00640 1 -- 172.21.15.46:0/197509909 learned_addr learned my addr 172.21.15.46:0/197509909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f072e400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f072815a1c0 0x7f072815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.163+0000 7f072d000640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728073a90 0x7f072813e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f072da00640 1 -- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f072815a1c0 msgr2=0x7f072815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f072da00640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f072815a1c0 0x7f072815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f072da00640 1 -- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728073a90 msgr2=0x7f072813e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f072da00640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728073a90 0x7f072813e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f072da00640 1 -- 172.21.15.46:0/197509909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07280b24f0 con 0x7f0728072100 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f072da00640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 0x7f07281fac00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c3f0 tx=0x7f071c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c017070 con 0x7f0728072100 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f071c0053c0 con 0x7f0728072100 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072815ab10 con 0x7f0728072100 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c003860 con 0x7f0728072100 2024-09-17T03:34:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07281538f0 con 0x7f0728072100 2024-09-17T03:34:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0728073ea0 con 0x7f0728072100 2024-09-17T03:34:22.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.167+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f071c01a070 con 0x7f0728072100 2024-09-17T03:34:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.171+0000 7f0725600640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0708074d30 0x7f0708077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.171+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f071c092420 con 0x7f0728072100 2024-09-17T03:34:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.171+0000 7f072d000640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0708074d30 0x7f0708077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.171+0000 7f072d000640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0708074d30 0x7f0708077240 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f07140af000 tx=0x7f071401b120 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.175+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f072815ab10 con 0x7f0728072100 2024-09-17T03:34:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.383+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07280b3740 con 0x7f0728072100 2024-09-17T03:34:22.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.383+0000 7f0725600640 1 -- 172.21.15.46:0/197509909 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5241 (secure 0 0 0) 0x7f071c05e850 con 0x7f0728072100 2024-09-17T03:34:22.385 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:22.385 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"btime":"2024-09-17T03:34:21:967289+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:34:18.364785+0000","modified":"2024-09-17T03:34:21.431279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248196":{"gid":248196,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/1168163211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1168163211},{"type":"v1","addr":"172.21.15.46:6837","nonce":1168163211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248196,"qdb_cluster":[248196]},"id":8}]} 2024-09-17T03:34:22.386 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-09-17T03:34:22.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0708074d30 msgr2=0x7f0708077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0708074d30 0x7f0708077240 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f07140af000 tx=0x7f071401b120 comp rx=0 tx=0).stop 2024-09-17T03:34:22.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 msgr2=0x7f07281fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 0x7f07281fac00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c3f0 tx=0x7f071c00c8b0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 shutdown_connections 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0708074d30 0x7f0708077240 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f072815a1c0 0x7f072815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0728073a90 0x7f072813e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 --2- 172.21.15.46:0/197509909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0728072100 0x7f07281fac00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 >> 172.21.15.46:0/197509909 conn(0x7f07280b2040 msgr2=0x7f072814eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 shutdown_connections 2024-09-17T03:34:22.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.387+0000 7f0730400640 1 -- 172.21.15.46:0/197509909 wait complete. 2024-09-17T03:34:22.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:34:22.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 -- 172.21.15.46:0/2477528159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c080b5c70 msgr2=0x7f7c0815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 --2- 172.21.15.46:0/2477528159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c080b5c70 0x7f7c0815bb60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7c080cca90 tx=0x7f7bf801c800 comp rx=0 tx=0).stop 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 -- 172.21.15.46:0/2477528159 shutdown_connections 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 --2- 172.21.15.46:0/2477528159 >> v2:172.21.15.99:3300/0 conn(0x7f7c080b4de0 0x7f7c080b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 --2- 172.21.15.46:0/2477528159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c080b5c70 0x7f7c0815bb60 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 -- 172.21.15.46:0/2477528159 >> 172.21.15.46:0/2477528159 conn(0x7f7c080b2040 msgr2=0x7f7c080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 -- 172.21.15.46:0/2477528159 shutdown_connections 2024-09-17T03:34:22.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 -- 172.21.15.46:0/2477528159 wait complete. 2024-09-17T03:34:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 Processor -- start 2024-09-17T03:34:22.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.567+0000 7f7c0d800640 1 -- start start 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c080b4de0 0x7f7c080ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c080b5c70 0x7f7c08150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 0x7f7c0815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08168970 con 0x7f7c081527b0 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081687f0 con 0x7f7c080b4de0 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7c08168af0 con 0x7f7c080b5c70 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 0x7f7c0815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c06a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c080b4de0 0x7f7c080ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 0x7f7c0815da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55896/0 (socket says 172.21.15.46:55896) 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c06a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c080b4de0 0x7f7c080ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:52510/0 (socket says 172.21.15.46:52510) 2024-09-17T03:34:22.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c080b5c70 0x7f7c08150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 -- 172.21.15.46:0/3878443996 learned_addr learned my addr 172.21.15.46:0/3878443996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 -- 172.21.15.46:0/3878443996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c080b5c70 msgr2=0x7f7c08150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c080b5c70 0x7f7c08150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 -- 172.21.15.46:0/3878443996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c080b4de0 msgr2=0x7f7c080ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c080b4de0 0x7f7c080ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 -- 172.21.15.46:0/3878443996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c080ae8b0 con 0x7f7c081527b0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c07400640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 0x7f7c0815da70 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000eab0 tx=0x7f7c00009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000ce40 con 0x7f7c081527b0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c00004c20 con 0x7f7c081527b0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c080aeb90 con 0x7f7c081527b0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00005020 con 0x7f7c081527b0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c080ab5c0 con 0x7f7c081527b0 2024-09-17T03:34:22.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.571+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c080ae620 con 0x7f7c081527b0 2024-09-17T03:34:22.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.575+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f7c000051e0 con 0x7f7c081527b0 2024-09-17T03:34:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.575+0000 7f7bfea00640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7be4074d00 0x7f7be4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.575+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f7c00093e30 con 0x7f7c081527b0 2024-09-17T03:34:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.575+0000 7f7c06a00640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7be4074d00 0x7f7be4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.575+0000 7f7c06a00640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7be4074d00 0x7f7be4077210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80af000 tx=0x7f7bf80a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:22.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.579+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7c000602a0 con 0x7f7c081527b0 2024-09-17T03:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.755+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c080ae370 con 0x7f7c081527b0 2024-09-17T03:34:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.755+0000 7f7bfea00640 1 -- 172.21.15.46:0/3878443996 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16695 (secure 0 0 0) 0x7f7c00064cd0 con 0x7f7c081527b0 2024-09-17T03:34:22.758 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:22.758 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":114,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:34:19.951710+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:34:16.113350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T03:34:17.334380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7be4074d00 msgr2=0x7f7be4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7be4074d00 0x7f7be4077210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80af000 tx=0x7f7bf80a7040 comp rx=0 tx=0).stop 2024-09-17T03:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 msgr2=0x7f7c0815da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 0x7f7c0815da70 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000eab0 tx=0x7f7c00009f70 comp rx=0 tx=0).stop 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 shutdown_connections 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7be4074d00 0x7f7be4077210 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c081527b0 0x7f7c0815da70 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7c080b5c70 0x7f7c08150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 --2- 172.21.15.46:0/3878443996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c080b4de0 0x7f7c080ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 >> 172.21.15.46:0/3878443996 conn(0x7f7c080b2040 msgr2=0x7f7c08159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 shutdown_connections 2024-09-17T03:34:22.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.759+0000 7f7c0d800640 1 -- 172.21.15.46:0/3878443996 wait complete. 2024-09-17T03:34:22.781 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:34:22.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:34:22.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 -- 172.21.15.46:0/2928204376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a08072100 msgr2=0x7f5a08155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 --2- 172.21.15.46:0/2928204376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a08072100 0x7f5a08155b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009870 tx=0x7f59f801c860 comp rx=0 tx=0).stop 2024-09-17T03:34:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 -- 172.21.15.46:0/2928204376 shutdown_connections 2024-09-17T03:34:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 --2- 172.21.15.46:0/2928204376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a08072100 0x7f5a08155b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 -- 172.21.15.46:0/2928204376 >> 172.21.15.46:0/2928204376 conn(0x7f5a080b2040 msgr2=0x7f5a080b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 -- 172.21.15.46:0/2928204376 shutdown_connections 2024-09-17T03:34:22.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 -- 172.21.15.46:0/2928204376 wait complete. 2024-09-17T03:34:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 Processor -- start 2024-09-17T03:34:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.939+0000 7f5a0f400640 1 -- start start 2024-09-17T03:34:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a08072100 0x7f5a08166b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 0x7f5a081f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a080b4de0 0x7f5a080b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a08169f40 con 0x7f5a08073a90 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a08169dc0 con 0x7f5a08072100 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0816a0c0 con 0x7f5a080b4de0 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 0x7f5a081f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 0x7f5a081f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55904/0 (socket says 172.21.15.46:55904) 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 -- 172.21.15.46:0/3535416062 learned_addr learned my addr 172.21.15.46:0/3535416062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0ca00640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a08072100 0x7f5a08166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0d400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a080b4de0 0x7f5a080b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 -- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a080b4de0 msgr2=0x7f5a080b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a080b4de0 0x7f5a080b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 -- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a08072100 msgr2=0x7f5a08166b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a08072100 0x7f5a08166b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 -- 172.21.15.46:0/3535416062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a081f96a0 con 0x7f5a08073a90 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0d400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a080b4de0 0x7f5a080b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f59ffe00640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 0x7f5a081f8bb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c430 tx=0x7f59f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0017070 con 0x7f5a08073a90 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f00053e0 con 0x7f5a08073a90 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a081f9980 con 0x7f5a08073a90 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0003860 con 0x7f5a08073a90 2024-09-17T03:34:22.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a08154220 con 0x7f5a08073a90 2024-09-17T03:34:22.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a08155b50 con 0x7f5a08073a90 2024-09-17T03:34:22.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.943+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f59f001a070 con 0x7f5a08073a90 2024-09-17T03:34:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.947+0000 7f5a05600640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f59e4074d00 0x7f59e4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.947+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f59f0092420 con 0x7f5a08073a90 2024-09-17T03:34:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.947+0000 7f5a0ca00640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f59e4074d00 0x7f59e4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.947+0000 7f5a0ca00640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f59e4074d00 0x7f59e4077210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f59f80af000 tx=0x7f59f801b0a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:22.951+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f59f005e8e0 con 0x7f5a08073a90 2024-09-17T03:34:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.123+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a081670c0 con 0x7f59e4074d00 2024-09-17T03:34:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.131+0000 7f5a05600640 1 -- 172.21.15.46:0/3535416062 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241556 (secure 0 0 0) 0x7f5a081670c0 con 0x7f59e4074d00 2024-09-17T03:34:23.134 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:34:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f59e4074d00 msgr2=0x7f59e4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f59e4074d00 0x7f59e4077210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f59f80af000 tx=0x7f59f801b0a0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 msgr2=0x7f5a081f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 0x7f5a081f8bb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f59f000c430 tx=0x7f59f000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 shutdown_connections 2024-09-17T03:34:23.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f59e4074d00 0x7f59e4077210 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a080b4de0 0x7f5a080b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a08073a90 0x7f5a081f8bb0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 --2- 172.21.15.46:0/3535416062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a08072100 0x7f5a08166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 >> 172.21.15.46:0/3535416062 conn(0x7f5a080b2040 msgr2=0x7f5a08158130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 shutdown_connections 2024-09-17T03:34:23.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.135+0000 7f5a0f400640 1 -- 172.21.15.46:0/3535416062 wait complete. 2024-09-17T03:34:23.156 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-17T03:34:21.536120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":76,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1049388,"kb_used_data":11828,"kb_used_omap":12,"kb_used_meta":1037363,"kb_avail":753925332,"statfs":{"total":773094113280,"available":772019539968,"internally_reserved":0,"allocated":12111872,"data_stored":5897567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1062260272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.607337"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.494988+0000","last_change":"2024-09-17T03:34:18.967638+0000","last_active":"2024-09-17T03:34:20.494988+0000","last_peered":"2024-09-17T03:34:20.494988+0000","last_clean":"2024-09-17T03:34:20.494988+0000","last_became_active":"2024-09-17T03:34:18.966926+0000","last_became_peered":"2024-09-17T03:34:18.966926+0000","last_unstale":"2024-09-17T03:34:20.494988+0000","last_undegraded":"2024-09-17T03:34:20.494988+0000","last_fullsized":"2024-09-17T03:34:20.494988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:15.647132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495510+0000","last_change":"2024-09-17T03:34:18.973403+0000","last_active":"2024-09-17T03:34:20.495510+0000","last_peered":"2024-09-17T03:34:20.495510+0000","last_clean":"2024-09-17T03:34:20.495510+0000","last_became_active":"2024-09-17T03:34:18.972784+0000","last_became_peered":"2024-09-17T03:34:18.972784+0000","last_unstale":"2024-09-17T03:34:20.495510+0000","last_undegraded":"2024-09-17T03:34:20.495510+0000","last_fullsized":"2024-09-17T03:34:20.495510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:45.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496664+0000","last_change":"2024-09-17T03:34:17.967902+0000","last_active":"2024-09-17T03:34:20.496664+0000","last_peered":"2024-09-17T03:34:20.496664+0000","last_clean":"2024-09-17T03:34:20.496664+0000","last_became_active":"2024-09-17T03:34:17.967529+0000","last_became_peered":"2024-09-17T03:34:17.967529+0000","last_unstale":"2024-09-17T03:34:20.496664+0000","last_undegraded":"2024-09-17T03:34:20.496664+0000","last_fullsized":"2024-09-17T03:34:20.496664+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:38.117876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.949965+0000","last_change":"2024-09-17T03:34:16.949965+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.949965+0000","last_undegraded":"2024-09-17T03:34:16.949965+0000","last_fullsized":"2024-09-17T03:34:16.949965+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.943480+0000","last_change":"2024-09-17T03:34:16.943480+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.943480+0000","last_undegraded":"2024-09-17T03:34:16.943480+0000","last_fullsized":"2024-09-17T03:34:16.943480+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942131+0000","last_change":"2024-09-17T03:34:16.942131+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942131+0000","last_undegraded":"2024-09-17T03:34:16.942131+0000","last_fullsized":"2024-09-17T03:34:16.942131+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495679+0000","last_change":"2024-09-17T03:34:17.968406+0000","last_active":"2024-09-17T03:34:20.495679+0000","last_peered":"2024-09-17T03:34:20.495679+0000","last_clean":"2024-09-17T03:34:20.495679+0000","last_became_active":"2024-09-17T03:34:17.968065+0000","last_became_peered":"2024-09-17T03:34:17.968065+0000","last_unstale":"2024-09-17T03:34:20.495679+0000","last_undegraded":"2024-09-17T03:34:20.495679+0000","last_fullsized":"2024-09-17T03:34:20.495679+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:58.697142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.935915+0000","last_change":"2024-09-17T03:34:16.935915+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.935915+0000","last_undegraded":"2024-09-17T03:34:16.935915+0000","last_fullsized":"2024-09-17T03:34:16.935915+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.948195+0000","last_change":"2024-09-17T03:34:16.948195+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.948195+0000","last_undegraded":"2024-09-17T03:34:16.948195+0000","last_fullsized":"2024-09-17T03:34:16.948195+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.494949+0000","last_change":"2024-09-17T03:34:18.968014+0000","last_active":"2024-09-17T03:34:20.494949+0000","last_peered":"2024-09-17T03:34:20.494949+0000","last_clean":"2024-09-17T03:34:20.494949+0000","last_became_active":"2024-09-17T03:34:18.967294+0000","last_became_peered":"2024-09-17T03:34:18.967294+0000","last_unstale":"2024-09-17T03:34:20.494949+0000","last_undegraded":"2024-09-17T03:34:20.494949+0000","last_fullsized":"2024-09-17T03:34:20.494949+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:19.020021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.934052+0000","last_change":"2024-09-17T03:34:16.934052+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.934052+0000","last_undegraded":"2024-09-17T03:34:16.934052+0000","last_fullsized":"2024-09-17T03:34:16.934052+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496762+0000","last_change":"2024-09-17T03:34:18.977053+0000","last_active":"2024-09-17T03:34:20.496762+0000","last_peered":"2024-09-17T03:34:20.496762+0000","last_clean":"2024-09-17T03:34:20.496762+0000","last_became_active":"2024-09-17T03:34:18.975249+0000","last_became_peered":"2024-09-17T03:34:18.975249+0000","last_unstale":"2024-09-17T03:34:20.496762+0000","last_undegraded":"2024-09-17T03:34:20.496762+0000","last_fullsized":"2024-09-17T03:34:20.496762+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:04.163445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942218+0000","last_change":"2024-09-17T03:34:16.942218+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942218+0000","last_undegraded":"2024-09-17T03:34:16.942218+0000","last_fullsized":"2024-09-17T03:34:16.942218+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.935794+0000","last_change":"2024-09-17T03:34:16.935794+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.935794+0000","last_undegraded":"2024-09-17T03:34:16.935794+0000","last_fullsized":"2024-09-17T03:34:16.935794+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.933724+0000","last_change":"2024-09-17T03:34:16.933724+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.933724+0000","last_undegraded":"2024-09-17T03:34:16.933724+0000","last_fullsized":"2024-09-17T03:34:16.933724+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496215+0000","last_change":"2024-09-17T03:34:18.972637+0000","last_active":"2024-09-17T03:34:20.496215+0000","last_peered":"2024-09-17T03:34:20.496215+0000","last_clean":"2024-09-17T03:34:20.496215+0000","last_became_active":"2024-09-17T03:34:18.972274+0000","last_became_peered":"2024-09-17T03:34:18.972274+0000","last_unstale":"2024-09-17T03:34:20.496215+0000","last_undegraded":"2024-09-17T03:34:20.496215+0000","last_fullsized":"2024-09-17T03:34:20.496215+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:59.813508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.934515+0000","last_change":"2024-09-17T03:34:16.934515+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.934515+0000","last_undegraded":"2024-09-17T03:34:16.934515+0000","last_fullsized":"2024-09-17T03:34:16.934515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.938912+0000","last_change":"2024-09-17T03:34:16.938912+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.938912+0000","last_undegraded":"2024-09-17T03:34:16.938912+0000","last_fullsized":"2024-09-17T03:34:16.938912+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.943119+0000","last_change":"2024-09-17T03:34:16.943119+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.943119+0000","last_undegraded":"2024-09-17T03:34:16.943119+0000","last_fullsized":"2024-09-17T03:34:16.943119+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942035+0000","last_change":"2024-09-17T03:34:16.942035+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942035+0000","last_undegraded":"2024-09-17T03:34:16.942035+0000","last_fullsized":"2024-09-17T03:34:16.942035+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.934038+0000","last_change":"2024-09-17T03:34:16.934038+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.934038+0000","last_undegraded":"2024-09-17T03:34:16.934038+0000","last_fullsized":"2024-09-17T03:34:16.934038+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942636+0000","last_change":"2024-09-17T03:34:16.942636+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942636+0000","last_undegraded":"2024-09-17T03:34:16.942636+0000","last_fullsized":"2024-09-17T03:34:16.942636+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.943077+0000","last_change":"2024-09-17T03:34:16.943077+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.943077+0000","last_undegraded":"2024-09-17T03:34:16.943077+0000","last_fullsized":"2024-09-17T03:34:16.943077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495931+0000","last_change":"2024-09-17T03:34:17.977152+0000","last_active":"2024-09-17T03:34:20.495931+0000","last_peered":"2024-09-17T03:34:20.495931+0000","last_clean":"2024-09-17T03:34:20.495931+0000","last_became_active":"2024-09-17T03:34:17.976753+0000","last_became_peered":"2024-09-17T03:34:17.976753+0000","last_unstale":"2024-09-17T03:34:20.495931+0000","last_undegraded":"2024-09-17T03:34:20.495931+0000","last_fullsized":"2024-09-17T03:34:20.495931+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:44.468930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.938342+0000","last_change":"2024-09-17T03:34:16.938342+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.938342+0000","last_undegraded":"2024-09-17T03:34:16.938342+0000","last_fullsized":"2024-09-17T03:34:16.938342+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942938+0000","last_change":"2024-09-17T03:34:16.942938+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942938+0000","last_undegraded":"2024-09-17T03:34:16.942938+0000","last_fullsized":"2024-09-17T03:34:16.942938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.937654+0000","last_change":"2024-09-17T03:34:16.937654+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.937654+0000","last_undegraded":"2024-09-17T03:34:16.937654+0000","last_fullsized":"2024-09-17T03:34:16.937654+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.933575+0000","last_change":"2024-09-17T03:34:16.933575+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.933575+0000","last_undegraded":"2024-09-17T03:34:16.933575+0000","last_fullsized":"2024-09-17T03:34:16.933575+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.936442+0000","last_change":"2024-09-17T03:34:16.936442+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.936442+0000","last_undegraded":"2024-09-17T03:34:16.936442+0000","last_fullsized":"2024-09-17T03:34:16.936442+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942980+0000","last_change":"2024-09-17T03:34:16.942980+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942980+0000","last_undegraded":"2024-09-17T03:34:16.942980+0000","last_fullsized":"2024-09-17T03:34:16.942980+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.932850+0000","last_change":"2024-09-17T03:34:16.932850+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.932850+0000","last_undegraded":"2024-09-17T03:34:16.932850+0000","last_fullsized":"2024-09-17T03:34:16.932850+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.936993+0000","last_change":"2024-09-17T03:34:16.936993+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.936993+0000","last_undegraded":"2024-09-17T03:34:16.936993+0000","last_fullsized":"2024-09-17T03:34:16.936993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495802+0000","last_change":"2024-09-17T03:34:17.971285+0000","last_active":"2024-09-17T03:34:20.495802+0000","last_peered":"2024-09-17T03:34:20.495802+0000","last_clean":"2024-09-17T03:34:20.495802+0000","last_became_active":"2024-09-17T03:34:17.970905+0000","last_became_peered":"2024-09-17T03:34:17.970905+0000","last_unstale":"2024-09-17T03:34:20.495802+0000","last_undegraded":"2024-09-17T03:34:20.495802+0000","last_fullsized":"2024-09-17T03:34:20.495802+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:49.385583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.948327+0000","last_change":"2024-09-17T03:34:16.948327+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.948327+0000","last_undegraded":"2024-09-17T03:34:16.948327+0000","last_fullsized":"2024-09-17T03:34:16.948327+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.942123+0000","last_change":"2024-09-17T03:34:16.942123+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.942123+0000","last_undegraded":"2024-09-17T03:34:16.942123+0000","last_fullsized":"2024-09-17T03:34:16.942123+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T03:34:16.932838+0000","last_change":"2024-09-17T03:34:16.932838+0000","last_active":"2024-09-17T03:34:16.925594+0000","last_peered":"2024-09-17T03:34:16.925594+0000","last_clean":"2024-09-17T03:34:16.925594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:16.932838+0000","last_undegraded":"2024-09-17T03:34:16.932838+0000","last_fullsized":"2024-09-17T03:34:16.932838+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:34:17.928553+0000","last_change":"2024-09-17T03:34:17.928553+0000","last_active":"2024-09-17T03:34:17.928553+0000","last_peered":"2024-09-17T03:34:17.928553+0000","last_clean":"2024-09-17T03:34:17.928553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:34:17.928553+0000","last_undegraded":"2024-09-17T03:34:17.928553+0000","last_fullsized":"2024-09-17T03:34:17.928553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495092+0000","last_change":"2024-09-17T03:34:17.977176+0000","last_active":"2024-09-17T03:34:20.495092+0000","last_peered":"2024-09-17T03:34:20.495092+0000","last_clean":"2024-09-17T03:34:20.495092+0000","last_became_active":"2024-09-17T03:34:17.976734+0000","last_became_peered":"2024-09-17T03:34:17.976734+0000","last_unstale":"2024-09-17T03:34:20.495092+0000","last_undegraded":"2024-09-17T03:34:20.495092+0000","last_fullsized":"2024-09-17T03:34:20.495092+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:40.285491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":275,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T03:34:16.950519+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:34:16.950519+0000","last_peered":"2024-09-17T03:34:16.950519+0000","last_clean":"2024-09-17T03:34:16.950519+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:34:16.950519+0000","last_undegraded":"2024-09-17T03:34:16.950519+0000","last_fullsized":"2024-09-17T03:34:16.950519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127388,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244452,"statfs":{"total":96636764160,"available":96506318848,"internally_reserved":0,"allocated":1904640,"data_stored":1156671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140744,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231096,"statfs":{"total":96636764160,"available":96492642304,"internally_reserved":0,"allocated":1359872,"data_stored":586799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149072,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222768,"statfs":{"total":96636764160,"available":96484114432,"internally_reserved":0,"allocated":1359872,"data_stored":586799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509604,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118544,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253296,"statfs":{"total":96636764160,"available":96515375104,"internally_reserved":0,"allocated":1372160,"data_stored":592644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140756,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231084,"statfs":{"total":96636764160,"available":96492630016,"internally_reserved":0,"allocated":1372160,"data_stored":592644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509603,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123284,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248556,"statfs":{"total":96636764160,"available":96510521344,"internally_reserved":0,"allocated":1966080,"data_stored":1183012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542308,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86640,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285200,"statfs":{"total":96636764160,"available":96548044800,"internally_reserved":0,"allocated":1404928,"data_stored":606354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":162960,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208880,"statfs":{"total":96636764160,"available":96469893120,"internally_reserved":0,"allocated":1372160,"data_stored":592644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:34:23.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 -- 172.21.15.46:0/3864642878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 msgr2=0x7f2dc415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3864642878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc415bb60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009840 tx=0x7f2db801c760 comp rx=0 tx=0).stop 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 -- 172.21.15.46:0/3864642878 shutdown_connections 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3864642878 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2dc41527b0 0x7f2dc4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3864642878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc415bb60 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 -- 172.21.15.46:0/3864642878 >> 172.21.15.46:0/3864642878 conn(0x7f2dc40b2040 msgr2=0x7f2dc40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 -- 172.21.15.46:0/3864642878 shutdown_connections 2024-09-17T03:34:23.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.355+0000 7f2dcc000640 1 -- 172.21.15.46:0/3864642878 wait complete. 2024-09-17T03:34:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 Processor -- start 2024-09-17T03:34:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 -- start start 2024-09-17T03:34:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2dc40b4de0 0x7f2dc41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc41f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc41527b0 0x7f2dc40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4168c50 con 0x7f2dc40b5c70 2024-09-17T03:34:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4168ad0 con 0x7f2dc41527b0 2024-09-17T03:34:23.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc4168dd0 con 0x7f2dc40b4de0 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc41f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc41f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55924/0 (socket says 172.21.15.46:55924) 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 -- 172.21.15.46:0/3721835101 learned_addr learned my addr 172.21.15.46:0/3721835101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc9600640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2dc40b4de0 0x7f2dc41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dca000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc41527b0 0x7f2dc40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 -- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2dc40b4de0 msgr2=0x7f2dc41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2dc40b4de0 0x7f2dc41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 -- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc41527b0 msgr2=0x7f2dc40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc41527b0 0x7f2dc40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 -- 172.21.15.46:0/3721835101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc41f8b70 con 0x7f2dc40b5c70 2024-09-17T03:34:23.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc8c00640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc41f8630 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c970 tx=0x7f2db000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0014070 con 0x7f2dc40b5c70 2024-09-17T03:34:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2db00077a0 con 0x7f2dc40b5c70 2024-09-17T03:34:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc4154050 con 0x7f2dc40b5c70 2024-09-17T03:34:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.359+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0007bf0 con 0x7f2dc40b5c70 2024-09-17T03:34:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.363+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc4154610 con 0x7f2dc40b5c70 2024-09-17T03:34:23.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.363+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc415bb60 con 0x7f2dc40b5c70 2024-09-17T03:34:23.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.363+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f2db0017070 con 0x7f2dc40b5c70 2024-09-17T03:34:23.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.363+0000 7f2dc1600640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2da4074d00 0x7f2da4077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.363+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f2db00932c0 con 0x7f2dc40b5c70 2024-09-17T03:34:23.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.363+0000 7f2dc9600640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2da4074d00 0x7f2da4077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.367+0000 7f2dc9600640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2da4074d00 0x7f2da4077210 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2db80af000 tx=0x7f2db801afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:23.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.367+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2db005f780 con 0x7f2dc40b5c70 2024-09-17T03:34:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.543+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2dc406b540 con 0x7f2da4074d00 2024-09-17T03:34:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.547+0000 7f2dc1600640 1 -- 172.21.15.46:0/3721835101 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252494 (secure 0 0 0) 0x7f2db80041c0 con 0x7f2da4074d00 2024-09-17T03:34:23.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:23.553 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2da4074d00 msgr2=0x7f2da4077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2da4074d00 0x7f2da4077210 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2db80af000 tx=0x7f2db801afa0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 msgr2=0x7f2dc41f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc41f8630 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c970 tx=0x7f2db000ce30 comp rx=0 tx=0).stop 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 shutdown_connections 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2da4074d00 0x7f2da4077210 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2dc41527b0 0x7f2dc40cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc40b5c70 0x7f2dc41f8630 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 --2- 172.21.15.46:0/3721835101 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2dc40b4de0 0x7f2dc41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 >> 172.21.15.46:0/3721835101 conn(0x7f2dc40b2040 msgr2=0x7f2dc40b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 shutdown_connections 2024-09-17T03:34:23.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.551+0000 7f2dcc000640 1 -- 172.21.15.46:0/3721835101 wait complete. 2024-09-17T03:34:23.573 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-17T03:34:23.536521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1049784,"kb_used_data":12220,"kb_used_omap":12,"kb_used_meta":1037363,"kb_avail":753924936,"statfs":{"total":773094113280,"available":772019134464,"internally_reserved":0,"allocated":12513280,"data_stored":6075852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1062260272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001442"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987560+0000","last_change":"2024-09-17T03:34:18.966035+0000","last_active":"2024-09-17T03:34:19.987560+0000","last_peered":"2024-09-17T03:34:19.987560+0000","last_clean":"2024-09-17T03:34:19.987560+0000","last_became_active":"2024-09-17T03:34:18.963772+0000","last_became_peered":"2024-09-17T03:34:18.963772+0000","last_unstale":"2024-09-17T03:34:19.987560+0000","last_undegraded":"2024-09-17T03:34:19.987560+0000","last_fullsized":"2024-09-17T03:34:19.987560+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:40.383278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.970469+0000","last_change":"2024-09-17T03:34:18.988453+0000","last_active":"2024-09-17T03:34:19.970469+0000","last_peered":"2024-09-17T03:34:19.970469+0000","last_clean":"2024-09-17T03:34:19.970469+0000","last_became_active":"2024-09-17T03:34:18.987926+0000","last_became_peered":"2024-09-17T03:34:18.987926+0000","last_unstale":"2024-09-17T03:34:19.970469+0000","last_undegraded":"2024-09-17T03:34:19.970469+0000","last_fullsized":"2024-09-17T03:34:19.970469+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:05.546628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.497278+0000","last_change":"2024-09-17T03:34:18.979765+0000","last_active":"2024-09-17T03:34:20.497278+0000","last_peered":"2024-09-17T03:34:20.497278+0000","last_clean":"2024-09-17T03:34:20.497278+0000","last_became_active":"2024-09-17T03:34:18.979449+0000","last_became_peered":"2024-09-17T03:34:18.979449+0000","last_unstale":"2024-09-17T03:34:20.497278+0000","last_undegraded":"2024-09-17T03:34:20.497278+0000","last_fullsized":"2024-09-17T03:34:20.497278+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:49.006748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986900+0000","last_change":"2024-09-17T03:34:18.956256+0000","last_active":"2024-09-17T03:34:19.986900+0000","last_peered":"2024-09-17T03:34:19.986900+0000","last_clean":"2024-09-17T03:34:19.986900+0000","last_became_active":"2024-09-17T03:34:18.955546+0000","last_became_peered":"2024-09-17T03:34:18.955546+0000","last_unstale":"2024-09-17T03:34:19.986900+0000","last_undegraded":"2024-09-17T03:34:19.986900+0000","last_fullsized":"2024-09-17T03:34:19.986900+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:18.701540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487316+0000","last_change":"2024-09-17T03:34:18.964799+0000","last_active":"2024-09-17T03:34:20.487316+0000","last_peered":"2024-09-17T03:34:20.487316+0000","last_clean":"2024-09-17T03:34:20.487316+0000","last_became_active":"2024-09-17T03:34:18.958602+0000","last_became_peered":"2024-09-17T03:34:18.958602+0000","last_unstale":"2024-09-17T03:34:20.487316+0000","last_undegraded":"2024-09-17T03:34:20.487316+0000","last_fullsized":"2024-09-17T03:34:20.487316+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:04.600883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.960535+0000","last_change":"2024-09-17T03:34:18.985817+0000","last_active":"2024-09-17T03:34:19.960535+0000","last_peered":"2024-09-17T03:34:19.960535+0000","last_clean":"2024-09-17T03:34:19.960535+0000","last_became_active":"2024-09-17T03:34:18.985513+0000","last_became_peered":"2024-09-17T03:34:18.985513+0000","last_unstale":"2024-09-17T03:34:19.960535+0000","last_undegraded":"2024-09-17T03:34:19.960535+0000","last_fullsized":"2024-09-17T03:34:19.960535+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:49.726183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.494988+0000","last_change":"2024-09-17T03:34:18.967638+0000","last_active":"2024-09-17T03:34:20.494988+0000","last_peered":"2024-09-17T03:34:20.494988+0000","last_clean":"2024-09-17T03:34:20.494988+0000","last_became_active":"2024-09-17T03:34:18.966926+0000","last_became_peered":"2024-09-17T03:34:18.966926+0000","last_unstale":"2024-09-17T03:34:20.494988+0000","last_undegraded":"2024-09-17T03:34:20.494988+0000","last_fullsized":"2024-09-17T03:34:20.494988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:15.647132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496398+0000","last_change":"2024-09-17T03:34:18.982993+0000","last_active":"2024-09-17T03:34:20.496398+0000","last_peered":"2024-09-17T03:34:20.496398+0000","last_clean":"2024-09-17T03:34:20.496398+0000","last_became_active":"2024-09-17T03:34:18.982127+0000","last_became_peered":"2024-09-17T03:34:18.982127+0000","last_unstale":"2024-09-17T03:34:20.496398+0000","last_undegraded":"2024-09-17T03:34:20.496398+0000","last_fullsized":"2024-09-17T03:34:20.496398+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:23.977087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987576+0000","last_change":"2024-09-17T03:34:18.958676+0000","last_active":"2024-09-17T03:34:19.987576+0000","last_peered":"2024-09-17T03:34:19.987576+0000","last_clean":"2024-09-17T03:34:19.987576+0000","last_became_active":"2024-09-17T03:34:18.958337+0000","last_became_peered":"2024-09-17T03:34:18.958337+0000","last_unstale":"2024-09-17T03:34:19.987576+0000","last_undegraded":"2024-09-17T03:34:19.987576+0000","last_fullsized":"2024-09-17T03:34:19.987576+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:39.319051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987144+0000","last_change":"2024-09-17T03:34:18.959889+0000","last_active":"2024-09-17T03:34:19.987144+0000","last_peered":"2024-09-17T03:34:19.987144+0000","last_clean":"2024-09-17T03:34:19.987144+0000","last_became_active":"2024-09-17T03:34:18.959561+0000","last_became_peered":"2024-09-17T03:34:18.959561+0000","last_unstale":"2024-09-17T03:34:19.987144+0000","last_undegraded":"2024-09-17T03:34:19.987144+0000","last_fullsized":"2024-09-17T03:34:19.987144+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:43.411054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.970318+0000","last_change":"2024-09-17T03:34:18.963753+0000","last_active":"2024-09-17T03:34:19.970318+0000","last_peered":"2024-09-17T03:34:19.970318+0000","last_clean":"2024-09-17T03:34:19.970318+0000","last_became_active":"2024-09-17T03:34:18.963387+0000","last_became_peered":"2024-09-17T03:34:18.963387+0000","last_unstale":"2024-09-17T03:34:19.970318+0000","last_undegraded":"2024-09-17T03:34:19.970318+0000","last_fullsized":"2024-09-17T03:34:19.970318+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:10.918863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987140+0000","last_change":"2024-09-17T03:34:18.956844+0000","last_active":"2024-09-17T03:34:19.987140+0000","last_peered":"2024-09-17T03:34:19.987140+0000","last_clean":"2024-09-17T03:34:19.987140+0000","last_became_active":"2024-09-17T03:34:18.956536+0000","last_became_peered":"2024-09-17T03:34:18.956536+0000","last_unstale":"2024-09-17T03:34:19.987140+0000","last_undegraded":"2024-09-17T03:34:19.987140+0000","last_fullsized":"2024-09-17T03:34:19.987140+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:53.783579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496866+0000","last_change":"2024-09-17T03:34:19.001823+0000","last_active":"2024-09-17T03:34:20.496866+0000","last_peered":"2024-09-17T03:34:20.496866+0000","last_clean":"2024-09-17T03:34:20.496866+0000","last_became_active":"2024-09-17T03:34:19.001461+0000","last_became_peered":"2024-09-17T03:34:19.001461+0000","last_unstale":"2024-09-17T03:34:20.496866+0000","last_undegraded":"2024-09-17T03:34:20.496866+0000","last_fullsized":"2024-09-17T03:34:20.496866+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:31.303366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496959+0000","last_change":"2024-09-17T03:34:18.980913+0000","last_active":"2024-09-17T03:34:20.496959+0000","last_peered":"2024-09-17T03:34:20.496959+0000","last_clean":"2024-09-17T03:34:20.496959+0000","last_became_active":"2024-09-17T03:34:18.980578+0000","last_became_peered":"2024-09-17T03:34:18.980578+0000","last_unstale":"2024-09-17T03:34:20.496959+0000","last_undegraded":"2024-09-17T03:34:20.496959+0000","last_fullsized":"2024-09-17T03:34:20.496959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:20.110611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.960484+0000","last_change":"2024-09-17T03:34:18.984253+0000","last_active":"2024-09-17T03:34:19.960484+0000","last_peered":"2024-09-17T03:34:19.960484+0000","last_clean":"2024-09-17T03:34:19.960484+0000","last_became_active":"2024-09-17T03:34:18.983931+0000","last_became_peered":"2024-09-17T03:34:18.983931+0000","last_unstale":"2024-09-17T03:34:19.960484+0000","last_undegraded":"2024-09-17T03:34:19.960484+0000","last_fullsized":"2024-09-17T03:34:19.960484+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:59.500357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986337+0000","last_change":"2024-09-17T03:34:18.964923+0000","last_active":"2024-09-17T03:34:19.986337+0000","last_peered":"2024-09-17T03:34:19.986337+0000","last_clean":"2024-09-17T03:34:19.986337+0000","last_became_active":"2024-09-17T03:34:18.964457+0000","last_became_peered":"2024-09-17T03:34:18.964457+0000","last_unstale":"2024-09-17T03:34:19.986337+0000","last_undegraded":"2024-09-17T03:34:19.986337+0000","last_fullsized":"2024-09-17T03:34:19.986337+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:11.691545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.970678+0000","last_change":"2024-09-17T03:34:18.973797+0000","last_active":"2024-09-17T03:34:19.970678+0000","last_peered":"2024-09-17T03:34:19.970678+0000","last_clean":"2024-09-17T03:34:19.970678+0000","last_became_active":"2024-09-17T03:34:18.973102+0000","last_became_peered":"2024-09-17T03:34:18.973102+0000","last_unstale":"2024-09-17T03:34:19.970678+0000","last_undegraded":"2024-09-17T03:34:19.970678+0000","last_fullsized":"2024-09-17T03:34:19.970678+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:51.159254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495233+0000","last_change":"2024-09-17T03:34:18.969265+0000","last_active":"2024-09-17T03:34:20.495233+0000","last_peered":"2024-09-17T03:34:20.495233+0000","last_clean":"2024-09-17T03:34:20.495233+0000","last_became_active":"2024-09-17T03:34:18.968898+0000","last_became_peered":"2024-09-17T03:34:18.968898+0000","last_unstale":"2024-09-17T03:34:20.495233+0000","last_undegraded":"2024-09-17T03:34:20.495233+0000","last_fullsized":"2024-09-17T03:34:20.495233+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:10.918863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.497439+0000","last_change":"2024-09-17T03:34:18.990289+0000","last_active":"2024-09-17T03:34:20.497439+0000","last_peered":"2024-09-17T03:34:20.497439+0000","last_clean":"2024-09-17T03:34:20.497439+0000","last_became_active":"2024-09-17T03:34:18.989977+0000","last_became_peered":"2024-09-17T03:34:18.989977+0000","last_unstale":"2024-09-17T03:34:20.497439+0000","last_undegraded":"2024-09-17T03:34:20.497439+0000","last_fullsized":"2024-09-17T03:34:20.497439+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:51.466818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.970272+0000","last_change":"2024-09-17T03:34:18.970379+0000","last_active":"2024-09-17T03:34:19.970272+0000","last_peered":"2024-09-17T03:34:19.970272+0000","last_clean":"2024-09-17T03:34:19.970272+0000","last_became_active":"2024-09-17T03:34:18.970085+0000","last_became_peered":"2024-09-17T03:34:18.970085+0000","last_unstale":"2024-09-17T03:34:19.970272+0000","last_undegraded":"2024-09-17T03:34:19.970272+0000","last_fullsized":"2024-09-17T03:34:19.970272+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:23:39.989977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969981+0000","last_change":"2024-09-17T03:34:18.963315+0000","last_active":"2024-09-17T03:34:19.969981+0000","last_peered":"2024-09-17T03:34:19.969981+0000","last_clean":"2024-09-17T03:34:19.969981+0000","last_became_active":"2024-09-17T03:34:18.962966+0000","last_became_peered":"2024-09-17T03:34:18.962966+0000","last_unstale":"2024-09-17T03:34:19.969981+0000","last_undegraded":"2024-09-17T03:34:19.969981+0000","last_fullsized":"2024-09-17T03:34:19.969981+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:12.779078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961993+0000","last_change":"2024-09-17T03:34:18.958517+0000","last_active":"2024-09-17T03:34:19.961993+0000","last_peered":"2024-09-17T03:34:19.961993+0000","last_clean":"2024-09-17T03:34:19.961993+0000","last_became_active":"2024-09-17T03:34:18.958080+0000","last_became_peered":"2024-09-17T03:34:18.958080+0000","last_unstale":"2024-09-17T03:34:19.961993+0000","last_undegraded":"2024-09-17T03:34:19.961993+0000","last_fullsized":"2024-09-17T03:34:19.961993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:23.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969734+0000","last_change":"2024-09-17T03:34:18.972348+0000","last_active":"2024-09-17T03:34:19.969734+0000","last_peered":"2024-09-17T03:34:19.969734+0000","last_clean":"2024-09-17T03:34:19.969734+0000","last_became_active":"2024-09-17T03:34:18.972019+0000","last_became_peered":"2024-09-17T03:34:18.972019+0000","last_unstale":"2024-09-17T03:34:19.969734+0000","last_undegraded":"2024-09-17T03:34:19.969734+0000","last_fullsized":"2024-09-17T03:34:19.969734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:04.957949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961275+0000","last_change":"2024-09-17T03:34:18.969269+0000","last_active":"2024-09-17T03:34:19.961275+0000","last_peered":"2024-09-17T03:34:19.961275+0000","last_clean":"2024-09-17T03:34:19.961275+0000","last_became_active":"2024-09-17T03:34:18.968933+0000","last_became_peered":"2024-09-17T03:34:18.968933+0000","last_unstale":"2024-09-17T03:34:19.961275+0000","last_undegraded":"2024-09-17T03:34:19.961275+0000","last_fullsized":"2024-09-17T03:34:19.961275+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:44.687903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.970623+0000","last_change":"2024-09-17T03:34:18.973443+0000","last_active":"2024-09-17T03:34:19.970623+0000","last_peered":"2024-09-17T03:34:19.970623+0000","last_clean":"2024-09-17T03:34:19.970623+0000","last_became_active":"2024-09-17T03:34:18.972679+0000","last_became_peered":"2024-09-17T03:34:18.972679+0000","last_unstale":"2024-09-17T03:34:19.970623+0000","last_undegraded":"2024-09-17T03:34:19.970623+0000","last_fullsized":"2024-09-17T03:34:19.970623+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:32.160944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987308+0000","last_change":"2024-09-17T03:34:18.958041+0000","last_active":"2024-09-17T03:34:19.987308+0000","last_peered":"2024-09-17T03:34:19.987308+0000","last_clean":"2024-09-17T03:34:19.987308+0000","last_became_active":"2024-09-17T03:34:18.957720+0000","last_became_peered":"2024-09-17T03:34:18.957720+0000","last_unstale":"2024-09-17T03:34:19.987308+0000","last_undegraded":"2024-09-17T03:34:19.987308+0000","last_fullsized":"2024-09-17T03:34:19.987308+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:20.108541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961121+0000","last_change":"2024-09-17T03:34:18.972185+0000","last_active":"2024-09-17T03:34:19.961121+0000","last_peered":"2024-09-17T03:34:19.961121+0000","last_clean":"2024-09-17T03:34:19.961121+0000","last_became_active":"2024-09-17T03:34:18.971866+0000","last_became_peered":"2024-09-17T03:34:18.971866+0000","last_unstale":"2024-09-17T03:34:19.961121+0000","last_undegraded":"2024-09-17T03:34:19.961121+0000","last_fullsized":"2024-09-17T03:34:19.961121+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:52.709122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962285+0000","last_change":"2024-09-17T03:34:18.963490+0000","last_active":"2024-09-17T03:34:19.962285+0000","last_peered":"2024-09-17T03:34:19.962285+0000","last_clean":"2024-09-17T03:34:19.962285+0000","last_became_active":"2024-09-17T03:34:18.961841+0000","last_became_peered":"2024-09-17T03:34:18.961841+0000","last_unstale":"2024-09-17T03:34:19.962285+0000","last_undegraded":"2024-09-17T03:34:19.962285+0000","last_fullsized":"2024-09-17T03:34:19.962285+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:55.457006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969614+0000","last_change":"2024-09-17T03:34:18.977607+0000","last_active":"2024-09-17T03:34:19.969614+0000","last_peered":"2024-09-17T03:34:19.969614+0000","last_clean":"2024-09-17T03:34:19.969614+0000","last_became_active":"2024-09-17T03:34:18.976158+0000","last_became_peered":"2024-09-17T03:34:18.976158+0000","last_unstale":"2024-09-17T03:34:19.969614+0000","last_undegraded":"2024-09-17T03:34:19.969614+0000","last_fullsized":"2024-09-17T03:34:19.969614+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:40.041605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495510+0000","last_change":"2024-09-17T03:34:18.973403+0000","last_active":"2024-09-17T03:34:20.495510+0000","last_peered":"2024-09-17T03:34:20.495510+0000","last_clean":"2024-09-17T03:34:20.495510+0000","last_became_active":"2024-09-17T03:34:18.972784+0000","last_became_peered":"2024-09-17T03:34:18.972784+0000","last_unstale":"2024-09-17T03:34:20.495510+0000","last_undegraded":"2024-09-17T03:34:20.495510+0000","last_fullsized":"2024-09-17T03:34:20.495510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:45.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986563+0000","last_change":"2024-09-17T03:34:18.963988+0000","last_active":"2024-09-17T03:34:19.986563+0000","last_peered":"2024-09-17T03:34:19.986563+0000","last_clean":"2024-09-17T03:34:19.986563+0000","last_became_active":"2024-09-17T03:34:18.963602+0000","last_became_peered":"2024-09-17T03:34:18.963602+0000","last_unstale":"2024-09-17T03:34:19.986563+0000","last_undegraded":"2024-09-17T03:34:19.986563+0000","last_fullsized":"2024-09-17T03:34:19.986563+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:26.544052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.960486+0000","last_change":"2024-09-17T03:34:18.971162+0000","last_active":"2024-09-17T03:34:19.960486+0000","last_peered":"2024-09-17T03:34:19.960486+0000","last_clean":"2024-09-17T03:34:19.960486+0000","last_became_active":"2024-09-17T03:34:18.970835+0000","last_became_peered":"2024-09-17T03:34:18.970835+0000","last_unstale":"2024-09-17T03:34:19.960486+0000","last_undegraded":"2024-09-17T03:34:19.960486+0000","last_fullsized":"2024-09-17T03:34:19.960486+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:32.953304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961519+0000","last_change":"2024-09-17T03:34:18.975433+0000","last_active":"2024-09-17T03:34:19.961519+0000","last_peered":"2024-09-17T03:34:19.961519+0000","last_clean":"2024-09-17T03:34:19.961519+0000","last_became_active":"2024-09-17T03:34:18.973877+0000","last_became_peered":"2024-09-17T03:34:18.973877+0000","last_unstale":"2024-09-17T03:34:19.961519+0000","last_undegraded":"2024-09-17T03:34:19.961519+0000","last_fullsized":"2024-09-17T03:34:19.961519+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:37.899274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496664+0000","last_change":"2024-09-17T03:34:17.967902+0000","last_active":"2024-09-17T03:34:20.496664+0000","last_peered":"2024-09-17T03:34:20.496664+0000","last_clean":"2024-09-17T03:34:20.496664+0000","last_became_active":"2024-09-17T03:34:17.967529+0000","last_became_peered":"2024-09-17T03:34:17.967529+0000","last_unstale":"2024-09-17T03:34:20.496664+0000","last_undegraded":"2024-09-17T03:34:20.496664+0000","last_fullsized":"2024-09-17T03:34:20.496664+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:38.117876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496523+0000","last_change":"2024-09-17T03:34:18.967767+0000","last_active":"2024-09-17T03:34:20.496523+0000","last_peered":"2024-09-17T03:34:20.496523+0000","last_clean":"2024-09-17T03:34:20.496523+0000","last_became_active":"2024-09-17T03:34:18.966276+0000","last_became_peered":"2024-09-17T03:34:18.966276+0000","last_unstale":"2024-09-17T03:34:20.496523+0000","last_undegraded":"2024-09-17T03:34:20.496523+0000","last_fullsized":"2024-09-17T03:34:20.496523+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:41.799092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496522+0000","last_change":"2024-09-17T03:34:17.970796+0000","last_active":"2024-09-17T03:34:20.496522+0000","last_peered":"2024-09-17T03:34:20.496522+0000","last_clean":"2024-09-17T03:34:20.496522+0000","last_became_active":"2024-09-17T03:34:17.970294+0000","last_became_peered":"2024-09-17T03:34:17.970294+0000","last_unstale":"2024-09-17T03:34:20.496522+0000","last_undegraded":"2024-09-17T03:34:20.496522+0000","last_fullsized":"2024-09-17T03:34:20.496522+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:54:19.498009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961166+0000","last_change":"2024-09-17T03:34:18.973240+0000","last_active":"2024-09-17T03:34:19.961166+0000","last_peered":"2024-09-17T03:34:19.961166+0000","last_clean":"2024-09-17T03:34:19.961166+0000","last_became_active":"2024-09-17T03:34:18.972907+0000","last_became_peered":"2024-09-17T03:34:18.972907+0000","last_unstale":"2024-09-17T03:34:19.961166+0000","last_undegraded":"2024-09-17T03:34:19.961166+0000","last_fullsized":"2024-09-17T03:34:19.961166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:01.060584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487666+0000","last_change":"2024-09-17T03:34:17.952012+0000","last_active":"2024-09-17T03:34:20.487666+0000","last_peered":"2024-09-17T03:34:20.487666+0000","last_clean":"2024-09-17T03:34:20.487666+0000","last_became_active":"2024-09-17T03:34:17.951685+0000","last_became_peered":"2024-09-17T03:34:17.951685+0000","last_unstale":"2024-09-17T03:34:20.487666+0000","last_undegraded":"2024-09-17T03:34:20.487666+0000","last_fullsized":"2024-09-17T03:34:20.487666+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:35.737967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.497501+0000","last_change":"2024-09-17T03:34:18.990248+0000","last_active":"2024-09-17T03:34:20.497501+0000","last_peered":"2024-09-17T03:34:20.497501+0000","last_clean":"2024-09-17T03:34:20.497501+0000","last_became_active":"2024-09-17T03:34:18.989922+0000","last_became_peered":"2024-09-17T03:34:18.989922+0000","last_unstale":"2024-09-17T03:34:20.497501+0000","last_undegraded":"2024-09-17T03:34:20.497501+0000","last_fullsized":"2024-09-17T03:34:20.497501+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:42.357610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487139+0000","last_change":"2024-09-17T03:34:17.956264+0000","last_active":"2024-09-17T03:34:20.487139+0000","last_peered":"2024-09-17T03:34:20.487139+0000","last_clean":"2024-09-17T03:34:20.487139+0000","last_became_active":"2024-09-17T03:34:17.955931+0000","last_became_peered":"2024-09-17T03:34:17.955931+0000","last_unstale":"2024-09-17T03:34:20.487139+0000","last_undegraded":"2024-09-17T03:34:20.487139+0000","last_fullsized":"2024-09-17T03:34:20.487139+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:33.815738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986952+0000","last_change":"2024-09-17T03:34:18.960283+0000","last_active":"2024-09-17T03:34:19.986952+0000","last_peered":"2024-09-17T03:34:19.986952+0000","last_clean":"2024-09-17T03:34:19.986952+0000","last_became_active":"2024-09-17T03:34:18.959890+0000","last_became_peered":"2024-09-17T03:34:18.959890+0000","last_unstale":"2024-09-17T03:34:19.986952+0000","last_undegraded":"2024-09-17T03:34:19.986952+0000","last_fullsized":"2024-09-17T03:34:19.986952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:57.962450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495679+0000","last_change":"2024-09-17T03:34:17.968406+0000","last_active":"2024-09-17T03:34:20.495679+0000","last_peered":"2024-09-17T03:34:20.495679+0000","last_clean":"2024-09-17T03:34:20.495679+0000","last_became_active":"2024-09-17T03:34:17.968065+0000","last_became_peered":"2024-09-17T03:34:17.968065+0000","last_unstale":"2024-09-17T03:34:20.495679+0000","last_undegraded":"2024-09-17T03:34:20.495679+0000","last_fullsized":"2024-09-17T03:34:20.495679+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:58.697142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969745+0000","last_change":"2024-09-17T03:34:18.988304+0000","last_active":"2024-09-17T03:34:19.969745+0000","last_peered":"2024-09-17T03:34:19.969745+0000","last_clean":"2024-09-17T03:34:19.969745+0000","last_became_active":"2024-09-17T03:34:18.987944+0000","last_became_peered":"2024-09-17T03:34:18.987944+0000","last_unstale":"2024-09-17T03:34:19.969745+0000","last_undegraded":"2024-09-17T03:34:19.969745+0000","last_fullsized":"2024-09-17T03:34:19.969745+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:16.410413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962380+0000","last_change":"2024-09-17T03:34:17.955131+0000","last_active":"2024-09-17T03:34:19.962380+0000","last_peered":"2024-09-17T03:34:19.962380+0000","last_clean":"2024-09-17T03:34:19.962380+0000","last_became_active":"2024-09-17T03:34:17.954824+0000","last_became_peered":"2024-09-17T03:34:17.954824+0000","last_unstale":"2024-09-17T03:34:19.962380+0000","last_undegraded":"2024-09-17T03:34:19.962380+0000","last_fullsized":"2024-09-17T03:34:19.962380+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:40.855750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986354+0000","last_change":"2024-09-17T03:34:18.982378+0000","last_active":"2024-09-17T03:34:19.986354+0000","last_peered":"2024-09-17T03:34:19.986354+0000","last_clean":"2024-09-17T03:34:19.986354+0000","last_became_active":"2024-09-17T03:34:18.982051+0000","last_became_peered":"2024-09-17T03:34:18.982051+0000","last_unstale":"2024-09-17T03:34:19.986354+0000","last_undegraded":"2024-09-17T03:34:19.986354+0000","last_fullsized":"2024-09-17T03:34:19.986354+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:37.667549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969360+0000","last_change":"2024-09-17T03:34:17.962471+0000","last_active":"2024-09-17T03:34:19.969360+0000","last_peered":"2024-09-17T03:34:19.969360+0000","last_clean":"2024-09-17T03:34:19.969360+0000","last_became_active":"2024-09-17T03:34:17.962149+0000","last_became_peered":"2024-09-17T03:34:17.962149+0000","last_unstale":"2024-09-17T03:34:19.969360+0000","last_undegraded":"2024-09-17T03:34:19.969360+0000","last_fullsized":"2024-09-17T03:34:19.969360+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:54.032921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.494949+0000","last_change":"2024-09-17T03:34:18.968014+0000","last_active":"2024-09-17T03:34:20.494949+0000","last_peered":"2024-09-17T03:34:20.494949+0000","last_clean":"2024-09-17T03:34:20.494949+0000","last_became_active":"2024-09-17T03:34:18.967294+0000","last_became_peered":"2024-09-17T03:34:18.967294+0000","last_unstale":"2024-09-17T03:34:20.494949+0000","last_undegraded":"2024-09-17T03:34:20.494949+0000","last_fullsized":"2024-09-17T03:34:20.494949+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:19.020021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986504+0000","last_change":"2024-09-17T03:34:17.965406+0000","last_active":"2024-09-17T03:34:19.986504+0000","last_peered":"2024-09-17T03:34:19.986504+0000","last_clean":"2024-09-17T03:34:19.986504+0000","last_became_active":"2024-09-17T03:34:17.964929+0000","last_became_peered":"2024-09-17T03:34:17.964929+0000","last_unstale":"2024-09-17T03:34:19.986504+0000","last_undegraded":"2024-09-17T03:34:19.986504+0000","last_fullsized":"2024-09-17T03:34:19.986504+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:56.066869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496762+0000","last_change":"2024-09-17T03:34:18.977053+0000","last_active":"2024-09-17T03:34:20.496762+0000","last_peered":"2024-09-17T03:34:20.496762+0000","last_clean":"2024-09-17T03:34:20.496762+0000","last_became_active":"2024-09-17T03:34:18.975249+0000","last_became_peered":"2024-09-17T03:34:18.975249+0000","last_unstale":"2024-09-17T03:34:20.496762+0000","last_undegraded":"2024-09-17T03:34:20.496762+0000","last_fullsized":"2024-09-17T03:34:20.496762+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:04.163445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487252+0000","last_change":"2024-09-17T03:34:17.958657+0000","last_active":"2024-09-17T03:34:20.487252+0000","last_peered":"2024-09-17T03:34:20.487252+0000","last_clean":"2024-09-17T03:34:20.487252+0000","last_became_active":"2024-09-17T03:34:17.956644+0000","last_became_peered":"2024-09-17T03:34:17.956644+0000","last_unstale":"2024-09-17T03:34:20.487252+0000","last_undegraded":"2024-09-17T03:34:20.487252+0000","last_fullsized":"2024-09-17T03:34:20.487252+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:17.060424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496420+0000","last_change":"2024-09-17T03:34:18.975049+0000","last_active":"2024-09-17T03:34:20.496420+0000","last_peered":"2024-09-17T03:34:20.496420+0000","last_clean":"2024-09-17T03:34:20.496420+0000","last_became_active":"2024-09-17T03:34:18.974605+0000","last_became_peered":"2024-09-17T03:34:18.974605+0000","last_unstale":"2024-09-17T03:34:20.496420+0000","last_undegraded":"2024-09-17T03:34:20.496420+0000","last_fullsized":"2024-09-17T03:34:20.496420+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:51.159254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962563+0000","last_change":"2024-09-17T03:34:17.960126+0000","last_active":"2024-09-17T03:34:19.962563+0000","last_peered":"2024-09-17T03:34:19.962563+0000","last_clean":"2024-09-17T03:34:19.962563+0000","last_became_active":"2024-09-17T03:34:17.958154+0000","last_became_peered":"2024-09-17T03:34:17.958154+0000","last_unstale":"2024-09-17T03:34:19.962563+0000","last_undegraded":"2024-09-17T03:34:19.962563+0000","last_fullsized":"2024-09-17T03:34:19.962563+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:50.156294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962289+0000","last_change":"2024-09-17T03:34:18.954604+0000","last_active":"2024-09-17T03:34:19.962289+0000","last_peered":"2024-09-17T03:34:19.962289+0000","last_clean":"2024-09-17T03:34:19.962289+0000","last_became_active":"2024-09-17T03:34:18.954313+0000","last_became_peered":"2024-09-17T03:34:18.954313+0000","last_unstale":"2024-09-17T03:34:19.962289+0000","last_undegraded":"2024-09-17T03:34:19.962289+0000","last_fullsized":"2024-09-17T03:34:19.962289+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:29.532716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986768+0000","last_change":"2024-09-17T03:34:17.964079+0000","last_active":"2024-09-17T03:34:19.986768+0000","last_peered":"2024-09-17T03:34:19.986768+0000","last_clean":"2024-09-17T03:34:19.986768+0000","last_became_active":"2024-09-17T03:34:17.963119+0000","last_became_peered":"2024-09-17T03:34:17.963119+0000","last_unstale":"2024-09-17T03:34:19.986768+0000","last_undegraded":"2024-09-17T03:34:19.986768+0000","last_fullsized":"2024-09-17T03:34:19.986768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:29.811128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496215+0000","last_change":"2024-09-17T03:34:18.972637+0000","last_active":"2024-09-17T03:34:20.496215+0000","last_peered":"2024-09-17T03:34:20.496215+0000","last_clean":"2024-09-17T03:34:20.496215+0000","last_became_active":"2024-09-17T03:34:18.972274+0000","last_became_peered":"2024-09-17T03:34:18.972274+0000","last_unstale":"2024-09-17T03:34:20.496215+0000","last_undegraded":"2024-09-17T03:34:20.496215+0000","last_fullsized":"2024-09-17T03:34:20.496215+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:59.813508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987229+0000","last_change":"2024-09-17T03:34:17.965870+0000","last_active":"2024-09-17T03:34:19.987229+0000","last_peered":"2024-09-17T03:34:19.987229+0000","last_clean":"2024-09-17T03:34:19.987229+0000","last_became_active":"2024-09-17T03:34:17.965529+0000","last_became_peered":"2024-09-17T03:34:17.965529+0000","last_unstale":"2024-09-17T03:34:19.987229+0000","last_undegraded":"2024-09-17T03:34:19.987229+0000","last_fullsized":"2024-09-17T03:34:19.987229+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:23.342226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962296+0000","last_change":"2024-09-17T03:34:18.962668+0000","last_active":"2024-09-17T03:34:19.962296+0000","last_peered":"2024-09-17T03:34:19.962296+0000","last_clean":"2024-09-17T03:34:19.962296+0000","last_became_active":"2024-09-17T03:34:18.962130+0000","last_became_peered":"2024-09-17T03:34:18.962130+0000","last_unstale":"2024-09-17T03:34:19.962296+0000","last_undegraded":"2024-09-17T03:34:19.962296+0000","last_fullsized":"2024-09-17T03:34:19.962296+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:02.762749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.960770+0000","last_change":"2024-09-17T03:34:17.958201+0000","last_active":"2024-09-17T03:34:19.960770+0000","last_peered":"2024-09-17T03:34:19.960770+0000","last_clean":"2024-09-17T03:34:19.960770+0000","last_became_active":"2024-09-17T03:34:17.956714+0000","last_became_peered":"2024-09-17T03:34:17.956714+0000","last_unstale":"2024-09-17T03:34:19.960770+0000","last_undegraded":"2024-09-17T03:34:19.960770+0000","last_fullsized":"2024-09-17T03:34:19.960770+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:08.237021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962060+0000","last_change":"2024-09-17T03:34:18.967020+0000","last_active":"2024-09-17T03:34:19.962060+0000","last_peered":"2024-09-17T03:34:19.962060+0000","last_clean":"2024-09-17T03:34:19.962060+0000","last_became_active":"2024-09-17T03:34:18.966631+0000","last_became_peered":"2024-09-17T03:34:18.966631+0000","last_unstale":"2024-09-17T03:34:19.962060+0000","last_undegraded":"2024-09-17T03:34:19.962060+0000","last_fullsized":"2024-09-17T03:34:19.962060+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:31.303366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495141+0000","last_change":"2024-09-17T03:34:17.971369+0000","last_active":"2024-09-17T03:34:20.495141+0000","last_peered":"2024-09-17T03:34:20.495141+0000","last_clean":"2024-09-17T03:34:20.495141+0000","last_became_active":"2024-09-17T03:34:17.971029+0000","last_became_peered":"2024-09-17T03:34:17.971029+0000","last_unstale":"2024-09-17T03:34:20.495141+0000","last_undegraded":"2024-09-17T03:34:20.495141+0000","last_fullsized":"2024-09-17T03:34:20.495141+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:17.149097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961345+0000","last_change":"2024-09-17T03:34:18.954600+0000","last_active":"2024-09-17T03:34:19.961345+0000","last_peered":"2024-09-17T03:34:19.961345+0000","last_clean":"2024-09-17T03:34:19.961345+0000","last_became_active":"2024-09-17T03:34:18.954264+0000","last_became_peered":"2024-09-17T03:34:18.954264+0000","last_unstale":"2024-09-17T03:34:19.961345+0000","last_undegraded":"2024-09-17T03:34:19.961345+0000","last_fullsized":"2024-09-17T03:34:19.961345+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:20.834048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":23,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487099+0000","last_change":"2024-09-17T03:34:17.960483+0000","last_active":"2024-09-17T03:34:20.487099+0000","last_peered":"2024-09-17T03:34:20.487099+0000","last_clean":"2024-09-17T03:34:20.487099+0000","last_became_active":"2024-09-17T03:34:17.960093+0000","last_became_peered":"2024-09-17T03:34:17.960093+0000","last_unstale":"2024-09-17T03:34:20.487099+0000","last_undegraded":"2024-09-17T03:34:20.487099+0000","last_fullsized":"2024-09-17T03:34:20.487099+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:57.576073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487465+0000","last_change":"2024-09-17T03:34:18.978997+0000","last_active":"2024-09-17T03:34:20.487465+0000","last_peered":"2024-09-17T03:34:20.487465+0000","last_clean":"2024-09-17T03:34:20.487465+0000","last_became_active":"2024-09-17T03:34:18.978610+0000","last_became_peered":"2024-09-17T03:34:18.978610+0000","last_unstale":"2024-09-17T03:34:20.487465+0000","last_undegraded":"2024-09-17T03:34:20.487465+0000","last_fullsized":"2024-09-17T03:34:20.487465+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:48.297167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987614+0000","last_change":"2024-09-17T03:34:17.961434+0000","last_active":"2024-09-17T03:34:19.987614+0000","last_peered":"2024-09-17T03:34:19.987614+0000","last_clean":"2024-09-17T03:34:19.987614+0000","last_became_active":"2024-09-17T03:34:17.960999+0000","last_became_peered":"2024-09-17T03:34:17.960999+0000","last_unstale":"2024-09-17T03:34:19.987614+0000","last_undegraded":"2024-09-17T03:34:19.987614+0000","last_fullsized":"2024-09-17T03:34:19.987614+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:58.226633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961550+0000","last_change":"2024-09-17T03:34:18.965734+0000","last_active":"2024-09-17T03:34:19.961550+0000","last_peered":"2024-09-17T03:34:19.961550+0000","last_clean":"2024-09-17T03:34:19.961550+0000","last_became_active":"2024-09-17T03:34:18.964887+0000","last_became_peered":"2024-09-17T03:34:18.964887+0000","last_unstale":"2024-09-17T03:34:19.961550+0000","last_undegraded":"2024-09-17T03:34:19.961550+0000","last_fullsized":"2024-09-17T03:34:19.961550+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:43.649033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487362+0000","last_change":"2024-09-17T03:34:17.947906+0000","last_active":"2024-09-17T03:34:20.487362+0000","last_peered":"2024-09-17T03:34:20.487362+0000","last_clean":"2024-09-17T03:34:20.487362+0000","last_became_active":"2024-09-17T03:34:17.947591+0000","last_became_peered":"2024-09-17T03:34:17.947591+0000","last_unstale":"2024-09-17T03:34:20.487362+0000","last_undegraded":"2024-09-17T03:34:20.487362+0000","last_fullsized":"2024-09-17T03:34:20.487362+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:14.252352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496680+0000","last_change":"2024-09-17T03:34:18.979587+0000","last_active":"2024-09-17T03:34:20.496680+0000","last_peered":"2024-09-17T03:34:20.496680+0000","last_clean":"2024-09-17T03:34:20.496680+0000","last_became_active":"2024-09-17T03:34:18.979227+0000","last_became_peered":"2024-09-17T03:34:18.979227+0000","last_unstale":"2024-09-17T03:34:20.496680+0000","last_undegraded":"2024-09-17T03:34:20.496680+0000","last_fullsized":"2024-09-17T03:34:20.496680+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:48.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495331+0000","last_change":"2024-09-17T03:34:17.972909+0000","last_active":"2024-09-17T03:34:20.495331+0000","last_peered":"2024-09-17T03:34:20.495331+0000","last_clean":"2024-09-17T03:34:20.495331+0000","last_became_active":"2024-09-17T03:34:17.972594+0000","last_became_peered":"2024-09-17T03:34:17.972594+0000","last_unstale":"2024-09-17T03:34:20.495331+0000","last_undegraded":"2024-09-17T03:34:20.495331+0000","last_fullsized":"2024-09-17T03:34:20.495331+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:15.663588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.497623+0000","last_change":"2024-09-17T03:34:18.978408+0000","last_active":"2024-09-17T03:34:20.497623+0000","last_peered":"2024-09-17T03:34:20.497623+0000","last_clean":"2024-09-17T03:34:20.497623+0000","last_became_active":"2024-09-17T03:34:18.978082+0000","last_became_peered":"2024-09-17T03:34:18.978082+0000","last_unstale":"2024-09-17T03:34:20.497623+0000","last_undegraded":"2024-09-17T03:34:20.497623+0000","last_fullsized":"2024-09-17T03:34:20.497623+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:06.160890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495931+0000","last_change":"2024-09-17T03:34:17.977152+0000","last_active":"2024-09-17T03:34:20.495931+0000","last_peered":"2024-09-17T03:34:20.495931+0000","last_clean":"2024-09-17T03:34:20.495931+0000","last_became_active":"2024-09-17T03:34:17.976753+0000","last_became_peered":"2024-09-17T03:34:17.976753+0000","last_unstale":"2024-09-17T03:34:20.495931+0000","last_undegraded":"2024-09-17T03:34:20.495931+0000","last_fullsized":"2024-09-17T03:34:20.495931+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:44.468930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496868+0000","last_change":"2024-09-17T03:34:18.979409+0000","last_active":"2024-09-17T03:34:20.496868+0000","last_peered":"2024-09-17T03:34:20.496868+0000","last_clean":"2024-09-17T03:34:20.496868+0000","last_became_active":"2024-09-17T03:34:18.979077+0000","last_became_peered":"2024-09-17T03:34:18.979077+0000","last_unstale":"2024-09-17T03:34:20.496868+0000","last_undegraded":"2024-09-17T03:34:20.496868+0000","last_fullsized":"2024-09-17T03:34:20.496868+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:25.786460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.960824+0000","last_change":"2024-09-17T03:34:17.959107+0000","last_active":"2024-09-17T03:34:19.960824+0000","last_peered":"2024-09-17T03:34:19.960824+0000","last_clean":"2024-09-17T03:34:19.960824+0000","last_became_active":"2024-09-17T03:34:17.958665+0000","last_became_peered":"2024-09-17T03:34:17.958665+0000","last_unstale":"2024-09-17T03:34:19.960824+0000","last_undegraded":"2024-09-17T03:34:19.960824+0000","last_fullsized":"2024-09-17T03:34:19.960824+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:25.671814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986741+0000","last_change":"2024-09-17T03:34:18.967003+0000","last_active":"2024-09-17T03:34:19.986741+0000","last_peered":"2024-09-17T03:34:19.986741+0000","last_clean":"2024-09-17T03:34:19.986741+0000","last_became_active":"2024-09-17T03:34:18.966630+0000","last_became_peered":"2024-09-17T03:34:18.966630+0000","last_unstale":"2024-09-17T03:34:19.986741+0000","last_undegraded":"2024-09-17T03:34:19.986741+0000","last_fullsized":"2024-09-17T03:34:19.986741+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:27:55.851072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.486983+0000","last_change":"2024-09-17T03:34:17.958278+0000","last_active":"2024-09-17T03:34:20.486983+0000","last_peered":"2024-09-17T03:34:20.486983+0000","last_clean":"2024-09-17T03:34:20.486983+0000","last_became_active":"2024-09-17T03:34:17.956303+0000","last_became_peered":"2024-09-17T03:34:17.956303+0000","last_unstale":"2024-09-17T03:34:20.486983+0000","last_undegraded":"2024-09-17T03:34:20.486983+0000","last_fullsized":"2024-09-17T03:34:20.486983+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:46.909219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.496799+0000","last_change":"2024-09-17T03:34:18.976109+0000","last_active":"2024-09-17T03:34:20.496799+0000","last_peered":"2024-09-17T03:34:20.496799+0000","last_clean":"2024-09-17T03:34:20.496799+0000","last_became_active":"2024-09-17T03:34:18.975778+0000","last_became_peered":"2024-09-17T03:34:18.975778+0000","last_unstale":"2024-09-17T03:34:20.496799+0000","last_undegraded":"2024-09-17T03:34:20.496799+0000","last_fullsized":"2024-09-17T03:34:20.496799+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:23.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.960414+0000","last_change":"2024-09-17T03:34:17.963131+0000","last_active":"2024-09-17T03:34:19.960414+0000","last_peered":"2024-09-17T03:34:19.960414+0000","last_clean":"2024-09-17T03:34:19.960414+0000","last_became_active":"2024-09-17T03:34:17.961875+0000","last_became_peered":"2024-09-17T03:34:17.961875+0000","last_unstale":"2024-09-17T03:34:19.960414+0000","last_undegraded":"2024-09-17T03:34:19.960414+0000","last_fullsized":"2024-09-17T03:34:19.960414+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:38.117876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961583+0000","last_change":"2024-09-17T03:34:18.970270+0000","last_active":"2024-09-17T03:34:19.961583+0000","last_peered":"2024-09-17T03:34:19.961583+0000","last_clean":"2024-09-17T03:34:19.961583+0000","last_became_active":"2024-09-17T03:34:18.969088+0000","last_became_peered":"2024-09-17T03:34:18.969088+0000","last_unstale":"2024-09-17T03:34:19.961583+0000","last_undegraded":"2024-09-17T03:34:19.961583+0000","last_fullsized":"2024-09-17T03:34:19.961583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:48.215704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986628+0000","last_change":"2024-09-17T03:34:17.967580+0000","last_active":"2024-09-17T03:34:19.986628+0000","last_peered":"2024-09-17T03:34:19.986628+0000","last_clean":"2024-09-17T03:34:19.986628+0000","last_became_active":"2024-09-17T03:34:17.966929+0000","last_became_peered":"2024-09-17T03:34:17.966929+0000","last_unstale":"2024-09-17T03:34:19.986628+0000","last_undegraded":"2024-09-17T03:34:19.986628+0000","last_fullsized":"2024-09-17T03:34:19.986628+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:05.598698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495503+0000","last_change":"2024-09-17T03:34:18.977297+0000","last_active":"2024-09-17T03:34:20.495503+0000","last_peered":"2024-09-17T03:34:20.495503+0000","last_clean":"2024-09-17T03:34:20.495503+0000","last_became_active":"2024-09-17T03:34:18.976962+0000","last_became_peered":"2024-09-17T03:34:18.976962+0000","last_unstale":"2024-09-17T03:34:20.495503+0000","last_undegraded":"2024-09-17T03:34:20.495503+0000","last_fullsized":"2024-09-17T03:34:20.495503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:32.160944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962433+0000","last_change":"2024-09-17T03:34:17.966894+0000","last_active":"2024-09-17T03:34:19.962433+0000","last_peered":"2024-09-17T03:34:19.962433+0000","last_clean":"2024-09-17T03:34:19.962433+0000","last_became_active":"2024-09-17T03:34:17.966582+0000","last_became_peered":"2024-09-17T03:34:17.966582+0000","last_unstale":"2024-09-17T03:34:19.962433+0000","last_undegraded":"2024-09-17T03:34:19.962433+0000","last_fullsized":"2024-09-17T03:34:19.962433+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:00.321715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987684+0000","last_change":"2024-09-17T03:34:18.970150+0000","last_active":"2024-09-17T03:34:19.987684+0000","last_peered":"2024-09-17T03:34:19.987684+0000","last_clean":"2024-09-17T03:34:19.987684+0000","last_became_active":"2024-09-17T03:34:18.969812+0000","last_became_peered":"2024-09-17T03:34:18.969812+0000","last_unstale":"2024-09-17T03:34:19.987684+0000","last_undegraded":"2024-09-17T03:34:19.987684+0000","last_fullsized":"2024-09-17T03:34:19.987684+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:53.803112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"114'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495626+0000","last_change":"2024-09-17T03:34:17.966013+0000","last_active":"2024-09-17T03:34:20.495626+0000","last_peered":"2024-09-17T03:34:20.495626+0000","last_clean":"2024-09-17T03:34:20.495626+0000","last_became_active":"2024-09-17T03:34:17.965158+0000","last_became_peered":"2024-09-17T03:34:17.965158+0000","last_unstale":"2024-09-17T03:34:20.495626+0000","last_undegraded":"2024-09-17T03:34:20.495626+0000","last_fullsized":"2024-09-17T03:34:20.495626+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:39.038646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.987175+0000","last_change":"2024-09-17T03:34:18.975171+0000","last_active":"2024-09-17T03:34:19.987175+0000","last_peered":"2024-09-17T03:34:19.987175+0000","last_clean":"2024-09-17T03:34:19.987175+0000","last_became_active":"2024-09-17T03:34:18.974855+0000","last_became_peered":"2024-09-17T03:34:18.974855+0000","last_unstale":"2024-09-17T03:34:19.987175+0000","last_undegraded":"2024-09-17T03:34:19.987175+0000","last_fullsized":"2024-09-17T03:34:19.987175+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:59.102467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986896+0000","last_change":"2024-09-17T03:34:17.952591+0000","last_active":"2024-09-17T03:34:19.986896+0000","last_peered":"2024-09-17T03:34:19.986896+0000","last_clean":"2024-09-17T03:34:19.986896+0000","last_became_active":"2024-09-17T03:34:17.952285+0000","last_became_peered":"2024-09-17T03:34:17.952285+0000","last_unstale":"2024-09-17T03:34:19.986896+0000","last_undegraded":"2024-09-17T03:34:19.986896+0000","last_fullsized":"2024-09-17T03:34:19.986896+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:06.370230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495244+0000","last_change":"2024-09-17T03:34:18.974997+0000","last_active":"2024-09-17T03:34:20.495244+0000","last_peered":"2024-09-17T03:34:20.495244+0000","last_clean":"2024-09-17T03:34:20.495244+0000","last_became_active":"2024-09-17T03:34:18.972673+0000","last_became_peered":"2024-09-17T03:34:18.972673+0000","last_unstale":"2024-09-17T03:34:20.495244+0000","last_undegraded":"2024-09-17T03:34:20.495244+0000","last_fullsized":"2024-09-17T03:34:20.495244+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:05.546628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.962369+0000","last_change":"2024-09-17T03:34:17.967204+0000","last_active":"2024-09-17T03:34:19.962369+0000","last_peered":"2024-09-17T03:34:19.962369+0000","last_clean":"2024-09-17T03:34:19.962369+0000","last_became_active":"2024-09-17T03:34:17.966855+0000","last_became_peered":"2024-09-17T03:34:17.966855+0000","last_unstale":"2024-09-17T03:34:19.962369+0000","last_undegraded":"2024-09-17T03:34:19.962369+0000","last_fullsized":"2024-09-17T03:34:19.962369+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:40.796133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969583+0000","last_change":"2024-09-17T03:34:18.967943+0000","last_active":"2024-09-17T03:34:19.969583+0000","last_peered":"2024-09-17T03:34:19.969583+0000","last_clean":"2024-09-17T03:34:19.969583+0000","last_became_active":"2024-09-17T03:34:18.967518+0000","last_became_peered":"2024-09-17T03:34:18.967518+0000","last_unstale":"2024-09-17T03:34:19.969583+0000","last_undegraded":"2024-09-17T03:34:19.969583+0000","last_fullsized":"2024-09-17T03:34:19.969583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:16.666547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495802+0000","last_change":"2024-09-17T03:34:17.971285+0000","last_active":"2024-09-17T03:34:20.495802+0000","last_peered":"2024-09-17T03:34:20.495802+0000","last_clean":"2024-09-17T03:34:20.495802+0000","last_became_active":"2024-09-17T03:34:17.970905+0000","last_became_peered":"2024-09-17T03:34:17.970905+0000","last_unstale":"2024-09-17T03:34:20.495802+0000","last_undegraded":"2024-09-17T03:34:20.495802+0000","last_fullsized":"2024-09-17T03:34:20.495802+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:49.385583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969894+0000","last_change":"2024-09-17T03:34:18.962364+0000","last_active":"2024-09-17T03:34:19.969894+0000","last_peered":"2024-09-17T03:34:19.969894+0000","last_clean":"2024-09-17T03:34:19.969894+0000","last_became_active":"2024-09-17T03:34:18.961999+0000","last_became_peered":"2024-09-17T03:34:18.961999+0000","last_unstale":"2024-09-17T03:34:19.969894+0000","last_undegraded":"2024-09-17T03:34:19.969894+0000","last_fullsized":"2024-09-17T03:34:19.969894+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:39.581970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.969601+0000","last_change":"2024-09-17T03:34:17.970118+0000","last_active":"2024-09-17T03:34:19.969601+0000","last_peered":"2024-09-17T03:34:19.969601+0000","last_clean":"2024-09-17T03:34:19.969601+0000","last_became_active":"2024-09-17T03:34:17.969528+0000","last_became_peered":"2024-09-17T03:34:17.969528+0000","last_unstale":"2024-09-17T03:34:19.969601+0000","last_undegraded":"2024-09-17T03:34:19.969601+0000","last_fullsized":"2024-09-17T03:34:19.969601+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:29.872962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.497219+0000","last_change":"2024-09-17T03:34:18.970911+0000","last_active":"2024-09-17T03:34:20.497219+0000","last_peered":"2024-09-17T03:34:20.497219+0000","last_clean":"2024-09-17T03:34:20.497219+0000","last_became_active":"2024-09-17T03:34:18.970578+0000","last_became_peered":"2024-09-17T03:34:18.970578+0000","last_unstale":"2024-09-17T03:34:20.497219+0000","last_undegraded":"2024-09-17T03:34:20.497219+0000","last_fullsized":"2024-09-17T03:34:20.497219+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:29.532716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.487070+0000","last_change":"2024-09-17T03:34:17.955847+0000","last_active":"2024-09-17T03:34:20.487070+0000","last_peered":"2024-09-17T03:34:20.487070+0000","last_clean":"2024-09-17T03:34:20.487070+0000","last_became_active":"2024-09-17T03:34:17.955541+0000","last_became_peered":"2024-09-17T03:34:17.955541+0000","last_unstale":"2024-09-17T03:34:20.487070+0000","last_undegraded":"2024-09-17T03:34:20.487070+0000","last_fullsized":"2024-09-17T03:34:20.487070+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:03.728323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986590+0000","last_change":"2024-09-17T03:34:18.975256+0000","last_active":"2024-09-17T03:34:19.986590+0000","last_peered":"2024-09-17T03:34:19.986590+0000","last_clean":"2024-09-17T03:34:19.986590+0000","last_became_active":"2024-09-17T03:34:18.974765+0000","last_became_peered":"2024-09-17T03:34:18.974765+0000","last_unstale":"2024-09-17T03:34:19.986590+0000","last_undegraded":"2024-09-17T03:34:19.986590+0000","last_fullsized":"2024-09-17T03:34:19.986590+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:00.824163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.986677+0000","last_change":"2024-09-17T03:34:17.967220+0000","last_active":"2024-09-17T03:34:19.986677+0000","last_peered":"2024-09-17T03:34:19.986677+0000","last_clean":"2024-09-17T03:34:19.986677+0000","last_became_active":"2024-09-17T03:34:17.966544+0000","last_became_peered":"2024-09-17T03:34:17.966544+0000","last_unstale":"2024-09-17T03:34:19.986677+0000","last_undegraded":"2024-09-17T03:34:19.986677+0000","last_fullsized":"2024-09-17T03:34:19.986677+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:40:49.743407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:19.961527+0000","last_change":"2024-09-17T03:34:18.967181+0000","last_active":"2024-09-17T03:34:19.961527+0000","last_peered":"2024-09-17T03:34:19.961527+0000","last_clean":"2024-09-17T03:34:19.961527+0000","last_became_active":"2024-09-17T03:34:18.962756+0000","last_became_peered":"2024-09-17T03:34:18.962756+0000","last_unstale":"2024-09-17T03:34:19.961527+0000","last_undegraded":"2024-09-17T03:34:19.961527+0000","last_fullsized":"2024-09-17T03:34:19.961527+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:17.928553+0000","last_clean_scrub_stamp":"2024-09-17T03:34:17.928553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:23.977087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.495092+0000","last_change":"2024-09-17T03:34:17.977176+0000","last_active":"2024-09-17T03:34:20.495092+0000","last_peered":"2024-09-17T03:34:20.495092+0000","last_clean":"2024-09-17T03:34:20.495092+0000","last_became_active":"2024-09-17T03:34:17.976734+0000","last_became_peered":"2024-09-17T03:34:17.976734+0000","last_unstale":"2024-09-17T03:34:20.495092+0000","last_undegraded":"2024-09-17T03:34:20.495092+0000","last_fullsized":"2024-09-17T03:34:20.495092+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:34:16.925594+0000","last_clean_scrub_stamp":"2024-09-17T03:34:16.925594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:40.285491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-17T03:34:20.497073+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:34:20.497073+0000","last_peered":"2024-09-17T03:34:20.497073+0000","last_clean":"2024-09-17T03:34:20.497073+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:34:20.497073+0000","last_undegraded":"2024-09-17T03:34:20.497073+0000","last_fullsized":"2024-09-17T03:34:20.497073+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127496,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244344,"statfs":{"total":96636764160,"available":96506208256,"internally_reserved":0,"allocated":2011136,"data_stored":1203062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509604,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140792,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231048,"statfs":{"total":96636764160,"available":96492593152,"internally_reserved":0,"allocated":1413120,"data_stored":612148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509604,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149112,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222728,"statfs":{"total":96636764160,"available":96484073472,"internally_reserved":0,"allocated":1409024,"data_stored":612078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509605,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118596,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253244,"statfs":{"total":96636764160,"available":96515321856,"internally_reserved":0,"allocated":1425408,"data_stored":613760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509604,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140796,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231044,"statfs":{"total":96636764160,"available":96492589056,"internally_reserved":0,"allocated":1409024,"data_stored":612078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509604,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123344,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248496,"statfs":{"total":96636764160,"available":96510459904,"internally_reserved":0,"allocated":2023424,"data_stored":1204170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542308,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86640,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94285200,"statfs":{"total":96636764160,"available":96548044800,"internally_reserved":0,"allocated":1404928,"data_stored":606354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542308,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163008,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208832,"statfs":{"total":96636764160,"available":96469843968,"internally_reserved":0,"allocated":1417216,"data_stored":612202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:34:23.577 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:34:23.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:34:23.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 -- 172.21.15.46:0/2560582855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0b3340 msgr2=0x7fb4fc0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 --2- 172.21.15.46:0/2560582855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0b3340 0x7fb4fc0b3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009c80 tx=0x7fb4f401cda0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 -- 172.21.15.46:0/2560582855 shutdown_connections 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 --2- 172.21.15.46:0/2560582855 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0b3340 0x7fb4fc0b3750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 --2- 172.21.15.46:0/2560582855 >> v2:172.21.15.99:3300/0 conn(0x7fb4fc0cefa0 0x7fb4fc0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 -- 172.21.15.46:0/2560582855 >> 172.21.15.46:0/2560582855 conn(0x7fb4fc0b2040 msgr2=0x7fb4fc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 -- 172.21.15.46:0/2560582855 shutdown_connections 2024-09-17T03:34:23.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 -- 172.21.15.46:0/2560582855 wait complete. 2024-09-17T03:34:23.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 Processor -- start 2024-09-17T03:34:23.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.779+0000 7fb502800640 1 -- start start 2024-09-17T03:34:23.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0749a0 0x7fb4fc157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 0x7fb4fc158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4fc0cefa0 0x7fb4fc158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc169e60 con 0x7fb4fc0b3340 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc169ce0 con 0x7fb4fc0cefa0 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb4fc169fe0 con 0x7fb4fc0749a0 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 0x7fb4fc158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 0x7fb4fc158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55938/0 (socket says 172.21.15.46:55938) 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 -- 172.21.15.46:0/1295576521 learned_addr learned my addr 172.21.15.46:0/1295576521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4fbe00640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4fc0cefa0 0x7fb4fc158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4fb400640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0749a0 0x7fb4fc157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 -- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0749a0 msgr2=0x7fb4fc157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0749a0 0x7fb4fc157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 -- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4fc0cefa0 msgr2=0x7fb4fc158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4fc0cefa0 0x7fb4fc158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 -- 172.21.15.46:0/1295576521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc0b1390 con 0x7fb4fc0b3340 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4fbe00640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4fc0cefa0 0x7fb4fc158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3e00640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 0x7fb4fc158750 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e770 tx=0x7fb4e400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400ccc0 con 0x7fb4fc0b3340 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4e400ee80 con 0x7fb4fc0b3340 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400ce60 con 0x7fb4fc0b3340 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc0b1630 con 0x7fb4fc0b3340 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc0b1bf0 con 0x7fb4fc0b3340 2024-09-17T03:34:23.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.783+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4fc0b3750 con 0x7fb4fc0b3340 2024-09-17T03:34:23.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.787+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fb4e4005280 con 0x7fb4fc0b3340 2024-09-17T03:34:23.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.787+0000 7fb4f3400640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb4d8074d00 0x7fb4d8077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:23.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.787+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fb4e4093a10 con 0x7fb4fc0b3340 2024-09-17T03:34:23.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.787+0000 7fb4fb400640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb4d8074d00 0x7fb4d8077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:23.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.787+0000 7fb4fb400640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb4d8074d00 0x7fb4d8077210 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec007fd0 tx=0x7fb4ec009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:23.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:23.791+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb4e405fe50 con 0x7fb4fc0b3340 2024-09-17T03:34:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.031+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb4fc0af560 con 0x7fb4fc0b3340 2024-09-17T03:34:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb4f3400640 1 -- 172.21.15.46:0/1295576521 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v42) ==== 190+0+147 (secure 0 0 0) 0x7fb4e4064880 con 0x7fb4fc0b3340 2024-09-17T03:34:24.035 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:34:24.035 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:34:24.035 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:34:24.036 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:34:24.036 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:34:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:34:24.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb4d8074d00 msgr2=0x7fb4d8077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb4d8074d00 0x7fb4d8077210 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec007fd0 tx=0x7fb4ec009040 comp rx=0 tx=0).stop 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 msgr2=0x7fb4fc158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 0x7fb4fc158750 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e770 tx=0x7fb4e400ec30 comp rx=0 tx=0).stop 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 shutdown_connections 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb4d8074d00 0x7fb4d8077210 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4fc0cefa0 0x7fb4fc158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4fc0b3340 0x7fb4fc158750 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 --2- 172.21.15.46:0/1295576521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4fc0749a0 0x7fb4fc157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 >> 172.21.15.46:0/1295576521 conn(0x7fb4fc0b2040 msgr2=0x7fb4fc159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 shutdown_connections 2024-09-17T03:34:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.035+0000 7fb502800640 1 -- 172.21.15.46:0/1295576521 wait complete. 2024-09-17T03:34:24.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:34:24.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f11a1000640 1 -- 172.21.15.46:0/1441059430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180003c20 msgr2=0x7f1180004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f11a1000640 1 --2- 172.21.15.46:0/1441059430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1180003c20 0x7f1180004070 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f118c001860 tx=0x7f118c01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f119a800640 1 -- 172.21.15.46:0/1441059430 reap_dead start 2024-09-17T03:34:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f11a1000640 1 -- 172.21.15.46:0/1441059430 shutdown_connections 2024-09-17T03:34:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f11a1000640 1 -- 172.21.15.46:0/1441059430 >> 172.21.15.46:0/1441059430 conn(0x7f119c0b2040 msgr2=0x7f119c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f11a1000640 1 -- 172.21.15.46:0/1441059430 shutdown_connections 2024-09-17T03:34:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.215+0000 7f11a1000640 1 -- 172.21.15.46:0/1441059430 wait complete. 2024-09-17T03:34:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 Processor -- start 2024-09-17T03:34:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 -- start start 2024-09-17T03:34:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1180003c20 0x7f119c13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c13f390 0x7f119c15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 0x7f119c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1688e0 con 0x7f119c15dee0 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f119c168760 con 0x7f119c13f390 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f11a1000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f119c168a60 con 0x7f1180003c20 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 0x7f119c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 0x7f119c15e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55948/0 (socket says 172.21.15.46:55948) 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 -- 172.21.15.46:0/1231068845 learned_addr learned my addr 172.21.15.46:0/1231068845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f1193e00640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c13f390 0x7f119c15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f1199e00640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1180003c20 0x7f119c13eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 -- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1180003c20 msgr2=0x7f119c13eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1180003c20 0x7f119c13eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 -- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c13f390 msgr2=0x7f119c15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c13f390 0x7f119c15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 -- 172.21.15.46:0/1231068845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c13da30 con 0x7f119c15dee0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.219+0000 7f119a800640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 0x7f119c15e2f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f119c15a900 tx=0x7f119400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194004b30 con 0x7f119c15dee0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1194004cd0 con 0x7f119c15dee0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194003be0 con 0x7f119c15dee0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c143fb0 con 0x7f119c15dee0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c13e0c0 con 0x7f119c15dee0 2024-09-17T03:34:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119c074db0 con 0x7f119c15dee0 2024-09-17T03:34:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f1194004030 con 0x7f119c15dee0 2024-09-17T03:34:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1192000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1178074d00 0x7f1178077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f1194093650 con 0x7f119c15dee0 2024-09-17T03:34:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.223+0000 7f1199e00640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1178074d00 0x7f1178077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.227+0000 7f1199e00640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1178074d00 0x7f1178077210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f118c001830 tx=0x7f118c019720 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.227+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f119405fa80 con 0x7f119c15dee0 2024-09-17T03:34:24.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f119c143d40 con 0x7f119c15dee0 2024-09-17T03:34:24.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f1192000640 1 -- 172.21.15.46:0/1231068845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5241 (secure 0 0 0) 0x7f11940644b0 con 0x7f119c15dee0 2024-09-17T03:34:24.440 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:24.440 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"btime":"2024-09-17T03:34:21:967289+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:34:18.364785+0000","modified":"2024-09-17T03:34:21.431279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248196":{"gid":248196,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/1168163211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1168163211},{"type":"v1","addr":"172.21.15.46:6837","nonce":1168163211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248196,"qdb_cluster":[248196]},"id":8}]} 2024-09-17T03:34:24.441 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-09-17T03:34:24.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1178074d00 msgr2=0x7f1178077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1178074d00 0x7f1178077210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f118c001830 tx=0x7f118c019720 comp rx=0 tx=0).stop 2024-09-17T03:34:24.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 msgr2=0x7f119c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:24.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 0x7f119c15e2f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f119c15a900 tx=0x7f119400f3c0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 shutdown_connections 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1178074d00 0x7f1178077210 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c15dee0 0x7f119c15e2f0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f119c13f390 0x7f119c15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 --2- 172.21.15.46:0/1231068845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1180003c20 0x7f119c13eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 >> 172.21.15.46:0/1231068845 conn(0x7f119c0b2040 msgr2=0x7f119c0b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 shutdown_connections 2024-09-17T03:34:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:24.439+0000 7f11a1000640 1 -- 172.21.15.46:0/1231068845 wait complete. 2024-09-17T03:34:24.865 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:34:18.364785+0000', 'modified': '2024-09-17T03:34:21.431279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 248196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_248196': {'gid': 248196, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6837/1168163211', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6836', 'nonce': 1168163211}, {'type': 'v1', 'addr': '172.21.15.46:6837', 'nonce': 1168163211}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 248196, 'qdb_cluster': [248196]} 2024-09-17T03:34:24.865 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:34:24.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T03:34:25.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 -- 172.21.15.46:0/2194011701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140b3340 msgr2=0x7fdb140b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/2194011701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140b3340 0x7fdb140b3750 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009bc0 tx=0x7fdb0c01cba0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 -- 172.21.15.46:0/2194011701 shutdown_connections 2024-09-17T03:34:25.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/2194011701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140b3340 0x7fdb140b3750 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/2194011701 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb140cefa0 0x7fdb140cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 -- 172.21.15.46:0/2194011701 >> 172.21.15.46:0/2194011701 conn(0x7fdb140b2040 msgr2=0x7fdb140b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 -- 172.21.15.46:0/2194011701 shutdown_connections 2024-09-17T03:34:25.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 -- 172.21.15.46:0/2194011701 wait complete. 2024-09-17T03:34:25.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.031+0000 7fdb1ac00640 1 Processor -- start 2024-09-17T03:34:25.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 -- start start 2024-09-17T03:34:25.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb140749a0 0x7fdb1413a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb140b3340 0x7fdb14138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 0x7fdb1415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14169dc0 con 0x7fdb140cefa0 2024-09-17T03:34:25.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14169c40 con 0x7fdb140749a0 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdb14169f40 con 0x7fdb140b3340 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 0x7fdb1415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 0x7fdb1415bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55968/0 (socket says 172.21.15.46:55968) 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 -- 172.21.15.46:0/1551701389 learned_addr learned my addr 172.21.15.46:0/1551701389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb140749a0 0x7fdb1413a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb0be00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb140b3340 0x7fdb14138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 -- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb140b3340 msgr2=0x7fdb14138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb140b3340 0x7fdb14138bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 -- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb140749a0 msgr2=0x7fdb1413a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb140749a0 0x7fdb1413a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 -- 172.21.15.46:0/1551701389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb141fc4c0 con 0x7fdb140cefa0 2024-09-17T03:34:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 0x7fdb1415bc40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009b10 tx=0x7fdb0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb10c00640 1 -- 172.21.15.46:0/1551701389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0bb070 con 0x7fdb140cefa0 2024-09-17T03:34:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb10c00640 1 -- 172.21.15.46:0/1551701389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0c004d40 con 0x7fdb140cefa0 2024-09-17T03:34:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb10c00640 1 -- 172.21.15.46:0/1551701389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0b2040 con 0x7fdb140cefa0 2024-09-17T03:34:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb141fca20 con 0x7fdb140cefa0 2024-09-17T03:34:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.035+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb141fc740 con 0x7fdb140cefa0 2024-09-17T03:34:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb10c00640 1 -- 172.21.15.46:0/1551701389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fdb0c004260 con 0x7fdb140cefa0 2024-09-17T03:34:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb10c00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdaf0074d00 0x7fdaf0077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb10c00640 1 -- 172.21.15.46:0/1551701389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fdb0c12cf60 con 0x7fdb140cefa0 2024-09-17T03:34:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdaf0074d00 0x7fdaf0077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb22436000 1 Processor -- start 2024-09-17T03:34:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb22436000 1 -- start start 2024-09-17T03:34:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.039+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdaf0074d00 0x7fdaf0077210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdb04000f70 tx=0x7fdb04009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588e5479410 con 0x5588e543cca0 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588e53b6b40 con 0x5588e5453950 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x5588e53a5f70 con 0x5588e5063460 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb0be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb13e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb0be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:52552/0 (socket says 172.21.15.46:52552) 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb0be00640 1 -- 172.21.15.46:0/3078834513 learned_addr learned my addr 172.21.15.46:0/3078834513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 -- 172.21.15.46:0/3078834513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 msgr2=0x5588e55a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 -- 172.21.15.46:0/3078834513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 msgr2=0x5588e55a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 -- 172.21.15.46:0/3078834513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e536d5b0 con 0x5588e543cca0 2024-09-17T03:34:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb18c00640 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55a5120 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c12d370 tx=0x7fdb0c0f5af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee35fb147b19d7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb03e00640 1 -- 172.21.15.46:0/3078834513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c11ae60 con 0x5588e543cca0 2024-09-17T03:34:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb03e00640 1 -- 172.21.15.46:0/3078834513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0c0f8e80 con 0x5588e543cca0 2024-09-17T03:34:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb03e00640 1 -- 172.21.15.46:0/3078834513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c102a70 con 0x5588e543cca0 2024-09-17T03:34:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- 172.21.15.46:0/3078834513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 msgr2=0x5588e55a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55a5120 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c12d370 tx=0x7fdb0c0f5af0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- 172.21.15.46:0/3078834513 shutdown_connections 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55a5120 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 --2- 172.21.15.46:0/3078834513 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- 172.21.15.46:0/3078834513 >> 172.21.15.46:0/3078834513 conn(0x5588e5529d90 msgr2=0x5588e5559690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- 172.21.15.46:0/3078834513 shutdown_connections 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- 172.21.15.46:0/3078834513 wait complete. 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 -- start start 2024-09-17T03:34:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.043+0000 7fdb22436000 1 Processor -- start 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55ab920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588e53405f0 con 0x5588e543cca0 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588e53a9d80 con 0x5588e5453950 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x5588e534c520 con 0x5588e5063460 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 10 client.0 _ll_register_callbacks cb 0x5588e5350830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb13e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55976/0 (socket says 172.21.15.46:55976) 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 -- 172.21.15.46:0/3965271412 learned_addr learned my addr 172.21.15.46:0/3965271412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb18c00640 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 -- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 msgr2=0x5588e55ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 -- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 msgr2=0x5588e55ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 -- 172.21.15.46:0/3965271412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e5344190 con 0x5588e543cca0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb0be00640 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55abe60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdafc012700 tx=0x7fdafc012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead29b88267b5051 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb00c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb00c00640 1 -- 172.21.15.46:0/3965271412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc011870 con 0x5588e543cca0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb00c00640 1 -- 172.21.15.46:0/3965271412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fdafc01daf0 con 0x5588e543cca0 2024-09-17T03:34:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588e533e110 con 0x5588e543cca0 2024-09-17T03:34:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb00c00640 1 -- 172.21.15.46:0/3965271412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fdafc01dd30 con 0x5588e543cca0 2024-09-17T03:34:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 10 client.248316 fetch_fsmap learned FSMap version 75 2024-09-17T03:34:25.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5588e542af10 con 0x5588e543cca0 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb00c00640 1 -- 172.21.15.46:0/3965271412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5588e533e110 con 0x5588e543cca0 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb00c00640 1 -- 172.21.15.46:0/3965271412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7fdafc024050 con 0x5588e543cca0 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 10 client.248316 fetch_fsmap finished waiting for FSMap version 75 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 10 client.248316 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 20 client.248316 populate_metadata read hostname 'smithi046' 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e50487d0 0x5588e55b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 4 client.248316 mds_command: new command op to 248196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 --> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588e53b8a90 con 0x5588e50487d0 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.047+0000 7fdb13e00640 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e50487d0 0x5588e55b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb13e00640 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e50487d0 0x5588e55b5d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8edcd71c4dec21d3 server_cookie=b6f285537c570d7c in_seq=0 out_seq=0 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb00c00640 10 client.248316 ms_handle_connect on v2:172.21.15.46:6836/1168163211 2024-09-17T03:34:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb00c00640 1 -- 172.21.15.46:0/3965271412 <== mds.0 v2:172.21.15.46:6836/1168163211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588e53b8a90 con 0x5588e50487d0 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb00c00640 10 client.248316 handle_command_reply: tid=0 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 client.248316 shutdown 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 msgr2=0x5588e55abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55abe60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdafc012700 tx=0x7fdafc012b00 comp rx=0 tx=0).stop 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 shutdown_connections 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e50487d0 0x5588e55b5d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e55ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e543cca0 0x5588e55abe60 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 --2- 172.21.15.46:0/3965271412 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e5063460 0x5588e55ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb00c00640 0 client.248316 ms_handle_reset on v2:172.21.15.46:6836/1168163211 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 >> 172.21.15.46:0/3965271412 conn(0x5588e5529d90 msgr2=0x5588e5560440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 shutdown_connections 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 1 -- 172.21.15.46:0/3965271412 wait complete. 2024-09-17T03:34:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.051+0000 7fdb22436000 20 client.248316 trim_cache size 0 max 16384 2024-09-17T03:34:25.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 Processor -- start 2024-09-17T03:34:25.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 -- start start 2024-09-17T03:34:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 0x5588e555a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e555a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e555ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14205230 con 0x5588e5063460 2024-09-17T03:34:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14204290 con 0x5588e5453950 2024-09-17T03:34:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdb140ce180 con 0x5588e543cca0 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 0x5588e555a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 0x5588e555a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55992/0 (socket says 172.21.15.46:55992) 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 -- 172.21.15.46:0/1118698350 learned_addr learned my addr 172.21.15.46:0/1118698350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb0be00640 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e555a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e555ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 -- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 msgr2=0x5588e555a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e555a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 -- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 msgr2=0x5588e555ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e555ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 -- 172.21.15.46:0/1118698350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e5401db0 con 0x5588e5063460 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e555ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 0x5588e555a3f0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x5588e53aabf0 tx=0x7fdb04011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c14693b0fe87f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb03400640 1 -- 172.21.15.46:0/1118698350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040039c0 con 0x5588e5063460 2024-09-17T03:34:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb03400640 1 -- 172.21.15.46:0/1118698350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb04003c50 con 0x5588e5063460 2024-09-17T03:34:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.071+0000 7fdb03400640 1 -- 172.21.15.46:0/1118698350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003e40 con 0x5588e5063460 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- 172.21.15.46:0/1118698350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 msgr2=0x5588e555a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 0x5588e555a3f0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x5588e53aabf0 tx=0x7fdb04011000 comp rx=0 tx=0).stop 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- 172.21.15.46:0/1118698350 shutdown_connections 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5453950 0x5588e555ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e555a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- 172.21.15.46:0/1118698350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5063460 0x5588e555a3f0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- 172.21.15.46:0/1118698350 >> 172.21.15.46:0/1118698350 conn(0x5588e5529d90 msgr2=0x5588e5558cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- 172.21.15.46:0/1118698350 shutdown_connections 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- 172.21.15.46:0/1118698350 wait complete. 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- start start 2024-09-17T03:34:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 Processor -- start 2024-09-17T03:34:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5063460 0x5588e561a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e561a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 0x5588e561aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588e5350760 con 0x5588e5453950 2024-09-17T03:34:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5588e53ae3b0 con 0x5588e5063460 2024-09-17T03:34:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x5588e542aa80 con 0x5588e543cca0 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb22436000 10 client.0 _ll_register_callbacks cb 0x7fdafc009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 0x5588e561aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 0x5588e561aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55996/0 (socket says 172.21.15.46:55996) 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 -- 172.21.15.46:0/1411262088 learned_addr learned my addr 172.21.15.46:0/1411262088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb0be00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e561a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5063460 0x5588e561a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 -- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 msgr2=0x5588e561a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e561a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 -- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5063460 msgr2=0x5588e561a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5063460 0x5588e561a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb18c00640 1 -- 172.21.15.46:0/1411262088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e536cab0 con 0x5588e5453950 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.075+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5063460 0x5588e561a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb18c00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 0x5588e561aab0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c01c610 tx=0x7fdb0c102b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bb3ecf8346081a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 1 -- 172.21.15.46:0/1411262088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c11ae60 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 1 -- 172.21.15.46:0/1411262088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fdb0c004550 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb04003c50 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 1 -- 172.21.15.46:0/1411262088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fdb0c0f8e80 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 10 client.248322 fetch_fsmap learned FSMap version 75 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5588e534b070 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 1 -- 172.21.15.46:0/1411262088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb04003c50 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 1 -- 172.21.15.46:0/1411262088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7fdb0c0b5030 con 0x5588e5453950 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 10 client.248322 fetch_fsmap finished waiting for FSMap version 75 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 10 client.248322 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:34:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 20 client.248322 populate_metadata read hostname 'smithi046' 2024-09-17T03:34:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e5036830 0x5588e55d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 4 client.248322 mds_command: new command op to 248196 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:34:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 --> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] -- command(tid 0: {"prefix": "status"}) -- 0x5588e53b4540 con 0x5588e5036830 2024-09-17T03:34:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e5036830 0x5588e55d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb13e00640 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e5036830 0x5588e55d56e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b6f9e834c7411e server_cookie=69b0a8e2737e6da1 in_seq=0 out_seq=0 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 10 client.248322 ms_handle_connect on v2:172.21.15.46:6836/1168163211 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 1 -- 172.21.15.46:0/1411262088 <== mds.0 v2:172.21.15.46:6836/1168163211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5588e53b4540 con 0x5588e5036830 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 10 client.248322 handle_command_reply: tid=0 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 client.248322 shutdown 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 msgr2=0x5588e561aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 0x5588e561aab0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c01c610 tx=0x7fdb0c102b20 comp rx=0 tx=0).stop 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 shutdown_connections 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:6836/1168163211,v1:172.21.15.46:6837/1168163211] conn(0x5588e5036830 0x5588e55d56e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588e5453950 0x5588e561aab0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x5588e543cca0 0x5588e561a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb00c00640 0 client.248322 ms_handle_reset on v2:172.21.15.46:6836/1168163211 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 --2- 172.21.15.46:0/1411262088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588e5063460 0x5588e561a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 >> 172.21.15.46:0/1411262088 conn(0x5588e5529d90 msgr2=0x5588e55b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 shutdown_connections 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 1 -- 172.21.15.46:0/1411262088 wait complete. 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb22436000 20 client.248322 trim_cache size 0 max 16384 2024-09-17T03:34:25.085 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 248196, 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.1454355600000001, 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 75, 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 114, 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 114, 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.532244113999999 2024-09-17T03:34:25.086 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:34:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdaf0074d00 msgr2=0x7fdaf0077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdaf0074d00 0x7fdaf0077210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdb04000f70 tx=0x7fdb04009040 comp rx=0 tx=0).stop 2024-09-17T03:34:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 msgr2=0x7fdb1415bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 0x7fdb1415bc40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009b10 tx=0x7fdb0c009260 comp rx=0 tx=0).stop 2024-09-17T03:34:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 shutdown_connections 2024-09-17T03:34:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdaf0074d00 0x7fdaf0077210 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb140cefa0 0x7fdb1415bc40 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdb140b3340 0x7fdb14138bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 --2- 172.21.15.46:0/1551701389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb140749a0 0x7fdb1413a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 >> 172.21.15.46:0/1551701389 conn(0x7fdb140b2040 msgr2=0x7fdb140b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 shutdown_connections 2024-09-17T03:34:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.079+0000 7fdb1ac00640 1 -- 172.21.15.46:0/1551701389 wait complete. 2024-09-17T03:34:25.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 -- 172.21.15.46:0/2805317117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 msgr2=0x7fc4600ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 --2- 172.21.15.46:0/2805317117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc4600ce190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc458009bc0 tx=0x7fc45801ca30 comp rx=0 tx=0).stop 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 -- 172.21.15.46:0/2805317117 shutdown_connections 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 --2- 172.21.15.46:0/2805317117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc4600ce190 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 -- 172.21.15.46:0/2805317117 >> 172.21.15.46:0/2805317117 conn(0x7fc4600b41f0 msgr2=0x7fc4600b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 -- 172.21.15.46:0/2805317117 shutdown_connections 2024-09-17T03:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.259+0000 7fc45fe00640 1 -- 172.21.15.46:0/2805317117 wait complete. 2024-09-17T03:34:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 Processor -- start 2024-09-17T03:34:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 -- start start 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 0x7fc46009a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc460155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc46009a7b0 0x7fc46015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc46016b4f0 con 0x7fc4600746e0 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc46016b370 con 0x7fc46015c920 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc46016b670 con 0x7fc46009a7b0 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 0x7fc46009a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 0x7fc46009a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56010/0 (socket says 172.21.15.46:56010) 2024-09-17T03:34:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.263+0000 7fc45ea00640 1 -- 172.21.15.46:0/1060800021 learned_addr learned my addr 172.21.15.46:0/1060800021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45ea00640 1 -- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc46009a7b0 msgr2=0x7fc46015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc46009a7b0 0x7fc46015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45ea00640 1 -- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 msgr2=0x7fc460155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc456000640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc460155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc460155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45ea00640 1 -- 172.21.15.46:0/1060800021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc460156160 con 0x7fc4600746e0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc456000640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc460155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45ea00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 0x7fc46009a180 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc45000c450 tx=0x7fc45000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500052d0 con 0x7fc4600746e0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc450005470 con 0x7fc4600746e0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450012870 con 0x7fc4600746e0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc460156e70 con 0x7fc4600746e0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4600d0310 con 0x7fc4600746e0 2024-09-17T03:34:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc460074c30 con 0x7fc4600746e0 2024-09-17T03:34:25.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.267+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fc450003600 con 0x7fc4600746e0 2024-09-17T03:34:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.271+0000 7fc457400640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc43c074d00 0x7fc43c077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.271+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fc450093720 con 0x7fc4600746e0 2024-09-17T03:34:25.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.271+0000 7fc456000640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc43c074d00 0x7fc43c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:25.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.271+0000 7fc456000640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc43c074d00 0x7fc43c077210 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc4600cb700 tx=0x7fc448009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:25.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.271+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc450060240 con 0x7fc4600746e0 2024-09-17T03:34:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc46015f570 con 0x7fc4600746e0 2024-09-17T03:34:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc457400640 1 -- 172.21.15.46:0/1060800021 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7fc45005fb40 con 0x7fc4600746e0 2024-09-17T03:34:25.488 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc43c074d00 msgr2=0x7fc43c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc43c074d00 0x7fc43c077210 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc4600cb700 tx=0x7fc448009040 comp rx=0 tx=0).stop 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 msgr2=0x7fc46009a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 0x7fc46009a180 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc45000c450 tx=0x7fc45000c910 comp rx=0 tx=0).stop 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 shutdown_connections 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc43c074d00 0x7fc43c077210 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc46009a7b0 0x7fc46015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46015c920 0x7fc460155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 --2- 172.21.15.46:0/1060800021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4600746e0 0x7fc46009a180 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 >> 172.21.15.46:0/1060800021 conn(0x7fc4600b41f0 msgr2=0x7fc460068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 shutdown_connections 2024-09-17T03:34:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25.487+0000 7fc45fe00640 1 -- 172.21.15.46:0/1060800021 wait complete. 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:34:25.511 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:34:25.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:25.512 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:34:25.521 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41623sec preferred_lft 41623sec 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:34:25.522 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:34:25.523 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:34:25.524 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.524 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:34:25.524 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:34:25.524 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:34:25.524 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:34:25.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:25.525 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:34:25.584 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:34:25.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:25.587 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:34:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:34:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:34:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:34:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:34:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:34:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:34:25.642 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:34:25.643 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:34:25.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:34:25.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:34:25.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:34:25.643 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:34:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:34:25.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:25.779 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:34:25.789 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:34:25.789 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:34:25.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:34:25.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:25.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:34:25.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:34:25.851 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:34:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:34:25.903 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:34:25.904 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:34:25.955 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:34:25.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:25.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:34:26.064 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:34:26.064 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:34:26.078 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:34:26.078 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:34:26.078 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:34:26.078 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:34:26.078 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:34:26.082 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:34:26.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:26.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:34:26.102 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:34:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:34:26.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:34:26.211 INFO:teuthology.orchestra.run.smithi046.stdout:client247187 2024-09-17T03:34:26.211 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:26.211 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:26.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:34:26.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 -- 172.21.15.46:0/1075488711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81700b4ce0 msgr2=0x7f81700b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 --2- 172.21.15.46:0/1075488711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81700b4ce0 0x7f81700b50f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f816800ad00 tx=0x7f816801df10 comp rx=0 tx=0).stop 2024-09-17T03:34:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 -- 172.21.15.46:0/1075488711 shutdown_connections 2024-09-17T03:34:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 --2- 172.21.15.46:0/1075488711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f81700b4ce0 0x7f81700b50f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 -- 172.21.15.46:0/1075488711 >> 172.21.15.46:0/1075488711 conn(0x7f81700b2040 msgr2=0x7f81700b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 -- 172.21.15.46:0/1075488711 shutdown_connections 2024-09-17T03:34:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.426+0000 7f8175800640 1 -- 172.21.15.46:0/1075488711 wait complete. 2024-09-17T03:34:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 Processor -- start 2024-09-17T03:34:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 -- start start 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 0x7f8170152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81700cf900 0x7f81700ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f817013d940 0x7f81700aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81701690c0 con 0x7f81700b4ce0 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8170168f40 con 0x7f81700cf900 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f8175800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8170169240 con 0x7f817013d940 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 0x7f8170152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:26.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 0x7f8170152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56022/0 (socket says 172.21.15.46:56022) 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 -- 172.21.15.46:0/1639837119 learned_addr learned my addr 172.21.15.46:0/1639837119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816f400640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f817013d940 0x7f81700aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816e000640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81700cf900 0x7f81700ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 -- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f817013d940 msgr2=0x7f81700aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f817013d940 0x7f81700aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 -- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81700cf900 msgr2=0x7f81700ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.430+0000 7f816ea00640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81700cf900 0x7f81700ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f816ea00640 1 -- 172.21.15.46:0/1639837119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81700afa00 con 0x7f81700b4ce0 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f816f400640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f817013d940 0x7f81700aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f816e000640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81700cf900 0x7f81700ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f816ea00640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 0x7f8170152ac0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f816000c430 tx=0x7f816000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160011790 con 0x7f81700b4ce0 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8160017070 con 0x7f81700b4ce0 2024-09-17T03:34:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81700cdad0 con 0x7f81700b4ce0 2024-09-17T03:34:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160011ce0 con 0x7f81700b4ce0 2024-09-17T03:34:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81700ce090 con 0x7f81700b4ce0 2024-09-17T03:34:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81700b50f0 con 0x7f81700b4ce0 2024-09-17T03:34:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f816001c460 con 0x7f81700b4ce0 2024-09-17T03:34:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8166a00640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8150074d30 0x7f8150077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.434+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f8160092ff0 con 0x7f81700b4ce0 2024-09-17T03:34:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.438+0000 7f816e000640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8150074d30 0x7f8150077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.438+0000 7f816e000640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8150074d30 0x7f8150077240 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f817015af40 tx=0x7f815c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:26.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.442+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f816005fab0 con 0x7f81700b4ce0 2024-09-17T03:34:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.654+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f81701533c0 con 0x7f81700b4ce0 2024-09-17T03:34:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.990+0000 7f8166a00640 1 -- 172.21.15.46:0/1639837119 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v76) ==== 136+0+0 (secure 0 0 0) 0x7f816005f3b0 con 0x7f81700b4ce0 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8150074d30 msgr2=0x7f8150077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8150074d30 0x7f8150077240 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f817015af40 tx=0x7f815c009040 comp rx=0 tx=0).stop 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 msgr2=0x7f8170152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 0x7f8170152ac0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f816000c430 tx=0x7f816000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 shutdown_connections 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8150074d30 0x7f8150077240 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f817013d940 0x7f81700aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81700cf900 0x7f81700ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 --2- 172.21.15.46:0/1639837119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81700b4ce0 0x7f8170152ac0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 >> 172.21.15.46:0/1639837119 conn(0x7f81700b2040 msgr2=0x7f81700b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 shutdown_connections 2024-09-17T03:34:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:26.994+0000 7f8175800640 1 -- 172.21.15.46:0/1639837119 wait complete. 2024-09-17T03:34:27.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 -- 172.21.15.46:0/934053014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c074b20 msgr2=0x7f645c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 --2- 172.21.15.46:0/934053014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c074b20 0x7f645c15cf90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f6454009c80 tx=0x7f645401cda0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 -- 172.21.15.46:0/934053014 shutdown_connections 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 --2- 172.21.15.46:0/934053014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c074b20 0x7f645c15cf90 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 --2- 172.21.15.46:0/934053014 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c157030 0x7f645c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 -- 172.21.15.46:0/934053014 >> 172.21.15.46:0/934053014 conn(0x7f645c0708c0 msgr2=0x7f645c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 -- 172.21.15.46:0/934053014 shutdown_connections 2024-09-17T03:34:27.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.174+0000 7f6462800640 1 -- 172.21.15.46:0/934053014 wait complete. 2024-09-17T03:34:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 Processor -- start 2024-09-17T03:34:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 -- start start 2024-09-17T03:34:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 0x7f645c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c160d50 0x7f645c1f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f645c1f9c40 0x7f645c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f645c16a7d0 con 0x7f645c157030 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f645c16a650 con 0x7f645c1f9c40 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6462800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f645c16a950 con 0x7f645c160d50 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f645b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 0x7f645c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f645b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 0x7f645c0b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56044/0 (socket says 172.21.15.46:56044) 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f645b400640 1 -- 172.21.15.46:0/1745228335 learned_addr learned my addr 172.21.15.46:0/1745228335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f645be00640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f645c1f9c40 0x7f645c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.178+0000 7f6452a00640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c160d50 0x7f645c1f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f645b400640 1 -- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c160d50 msgr2=0x7f645c1f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f645b400640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c160d50 0x7f645c1f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f645b400640 1 -- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f645c1f9c40 msgr2=0x7f645c14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f645b400640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f645c1f9c40 0x7f645c14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f645b400640 1 -- 172.21.15.46:0/1745228335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c152be0 con 0x7f645c157030 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6452a00640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c160d50 0x7f645c1f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f645b400640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 0x7f645c0b48d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f644c0058d0 tx=0x7f644c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c017070 con 0x7f645c157030 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f644c009d20 con 0x7f645c157030 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c152ec0 con 0x7f645c157030 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0106c0 con 0x7f645c157030 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c0b2400 con 0x7f645c157030 2024-09-17T03:34:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c0d2260 con 0x7f645c157030 2024-09-17T03:34:27.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.182+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f644c01a070 con 0x7f645c157030 2024-09-17T03:34:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.186+0000 7f6453e00640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6438074d00 0x7f6438077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.186+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f644c093270 con 0x7f645c157030 2024-09-17T03:34:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.186+0000 7f6452a00640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6438074d00 0x7f6438077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.186+0000 7f6452a00640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6438074d00 0x7f6438077210 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f645c14c280 tx=0x7f6444009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.190+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f644c05f6a0 con 0x7f645c157030 2024-09-17T03:34:27.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.382+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f645c06b540 con 0x7f6438074d00 2024-09-17T03:34:27.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6453e00640 1 -- 172.21.15.46:0/1745228335 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6444001c00 con 0x7f6438074d00 2024-09-17T03:34:27.388 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:34:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:34:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:34:27.389 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:34:27.389 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:34:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6438074d00 msgr2=0x7f6438077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6438074d00 0x7f6438077210 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f645c14c280 tx=0x7f6444009040 comp rx=0 tx=0).stop 2024-09-17T03:34:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 msgr2=0x7f645c0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 0x7f645c0b48d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f644c0058d0 tx=0x7f644c00dde0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 shutdown_connections 2024-09-17T03:34:27.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6438074d00 0x7f6438077210 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f645c1f9c40 0x7f645c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f645c160d50 0x7f645c1f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 --2- 172.21.15.46:0/1745228335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f645c157030 0x7f645c0b48d0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 >> 172.21.15.46:0/1745228335 conn(0x7f645c0708c0 msgr2=0x7f645c15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 shutdown_connections 2024-09-17T03:34:27.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.386+0000 7f6462800640 1 -- 172.21.15.46:0/1745228335 wait complete. 2024-09-17T03:34:27.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 -- 172.21.15.46:0/2054616777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00440d29c0 msgr2=0x7f00440d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 --2- 172.21.15.46:0/2054616777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00440d29c0 0x7f00440d2dd0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f002c009840 tx=0x7f002c01c790 comp rx=0 tx=0).stop 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 -- 172.21.15.46:0/2054616777 shutdown_connections 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 --2- 172.21.15.46:0/2054616777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00440d29c0 0x7f00440d2dd0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 -- 172.21.15.46:0/2054616777 >> 172.21.15.46:0/2054616777 conn(0x7f00440708c0 msgr2=0x7f0044070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 -- 172.21.15.46:0/2054616777 shutdown_connections 2024-09-17T03:34:27.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.570+0000 7f004a800640 1 -- 172.21.15.46:0/2054616777 wait complete. 2024-09-17T03:34:27.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 Processor -- start 2024-09-17T03:34:27.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 -- start start 2024-09-17T03:34:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00440d29c0 0x7f00441fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0044155ad0 0x7f00440b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 0x7f004414a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f004416db30 con 0x7f004415f180 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f004416d9b0 con 0x7f00440d29c0 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f004a800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f004416dcb0 con 0x7f0044155ad0 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f0043e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 0x7f004414a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f0043e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 0x7f004414a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56066/0 (socket says 172.21.15.46:56066) 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f0043e00640 1 -- 172.21.15.46:0/2708395846 learned_addr learned my addr 172.21.15.46:0/2708395846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f003be00640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0044155ad0 0x7f00440b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.574+0000 7f0043400640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00440d29c0 0x7f00441fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043e00640 1 -- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0044155ad0 msgr2=0x7f00440b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043e00640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0044155ad0 0x7f00440b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043e00640 1 -- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00440d29c0 msgr2=0x7f00441fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043e00640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00440d29c0 0x7f00441fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043e00640 1 -- 172.21.15.46:0/2708395846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441fa6d0 con 0x7f004415f180 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043400640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00440d29c0 0x7f00441fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f0043e00640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 0x7f004414a790 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c760 tx=0x7f003c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c003b80 con 0x7f004415f180 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f003c003d20 con 0x7f004415f180 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c00aab0 con 0x7f004415f180 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441f9b50 con 0x7f004415f180 2024-09-17T03:34:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441fa110 con 0x7f004415f180 2024-09-17T03:34:27.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00440d2dd0 con 0x7f004415f180 2024-09-17T03:34:27.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f003c00ac50 con 0x7f004415f180 2024-09-17T03:34:27.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f003b400640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0020074d00 0x7f0020077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.578+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f003c092e20 con 0x7f004415f180 2024-09-17T03:34:27.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.582+0000 7f0043400640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0020074d00 0x7f0020077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.582+0000 7f0043400640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0020074d00 0x7f0020077210 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0034000f70 tx=0x7f0034009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:27.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.586+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f003c05f930 con 0x7f004415f180 2024-09-17T03:34:27.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0044151fa0 con 0x7f004415f180 2024-09-17T03:34:27.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f003b400640 1 -- 172.21.15.46:0/2708395846 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f003c05f230 con 0x7f004415f180 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0020074d00 msgr2=0x7f0020077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0020074d00 0x7f0020077210 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0034000f70 tx=0x7f0034009040 comp rx=0 tx=0).stop 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 msgr2=0x7f004414a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 0x7f004414a790 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c760 tx=0x7f003c00cc20 comp rx=0 tx=0).stop 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 shutdown_connections 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0020074d00 0x7f0020077210 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f004415f180 0x7f004414a790 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0044155ad0 0x7f00440b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 --2- 172.21.15.46:0/2708395846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00440d29c0 0x7f00441fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 >> 172.21.15.46:0/2708395846 conn(0x7f00440708c0 msgr2=0x7f004414dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 shutdown_connections 2024-09-17T03:34:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.762+0000 7f004a800640 1 -- 172.21.15.46:0/2708395846 wait complete. 2024-09-17T03:34:27.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:34:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 -- 172.21.15.46:0/2776289700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 msgr2=0x7f52fc06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 --2- 172.21.15.46:0/2776289700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc06d200 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0ce170 tx=0x7f52e401c830 comp rx=0 tx=0).stop 2024-09-17T03:34:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 -- 172.21.15.46:0/2776289700 shutdown_connections 2024-09-17T03:34:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 --2- 172.21.15.46:0/2776289700 >> v2:172.21.15.99:3300/0 conn(0x7f52fc06fc90 0x7f52fc0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 --2- 172.21.15.46:0/2776289700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc06d200 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 -- 172.21.15.46:0/2776289700 >> 172.21.15.46:0/2776289700 conn(0x7f52fc0b35f0 msgr2=0x7f52fc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 -- 172.21.15.46:0/2776289700 shutdown_connections 2024-09-17T03:34:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.942+0000 7f5302400640 1 -- 172.21.15.46:0/2776289700 wait complete. 2024-09-17T03:34:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 Processor -- start 2024-09-17T03:34:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 -- start start 2024-09-17T03:34:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc0afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52fc06fc90 0x7f52fc164890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52fc159130 0x7f52fc164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc168d80 con 0x7f52fc06cdf0 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc168c00 con 0x7f52fc159130 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.946+0000 7f5302400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f52fc168f00 con 0x7f52fc06fc90 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc0afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc0afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56088/0 (socket says 172.21.15.46:56088) 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 -- 172.21.15.46:0/2578235075 learned_addr learned my addr 172.21.15.46:0/2578235075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52f2a00640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52fc06fc90 0x7f52fc164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fbe00640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52fc159130 0x7f52fc164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 -- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52fc06fc90 msgr2=0x7f52fc164890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52fc06fc90 0x7f52fc164890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 -- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52fc159130 msgr2=0x7f52fc164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52fc159130 0x7f52fc164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 -- 172.21.15.46:0/2578235075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc15deb0 con 0x7f52fc06cdf0 2024-09-17T03:34:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52f2a00640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52fc06fc90 0x7f52fc164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52fb400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc0afce0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b410 tx=0x7f52ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec015030 con 0x7f52fc06cdf0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52ec004020 con 0x7f52fc06cdf0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0044b0 con 0x7f52fc06cdf0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc15e190 con 0x7f52fc06cdf0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc0b0220 con 0x7f52fc06cdf0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52fc06d200 con 0x7f52fc06cdf0 2024-09-17T03:34:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.950+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f52ec018070 con 0x7f52fc06cdf0 2024-09-17T03:34:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.954+0000 7f52f3e00640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52d8074d00 0x7f52d8077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.954+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f52ec092d20 con 0x7f52fc06cdf0 2024-09-17T03:34:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.954+0000 7f52f2a00640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52d8074d00 0x7f52d8077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.954+0000 7f52f2a00640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52d8074d00 0x7f52d8077210 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f52e40af000 tx=0x7f52e401b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:27.958+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f52ec05f7a0 con 0x7f52fc06cdf0 2024-09-17T03:34:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f52fc15e6b0 con 0x7f52fc06cdf0 2024-09-17T03:34:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f52f3e00640 1 -- 172.21.15.46:0/2578235075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v52) ==== 116+0+2 (secure 0 0 0) 0x7f52ec05f0a0 con 0x7f52fc06cdf0 2024-09-17T03:34:28.136 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:34:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52d8074d00 msgr2=0x7f52d8077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52d8074d00 0x7f52d8077210 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f52e40af000 tx=0x7f52e401b070 comp rx=0 tx=0).stop 2024-09-17T03:34:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 msgr2=0x7f52fc0afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc0afce0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00b410 tx=0x7f52ec00b8d0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 shutdown_connections 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f52d8074d00 0x7f52d8077210 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52fc159130 0x7f52fc164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f52fc06fc90 0x7f52fc164890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 --2- 172.21.15.46:0/2578235075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52fc06cdf0 0x7f52fc0afce0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 >> 172.21.15.46:0/2578235075 conn(0x7f52fc0b35f0 msgr2=0x7f52fc15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 shutdown_connections 2024-09-17T03:34:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.134+0000 7f5302400640 1 -- 172.21.15.46:0/2578235075 wait complete. 2024-09-17T03:34:28.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:34:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 -- 172.21.15.46:0/3052172948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e940b5c70 msgr2=0x7f7e9415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 --2- 172.21.15.46:0/3052172948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e940b5c70 0x7f7e9415bb60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7e94074b20 tx=0x7f7e8801c760 comp rx=0 tx=0).stop 2024-09-17T03:34:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 -- 172.21.15.46:0/3052172948 shutdown_connections 2024-09-17T03:34:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 --2- 172.21.15.46:0/3052172948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e940b5c70 0x7f7e9415bb60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 -- 172.21.15.46:0/3052172948 >> 172.21.15.46:0/3052172948 conn(0x7f7e940b2040 msgr2=0x7f7e940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 -- 172.21.15.46:0/3052172948 shutdown_connections 2024-09-17T03:34:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.318+0000 7f7e9b800640 1 -- 172.21.15.46:0/3052172948 wait complete. 2024-09-17T03:34:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 Processor -- start 2024-09-17T03:34:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 -- start start 2024-09-17T03:34:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e940b4de0 0x7f7e9415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940b5c70 0x7f7e9414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 0x7f7e94151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94169b30 con 0x7f7e941527b0 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941699b0 con 0x7f7e940b5c70 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7e94169cb0 con 0x7f7e940b4de0 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 0x7f7e94151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 0x7f7e94151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56100/0 (socket says 172.21.15.46:56100) 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 -- 172.21.15.46:0/2555569732 learned_addr learned my addr 172.21.15.46:0/2555569732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e98e00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e940b4de0 0x7f7e9415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e93e00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940b5c70 0x7f7e9414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 -- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e940b4de0 msgr2=0x7f7e9415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e940b4de0 0x7f7e9415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 -- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940b5c70 msgr2=0x7f7e9414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940b5c70 0x7f7e9414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 -- 172.21.15.46:0/2555569732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e940ae8e0 con 0x7f7e941527b0 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e98e00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e940b4de0 0x7f7e9415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e93e00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940b5c70 0x7f7e9414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e99800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 0x7f7e94151630 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c6b0 tx=0x7f7e8c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c003b80 con 0x7f7e941527b0 2024-09-17T03:34:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e8c003d20 con 0x7f7e941527b0 2024-09-17T03:34:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9415deb0 con 0x7f7e941527b0 2024-09-17T03:34:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c005330 con 0x7f7e941527b0 2024-09-17T03:34:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.322+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e940aee80 con 0x7f7e941527b0 2024-09-17T03:34:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.326+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9415bb60 con 0x7f7e941527b0 2024-09-17T03:34:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.326+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f7e8c0054d0 con 0x7f7e941527b0 2024-09-17T03:34:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.326+0000 7f7e90c00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7e74074d00 0x7f7e74077210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.326+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f7e8c092e70 con 0x7f7e941527b0 2024-09-17T03:34:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.326+0000 7f7e98e00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7e74074d00 0x7f7e74077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.326+0000 7f7e98e00640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7e74074d00 0x7f7e74077210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7e880af000 tx=0x7f7e8801afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.330+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7e8c05fa00 con 0x7f7e941527b0 2024-09-17T03:34:28.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.506+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f7e9414f6c0 con 0x7f7e941527b0 2024-09-17T03:34:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e90c00640 1 -- 172.21.15.46:0/2555569732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v53)=0 v53) ==== 131+0+0 (secure 0 0 0) 0x7f7e8c05f300 con 0x7f7e941527b0 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7e74074d00 msgr2=0x7f7e74077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7e74074d00 0x7f7e74077210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7e880af000 tx=0x7f7e8801afa0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 msgr2=0x7f7e94151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 0x7f7e94151630 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c6b0 tx=0x7f7e8c00cb70 comp rx=0 tx=0).stop 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 shutdown_connections 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7e74074d00 0x7f7e74077210 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e941527b0 0x7f7e94151630 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e940b5c70 0x7f7e9414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 --2- 172.21.15.46:0/2555569732 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7e940b4de0 0x7f7e9415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 >> 172.21.15.46:0/2555569732 conn(0x7f7e940b2040 msgr2=0x7f7e940b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 shutdown_connections 2024-09-17T03:34:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.510+0000 7f7e9b800640 1 -- 172.21.15.46:0/2555569732 wait complete. 2024-09-17T03:34:28.534 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.530+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.534 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.530+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.530+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.530+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.530+0000 7fbe51800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:34:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7294200640 1 -- 172.21.15.46:0/2671896447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72740039c0 msgr2=0x7f7274003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7294200640 1 --2- 172.21.15.46:0/2671896447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72740039c0 0x7f7274003e30 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f728000c610 tx=0x7f728001d0e0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7292200640 1 -- 172.21.15.46:0/2671896447 reap_dead start 2024-09-17T03:34:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7294200640 1 -- 172.21.15.46:0/2671896447 shutdown_connections 2024-09-17T03:34:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7294200640 1 -- 172.21.15.46:0/2671896447 >> 172.21.15.46:0/2671896447 conn(0x7f728c0b2c40 msgr2=0x7f728c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7294200640 1 -- 172.21.15.46:0/2671896447 shutdown_connections 2024-09-17T03:34:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.698+0000 7f7294200640 1 -- 172.21.15.46:0/2671896447 wait complete. 2024-09-17T03:34:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 Processor -- start 2024-09-17T03:34:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 -- start start 2024-09-17T03:34:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72740039c0 0x7f728c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f728c1ec900 0x7f728c1fa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 0x7f728c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1675a0 con 0x7f728c1f7250 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f728c167420 con 0x7f72740039c0 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7294200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f728c167720 con 0x7f728c1ec900 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7292200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 0x7f728c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7292200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 0x7f728c15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56124/0 (socket says 172.21.15.46:56124) 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7292200640 1 -- 172.21.15.46:0/420882917 learned_addr learned my addr 172.21.15.46:0/420882917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.702+0000 7f7290e00640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f728c1ec900 0x7f728c1fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7291800640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72740039c0 0x7f728c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7292200640 1 -- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f728c1ec900 msgr2=0x7f728c1fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7292200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f728c1ec900 0x7f728c1fa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7292200640 1 -- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72740039c0 msgr2=0x7f728c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7292200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72740039c0 0x7f728c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7292200640 1 -- 172.21.15.46:0/420882917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c15de30 con 0x7f728c1f7250 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7291800640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72740039c0 0x7f728c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7292200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 0x7f728c15bcb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f728c070e20 tx=0x7f7284011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284004b30 con 0x7f728c1f7250 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7284004cd0 con 0x7f728c1f7250 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284003be0 con 0x7f728c1f7250 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c15e110 con 0x7f728c1f7250 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1f6a50 con 0x7f728c1f7250 2024-09-17T03:34:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c1606b0 con 0x7f728c1f7250 2024-09-17T03:34:28.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.706+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f7284004030 con 0x7f728c1f7250 2024-09-17T03:34:28.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.710+0000 7f7289600640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f726c074d30 0x7f726c077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.710+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f7284093cd0 con 0x7f728c1f7250 2024-09-17T03:34:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.710+0000 7f7291800640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f726c074d30 0x7f726c077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.710+0000 7f7291800640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f726c074d30 0x7f726c077240 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f72800ac000 tx=0x7f7280019e80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.710+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f72840607d0 con 0x7f728c1f7250 2024-09-17T03:34:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.910+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f728c156b30 con 0x7f726c074d30 2024-09-17T03:34:28.922 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.918+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.922 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.918+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.922 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.918+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.922 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.918+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.922 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:34:28.918+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:34:28.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7289600640 1 -- 172.21.15.46:0/420882917 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f728c156b30 con 0x7f726c074d30 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f726c074d30 msgr2=0x7f726c077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f726c074d30 0x7f726c077240 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f72800ac000 tx=0x7f7280019e80 comp rx=0 tx=0).stop 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 msgr2=0x7f728c15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 0x7f728c15bcb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f728c070e20 tx=0x7f7284011ef0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 shutdown_connections 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f726c074d30 0x7f726c077240 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c1f7250 0x7f728c15bcb0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f728c1ec900 0x7f728c1fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 --2- 172.21.15.46:0/420882917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72740039c0 0x7f728c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 >> 172.21.15.46:0/420882917 conn(0x7f728c0b2c40 msgr2=0x7f728c13efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 shutdown_connections 2024-09-17T03:34:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:28.954+0000 7f7294200640 1 -- 172.21.15.46:0/420882917 wait complete. 2024-09-17T03:34:28.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:34:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.138+0000 7f7feba00640 1 -- 172.21.15.46:0/4113644627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 msgr2=0x7f7fc800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.138+0000 7f7feba00640 1 --2- 172.21.15.46:0/4113644627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fc800b9d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c640 tx=0x7f7fdc01f9c0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.138+0000 7f7feba00640 1 -- 172.21.15.46:0/4113644627 shutdown_connections 2024-09-17T03:34:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.138+0000 7f7feba00640 1 --2- 172.21.15.46:0/4113644627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fc800b9d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.138+0000 7f7feba00640 1 -- 172.21.15.46:0/4113644627 >> 172.21.15.46:0/4113644627 conn(0x7f7fe40b2040 msgr2=0x7f7fe40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 -- 172.21.15.46:0/4113644627 shutdown_connections 2024-09-17T03:34:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 -- 172.21.15.46:0/4113644627 wait complete. 2024-09-17T03:34:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 Processor -- start 2024-09-17T03:34:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 -- start start 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fe4135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe40b0f70 0x7f7fe40af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fe41348b0 0x7f7fe4134cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe415ecd0 con 0x7f7fc8005280 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe40b5890 con 0x7f7fe40b0f70 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.142+0000 7f7feba00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe415ee50 con 0x7f7fe41348b0 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fe4135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fe4135450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56144/0 (socket says 172.21.15.46:56144) 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 -- 172.21.15.46:0/2683392006 learned_addr learned my addr 172.21.15.46:0/2683392006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9a00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fe41348b0 0x7f7fe4134cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fdbe00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe40b0f70 0x7f7fe40af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 -- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fe41348b0 msgr2=0x7f7fe4134cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fe41348b0 0x7f7fe4134cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 -- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe40b0f70 msgr2=0x7f7fe40af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe40b0f70 0x7f7fe40af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 -- 172.21.15.46:0/2683392006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe4135200 con 0x7f7fc8005280 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fdbe00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe40b0f70 0x7f7fe40af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:34:29.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe9000640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fe4135450 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4003ed0 tx=0x7f7fd4001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4015070 con 0x7f7fc8005280 2024-09-17T03:34:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fd4011840 con 0x7f7fc8005280 2024-09-17T03:34:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe40aef10 con 0x7f7fc8005280 2024-09-17T03:34:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4011c70 con 0x7f7fc8005280 2024-09-17T03:34:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe40aba10 con 0x7f7fc8005280 2024-09-17T03:34:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe415b2d0 con 0x7f7fc8005280 2024-09-17T03:34:29.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.146+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7f7fd4018070 con 0x7f7fc8005280 2024-09-17T03:34:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.150+0000 7f7fe1600640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fc0074d30 0x7f7fc0077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.150+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f7fd4093ce0 con 0x7f7fc8005280 2024-09-17T03:34:29.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.150+0000 7f7fdbe00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fc0074d30 0x7f7fc0077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.150+0000 7f7fdbe00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fc0074d30 0x7f7fc0077240 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4138f30 tx=0x7f7fcc0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:29.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.154+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7fe4135200 con 0x7f7fc8005280 2024-09-17T03:34:29.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.354+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7fe40afa40 con 0x7f7fc0074d30 2024-09-17T03:34:29.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.358+0000 7f7fe1600640 1 -- 172.21.15.46:0/2683392006 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f7fe40afa40 con 0x7f7fc0074d30 2024-09-17T03:34:29.361 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.358+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fc0074d30 msgr2=0x7f7fc0077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.358+0000 7f7feba00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fc0074d30 0x7f7fc0077240 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4138f30 tx=0x7f7fcc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.358+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 msgr2=0x7f7fe4135450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.358+0000 7f7feba00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fe4135450 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4003ed0 tx=0x7f7fd4001720 comp rx=0 tx=0).stop 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 shutdown_connections 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7fc0074d30 0x7f7fc0077240 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fe41348b0 0x7f7fe4134cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe40b0f70 0x7f7fe40af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 --2- 172.21.15.46:0/2683392006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fc8005280 0x7f7fe4135450 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 >> 172.21.15.46:0/2683392006 conn(0x7f7fe40b2040 msgr2=0x7f7fe41599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 shutdown_connections 2024-09-17T03:34:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.362+0000 7f7feba00640 1 -- 172.21.15.46:0/2683392006 wait complete. 2024-09-17T03:34:29.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:34:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 -- 172.21.15.46:0/4082320406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa214003a10 msgr2=0x7fa214003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 --2- 172.21.15.46:0/4082320406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa214003a10 0x7fa214003e40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c310 tx=0x7fa22001cf50 comp rx=0 tx=0).stop 2024-09-17T03:34:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 -- 172.21.15.46:0/4082320406 shutdown_connections 2024-09-17T03:34:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 --2- 172.21.15.46:0/4082320406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa214003a10 0x7fa214003e40 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 -- 172.21.15.46:0/4082320406 >> 172.21.15.46:0/4082320406 conn(0x7fa2300b2040 msgr2=0x7fa2300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:29.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 -- 172.21.15.46:0/4082320406 shutdown_connections 2024-09-17T03:34:29.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 -- 172.21.15.46:0/4082320406 wait complete. 2024-09-17T03:34:29.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 Processor -- start 2024-09-17T03:34:29.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.546+0000 7fa236200640 1 -- start start 2024-09-17T03:34:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa214003a10 0x7fa2301f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa230151ae0 0x7fa230144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 0x7fa2300b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301529c0 con 0x7fa2300b3cd0 2024-09-17T03:34:29.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa230152840 con 0x7fa214003a10 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa23015f180 con 0x7fa230151ae0 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 0x7fa2300b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 0x7fa2300b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56156/0 (socket says 172.21.15.46:56156) 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 -- 172.21.15.46:0/1361818764 learned_addr learned my addr 172.21.15.46:0/1361818764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22f400640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa214003a10 0x7fa2301f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa227e00640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa230151ae0 0x7fa230144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 -- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa230151ae0 msgr2=0x7fa230144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa230151ae0 0x7fa230144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 -- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa214003a10 msgr2=0x7fa2301f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa214003a10 0x7fa2301f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 -- 172.21.15.46:0/1361818764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2300b4620 con 0x7fa2300b3cd0 2024-09-17T03:34:29.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa22fe00640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 0x7fa2300b40e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa230071130 tx=0x7fa22800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228010ea0 con 0x7fa2300b3cd0 2024-09-17T03:34:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa228015070 con 0x7fa2300b3cd0 2024-09-17T03:34:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228003be0 con 0x7fa2300b3cd0 2024-09-17T03:34:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301fa680 con 0x7fa2300b3cd0 2024-09-17T03:34:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301fac40 con 0x7fa2300b3cd0 2024-09-17T03:34:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.550+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23015f6e0 con 0x7fa2300b3cd0 2024-09-17T03:34:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.554+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) ==== 99682+0+0 (secure 0 0 0) 0x7fa228004050 con 0x7fa2300b3cd0 2024-09-17T03:34:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.554+0000 7fa227400640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa20c074d30 0x7fa20c077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.554+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fa228093ef0 con 0x7fa2300b3cd0 2024-09-17T03:34:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.554+0000 7fa22f400640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa20c074d30 0x7fa20c077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:29.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.554+0000 7fa22f400640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa20c074d30 0x7fa20c077240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa2200ac000 tx=0x7fa220019d90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.558+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa2300b4620 con 0x7fa2300b3cd0 2024-09-17T03:34:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.758+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa23006b540 con 0x7fa20c074d30 2024-09-17T03:34:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.762+0000 7fa227400640 1 -- 172.21.15.46:0/1361818764 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa220001600 con 0x7fa20c074d30 2024-09-17T03:34:29.766 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa20c074d30 msgr2=0x7fa20c077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa20c074d30 0x7fa20c077240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa2200ac000 tx=0x7fa220019d90 comp rx=0 tx=0).stop 2024-09-17T03:34:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 msgr2=0x7fa2300b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 0x7fa2300b40e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa230071130 tx=0x7fa22800e720 comp rx=0 tx=0).stop 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 shutdown_connections 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa20c074d30 0x7fa20c077240 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2300b3cd0 0x7fa2300b40e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa230151ae0 0x7fa230144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 --2- 172.21.15.46:0/1361818764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa214003a10 0x7fa2301f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 >> 172.21.15.46:0/1361818764 conn(0x7fa2300b2040 msgr2=0x7fa2300b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 shutdown_connections 2024-09-17T03:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:29.766+0000 7fa236200640 1 -- 172.21.15.46:0/1361818764 wait complete. 2024-09-17T03:34:29.789 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:29.789 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:29.789 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:29.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:29.799 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:34:30.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:34:30.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:34:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:34:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:34:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.52993 s, 126 MB/s 2024-09-17T03:34:38.388 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:38.388 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:38.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:38.397 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:34:46.686 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:34:46.686 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:34:46.686 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.23736 s, 130 MB/s 2024-09-17T03:34:46.688 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:34:46.688 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:34:46.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:46.697 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:34:55.273 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:34:55.273 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:34:55.273 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.52476 s, 126 MB/s 2024-09-17T03:34:55.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 -- 172.21.15.46:0/2507034121 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c200b54f0 msgr2=0x7f1c2015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 --2- 172.21.15.46:0/2507034121 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c200b54f0 0x7f1c2015bb60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009870 tx=0x7f1c1801c750 comp rx=0 tx=0).stop 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 -- 172.21.15.46:0/2507034121 shutdown_connections 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 --2- 172.21.15.46:0/2507034121 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c200b54f0 0x7f1c2015bb60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 -- 172.21.15.46:0/2507034121 >> 172.21.15.46:0/2507034121 conn(0x7f1c200b2040 msgr2=0x7f1c200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 -- 172.21.15.46:0/2507034121 shutdown_connections 2024-09-17T03:34:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 -- 172.21.15.46:0/2507034121 wait complete. 2024-09-17T03:34:55.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 Processor -- start 2024-09-17T03:34:55.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.438+0000 7f1c25600640 1 -- start start 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 0x7f1c20148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c201527b0 0x7f1c200cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c20149970 0x7f1c200cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20169bd0 con 0x7f1c200b5c70 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20169a50 con 0x7f1c201527b0 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1c20169d50 con 0x7f1c20149970 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 0x7f1c20148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 0x7f1c20148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53474/0 (socket says 172.21.15.46:53474) 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 -- 172.21.15.46:0/994090935 learned_addr learned my addr 172.21.15.46:0/994090935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1f400640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c20149970 0x7f1c200cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c16000640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c201527b0 0x7f1c200cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 -- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c20149970 msgr2=0x7f1c200cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c20149970 0x7f1c200cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 -- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c201527b0 msgr2=0x7f1c200cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c201527b0 0x7f1c200cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 -- 172.21.15.46:0/994090935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c200cf1c0 con 0x7f1c200b5c70 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c1ea00640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 0x7f1c20148580 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000c780 tx=0x7f1c1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10003b80 con 0x7f1c200b5c70 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c10003d20 con 0x7f1c200b5c70 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c200cf6d0 con 0x7f1c200b5c70 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000aab0 con 0x7f1c200b5c70 2024-09-17T03:34:55.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c20149170 con 0x7f1c200b5c70 2024-09-17T03:34:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2015bb60 con 0x7f1c200b5c70 2024-09-17T03:34:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.442+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 44) ==== 99784+0+0 (secure 0 0 0) 0x7f1c1000ac50 con 0x7f1c200b5c70 2024-09-17T03:34:55.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.446+0000 7f1c17400640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bfc074e50 0x7f1bfc077360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.446+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f1c10092df0 con 0x7f1c200b5c70 2024-09-17T03:34:55.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.446+0000 7f1c16000640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bfc074e50 0x7f1bfc077360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.446+0000 7f1c16000640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bfc074e50 0x7f1bfc077360 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1c08000c10 tx=0x7f1c08001730 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.450+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1c1005f2b0 con 0x7f1c200b5c70 2024-09-17T03:34:55.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.654+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f1c2006b540 con 0x7f1bfc074e50 2024-09-17T03:34:55.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.666+0000 7f1c17400640 1 -- 172.21.15.46:0/994090935 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1c0800f820 con 0x7f1bfc074e50 2024-09-17T03:34:55.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bfc074e50 msgr2=0x7f1bfc077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bfc074e50 0x7f1bfc077360 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1c08000c10 tx=0x7f1c08001730 comp rx=0 tx=0).stop 2024-09-17T03:34:55.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 msgr2=0x7f1c20148580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 0x7f1c20148580 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000c780 tx=0x7f1c1000cc40 comp rx=0 tx=0).stop 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 shutdown_connections 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1bfc074e50 0x7f1bfc077360 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c20149970 0x7f1c200cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c201527b0 0x7f1c200cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 --2- 172.21.15.46:0/994090935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200b5c70 0x7f1c20148580 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 >> 172.21.15.46:0/994090935 conn(0x7f1c200b2040 msgr2=0x7f1c20166a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 shutdown_connections 2024-09-17T03:34:55.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.670+0000 7f1c25600640 1 -- 172.21.15.46:0/994090935 wait complete. 2024-09-17T03:34:55.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 -- 172.21.15.46:0/744469362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8204074300 msgr2=0x7f820415f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 --2- 172.21.15.46:0/744469362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8204074300 0x7f820415f170 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8204071970 tx=0x7f81ec01c890 comp rx=0 tx=0).stop 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 -- 172.21.15.46:0/744469362 shutdown_connections 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 --2- 172.21.15.46:0/744469362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8204074300 0x7f820415f170 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 --2- 172.21.15.46:0/744469362 >> v2:172.21.15.99:3300/0 conn(0x7f820415c5d0 0x7f8204073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 -- 172.21.15.46:0/744469362 >> 172.21.15.46:0/744469362 conn(0x7f82040b2c40 msgr2=0x7f82040b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 -- 172.21.15.46:0/744469362 shutdown_connections 2024-09-17T03:34:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 -- 172.21.15.46:0/744469362 wait complete. 2024-09-17T03:34:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 Processor -- start 2024-09-17T03:34:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.854+0000 7f820a400640 1 -- start start 2024-09-17T03:34:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8204074300 0x7f82041fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 0x7f820414d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f820415f6b0 0x7f820415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f820406f980 con 0x7f820415f6b0 2024-09-17T03:34:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82040b3d50 con 0x7f8204074300 2024-09-17T03:34:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f820406fb00 con 0x7f820415c5d0 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f8203e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f820415f6b0 0x7f820415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 0x7f820414d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f8203400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8204074300 0x7f82041fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 0x7f820414d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:38290/0 (socket says 172.21.15.46:38290) 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f8203400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8204074300 0x7f82041fcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:33614/0 (socket says 172.21.15.46:33614) 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 -- 172.21.15.46:0/1872603222 learned_addr learned my addr 172.21.15.46:0/1872603222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 -- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8204074300 msgr2=0x7f82041fcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8204074300 0x7f82041fcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 -- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f820415f6b0 msgr2=0x7f820415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f820415f6b0 0x7f820415a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 -- 172.21.15.46:0/1872603222 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82040cea70 con 0x7f820415c5d0 2024-09-17T03:34:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f8203e00640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f820415f6b0 0x7f820415a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:34:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fbe00640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 0x7f820414d2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0194a0 tx=0x7f81ec004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0bc070 con 0x7f820415c5d0 2024-09-17T03:34:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81ec005320 con 0x7f820415c5d0 2024-09-17T03:34:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0b2040 con 0x7f820415c5d0 2024-09-17T03:34:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82040cdcb0 con 0x7f820415c5d0 2024-09-17T03:34:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f82040ce270 con 0x7f820415c5d0 2024-09-17T03:34:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.858+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820415f170 con 0x7f820415c5d0 2024-09-17T03:34:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.862+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 44) ==== 99784+0+0 (secure 0 0 0) 0x7f81ec0a5a40 con 0x7f820415c5d0 2024-09-17T03:34:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.862+0000 7f81fb400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81e0074e50 0x7f81e0077360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:34:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.862+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f81ec12cd00 con 0x7f820415c5d0 2024-09-17T03:34:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.862+0000 7f8203400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81e0074e50 0x7f81e0077360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:34:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.862+0000 7f8203400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81e0074e50 0x7f81e0077360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f81f4004630 tx=0x7f81f400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:34:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:55.862+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f81ec0164b0 con 0x7f820415c5d0 2024-09-17T03:34:56.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.066+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f82040cd370 con 0x7f81e0074e50 2024-09-17T03:34:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f81fb400640 1 -- 172.21.15.46:0/1872603222 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f82040cd370 con 0x7f81e0074e50 2024-09-17T03:34:56.073 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81e0074e50 msgr2=0x7f81e0077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81e0074e50 0x7f81e0077360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f81f4004630 tx=0x7f81f400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 msgr2=0x7f820414d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 0x7f820414d2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0194a0 tx=0x7f81ec004600 comp rx=0 tx=0).stop 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 shutdown_connections 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f81e0074e50 0x7f81e0077360 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f820415f6b0 0x7f820415a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f820415c5d0 0x7f820414d2a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:56.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 --2- 172.21.15.46:0/1872603222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8204074300 0x7f82041fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:34:56.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 >> 172.21.15.46:0/1872603222 conn(0x7f82040b2c40 msgr2=0x7f8204154a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:34:56.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 shutdown_connections 2024-09-17T03:34:56.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:34:56.070+0000 7f820a400640 1 -- 172.21.15.46:0/1872603222 wait complete. 2024-09-17T03:34:56.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:56.096 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:56.096 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:34:56.105 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:56.105 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:34:56.105 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:56.106 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:34:58.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:34:58.108 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:58.108 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:34:58.119 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:34:58.119 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:34:58.119 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:34:58.120 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:35:00.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:00.122 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:35:00.122 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:35:00.133 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:35:00.133 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T03:35:00.133 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:00.134 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T03:35:00.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone19820 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 -- 172.21.15.46:0/3459894987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b5c70 msgr2=0x7f63f815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 --2- 172.21.15.46:0/3459894987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b5c70 0x7f63f815bb60 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f63f80cca90 tx=0x7f63e801cac0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 -- 172.21.15.46:0/3459894987 shutdown_connections 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 --2- 172.21.15.46:0/3459894987 >> v2:172.21.15.99:3300/0 conn(0x7f63f80b4de0 0x7f63f80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 --2- 172.21.15.46:0/3459894987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b5c70 0x7f63f815bb60 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 -- 172.21.15.46:0/3459894987 >> 172.21.15.46:0/3459894987 conn(0x7f63f80b2040 msgr2=0x7f63f80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 -- 172.21.15.46:0/3459894987 shutdown_connections 2024-09-17T03:35:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 -- 172.21.15.46:0/3459894987 wait complete. 2024-09-17T03:35:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 Processor -- start 2024-09-17T03:35:00.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.350+0000 7f63fce00640 1 -- start start 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 0x7f63f81fa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63f80b5c70 0x7f63f8149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63f81527b0 0x7f63f815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8168910 con 0x7f63f80b4de0 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8168790 con 0x7f63f81527b0 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f63f8168a90 con 0x7f63f80b5c70 2024-09-17T03:35:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 0x7f63f81fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 0x7f63f81fa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53498/0 (socket says 172.21.15.46:53498) 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 -- 172.21.15.46:0/217876731 learned_addr learned my addr 172.21.15.46:0/217876731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f6800640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63f81527b0 0x7f63f815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63ed400640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63f80b5c70 0x7f63f8149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 -- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63f80b5c70 msgr2=0x7f63f8149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63f80b5c70 0x7f63f8149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 -- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63f81527b0 msgr2=0x7f63f815a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63f81527b0 0x7f63f815a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 -- 172.21.15.46:0/217876731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f8157bd0 con 0x7f63f80b4de0 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63ed400640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63f80b5c70 0x7f63f8149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63f5e00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 0x7f63f81fa810 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f63e801c530 tx=0x7f63e8005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e80b2040 con 0x7f63f80b4de0 2024-09-17T03:35:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63e80bb070 con 0x7f63f80b4de0 2024-09-17T03:35:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81fad50 con 0x7f63f80b4de0 2024-09-17T03:35:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8004370 con 0x7f63f80b4de0 2024-09-17T03:35:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f8158190 con 0x7f63f80b4de0 2024-09-17T03:35:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.354+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f80b51f0 con 0x7f63f80b4de0 2024-09-17T03:35:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.358+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 44) ==== 99784+0+0 (secure 0 0 0) 0x7f63e8004510 con 0x7f63f80b4de0 2024-09-17T03:35:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.358+0000 7f63eea00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63d4074e20 0x7f63d4077330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.358+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f63e812dbe0 con 0x7f63f80b4de0 2024-09-17T03:35:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.358+0000 7f63ed400640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63d4074e20 0x7f63d4077330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.358+0000 7f63ed400640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63d4074e20 0x7f63d4077330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000f70 tx=0x7f63e0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:00.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.362+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f63e8016770 con 0x7f63f80b4de0 2024-09-17T03:35:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.570+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone19820", "target": ["mon-mgr", ""]}) -- 0x7f63f806b540 con 0x7f63d4074e20 2024-09-17T03:35:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.626+0000 7f63eea00640 1 -- 172.21.15.46:0/217876731 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63e0001d20 con 0x7f63d4074e20 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63d4074e20 msgr2=0x7f63d4077330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63d4074e20 0x7f63d4077330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000f70 tx=0x7f63e0009040 comp rx=0 tx=0).stop 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 msgr2=0x7f63f81fa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 0x7f63f81fa810 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f63e801c530 tx=0x7f63e8005ae0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 shutdown_connections 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f63d4074e20 0x7f63d4077330 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63f81527b0 0x7f63f815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63f80b5c70 0x7f63f8149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 --2- 172.21.15.46:0/217876731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f80b4de0 0x7f63f81fa810 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 >> 172.21.15.46:0/217876731 conn(0x7f63f80b2040 msgr2=0x7f63f815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 shutdown_connections 2024-09-17T03:35:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.630+0000 7f63fce00640 1 -- 172.21.15.46:0/217876731 wait complete. 2024-09-17T03:35:00.636 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.634+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.636 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.634+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.636 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.634+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.636 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.634+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.636 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.634+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.655 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.650+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.655 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.650+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.655 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.650+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.655 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.650+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.655 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:00.650+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:00.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone18021 2024-09-17T03:35:00.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:35:00.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:35:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 -- 172.21.15.46:0/3327746185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac1527b0 msgr2=0x7f19ac15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 --2- 172.21.15.46:0/3327746185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac1527b0 0x7f19ac15bb60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1994009900 tx=0x7f199401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 -- 172.21.15.46:0/3327746185 shutdown_connections 2024-09-17T03:35:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 --2- 172.21.15.46:0/3327746185 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 0x7f19ac0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 --2- 172.21.15.46:0/3327746185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac1527b0 0x7f19ac15bb60 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 -- 172.21.15.46:0/3327746185 >> 172.21.15.46:0/3327746185 conn(0x7f19ac0b2040 msgr2=0x7f19ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 -- 172.21.15.46:0/3327746185 shutdown_connections 2024-09-17T03:35:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.826+0000 7f19b1e00640 1 -- 172.21.15.46:0/3327746185 wait complete. 2024-09-17T03:35:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 Processor -- start 2024-09-17T03:35:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 -- start start 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 0x7f19ac1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac0b5c70 0x7f19ac156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 0x7f19ac0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac168950 con 0x7f19ac0b5c70 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac1687d0 con 0x7f19ac1527b0 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19b1e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac168ad0 con 0x7f19ac0b4de0 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19a2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac0b5c70 0x7f19ac156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19ab400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 0x7f19ac0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19aaa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 0x7f19ac1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19ab400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 0x7f19ac0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:33662/0 (socket says 172.21.15.46:33662) 2024-09-17T03:35:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19ab400640 1 -- 172.21.15.46:0/71729569 learned_addr learned my addr 172.21.15.46:0/71729569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19aaa00640 1 -- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 msgr2=0x7f19ac0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19aaa00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 0x7f19ac0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19aaa00640 1 -- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac0b5c70 msgr2=0x7f19ac156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19aaa00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac0b5c70 0x7f19ac156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19aaa00640 1 -- 172.21.15.46:0/71729569 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac154890 con 0x7f19ac0b4de0 2024-09-17T03:35:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.830+0000 7f19ab400640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 0x7f19ac0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:35:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19aaa00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 0x7f19ac1491a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c760 tx=0x7f199c00cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c003b80 con 0x7f19ac0b4de0 2024-09-17T03:35:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f199c003d20 con 0x7f19ac0b4de0 2024-09-17T03:35:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac154b70 con 0x7f19ac0b4de0 2024-09-17T03:35:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c005330 con 0x7f19ac0b4de0 2024-09-17T03:35:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac0cdd30 con 0x7f19ac0b4de0 2024-09-17T03:35:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ac0b51f0 con 0x7f19ac0b4de0 2024-09-17T03:35:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.834+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 44) ==== 99784+0+0 (secure 0 0 0) 0x7f199c0054d0 con 0x7f19ac0b4de0 2024-09-17T03:35:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.838+0000 7f19a3e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1988074e20 0x7f1988077330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.838+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f199c092f10 con 0x7f19ac0b4de0 2024-09-17T03:35:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.838+0000 7f19a2000640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1988074e20 0x7f1988077330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.838+0000 7f19a2000640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1988074e20 0x7f1988077330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f19940af000 tx=0x7f199401b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:00.838+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19ac0b5730 con 0x7f19ac0b4de0 2024-09-17T03:35:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.046+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone18021", "target": ["mon-mgr", ""]}) -- 0x7f19ac06b540 con 0x7f1988074e20 2024-09-17T03:35:01.635 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:01.630+0000 7fbddca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:01.635 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:01.630+0000 7fbddca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:01.635 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:01.630+0000 7fbddca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:01.635 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:01.630+0000 7fbddca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:01.635 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:01.630+0000 7fbddca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.726+0000 7f19a3e00640 1 -- 172.21.15.46:0/71729569 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19940b3040 con 0x7f1988074e20 2024-09-17T03:35:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1988074e20 msgr2=0x7f1988077330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1988074e20 0x7f1988077330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f19940af000 tx=0x7f199401b0f0 comp rx=0 tx=0).stop 2024-09-17T03:35:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 msgr2=0x7f19ac1491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:01.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 0x7f19ac1491a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c760 tx=0x7f199c00cb60 comp rx=0 tx=0).stop 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 shutdown_connections 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1988074e20 0x7f1988077330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19ac1527b0 0x7f19ac0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19ac0b5c70 0x7f19ac156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 --2- 172.21.15.46:0/71729569 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f19ac0b4de0 0x7f19ac1491a0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c760 tx=0x7f199c00cb60 comp rx=0 tx=0).stop 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 >> 172.21.15.46:0/71729569 conn(0x7f19ac0b2040 msgr2=0x7f19ac151840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 shutdown_connections 2024-09-17T03:35:01.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:01.730+0000 7f19b1e00640 1 -- 172.21.15.46:0/71729569 wait complete. 2024-09-17T03:35:01.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80982 2024-09-17T03:35:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.106+0000 7f4447200640 1 -- 172.21.15.46:0/569417658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44400aeff0 msgr2=0x7f444015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.106+0000 7f4447200640 1 --2- 172.21.15.46:0/569417658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44400aeff0 0x7f444015eef0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4440071cb0 tx=0x7f443801ccf0 comp rx=0 tx=0).stop 2024-09-17T03:35:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.106+0000 7f4447200640 1 -- 172.21.15.46:0/569417658 shutdown_connections 2024-09-17T03:35:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.106+0000 7f4447200640 1 --2- 172.21.15.46:0/569417658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44400aeff0 0x7f444015eef0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.106+0000 7f4447200640 1 --2- 172.21.15.46:0/569417658 >> v2:172.21.15.99:3300/0 conn(0x7f4440155b00 0x7f44400cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.106+0000 7f4447200640 1 -- 172.21.15.46:0/569417658 >> 172.21.15.46:0/569417658 conn(0x7f44400b2040 msgr2=0x7f44400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.110+0000 7f4447200640 1 -- 172.21.15.46:0/569417658 shutdown_connections 2024-09-17T03:35:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.110+0000 7f4447200640 1 -- 172.21.15.46:0/569417658 wait complete. 2024-09-17T03:35:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.114+0000 7f4447200640 1 Processor -- start 2024-09-17T03:35:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.114+0000 7f4447200640 1 -- start start 2024-09-17T03:35:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4447200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440155b00 0x7f44401ff500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4447200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 0x7f444006e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4447200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444014dd60 0x7f444006e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4447200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f444016c0a0 con 0x7f4440163060 2024-09-17T03:35:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4447200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f444016bf20 con 0x7f4440155b00 2024-09-17T03:35:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4447200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f444016c220 con 0x7f444014dd60 2024-09-17T03:35:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 0x7f444006e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 0x7f444006e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53570/0 (socket says 172.21.15.46:53570) 2024-09-17T03:35:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 -- 172.21.15.46:0/769135725 learned_addr learned my addr 172.21.15.46:0/769135725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f4445200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444014dd60 0x7f444006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 -- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444014dd60 msgr2=0x7f444006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444014dd60 0x7f444006e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 -- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440155b00 msgr2=0x7f44401ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440155b00 0x7f44401ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.118+0000 7f443f400640 1 -- 172.21.15.46:0/769135725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44401fc9a0 con 0x7f4440163060 2024-09-17T03:35:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.122+0000 7f443f400640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 0x7f444006e1d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c3c0 tx=0x7f442c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.134+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c017070 con 0x7f4440163060 2024-09-17T03:35:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.134+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442c003600 con 0x7f4440163060 2024-09-17T03:35:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.134+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c017070 con 0x7f4440163060 2024-09-17T03:35:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.134+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401fcc80 con 0x7f4440163060 2024-09-17T03:35:02.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.134+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401fd240 con 0x7f4440163060 2024-09-17T03:35:02.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.138+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444015eef0 con 0x7f4440163060 2024-09-17T03:35:02.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.142+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 45) ==== 99937+0+0 (secure 0 0 0) 0x7f442c01a070 con 0x7f4440163060 2024-09-17T03:35:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.142+0000 7f4437e00640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4420075200 0x7f4420077710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.142+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f442c092810 con 0x7f4440163060 2024-09-17T03:35:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.150+0000 7f443fe00640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4420075200 0x7f4420077710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.150+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f442c05ecc0 con 0x7f4440163060 2024-09-17T03:35:02.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.162+0000 7f443fe00640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4420075200 0x7f4420077710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f443000a860 tx=0x7f4430005b60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:02.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:02.642+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80982", "target": ["mon-mgr", ""]}) -- 0x7f444006b540 con 0x7f4420075200 2024-09-17T03:35:03.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.386+0000 7f4437e00640 1 -- 172.21.15.46:0/769135725 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f443000ae40 con 0x7f4420075200 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.398+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4420075200 msgr2=0x7f4420077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.398+0000 7f4447200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4420075200 0x7f4420077710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f443000a860 tx=0x7f4430005b60 comp rx=0 tx=0).stop 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.398+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 msgr2=0x7f444006e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.398+0000 7f4447200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 0x7f444006e1d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c3c0 tx=0x7f442c00c880 comp rx=0 tx=0).stop 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 shutdown_connections 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4420075200 0x7f4420077710 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f444014dd60 0x7f444006e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440163060 0x7f444006e1d0 secure :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c3c0 tx=0x7f442c00c880 comp rx=0 tx=0).stop 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 --2- 172.21.15.46:0/769135725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440155b00 0x7f44401ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 >> 172.21.15.46:0/769135725 conn(0x7f44400b2040 msgr2=0x7f44400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:03.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 shutdown_connections 2024-09-17T03:35:03.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:03.402+0000 7f4447200640 1 -- 172.21.15.46:0/769135725 wait complete. 2024-09-17T03:35:03.430 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:03.426+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:03.430 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:03.426+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:03.430 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:03.426+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:03.430 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:03.426+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:03.430 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:03.426+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:04.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone19820 2024-09-17T03:35:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 -- 172.21.15.46:0/2478547213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 msgr2=0x7fdadc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 --2- 172.21.15.46:0/2478547213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdadc003e60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c680 tx=0x7fdaf001d3d0 comp rx=0 tx=0).stop 2024-09-17T03:35:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 -- 172.21.15.46:0/2478547213 shutdown_connections 2024-09-17T03:35:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 --2- 172.21.15.46:0/2478547213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdadc003e60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 -- 172.21.15.46:0/2478547213 >> 172.21.15.46:0/2478547213 conn(0x7fdaf806cdf0 msgr2=0x7fdaf806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 -- 172.21.15.46:0/2478547213 shutdown_connections 2024-09-17T03:35:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.902+0000 7fdaff200640 1 -- 172.21.15.46:0/2478547213 wait complete. 2024-09-17T03:35:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.906+0000 7fdaff200640 1 Processor -- start 2024-09-17T03:35:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.906+0000 7fdaff200640 1 -- start start 2024-09-17T03:35:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaff200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdaf814a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaff200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf80ab3c0 0x7fdaf814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaff200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdaf814d440 0x7fdaf80b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaff200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8073e40 con 0x7fdadc003a10 2024-09-17T03:35:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaff200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8073cc0 con 0x7fdaf80ab3c0 2024-09-17T03:35:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaff200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf8166700 con 0x7fdaf814d440 2024-09-17T03:35:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaf7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdaf814a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaf7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdaf814a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52570/0 (socket says 172.21.15.46:52570) 2024-09-17T03:35:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaf7e00640 1 -- 172.21.15.46:0/354827419 learned_addr learned my addr 172.21.15.46:0/354827419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdaf7400640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf80ab3c0 0x7fdaf814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.910+0000 7fdafd200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdaf814d440 0x7fdaf80b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaf7e00640 1 -- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdaf814d440 msgr2=0x7fdaf80b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaf7e00640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdaf814d440 0x7fdaf80b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaf7e00640 1 -- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf80ab3c0 msgr2=0x7fdaf814cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaf7e00640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf80ab3c0 0x7fdaf814cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaf7e00640 1 -- 172.21.15.46:0/354827419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf80aa600 con 0x7fdadc003a10 2024-09-17T03:35:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaf7e00640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdaf814a510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c680 tx=0x7fdaf000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003980 con 0x7fdadc003a10 2024-09-17T03:35:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.914+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaf0003b20 con 0x7fdadc003a10 2024-09-17T03:35:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.918+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003d30 con 0x7fdadc003a10 2024-09-17T03:35:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.918+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf80aa8e0 con 0x7fdadc003a10 2024-09-17T03:35:04.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.922+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf80aaea0 con 0x7fdadc003a10 2024-09-17T03:35:04.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.926+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf8166c60 con 0x7fdadc003a10 2024-09-17T03:35:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.926+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7fdaf00bd020 con 0x7fdadc003a10 2024-09-17T03:35:04.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.930+0000 7fdaefe00640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdad8075060 0x7fdad8077570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:04.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.930+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7fdaf00a9050 con 0x7fdadc003a10 2024-09-17T03:35:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.938+0000 7fdaf7400640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdad8075060 0x7fdad8077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.938+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdaf0017130 con 0x7fdadc003a10 2024-09-17T03:35:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:04.950+0000 7fdaf7400640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdad8075060 0x7fdad8077570 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80cf670 tx=0x7fdae80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:05.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:05.430+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone19820", "target": ["mon-mgr", ""]}) -- 0x7fdaf814dc70 con 0x7fdad8075060 2024-09-17T03:35:05.485 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:05.482+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:05.485 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:05.482+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:05.485 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:05.482+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:05.485 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:05.482+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:05.485 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:05.482+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:06.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:06.258+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7fdaf00b5070 con 0x7fdadc003a10 2024-09-17T03:35:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.010+0000 7fdaefe00640 1 -- 172.21.15.46:0/354827419 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdaf814dc70 con 0x7fdad8075060 2024-09-17T03:35:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.014+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdad8075060 msgr2=0x7fdad8077570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.014+0000 7fdaff200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdad8075060 0x7fdad8077570 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80cf670 tx=0x7fdae80091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.014+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 msgr2=0x7fdaf814a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:07.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.014+0000 7fdaff200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdaf814a510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c680 tx=0x7fdaf000c330 comp rx=0 tx=0).stop 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 shutdown_connections 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdad8075060 0x7fdad8077570 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdaf814d440 0x7fdaf80b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf80ab3c0 0x7fdaf814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 --2- 172.21.15.46:0/354827419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdadc003a10 0x7fdaf814a510 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 >> 172.21.15.46:0/354827419 conn(0x7fdaf806cdf0 msgr2=0x7fdaf8164f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:07.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 shutdown_connections 2024-09-17T03:35:07.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.018+0000 7fdaff200640 1 -- 172.21.15.46:0/354827419 wait complete. 2024-09-17T03:35:07.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone18021 2024-09-17T03:35:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.510+0000 7f96f3e00640 1 -- 172.21.15.46:0/3593740087 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f40d0a40 msgr2=0x7f96f4070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.510+0000 7f96f3e00640 1 --2- 172.21.15.46:0/3593740087 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f40d0a40 0x7f96f4070180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009bf0 tx=0x7f96ec01cb50 comp rx=0 tx=0).stop 2024-09-17T03:35:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.510+0000 7f96f3e00640 1 -- 172.21.15.46:0/3593740087 shutdown_connections 2024-09-17T03:35:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.510+0000 7f96f3e00640 1 --2- 172.21.15.46:0/3593740087 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f40d0a40 0x7f96f4070180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.510+0000 7f96f3e00640 1 -- 172.21.15.46:0/3593740087 >> 172.21.15.46:0/3593740087 conn(0x7f96f406e4a0 msgr2=0x7f96f406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.510+0000 7f96f3e00640 1 -- 172.21.15.46:0/3593740087 shutdown_connections 2024-09-17T03:35:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.514+0000 7f96f3e00640 1 -- 172.21.15.46:0/3593740087 wait complete. 2024-09-17T03:35:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.518+0000 7f96f3e00640 1 Processor -- start 2024-09-17T03:35:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.518+0000 7f96f3e00640 1 -- start start 2024-09-17T03:35:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f406fd70 0x7f96f415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96f415ec80 0x7f96f40afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f4160d20 0x7f96f4161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96f416d340 con 0x7f96f4160d20 2024-09-17T03:35:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96f416d1c0 con 0x7f96f415ec80 2024-09-17T03:35:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f96f416d4c0 con 0x7f96f406fd70 2024-09-17T03:35:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f406fd70 0x7f96f415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f2a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f406fd70 0x7f96f415d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:44798/0 (socket says 172.21.15.46:44798) 2024-09-17T03:35:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f2a00640 1 -- 172.21.15.46:0/4270833679 learned_addr learned my addr 172.21.15.46:0/4270833679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f3400640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f4160d20 0x7f96f4161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.522+0000 7f96f2000640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96f415ec80 0x7f96f40afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.526+0000 7f96f3400640 1 -- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f406fd70 msgr2=0x7f96f415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.526+0000 7f96f3400640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f406fd70 0x7f96f415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.526+0000 7f96f3400640 1 -- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96f415ec80 msgr2=0x7f96f40afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.526+0000 7f96f3400640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96f415ec80 0x7f96f40afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.526+0000 7f96f3400640 1 -- 172.21.15.46:0/4270833679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f4161670 con 0x7f96f4160d20 2024-09-17T03:35:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.526+0000 7f96f3400640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f4160d20 0x7f96f4161130 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c390 tx=0x7f96e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.530+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4017070 con 0x7f96f4160d20 2024-09-17T03:35:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.530+0000 7f96f3e00640 1 -- 172.21.15.46:0/4270833679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f40b1a50 con 0x7f96f4160d20 2024-09-17T03:35:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.530+0000 7f96f3e00640 1 -- 172.21.15.46:0/4270833679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f40b2230 con 0x7f96f4160d20 2024-09-17T03:35:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.534+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96e40053c0 con 0x7f96f4160d20 2024-09-17T03:35:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.534+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4017070 con 0x7f96f4160d20 2024-09-17T03:35:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.534+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc0052a0 con 0x7f96f4160d20 2024-09-17T03:35:07.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.534+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f96e401a070 con 0x7f96f4160d20 2024-09-17T03:35:07.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.538+0000 7f96eaa00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96d4075320 0x7f96d4077830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.538+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f96e4091fe0 con 0x7f96f4160d20 2024-09-17T03:35:07.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.542+0000 7f96f2a00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96d4075320 0x7f96d4077830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.546+0000 7f96f2a00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96d4075320 0x7f96d4077830 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f96e0001700 tx=0x7f96e0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:07.554+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f96f4161670 con 0x7f96f4160d20 2024-09-17T03:35:08.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.086+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone18021", "target": ["mon-mgr", ""]}) -- 0x7f96cc000bd0 con 0x7f96d4075320 2024-09-17T03:35:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96eaa00640 1 -- 172.21.15.46:0/4270833679 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f96cc000bd0 con 0x7f96d4075320 2024-09-17T03:35:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96d4075320 msgr2=0x7f96d4077830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96d4075320 0x7f96d4077830 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f96e0001700 tx=0x7f96e0009040 comp rx=0 tx=0).stop 2024-09-17T03:35:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f4160d20 msgr2=0x7f96f4161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f4160d20 0x7f96f4161130 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c390 tx=0x7f96e400c850 comp rx=0 tx=0).stop 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 shutdown_connections 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f96d4075320 0x7f96d4077830 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96f4160d20 0x7f96f4161130 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96f415ec80 0x7f96f40afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 --2- 172.21.15.46:0/4270833679 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f96f406fd70 0x7f96f415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 >> 172.21.15.46:0/4270833679 conn(0x7f96f406e4a0 msgr2=0x7f96f4155ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 shutdown_connections 2024-09-17T03:35:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.486+0000 7f96dfe00640 1 -- 172.21.15.46:0/4270833679 wait complete. 2024-09-17T03:35:08.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80982 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361c000640 1 -- 172.21.15.46:0/3936008507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35fc005160 msgr2=0x7f35fc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361c000640 1 --2- 172.21.15.46:0/3936008507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35fc005160 0x7f35fc00b9d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f36140ce190 tx=0x7f360c01d610 comp rx=0 tx=0).stop 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361a000640 1 -- 172.21.15.46:0/3936008507 reap_dead start 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361c000640 1 -- 172.21.15.46:0/3936008507 shutdown_connections 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361c000640 1 -- 172.21.15.46:0/3936008507 >> 172.21.15.46:0/3936008507 conn(0x7f36140b35f0 msgr2=0x7f36140b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361c000640 1 -- 172.21.15.46:0/3936008507 shutdown_connections 2024-09-17T03:35:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.854+0000 7f361c000640 1 -- 172.21.15.46:0/3936008507 wait complete. 2024-09-17T03:35:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 Processor -- start 2024-09-17T03:35:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 -- start start 2024-09-17T03:35:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35fc005160 0x7f36140d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 0x7f36141576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36141f14a0 0x7f36141f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36140b5b60 con 0x7f3614157de0 2024-09-17T03:35:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f361406d550 con 0x7f36141f14a0 2024-09-17T03:35:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361c000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f36140b5ce0 con 0x7f35fc005160 2024-09-17T03:35:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 0x7f36141576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 0x7f36141576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52606/0 (socket says 172.21.15.46:52606) 2024-09-17T03:35:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 -- 172.21.15.46:0/3875970565 learned_addr learned my addr 172.21.15.46:0/3875970565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f361a000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36141f14a0 0x7f36141f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 -- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35fc005160 msgr2=0x7f36140d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35fc005160 0x7f36140d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 -- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36141f14a0 msgr2=0x7f36141f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.858+0000 7f3618c00640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36141f14a0 0x7f36141f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f3618c00640 1 -- 172.21.15.46:0/3875970565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36140cf3e0 con 0x7f3614157de0 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f3618c00640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 0x7f36141576f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3600001a40 tx=0x7f36000127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600013860 con 0x7f3614157de0 2024-09-17T03:35:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3600013a20 con 0x7f3614157de0 2024-09-17T03:35:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600010480 con 0x7f3614157de0 2024-09-17T03:35:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36140cf6c0 con 0x7f3614157de0 2024-09-17T03:35:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36140cfc80 con 0x7f3614157de0 2024-09-17T03:35:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36140b6200 con 0x7f3614157de0 2024-09-17T03:35:08.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.862+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f3600010620 con 0x7f3614157de0 2024-09-17T03:35:08.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.866+0000 7f3611600640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f4075120 0x7f35f4077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:08.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.866+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f3600096fd0 con 0x7f3614157de0 2024-09-17T03:35:08.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.866+0000 7f3619600640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f4075120 0x7f35f4077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.874+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f361415c0d0 con 0x7f3614157de0 2024-09-17T03:35:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:08.878+0000 7f3619600640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f4075120 0x7f35f4077630 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f360800d440 tx=0x7f3608005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.306+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80982", "target": ["mon-mgr", ""]}) -- 0x7f361406b540 con 0x7f35f4075120 2024-09-17T03:35:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f3611600640 1 -- 172.21.15.46:0/3875970565 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3608005c60 con 0x7f35f4075120 2024-09-17T03:35:09.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f4075120 msgr2=0x7f35f4077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f4075120 0x7f35f4077630 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f360800d440 tx=0x7f3608005860 comp rx=0 tx=0).stop 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 msgr2=0x7f36141576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 0x7f36141576f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3600001a40 tx=0x7f36000127b0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 shutdown_connections 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35f4075120 0x7f35f4077630 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36141f14a0 0x7f36141f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3614157de0 0x7f36141576f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 --2- 172.21.15.46:0/3875970565 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35fc005160 0x7f36140d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 >> 172.21.15.46:0/3875970565 conn(0x7f36140b35f0 msgr2=0x7f361415b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 shutdown_connections 2024-09-17T03:35:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.446+0000 7f361c000640 1 -- 172.21.15.46:0/3875970565 wait complete. 2024-09-17T03:35:09.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone19820 2024-09-17T03:35:09.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 -- 172.21.15.46:0/4122096817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815aa50 msgr2=0x7f95480cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 --2- 172.21.15.46:0/4122096817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815aa50 0x7f95480cd260 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f95480cc4f0 tx=0x7f953801d530 comp rx=0 tx=0).stop 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 -- 172.21.15.46:0/4122096817 shutdown_connections 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 --2- 172.21.15.46:0/4122096817 >> v2:172.21.15.99:3300/0 conn(0x7f954815e4e0 0x7f954815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 --2- 172.21.15.46:0/4122096817 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95480cd7a0 0x7f95480cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 --2- 172.21.15.46:0/4122096817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815aa50 0x7f95480cd260 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 -- 172.21.15.46:0/4122096817 >> 172.21.15.46:0/4122096817 conn(0x7f95480b2040 msgr2=0x7f95480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 -- 172.21.15.46:0/4122096817 shutdown_connections 2024-09-17T03:35:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.634+0000 7f954cc00640 1 -- 172.21.15.46:0/4122096817 wait complete. 2024-09-17T03:35:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 Processor -- start 2024-09-17T03:35:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 -- start start 2024-09-17T03:35:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95480cd7a0 0x7f95480af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954815aa50 0x7f9548153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 0x7f9548153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9548167670 con 0x7f954815e4e0 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95480b40c0 con 0x7f954815aa50 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f954cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f95481677f0 con 0x7f95480cd7a0 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.638+0000 7f9546800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 0x7f9548153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 0x7f9548153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52638/0 (socket says 172.21.15.46:52638) 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 -- 172.21.15.46:0/2564795742 learned_addr learned my addr 172.21.15.46:0/2564795742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f953fe00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954815aa50 0x7f9548153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9545e00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95480cd7a0 0x7f95480af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 -- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95480cd7a0 msgr2=0x7f95480af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95480cd7a0 0x7f95480af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 -- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954815aa50 msgr2=0x7f9548153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954815aa50 0x7f9548153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 -- 172.21.15.46:0/2564795742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954814f5e0 con 0x7f954815e4e0 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f953fe00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954815aa50 0x7f9548153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f9546800640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 0x7f9548153590 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f954000ea80 tx=0x7f954000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540004b30 con 0x7f954815e4e0 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9540004cf0 con 0x7f954815e4e0 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540005280 con 0x7f954815e4e0 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95480afa00 con 0x7f954815e4e0 2024-09-17T03:35:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95480aa5e0 con 0x7f954815e4e0 2024-09-17T03:35:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954815d900 con 0x7f954815e4e0 2024-09-17T03:35:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.642+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f9540005420 con 0x7f954815e4e0 2024-09-17T03:35:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.646+0000 7f953e000640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f95240750f0 0x7f9524077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.646+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f9540093e10 con 0x7f954815e4e0 2024-09-17T03:35:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.646+0000 7f9545e00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f95240750f0 0x7f9524077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.646+0000 7f9545e00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f95240750f0 0x7f9524077600 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f95380af000 tx=0x7f953801bd70 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.650+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9540060240 con 0x7f954815e4e0 2024-09-17T03:35:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.858+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone19820", "target": ["mon-mgr", ""]}) -- 0x7f954806b540 con 0x7f95240750f0 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f953e000640 1 -- 172.21.15.46:0/2564795742 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f9538008e50 con 0x7f95240750f0 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:35:09.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:09.869 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:35:09.869 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:35:09.869 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:35:09.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:09.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:09.869 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:35:09.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f95240750f0 msgr2=0x7f9524077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f95240750f0 0x7f9524077600 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f95380af000 tx=0x7f953801bd70 comp rx=0 tx=0).stop 2024-09-17T03:35:09.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 msgr2=0x7f9548153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 0x7f9548153590 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f954000ea80 tx=0x7f954000ef40 comp rx=0 tx=0).stop 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 shutdown_connections 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f95240750f0 0x7f9524077600 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f954815e4e0 0x7f9548153590 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954815aa50 0x7f9548153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 --2- 172.21.15.46:0/2564795742 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95480cd7a0 0x7f95480af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 >> 172.21.15.46:0/2564795742 conn(0x7f95480b2040 msgr2=0x7f9548158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 shutdown_connections 2024-09-17T03:35:09.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:09.866+0000 7f954cc00640 1 -- 172.21.15.46:0/2564795742 wait complete. 2024-09-17T03:35:09.891 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:35:09.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone18021 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 -- 172.21.15.46:0/144554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654069500 msgr2=0x7f16541603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 --2- 172.21.15.46:0/144554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654069500 0x7f16541603e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f1640009840 tx=0x7f164001c7f0 comp rx=0 tx=0).stop 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 -- 172.21.15.46:0/144554739 shutdown_connections 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 --2- 172.21.15.46:0/144554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654069500 0x7f16541603e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 -- 172.21.15.46:0/144554739 >> 172.21.15.46:0/144554739 conn(0x7f16540b35f0 msgr2=0x7f16540b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 -- 172.21.15.46:0/144554739 shutdown_connections 2024-09-17T03:35:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 -- 172.21.15.46:0/144554739 wait complete. 2024-09-17T03:35:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 Processor -- start 2024-09-17T03:35:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.054+0000 7f165b400640 1 -- start start 2024-09-17T03:35:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 0x7f1654151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1654160920 0x7f1654159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16541521c0 0x7f1654159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f165416ada0 con 0x7f165415c7b0 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f165416ac20 con 0x7f1654160920 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f165416af20 con 0x7f16541521c0 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 0x7f1654151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 0x7f1654151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52652/0 (socket says 172.21.15.46:52652) 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 -- 172.21.15.46:0/1247388898 learned_addr learned my addr 172.21.15.46:0/1247388898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1659400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16541521c0 0x7f1654159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1653e00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1654160920 0x7f1654159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 -- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16541521c0 msgr2=0x7f1654159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16541521c0 0x7f1654159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:10.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 -- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1654160920 msgr2=0x7f1654159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:10.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1654160920 0x7f1654159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:10.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 -- 172.21.15.46:0/1247388898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165415b930 con 0x7f165415c7b0 2024-09-17T03:35:10.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1658a00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 0x7f1654151670 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f164800c3f0 tx=0x7f164800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:10.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648017070 con 0x7f165415c7b0 2024-09-17T03:35:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1648003600 con 0x7f165415c7b0 2024-09-17T03:35:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16540b49c0 con 0x7f165415c7b0 2024-09-17T03:35:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648003a50 con 0x7f165415c7b0 2024-09-17T03:35:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541f8360 con 0x7f165415c7b0 2024-09-17T03:35:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.058+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16541603e0 con 0x7f165415c7b0 2024-09-17T03:35:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.062+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f164801a070 con 0x7f165415c7b0 2024-09-17T03:35:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.062+0000 7f1650c00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1634075120 0x7f1634077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.062+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f16480928b0 con 0x7f165415c7b0 2024-09-17T03:35:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.062+0000 7f1653e00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1634075120 0x7f1634077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.062+0000 7f1653e00640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1634075120 0x7f1634077630 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f16400af000 tx=0x7f164001b030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:10.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.066+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1648021020 con 0x7f165415c7b0 2024-09-17T03:35:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:10.274+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone18021", "target": ["mon-mgr", ""]}) -- 0x7f165406b540 con 0x7f1634075120 2024-09-17T03:35:14.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f1650c00640 1 -- 172.21.15.46:0/1247388898 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f1640004200 con 0x7f1634075120 2024-09-17T03:35:14.524 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:35:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:35:14.524 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:14.525 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:35:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1634075120 msgr2=0x7f1634077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1634075120 0x7f1634077630 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f16400af000 tx=0x7f164001b030 comp rx=0 tx=0).stop 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 msgr2=0x7f1654151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 0x7f1654151670 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f164800c3f0 tx=0x7f164800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 shutdown_connections 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1634075120 0x7f1634077630 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16541521c0 0x7f1654159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1654160920 0x7f1654159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 --2- 172.21.15.46:0/1247388898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f165415c7b0 0x7f1654151670 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 >> 172.21.15.46:0/1247388898 conn(0x7f16540b35f0 msgr2=0x7f16540af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 shutdown_connections 2024-09-17T03:35:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.522+0000 7f165b400640 1 -- 172.21.15.46:0/1247388898 wait complete. 2024-09-17T03:35:14.552 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:35:14.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone80982 2024-09-17T03:35:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 -- 172.21.15.46:0/3377494680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 msgr2=0x7f2e04074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 --2- 172.21.15.46:0/3377494680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04074e30 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2e04071cb0 tx=0x7f2dec01c890 comp rx=0 tx=0).stop 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 -- 172.21.15.46:0/3377494680 shutdown_connections 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 --2- 172.21.15.46:0/3377494680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04074e30 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 --2- 172.21.15.46:0/3377494680 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b4de0 0x7f2e040b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 --2- 172.21.15.46:0/3377494680 >> v2:172.21.15.99:3300/0 conn(0x7f2e040b5c70 0x7f2e0415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 -- 172.21.15.46:0/3377494680 >> 172.21.15.46:0/3377494680 conn(0x7f2e040b2040 msgr2=0x7f2e040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 -- 172.21.15.46:0/3377494680 shutdown_connections 2024-09-17T03:35:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 -- 172.21.15.46:0/3377494680 wait complete. 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 Processor -- start 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.718+0000 7f2e09400640 1 -- start start 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e040b4de0 0x7f2e041561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b5c70 0x7f2e04144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04166e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0415f6b0 con 0x7f2e041527b0 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0415f530 con 0x7f2e040b4de0 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0415f830 con 0x7f2e040b5c70 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04166e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04166e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35046/0 (socket says 172.21.15.46:35046) 2024-09-17T03:35:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 -- 172.21.15.46:0/1249303138 learned_addr learned my addr 172.21.15.46:0/1249303138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e02800640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e040b4de0 0x7f2e041561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2dfbe00640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b5c70 0x7f2e04144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 -- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b5c70 msgr2=0x7f2e04144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b5c70 0x7f2e04144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 -- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e040b4de0 msgr2=0x7f2e041561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e040b4de0 0x7f2e041561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 -- 172.21.15.46:0/1249303138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e041450f0 con 0x7f2e041527b0 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e02800640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e040b4de0 0x7f2e041561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:14.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2dfbe00640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b5c70 0x7f2e04144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e03200640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04166e80 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00cab0 tx=0x7f2dfc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc007d90 con 0x7f2e041527b0 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dfc00f560 con 0x7f2e041527b0 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00f9b0 con 0x7f2e041527b0 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e04167fc0 con 0x7f2e041527b0 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e041663a0 con 0x7f2e041527b0 2024-09-17T03:35:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.722+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0415fd50 con 0x7f2e041527b0 2024-09-17T03:35:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.726+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f2dfc00fb50 con 0x7f2e041527b0 2024-09-17T03:35:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.726+0000 7f2dfaa00640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2de00750f0 0x7f2de0077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.726+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f2dfc093c70 con 0x7f2e041527b0 2024-09-17T03:35:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.726+0000 7f2e02800640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2de00750f0 0x7f2de0077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.726+0000 7f2e02800640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2de00750f0 0x7f2de0077600 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2df4005fd0 tx=0x7f2df400a540 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.730+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2dfc014070 con 0x7f2e041527b0 2024-09-17T03:35:14.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.938+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone80982", "target": ["mon-mgr", ""]}) -- 0x7f2e0406b540 con 0x7f2de00750f0 2024-09-17T03:35:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.950+0000 7f2dfaa00640 1 -- 172.21.15.46:0/1249303138 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f2df400a2f0 con 0x7f2de00750f0 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:35:14.955 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:35:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:14.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:14.956 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:35:14.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2de00750f0 msgr2=0x7f2de0077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2de00750f0 0x7f2de0077600 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2df4005fd0 tx=0x7f2df400a540 comp rx=0 tx=0).stop 2024-09-17T03:35:14.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 msgr2=0x7f2e04166e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:14.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04166e80 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00cab0 tx=0x7f2dfc00cf70 comp rx=0 tx=0).stop 2024-09-17T03:35:14.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 shutdown_connections 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2de00750f0 0x7f2de0077600 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e041527b0 0x7f2e04166e80 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e040b5c70 0x7f2e04144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 --2- 172.21.15.46:0/1249303138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e040b4de0 0x7f2e041561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 >> 172.21.15.46:0/1249303138 conn(0x7f2e040b2040 msgr2=0x7f2e040b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 shutdown_connections 2024-09-17T03:35:14.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:14.954+0000 7f2e09400640 1 -- 172.21.15.46:0/1249303138 wait complete. 2024-09-17T03:35:14.979 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:35:14.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f83000640 1 -- 172.21.15.46:0/3142014687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f64005310 msgr2=0x7f8f6400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f83000640 1 --2- 172.21.15.46:0/3142014687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f64005310 0x7f8f6400b9d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cf890 tx=0x7f8f7401d610 comp rx=0 tx=0).stop 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f82600640 1 -- 172.21.15.46:0/3142014687 reap_dead start 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f83000640 1 -- 172.21.15.46:0/3142014687 shutdown_connections 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f83000640 1 -- 172.21.15.46:0/3142014687 >> 172.21.15.46:0/3142014687 conn(0x7f8f7c0b4560 msgr2=0x7f8f7c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f83000640 1 -- 172.21.15.46:0/3142014687 shutdown_connections 2024-09-17T03:35:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.134+0000 7f8f83000640 1 -- 172.21.15.46:0/3142014687 wait complete. 2024-09-17T03:35:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 Processor -- start 2024-09-17T03:35:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 -- start start 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f64005310 0x7f8f7c146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c1e5de0 0x7f8f7c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 0x7f8f7c1f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0b55f0 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0b5470 con 0x7f8f7c1e5de0 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f83000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0b5770 con 0x7f8f64005310 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 0x7f8f7c1f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 0x7f8f7c1f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35064/0 (socket says 172.21.15.46:35064) 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 -- 172.21.15.46:0/3008519016 learned_addr learned my addr 172.21.15.46:0/3008519016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f81c00640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f64005310 0x7f8f7c146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f81200640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c1e5de0 0x7f8f7c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 -- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f64005310 msgr2=0x7f8f7c146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f64005310 0x7f8f7c146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 -- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c1e5de0 msgr2=0x7f8f7c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c1e5de0 0x7f8f7c15b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f82600640 1 -- 172.21.15.46:0/3008519016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c15be00 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.138+0000 7f8f81200640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c1e5de0 0x7f8f7c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f82600640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 0x7f8f7c1f8810 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f8f74002240 tx=0x7f8f740aae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74009bf0 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f740a9450 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f740b1830 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c06edd0 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c06d270 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c160780 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f8f740a7040 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.142+0000 7f8f79600640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f5c0750f0 0x7f8f5c077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.146+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f8f7412e590 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.146+0000 7f8f81c00640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f5c0750f0 0x7f8f5c077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.146+0000 7f8f81c00640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f5c0750f0 0x7f8f5c077600 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8f70001130 tx=0x7f8f70001e10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.150+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8f7c15be00 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.382+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8f7c1e6b60 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.382+0000 7f8f79600640 1 -- 172.21.15.46:0/3008519016 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1925 (secure 0 0 0) 0x7f8f740ad030 con 0x7f8f7c15b9f0 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:35:15.387 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 873, 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:35:15.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 116, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:35:15.389 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 1996, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 8238921711, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 17634275328, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 755459837952, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 423446438, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 417905986, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 104, 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 116 2024-09-17T03:35:15.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 76, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:34:26:988878+0000", 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 8, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "a", 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:35:15.391 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 248196 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:35:15.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 19, 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:34:07.532987+0000", 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:35:15.393 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f5c0750f0 msgr2=0x7f8f5c077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f5c0750f0 0x7f8f5c077600 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8f70001130 tx=0x7f8f70001e10 comp rx=0 tx=0).stop 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 msgr2=0x7f8f7c1f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 0x7f8f7c1f8810 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f8f74002240 tx=0x7f8f740aae50 comp rx=0 tx=0).stop 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 shutdown_connections 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f5c0750f0 0x7f8f5c077600 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c15b9f0 0x7f8f7c1f8810 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c1e5de0 0x7f8f7c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 --2- 172.21.15.46:0/3008519016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f64005310 0x7f8f7c146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 >> 172.21.15.46:0/3008519016 conn(0x7f8f7c0b4560 msgr2=0x7f8f7c15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 shutdown_connections 2024-09-17T03:35:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.386+0000 7f8f83000640 1 -- 172.21.15.46:0/3008519016 wait complete. 2024-09-17T03:35:15.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:15.409 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone19820/9a96565a-633c-4e0e-b651-3d129341f36b 2024-09-17T03:35:15.409 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:35:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone19820/9a96565a-633c-4e0e-b651-3d129341f36b': No such file or directory 2024-09-17T03:35:15.421 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T03:35:15.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 -- 172.21.15.46:0/3211406774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e01527b0 msgr2=0x7f07e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 --2- 172.21.15.46:0/3211406774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e01527b0 0x7f07e015bb60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009840 tx=0x7f07c801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 -- 172.21.15.46:0/3211406774 shutdown_connections 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 --2- 172.21.15.46:0/3211406774 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b4de0 0x7f07e00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 --2- 172.21.15.46:0/3211406774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e01527b0 0x7f07e015bb60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 -- 172.21.15.46:0/3211406774 >> 172.21.15.46:0/3211406774 conn(0x7f07e00b2040 msgr2=0x7f07e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 -- 172.21.15.46:0/3211406774 shutdown_connections 2024-09-17T03:35:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 -- 172.21.15.46:0/3211406774 wait complete. 2024-09-17T03:35:15.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 Processor -- start 2024-09-17T03:35:15.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.622+0000 7f07e5a00640 1 -- start start 2024-09-17T03:35:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b4de0 0x7f07e01fb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b5c70 0x7f07e015a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 0x7f07e014fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0168a00 con 0x7f07e01527b0 2024-09-17T03:35:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0168880 con 0x7f07e00b4de0 2024-09-17T03:35:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f07e0168b80 con 0x7f07e00b5c70 2024-09-17T03:35:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 0x7f07e014fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 0x7f07e014fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35068/0 (socket says 172.21.15.46:35068) 2024-09-17T03:35:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 -- 172.21.15.46:0/167282761 learned_addr learned my addr 172.21.15.46:0/167282761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07dea00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b4de0 0x7f07e01fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07d6000640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b5c70 0x7f07e015a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 -- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b5c70 msgr2=0x7f07e015a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b5c70 0x7f07e015a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 -- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b4de0 msgr2=0x7f07e01fb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b4de0 0x7f07e01fb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 -- 172.21.15.46:0/167282761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e00b2930 con 0x7f07e01527b0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07df400640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 0x7f07e014fb20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f07d800e950 tx=0x7f07d800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d800cd20 con 0x7f07e01527b0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d8004030 con 0x7f07e01527b0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8004480 con 0x7f07e01527b0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e0152200 con 0x7f07e01527b0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01fc2b0 con 0x7f07e01527b0 2024-09-17T03:35:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.626+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e00b51f0 con 0x7f07e01527b0 2024-09-17T03:35:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.630+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f07d8003600 con 0x7f07e01527b0 2024-09-17T03:35:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.630+0000 7f07d7400640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc075120 0x7f07bc077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.630+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f07d8093ad0 con 0x7f07e01527b0 2024-09-17T03:35:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.630+0000 7f07dea00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc075120 0x7f07bc077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.630+0000 7f07dea00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc075120 0x7f07bc077630 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f07d0001e40 tx=0x7f07d0009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:15.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.634+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07d805ff80 con 0x7f07e01527b0 2024-09-17T03:35:15.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.830+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f07e006b540 con 0x7f07bc075120 2024-09-17T03:35:15.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.830+0000 7f07d7400640 1 -- 172.21.15.46:0/167282761 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f07d0001ae0 con 0x7f07bc075120 2024-09-17T03:35:15.833 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:35:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:35:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:15.834 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc075120 msgr2=0x7f07bc077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc075120 0x7f07bc077630 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f07d0001e40 tx=0x7f07d0009040 comp rx=0 tx=0).stop 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 msgr2=0x7f07e014fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 0x7f07e014fb20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f07d800e950 tx=0x7f07d800ee10 comp rx=0 tx=0).stop 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 shutdown_connections 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07bc075120 0x7f07bc077630 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07e01527b0 0x7f07e014fb20 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07e00b5c70 0x7f07e015a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 --2- 172.21.15.46:0/167282761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07e00b4de0 0x7f07e01fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 >> 172.21.15.46:0/167282761 conn(0x7f07e00b2040 msgr2=0x7f07e00b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 shutdown_connections 2024-09-17T03:35:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:15.834+0000 7f07e5a00640 1 -- 172.21.15.46:0/167282761 wait complete. 2024-09-17T03:35:15.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 -- 172.21.15.46:0/1016630734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c074b20 msgr2=0x7fda8c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 --2- 172.21.15.46:0/1016630734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c074b20 0x7fda8c15cf90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fda84009c80 tx=0x7fda8401cda0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 -- 172.21.15.46:0/1016630734 shutdown_connections 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 --2- 172.21.15.46:0/1016630734 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c074b20 0x7fda8c15cf90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 --2- 172.21.15.46:0/1016630734 >> v2:172.21.15.99:3300/0 conn(0x7fda8c157030 0x7fda8c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 -- 172.21.15.46:0/1016630734 >> 172.21.15.46:0/1016630734 conn(0x7fda8c0708c0 msgr2=0x7fda8c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 -- 172.21.15.46:0/1016630734 shutdown_connections 2024-09-17T03:35:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 -- 172.21.15.46:0/1016630734 wait complete. 2024-09-17T03:35:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 Processor -- start 2024-09-17T03:35:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.018+0000 7fda90e00640 1 -- start start 2024-09-17T03:35:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c157030 0x7fda8c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8c160d50 0x7fda8c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda8c150ef0 0x7fda8c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c16a7e0 con 0x7fda8c160d50 2024-09-17T03:35:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c16a660 con 0x7fda8c150ef0 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c16a960 con 0x7fda8c157030 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8c160d50 0x7fda8c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda89e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c157030 0x7fda8c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda89e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c157030 0x7fda8c13bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:47408/0 (socket says 172.21.15.46:47408) 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda89e00640 1 -- 172.21.15.46:0/1683221718 learned_addr learned my addr 172.21.15.46:0/1683221718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda8a800640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda8c150ef0 0x7fda8c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 -- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c157030 msgr2=0x7fda8c13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c157030 0x7fda8c13bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 -- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda8c150ef0 msgr2=0x7fda8c155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda8c150ef0 0x7fda8c155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 -- 172.21.15.46:0/1683221718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c1f7f70 con 0x7fda8c160d50 2024-09-17T03:35:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda83e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8c160d50 0x7fda8c1554c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c390 tx=0x7fda7400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74017070 con 0x7fda8c160d50 2024-09-17T03:35:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda740053c0 con 0x7fda8c160d50 2024-09-17T03:35:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c1f71b0 con 0x7fda8c160d50 2024-09-17T03:35:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74003840 con 0x7fda8c160d50 2024-09-17T03:35:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c1f7770 con 0x7fda8c160d50 2024-09-17T03:35:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.022+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c15cf90 con 0x7fda8c160d50 2024-09-17T03:35:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.026+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7fda7401a070 con 0x7fda8c160d50 2024-09-17T03:35:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.026+0000 7fda82000640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda68075120 0x7fda68077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.026+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7fda740922e0 con 0x7fda8c160d50 2024-09-17T03:35:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.026+0000 7fda89e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda68075120 0x7fda68077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.026+0000 7fda89e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda68075120 0x7fda68077630 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009c00 tx=0x7fda7c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:16.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.030+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda7405e840 con 0x7fda8c160d50 2024-09-17T03:35:16.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.230+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda8c1541a0 con 0x7fda68075120 2024-09-17T03:35:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.234+0000 7fda82000640 1 -- 172.21.15.46:0/1683221718 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7fda8c1541a0 con 0x7fda68075120 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone80982" 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone18021" 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:16.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone19820" 2024-09-17T03:35:16.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:16.239 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda68075120 msgr2=0x7fda68077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda68075120 0x7fda68077630 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009c00 tx=0x7fda7c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8c160d50 msgr2=0x7fda8c1554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8c160d50 0x7fda8c1554c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c390 tx=0x7fda7400c850 comp rx=0 tx=0).stop 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 shutdown_connections 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda68075120 0x7fda68077630 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda8c150ef0 0x7fda8c155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda8c160d50 0x7fda8c1554c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 --2- 172.21.15.46:0/1683221718 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda8c157030 0x7fda8c13bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 >> 172.21.15.46:0/1683221718 conn(0x7fda8c0708c0 msgr2=0x7fda8c14fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 shutdown_connections 2024-09-17T03:35:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.238+0000 7fda90e00640 1 -- 172.21.15.46:0/1683221718 wait complete. 2024-09-17T03:35:16.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06d2800640 1 -- 172.21.15.46:0/3916753782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b0003c20 msgr2=0x7f06b0004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06d2800640 1 --2- 172.21.15.46:0/3916753782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b0003c20 0x7f06b0004070 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f06bc001860 tx=0x7f06bc01c8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06cbe00640 1 -- 172.21.15.46:0/3916753782 reap_dead start 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06d2800640 1 -- 172.21.15.46:0/3916753782 shutdown_connections 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06d2800640 1 -- 172.21.15.46:0/3916753782 >> 172.21.15.46:0/3916753782 conn(0x7f06cc0b2040 msgr2=0x7f06cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06d2800640 1 -- 172.21.15.46:0/3916753782 shutdown_connections 2024-09-17T03:35:16.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.422+0000 7f06d2800640 1 -- 172.21.15.46:0/3916753782 wait complete. 2024-09-17T03:35:16.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 Processor -- start 2024-09-17T03:35:16.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 -- start start 2024-09-17T03:35:16.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 0x7f06cc140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06cc15ea30 0x7f06cc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06cc160070 0x7f06cc160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc167bd0 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc167a50 con 0x7f06cc160070 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc167d50 con 0x7f06cc15ea30 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 0x7f06cc140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 0x7f06cc140a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35084/0 (socket says 172.21.15.46:35084) 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 -- 172.21.15.46:0/2828321395 learned_addr learned my addr 172.21.15.46:0/2828321395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06c2a00640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06cc15ea30 0x7f06cc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cbe00640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06cc160070 0x7f06cc160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 -- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06cc15ea30 msgr2=0x7f06cc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06cc15ea30 0x7f06cc15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 -- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06cc160070 msgr2=0x7f06cc160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06cc160070 0x7f06cc160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 -- 172.21.15.46:0/2828321395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc15f780 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cbe00640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06cc160070 0x7f06cc160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:16.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06cb400640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 0x7f06cc140a70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f06bc001860 tx=0x7f06bc0a6cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:16.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc001480 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06bc0b5070 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc15fa60 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0068d0 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.426+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc15e510 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.430+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc074e30 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.430+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f06bc006310 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.430+0000 7f06c3e00640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f06a80750f0 0x7f06a8077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.430+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f06bc12da80 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.430+0000 7f06c2a00640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f06a80750f0 0x7f06a8077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.434+0000 7f06c2a00640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f06a80750f0 0x7f06a8077600 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0cf650 tx=0x7f06b40091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:16.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.434+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f06bc016690 con 0x7f06cc0aa4d0 2024-09-17T03:35:16.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.642+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06cc155f00 con 0x7f06a80750f0 2024-09-17T03:35:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06c3e00640 1 -- 172.21.15.46:0/2828321395 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f06cc155f00 con 0x7f06a80750f0 2024-09-17T03:35:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:16.660 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:35:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:35:16.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:16.661 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f06a80750f0 msgr2=0x7f06a8077600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f06a80750f0 0x7f06a8077600 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0cf650 tx=0x7f06b40091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 msgr2=0x7f06cc140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 0x7f06cc140a70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f06bc001860 tx=0x7f06bc0a6cb0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 shutdown_connections 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f06a80750f0 0x7f06a8077600 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06cc160070 0x7f06cc160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f06cc15ea30 0x7f06cc15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 --2- 172.21.15.46:0/2828321395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0aa4d0 0x7f06cc140a70 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 >> 172.21.15.46:0/2828321395 conn(0x7f06cc0b2040 msgr2=0x7f06cc152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 shutdown_connections 2024-09-17T03:35:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.658+0000 7f06d2800640 1 -- 172.21.15.46:0/2828321395 wait complete. 2024-09-17T03:35:16.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:35:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 -- 172.21.15.46:0/676048112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 msgr2=0x7fa1340b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 --2- 172.21.15.46:0/676048112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1340b51f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa1340b2fd0 tx=0x7fa11c01c650 comp rx=0 tx=0).stop 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 -- 172.21.15.46:0/676048112 shutdown_connections 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 --2- 172.21.15.46:0/676048112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1340b51f0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 --2- 172.21.15.46:0/676048112 >> v2:172.21.15.99:3300/0 conn(0x7fa134072100 0x7fa134155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 -- 172.21.15.46:0/676048112 >> 172.21.15.46:0/676048112 conn(0x7fa1340b2040 msgr2=0x7fa1340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 -- 172.21.15.46:0/676048112 shutdown_connections 2024-09-17T03:35:16.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 -- 172.21.15.46:0/676048112 wait complete. 2024-09-17T03:35:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 Processor -- start 2024-09-17T03:35:16.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.846+0000 7fa13a600640 1 -- start start 2024-09-17T03:35:16.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa134072100 0x7fa13414ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1341f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f0e0 0x7fa1341572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341693b0 con 0x7fa1340b4de0 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa134169230 con 0x7fa134072100 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa134169530 con 0x7fa13415f0e0 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1341f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1341f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35094/0 (socket says 172.21.15.46:35094) 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 -- 172.21.15.46:0/1426433215 learned_addr learned my addr 172.21.15.46:0/1426433215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa133e00640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f0e0 0x7fa1341572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa133400640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa134072100 0x7fa13414ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 -- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f0e0 msgr2=0x7fa1341572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f0e0 0x7fa1341572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 -- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa134072100 msgr2=0x7fa13414ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa134072100 0x7fa13414ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 -- 172.21.15.46:0/1426433215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1341f7bb0 con 0x7fa1340b4de0 2024-09-17T03:35:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12be00640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1341f71d0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa11c01ad00 tx=0x7fa11c0aa590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0aab10 con 0x7fa1340b4de0 2024-09-17T03:35:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa11c0aada0 con 0x7fa1340b4de0 2024-09-17T03:35:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1341f7e90 con 0x7fa1340b4de0 2024-09-17T03:35:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003860 con 0x7fa1340b4de0 2024-09-17T03:35:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1340aada0 con 0x7fa1340b4de0 2024-09-17T03:35:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.850+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1341f7920 con 0x7fa1340b4de0 2024-09-17T03:35:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.854+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7fa11c0a8050 con 0x7fa1340b4de0 2024-09-17T03:35:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.854+0000 7fa12b400640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa110075120 0x7fa110077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.854+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7fa11c12e650 con 0x7fa1340b4de0 2024-09-17T03:35:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.854+0000 7fa133400640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa110075120 0x7fa110077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:16.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.854+0000 7fa133400640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa110075120 0x7fa110077630 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa124005fd0 tx=0x7fa124005d10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:16.858+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa11c0a6040 con 0x7fa1340b4de0 2024-09-17T03:35:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.062+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa13406b540 con 0x7fa110075120 2024-09-17T03:35:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.094+0000 7fa12b400640 1 -- 172.21.15.46:0/1426433215 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa12400ea60 con 0x7fa110075120 2024-09-17T03:35:17.098 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa110075120 msgr2=0x7fa110077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa110075120 0x7fa110077630 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa124005fd0 tx=0x7fa124005d10 comp rx=0 tx=0).stop 2024-09-17T03:35:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 msgr2=0x7fa1341f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1341f71d0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa11c01ad00 tx=0x7fa11c0aa590 comp rx=0 tx=0).stop 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 shutdown_connections 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa110075120 0x7fa110077630 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa13415f0e0 0x7fa1341572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1340b4de0 0x7fa1341f71d0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 --2- 172.21.15.46:0/1426433215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa134072100 0x7fa13414ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 >> 172.21.15.46:0/1426433215 conn(0x7fa1340b2040 msgr2=0x7fa1340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 shutdown_connections 2024-09-17T03:35:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.098+0000 7fa13a600640 1 -- 172.21.15.46:0/1426433215 wait complete. 2024-09-17T03:35:17.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:35:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 -- 172.21.15.46:0/898179236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 msgr2=0x7f8f68160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 --2- 172.21.15.46:0/898179236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f68160740 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f8f680b3040 tx=0x7f8f6001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 -- 172.21.15.46:0/898179236 shutdown_connections 2024-09-17T03:35:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 --2- 172.21.15.46:0/898179236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f68160740 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 --2- 172.21.15.46:0/898179236 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f680cefa0 0x7f8f680cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 --2- 172.21.15.46:0/898179236 >> v2:172.21.15.99:3300/0 conn(0x7f8f680749a0 0x7f8f68074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 -- 172.21.15.46:0/898179236 >> 172.21.15.46:0/898179236 conn(0x7f8f680b2040 msgr2=0x7f8f680b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 -- 172.21.15.46:0/898179236 shutdown_connections 2024-09-17T03:35:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 -- 172.21.15.46:0/898179236 wait complete. 2024-09-17T03:35:17.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 Processor -- start 2024-09-17T03:35:17.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.282+0000 7f8f6de00640 1 -- start start 2024-09-17T03:35:17.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f680749a0 0x7f8f681f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f680cefa0 0x7f8f68166bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f6813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680b3ee0 con 0x7f8f68160330 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680b3d60 con 0x7f8f680749a0 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8f680b4060 con 0x7f8f680cefa0 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f6813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f6813ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35116/0 (socket says 172.21.15.46:35116) 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 -- 172.21.15.46:0/2638488992 learned_addr learned my addr 172.21.15.46:0/2638488992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f5e000640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f680cefa0 0x7f8f68166bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f66a00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f680749a0 0x7f8f681f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 -- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f680cefa0 msgr2=0x7f8f68166bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f680cefa0 0x7f8f68166bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 -- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f680749a0 msgr2=0x7f8f681f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f680749a0 0x7f8f681f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 -- 172.21.15.46:0/2638488992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f681fa010 con 0x7f8f68160330 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f66a00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f680749a0 0x7f8f681f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f67400640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f6813ef70 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8f600b0040 tx=0x7f8f60005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600b2040 con 0x7f8f68160330 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f60005400 con 0x7f8f68160330 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f68167110 con 0x7f8f68160330 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f600046d0 con 0x7f8f68160330 2024-09-17T03:35:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f68165e10 con 0x7f8f68160330 2024-09-17T03:35:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f680cf3b0 con 0x7f8f68160330 2024-09-17T03:35:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.286+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f8f600be060 con 0x7f8f68160330 2024-09-17T03:35:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.290+0000 7f8f5fe00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f44075120 0x7f8f44077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.290+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f8f6012cb50 con 0x7f8f68160330 2024-09-17T03:35:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.290+0000 7f8f66a00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f44075120 0x7f8f44077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.290+0000 7f8f66a00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f44075120 0x7f8f44077630 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009df0 tx=0x7f8f580093e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.294+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8f600164b0 con 0x7f8f68160330 2024-09-17T03:35:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.494+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8f6806b540 con 0x7f8f44075120 2024-09-17T03:35:17.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.514+0000 7f8f5fe00640 1 -- 172.21.15.46:0/2638488992 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8f580099e0 con 0x7f8f44075120 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f44075120 msgr2=0x7f8f44077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f44075120 0x7f8f44077630 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009df0 tx=0x7f8f580093e0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 msgr2=0x7f8f6813ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f6813ef70 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8f600b0040 tx=0x7f8f60005e50 comp rx=0 tx=0).stop 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 shutdown_connections 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8f44075120 0x7f8f44077630 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f68160330 0x7f8f6813ef70 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8f680cefa0 0x7f8f68166bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 --2- 172.21.15.46:0/2638488992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f680749a0 0x7f8f681f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 >> 172.21.15.46:0/2638488992 conn(0x7f8f680b2040 msgr2=0x7f8f68159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:17.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 shutdown_connections 2024-09-17T03:35:17.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.518+0000 7f8f6de00640 1 -- 172.21.15.46:0/2638488992 wait complete. 2024-09-17T03:35:17.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:17.530+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:17.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:17.530+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:17.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:17.530+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:17.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:17.530+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:17.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:17.530+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:17.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone80982 2024-09-17T03:35:17.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 -- 172.21.15.46:0/655168643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 msgr2=0x7f0f2c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 --2- 172.21.15.46:0/655168643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b5a80 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0b6a50 tx=0x7f0f1c01c830 comp rx=0 tx=0).stop 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 -- 172.21.15.46:0/655168643 shutdown_connections 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 --2- 172.21.15.46:0/655168643 >> v2:172.21.15.99:3300/0 conn(0x7f0f2c074b20 0x7f0f2c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 --2- 172.21.15.46:0/655168643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b5a80 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 -- 172.21.15.46:0/655168643 >> 172.21.15.46:0/655168643 conn(0x7f0f2c0708c0 msgr2=0x7f0f2c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 -- 172.21.15.46:0/655168643 shutdown_connections 2024-09-17T03:35:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 -- 172.21.15.46:0/655168643 wait complete. 2024-09-17T03:35:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 Processor -- start 2024-09-17T03:35:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.710+0000 7f0f32600640 1 -- start start 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c160d50 0x7f0f2c14af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c14ebe0 0x7f0f2c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c16a740 con 0x7f0f2c157030 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c16a5c0 con 0x7f0f2c160d50 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c16a8c0 con 0x7f0f2c14ebe0 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35160/0 (socket says 172.21.15.46:35160) 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 -- 172.21.15.46:0/112669384 learned_addr learned my addr 172.21.15.46:0/112669384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2be00640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c14ebe0 0x7f0f2c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f22a00640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c160d50 0x7f0f2c14af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 -- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c14ebe0 msgr2=0x7f0f2c14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c14ebe0 0x7f0f2c14eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 -- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c160d50 msgr2=0x7f0f2c14af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c160d50 0x7f0f2c14af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 -- 172.21.15.46:0/112669384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c14f530 con 0x7f0f2c157030 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f2b400640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b4a00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0b0040 tx=0x7f0f1c0a6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c004340 con 0x7f0f2c157030 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f1c005020 con 0x7f0f2c157030 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c005470 con 0x7f0f2c157030 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c1f9a20 con 0x7f0f2c157030 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1f9fe0 con 0x7f0f2c157030 2024-09-17T03:35:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.714+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c15cf90 con 0x7f0f2c157030 2024-09-17T03:35:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.718+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f0f1c0be060 con 0x7f0f2c157030 2024-09-17T03:35:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.718+0000 7f0f23e00640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f08075120 0x7f0f08077630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.718+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f0f1c12d200 con 0x7f0f2c157030 2024-09-17T03:35:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.718+0000 7f0f22a00640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f08075120 0x7f0f08077630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.718+0000 7f0f22a00640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f08075120 0x7f0f08077630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c14eb50 tx=0x7f0f14009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:17.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:17.722+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0f2c14f530 con 0x7f0f2c157030 2024-09-17T03:35:18.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.110+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone80982", "target": ["mon-mgr", ""]}) -- 0x7f0f2c06b540 con 0x7f0f08075120 2024-09-17T03:35:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.390+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f0f1c0f5930 con 0x7f0f2c157030 2024-09-17T03:35:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.394+0000 7f0f23e00640 1 -- 172.21.15.46:0/112669384 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f14009550 con 0x7f0f08075120 2024-09-17T03:35:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f08075120 msgr2=0x7f0f08077630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f08075120 0x7f0f08077630 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c14eb50 tx=0x7f0f14009060 comp rx=0 tx=0).stop 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 msgr2=0x7f0f2c0b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b4a00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0b0040 tx=0x7f0f1c0a6750 comp rx=0 tx=0).stop 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 shutdown_connections 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0f08075120 0x7f0f08077630 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0f2c14ebe0 0x7f0f2c14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f2c160d50 0x7f0f2c14af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 --2- 172.21.15.46:0/112669384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f2c157030 0x7f0f2c0b4a00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 >> 172.21.15.46:0/112669384 conn(0x7f0f2c0708c0 msgr2=0x7f0f2c14baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 shutdown_connections 2024-09-17T03:35:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.398+0000 7f0f32600640 1 -- 172.21.15.46:0/112669384 wait complete. 2024-09-17T03:35:18.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone18021 2024-09-17T03:35:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 -- 172.21.15.46:0/3865698363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 msgr2=0x7f1cc0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 --2- 172.21.15.46:0/3865698363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0074db0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009900 tx=0x7f1cb001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 -- 172.21.15.46:0/3865698363 shutdown_connections 2024-09-17T03:35:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 --2- 172.21.15.46:0/3865698363 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc00cefa0 0x7f1cc00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 --2- 172.21.15.46:0/3865698363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0074db0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 -- 172.21.15.46:0/3865698363 >> 172.21.15.46:0/3865698363 conn(0x7f1cc00b2040 msgr2=0x7f1cc00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 -- 172.21.15.46:0/3865698363 shutdown_connections 2024-09-17T03:35:18.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.582+0000 7f1cc5000640 1 -- 172.21.15.46:0/3865698363 wait complete. 2024-09-17T03:35:18.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 Processor -- start 2024-09-17T03:35:18.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 -- start start 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc00cefa0 0x7f1cc013a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc0160330 0x7f1cc013a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc016a060 con 0x7f1cc00749a0 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0169ee0 con 0x7f1cc00cefa0 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cc5000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc016a1e0 con 0x7f1cc0160330 2024-09-17T03:35:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35186/0 (socket says 172.21.15.46:35186) 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 -- 172.21.15.46:0/1084378318 learned_addr learned my addr 172.21.15.46:0/1084378318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 -- 172.21.15.46:0/1084378318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc0160330 msgr2=0x7f1cc013a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc0160330 0x7f1cc013a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 -- 172.21.15.46:0/1084378318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc00cefa0 msgr2=0x7f1cc013a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc00cefa0 0x7f1cc013a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.586+0000 7f1cbde00640 1 -- 172.21.15.46:0/1084378318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc013b9c0 con 0x7f1cc00749a0 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cbde00640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0141cf0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b0040 tx=0x7f1cb00041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00b2040 con 0x7f1cc00749a0 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb0003600 con 0x7f1cc00749a0 2024-09-17T03:35:18.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc013bca0 con 0x7f1cc00749a0 2024-09-17T03:35:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0003a10 con 0x7f1cc00749a0 2024-09-17T03:35:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01f7850 con 0x7f1cc00749a0 2024-09-17T03:35:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc01f7a80 con 0x7f1cc00749a0 2024-09-17T03:35:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f1cb00bf060 con 0x7f1cc00749a0 2024-09-17T03:35:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.590+0000 7f1cb6000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca00751b0 0x7f1ca00776c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.594+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5795+0+0 (secure 0 0 0) 0x7f1cb012ce90 con 0x7f1cc00749a0 2024-09-17T03:35:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.594+0000 7f1cbd400640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca00751b0 0x7f1ca00776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:18.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.594+0000 7f1cbd400640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca00751b0 0x7f1ca00776c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1cc013b310 tx=0x7f1cac006c00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:18.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.598+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1cb00164d0 con 0x7f1cc00749a0 2024-09-17T03:35:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:18.794+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone18021", "target": ["mon-mgr", ""]}) -- 0x7f1cc006b540 con 0x7f1ca00751b0 2024-09-17T03:35:24.574 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:24.570+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:24.575 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:24.570+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:24.575 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:24.570+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:24.575 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:24.570+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:24.575 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:35:24.570+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:35:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.598+0000 7f1cb6000640 1 -- 172.21.15.46:0/1084378318 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cac013430 con 0x7f1ca00751b0 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca00751b0 msgr2=0x7f1ca00776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca00751b0 0x7f1ca00776c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1cc013b310 tx=0x7f1cac006c00 comp rx=0 tx=0).stop 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 msgr2=0x7f1cc0141cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0141cf0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b0040 tx=0x7f1cb00041a0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 shutdown_connections 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1ca00751b0 0x7f1ca00776c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1cc0160330 0x7f1cc013a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc00cefa0 0x7f1cc013a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 --2- 172.21.15.46:0/1084378318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cc00749a0 0x7f1cc0141cf0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 >> 172.21.15.46:0/1084378318 conn(0x7f1cc00b2040 msgr2=0x7f1cc013ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:24.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 shutdown_connections 2024-09-17T03:35:24.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.602+0000 7f1cc5000640 1 -- 172.21.15.46:0/1084378318 wait complete. 2024-09-17T03:35:24.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone19820 2024-09-17T03:35:24.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 -- 172.21.15.46:0/4246324146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b3130 msgr2=0x7f7dcc0b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:24.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 --2- 172.21.15.46:0/4246324146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc0b3540 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009bf0 tx=0x7f7dc401cae0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 -- 172.21.15.46:0/4246324146 shutdown_connections 2024-09-17T03:35:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 --2- 172.21.15.46:0/4246324146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc0b3540 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 -- 172.21.15.46:0/4246324146 >> 172.21.15.46:0/4246324146 conn(0x7f7dcc06cdf0 msgr2=0x7f7dcc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 -- 172.21.15.46:0/4246324146 shutdown_connections 2024-09-17T03:35:24.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 -- 172.21.15.46:0/4246324146 wait complete. 2024-09-17T03:35:24.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.798+0000 7f7dd1800640 1 Processor -- start 2024-09-17T03:35:24.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 -- start start 2024-09-17T03:35:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b1dc0 0x7f7dcc150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:24.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7dcc155a00 0x7f7dcc147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc169f10 con 0x7f7dcc0b3130 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc169d90 con 0x7f7dcc0b1dc0 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc16a090 con 0x7f7dcc155a00 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42084/0 (socket says 172.21.15.46:42084) 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 -- 172.21.15.46:0/463992510 learned_addr learned my addr 172.21.15.46:0/463992510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dcbe00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b1dc0 0x7f7dcc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd0e00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7dcc155a00 0x7f7dcc147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 -- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7dcc155a00 msgr2=0x7f7dcc147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7dcc155a00 0x7f7dcc147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 -- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b1dc0 msgr2=0x7f7dcc150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b1dc0 0x7f7dcc150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 -- 172.21.15.46:0/463992510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc14a940 con 0x7f7dcc0b3130 2024-09-17T03:35:24.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc3e00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc152b50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f7db400c430 tx=0x7f7db400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4011790 con 0x7f7dcc0b3130 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db4017070 con 0x7f7dcc0b3130 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc14abe0 con 0x7f7dcc0b3130 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4011ca0 con 0x7f7dcc0b3130 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.802+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc14b1a0 con 0x7f7dcc0b3130 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.806+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc14f2a0 con 0x7f7dcc0b3130 2024-09-17T03:35:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.806+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f7db401c460 con 0x7f7dcc0b3130 2024-09-17T03:35:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.806+0000 7f7dc8c00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7da8075180 0x7f7da8077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.806+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5851+0+0 (secure 0 0 0) 0x7f7db4092f00 con 0x7f7dcc0b3130 2024-09-17T03:35:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.806+0000 7f7dcbe00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7da8075180 0x7f7da8077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:24.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.806+0000 7f7dcbe00640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7da8075180 0x7f7da8077690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc000c10 tx=0x7f7dbc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:24.810+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7db405f3b0 con 0x7f7dcc0b3130 2024-09-17T03:35:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:25.010+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone19820", "target": ["mon-mgr", ""]}) -- 0x7f7dcc06b540 con 0x7f7da8075180 2024-09-17T03:35:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:26.414+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f7db405b620 con 0x7f7dcc0b3130 2024-09-17T03:35:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.562+0000 7f7dc8c00640 1 -- 172.21.15.46:0/463992510 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7dbc001450 con 0x7f7da8075180 2024-09-17T03:35:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7da8075180 msgr2=0x7f7da8077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7da8075180 0x7f7da8077690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc000c10 tx=0x7f7dbc009040 comp rx=0 tx=0).stop 2024-09-17T03:35:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 msgr2=0x7f7dcc152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc152b50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f7db400c430 tx=0x7f7db400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 shutdown_connections 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7da8075180 0x7f7da8077690 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7dcc155a00 0x7f7dcc147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dcc0b3130 0x7f7dcc152b50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 --2- 172.21.15.46:0/463992510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dcc0b1dc0 0x7f7dcc150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 >> 172.21.15.46:0/463992510 conn(0x7f7dcc06cdf0 msgr2=0x7f7dcc150800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 shutdown_connections 2024-09-17T03:35:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.566+0000 7f7dd1800640 1 -- 172.21.15.46:0/463992510 wait complete. 2024-09-17T03:35:29.601 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:35:29.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:35:29.682 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:35:29.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:35:29.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:35:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 -- 172.21.15.46:0/1696188219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0074950 msgr2=0x7fe7e00b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 --2- 172.21.15.46:0/1696188219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0074950 0x7fe7e00b5260 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009bc0 tx=0x7fe7d801cb10 comp rx=0 tx=0).stop 2024-09-17T03:35:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 -- 172.21.15.46:0/1696188219 shutdown_connections 2024-09-17T03:35:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 --2- 172.21.15.46:0/1696188219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0074950 0x7fe7e00b5260 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 -- 172.21.15.46:0/1696188219 >> 172.21.15.46:0/1696188219 conn(0x7fe7e00b4560 msgr2=0x7fe7e00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 -- 172.21.15.46:0/1696188219 shutdown_connections 2024-09-17T03:35:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.854+0000 7fe7e7000640 1 -- 172.21.15.46:0/1696188219 wait complete. 2024-09-17T03:35:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 Processor -- start 2024-09-17T03:35:29.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 -- start start 2024-09-17T03:35:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e01536f0 0x7fe7e00d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 0x7fe7e0143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7e0154880 0x7fe7e0154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e016aec0 con 0x7fe7e0160060 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e016ad40 con 0x7fe7e01536f0 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e7000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e016b040 con 0x7fe7e0154880 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 0x7fe7e0143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 0x7fe7e0143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42102/0 (socket says 172.21.15.46:42102) 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 -- 172.21.15.46:0/354037076 learned_addr learned my addr 172.21.15.46:0/354037076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5c00640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e01536f0 0x7fe7e00d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e6600640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7e0154880 0x7fe7e0154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 -- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7e0154880 msgr2=0x7fe7e0154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7e0154880 0x7fe7e0154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 -- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e01536f0 msgr2=0x7fe7e00d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e01536f0 0x7fe7e00d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 -- 172.21.15.46:0/354037076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e00d1050 con 0x7fe7e0160060 2024-09-17T03:35:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.858+0000 7fe7e5200640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 0x7fe7e0143bf0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c430 tx=0x7fe7cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc011790 con 0x7fe7e0160060 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7cc017070 con 0x7fe7e0160060 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e00d1330 con 0x7fe7e0160060 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc011d00 con 0x7fe7e0160060 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e00d18f0 con 0x7fe7e0160060 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e00d0dc0 con 0x7fe7e0160060 2024-09-17T03:35:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fe7cc01c460 con 0x7fe7e0160060 2024-09-17T03:35:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7dd600640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe7c0075240 0x7fe7c0077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7fe7cc092fb0 con 0x7fe7e0160060 2024-09-17T03:35:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.862+0000 7fe7e5c00640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe7c0075240 0x7fe7c0077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.866+0000 7fe7e5c00640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe7c0075240 0x7fe7c0077750 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4007fd0 tx=0x7fe7d4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:29.866+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7cc05f3e0 con 0x7fe7e0160060 2024-09-17T03:35:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe7e00b5260 con 0x7fe7e0160060 2024-09-17T03:35:30.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7dd600640 1 -- 172.21.15.46:0/354037076 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v53)=0 v53) ==== 131+0+0 (secure 0 0 0) 0x7fe7cc063e10 con 0x7fe7e0160060 2024-09-17T03:35:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe7c0075240 msgr2=0x7fe7c0077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe7c0075240 0x7fe7c0077750 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4007fd0 tx=0x7fe7d4009040 comp rx=0 tx=0).stop 2024-09-17T03:35:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 msgr2=0x7fe7e0143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 0x7fe7e0143bf0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c430 tx=0x7fe7cc00c8f0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 shutdown_connections 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe7c0075240 0x7fe7c0077750 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe7e0154880 0x7fe7e0154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e0160060 0x7fe7e0143bf0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 --2- 172.21.15.46:0/354037076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e01536f0 0x7fe7e00d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 >> 172.21.15.46:0/354037076 conn(0x7fe7e00b4560 msgr2=0x7fe7e015f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 shutdown_connections 2024-09-17T03:35:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.046+0000 7fe7e7000640 1 -- 172.21.15.46:0/354037076 wait complete. 2024-09-17T03:35:30.068 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:35:30.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:35:30.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:35:30.281 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:35:30.281 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:35:30.281 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:35:30.281 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:30.290 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:35:30.290 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:30.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 -- 172.21.15.46:0/1982013598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500d0550 msgr2=0x7f0a500b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 --2- 172.21.15.46:0/1982013598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500d0550 0x7f0a500b3130 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009840 tx=0x7f0a3801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 -- 172.21.15.46:0/1982013598 shutdown_connections 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 --2- 172.21.15.46:0/1982013598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a500b3670 0x7f0a50072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 --2- 172.21.15.46:0/1982013598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500d0550 0x7f0a500b3130 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 -- 172.21.15.46:0/1982013598 >> 172.21.15.46:0/1982013598 conn(0x7f0a5006cdf0 msgr2=0x7f0a5006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:30.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 -- 172.21.15.46:0/1982013598 shutdown_connections 2024-09-17T03:35:30.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 -- 172.21.15.46:0/1982013598 wait complete. 2024-09-17T03:35:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 Processor -- start 2024-09-17T03:35:30.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.550+0000 7f0a55800640 1 -- start start 2024-09-17T03:35:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a500b3670 0x7f0a501f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a500b5040 0x7f0a500b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 0x7f0a50142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50165d40 con 0x7f0a500b1100 2024-09-17T03:35:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50165bc0 con 0x7f0a500b3670 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0a50165ec0 con 0x7f0a500b5040 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 0x7f0a50142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 0x7f0a50142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42120/0 (socket says 172.21.15.46:42120) 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 -- 172.21.15.46:0/96445862 learned_addr learned my addr 172.21.15.46:0/96445862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a4fe00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a500b3670 0x7f0a501f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a47400640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a500b5040 0x7f0a500b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 -- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a500b5040 msgr2=0x7f0a500b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a500b5040 0x7f0a500b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 -- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a500b3670 msgr2=0x7f0a501f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a500b3670 0x7f0a501f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 -- 172.21.15.46:0/96445862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5013c1b0 con 0x7f0a500b1100 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a4fe00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a500b3670 0x7f0a501f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a54e00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 0x7f0a50142d80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800ea80 tx=0x7f0a4800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48004b30 con 0x7f0a500b1100 2024-09-17T03:35:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a48004cf0 con 0x7f0a500b1100 2024-09-17T03:35:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48005280 con 0x7f0a500b1100 2024-09-17T03:35:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a50144400 con 0x7f0a500b1100 2024-09-17T03:35:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a50142580 con 0x7f0a500b1100 2024-09-17T03:35:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.554+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a50072030 con 0x7f0a500b1100 2024-09-17T03:35:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.558+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f0a48005420 con 0x7f0a500b1100 2024-09-17T03:35:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.558+0000 7f0a4cc00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0a2c075210 0x7f0a2c077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.558+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7f0a48093f00 con 0x7f0a500b1100 2024-09-17T03:35:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.558+0000 7f0a4fe00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0a2c075210 0x7f0a2c077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.558+0000 7f0a4fe00640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0a2c075210 0x7f0a2c077720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a40007a60 tx=0x7f0a40009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.562+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0a5013c1b0 con 0x7f0a500b1100 2024-09-17T03:35:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.750+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a50077240 con 0x7f0a500b1100 2024-09-17T03:35:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.750+0000 7f0a4cc00640 1 -- 172.21.15.46:0/96445862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7f0a48060980 con 0x7f0a500b1100 2024-09-17T03:35:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0a2c075210 msgr2=0x7f0a2c077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0a2c075210 0x7f0a2c077720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a40007a60 tx=0x7f0a40009040 comp rx=0 tx=0).stop 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 msgr2=0x7f0a50142d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 0x7f0a50142d80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800ea80 tx=0x7f0a4800ef40 comp rx=0 tx=0).stop 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 shutdown_connections 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0a2c075210 0x7f0a2c077720 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a500b1100 0x7f0a50142d80 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0a500b5040 0x7f0a500b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 --2- 172.21.15.46:0/96445862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a500b3670 0x7f0a501f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 >> 172.21.15.46:0/96445862 conn(0x7f0a5006cdf0 msgr2=0x7f0a50159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 shutdown_connections 2024-09-17T03:35:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:30.754+0000 7f0a55800640 1 -- 172.21.15.46:0/96445862 wait complete. 2024-09-17T03:35:30.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:35:30.765 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:35:30.772 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:35:30.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:30.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:35:30.901 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:35:30.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:30.901 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:35:30.920 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:35:30.920 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:35:30.921 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:30.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:35:30.941 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:35:30.941 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:35:30.941 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:30.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:35:30.977 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:35:30.977 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:35:30.977 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:30.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:35:30.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:31.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 -- 172.21.15.46:0/1976181637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 msgr2=0x7f684c074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 --2- 172.21.15.46:0/1976181637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c074db0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f683c009900 tx=0x7f683c01c8e0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 -- 172.21.15.46:0/1976181637 shutdown_connections 2024-09-17T03:35:31.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 --2- 172.21.15.46:0/1976181637 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c0cefa0 0x7f684c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 --2- 172.21.15.46:0/1976181637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c074db0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 -- 172.21.15.46:0/1976181637 >> 172.21.15.46:0/1976181637 conn(0x7f684c0b2040 msgr2=0x7f684c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:31.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 -- 172.21.15.46:0/1976181637 shutdown_connections 2024-09-17T03:35:31.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.178+0000 7f6852200640 1 -- 172.21.15.46:0/1976181637 wait complete. 2024-09-17T03:35:31.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 Processor -- start 2024-09-17T03:35:31.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 -- start start 2024-09-17T03:35:31.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c0aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0cefa0 0x7f684c1fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c160330 0x7f684c1fa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f684c16a060 con 0x7f684c0749a0 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f684c169ee0 con 0x7f684c0cefa0 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6852200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f684c16a1e0 con 0x7f684c160330 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c0aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c0aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42140/0 (socket says 172.21.15.46:42140) 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 -- 172.21.15.46:0/16452447 learned_addr learned my addr 172.21.15.46:0/16452447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684be00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c160330 0x7f684c1fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6842a00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0cefa0 0x7f684c1fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 -- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c160330 msgr2=0x7f684c1fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c160330 0x7f684c1fa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 -- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0cefa0 msgr2=0x7f684c1fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0cefa0 0x7f684c1fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 -- 172.21.15.46:0/16452447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c0aacd0 con 0x7f684c0749a0 2024-09-17T03:35:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684be00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c160330 0x7f684c1fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6842a00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0cefa0 0x7f684c1fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f684b400640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c0aa4d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f683c009850 tx=0x7f683c0a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.182+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c0b2040 con 0x7f684c0749a0 2024-09-17T03:35:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f683c0042b0 con 0x7f684c0749a0 2024-09-17T03:35:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c005020 con 0x7f684c0749a0 2024-09-17T03:35:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c0aaa10 con 0x7f684c0749a0 2024-09-17T03:35:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c145260 con 0x7f684c0749a0 2024-09-17T03:35:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c074db0 con 0x7f684c0749a0 2024-09-17T03:35:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f683c0bf030 con 0x7f684c0749a0 2024-09-17T03:35:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6843e00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6828075200 0x7f6828077710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.186+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7f683c12d800 con 0x7f684c0749a0 2024-09-17T03:35:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.190+0000 7f6842a00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6828075200 0x7f6828077710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.190+0000 7f6842a00640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6828075200 0x7f6828077710 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f6834005de0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.194+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f683c0c6020 con 0x7f684c0749a0 2024-09-17T03:35:31.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.402+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f684c0b1050 con 0x7f684c0749a0 2024-09-17T03:35:31.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.402+0000 7f6843e00640 1 -- 172.21.15.46:0/16452447 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5241 (secure 0 0 0) 0x7f683c0164d0 con 0x7f684c0749a0 2024-09-17T03:35:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:31.402 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"btime":"2024-09-17T03:34:26:988878+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:34:18.364785+0000","modified":"2024-09-17T03:34:21.431279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248196":{"gid":248196,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/1168163211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1168163211},{"type":"v1","addr":"172.21.15.46:6837","nonce":1168163211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248196,"qdb_cluster":[248196]},"id":8}]} 2024-09-17T03:35:31.402 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6828075200 msgr2=0x7f6828077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6828075200 0x7f6828077710 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6834005fd0 tx=0x7f6834005de0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 msgr2=0x7f684c0aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c0aa4d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f683c009850 tx=0x7f683c0a5880 comp rx=0 tx=0).stop 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 shutdown_connections 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6828075200 0x7f6828077710 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f684c160330 0x7f684c1fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f684c0cefa0 0x7f684c1fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 --2- 172.21.15.46:0/16452447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0749a0 0x7f684c0aa4d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 >> 172.21.15.46:0/16452447 conn(0x7f684c0b2040 msgr2=0x7f684c160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 shutdown_connections 2024-09-17T03:35:31.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.406+0000 7f6852200640 1 -- 172.21.15.46:0/16452447 wait complete. 2024-09-17T03:35:31.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 -- 172.21.15.46:0/2821174583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b5c70 msgr2=0x7fbbf415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 --2- 172.21.15.46:0/2821174583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b5c70 0x7fbbf415bb90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009870 tx=0x7fbbe401c750 comp rx=0 tx=0).stop 2024-09-17T03:35:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 -- 172.21.15.46:0/2821174583 shutdown_connections 2024-09-17T03:35:31.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 --2- 172.21.15.46:0/2821174583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b5c70 0x7fbbf415bb90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 -- 172.21.15.46:0/2821174583 >> 172.21.15.46:0/2821174583 conn(0x7fbbf40b2040 msgr2=0x7fbbf40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:31.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 -- 172.21.15.46:0/2821174583 shutdown_connections 2024-09-17T03:35:31.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 -- 172.21.15.46:0/2821174583 wait complete. 2024-09-17T03:35:31.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 Processor -- start 2024-09-17T03:35:31.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.590+0000 7fbbf9600640 1 -- start start 2024-09-17T03:35:31.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b4de0 0x7fbbf4152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 0x7fbbf4154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf41527b0 0x7fbbf4158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4169cc0 con 0x7fbbf40b5c70 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4169b40 con 0x7fbbf41527b0 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf4169e40 con 0x7fbbf40b4de0 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 0x7fbbf4154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 0x7fbbf4154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42154/0 (socket says 172.21.15.46:42154) 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 -- 172.21.15.46:0/3150178669 learned_addr learned my addr 172.21.15.46:0/3150178669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf2a00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b4de0 0x7fbbf4152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf3400640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf41527b0 0x7fbbf4158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 -- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b4de0 msgr2=0x7fbbf4152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b4de0 0x7fbbf4152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 -- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf41527b0 msgr2=0x7fbbf4158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf41527b0 0x7fbbf4158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 -- 172.21.15.46:0/3150178669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf414ccf0 con 0x7fbbf40b5c70 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf3400640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf41527b0 0x7fbbf4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbebe00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 0x7fbbf4154600 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c4b0 tx=0x7fbbdc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc016020 con 0x7fbbf40b5c70 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbdc012920 con 0x7fbbf40b5c70 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc012d50 con 0x7fbbf40b5c70 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf4153410 con 0x7fbbf40b5c70 2024-09-17T03:35:31.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf41539d0 con 0x7fbbf40b5c70 2024-09-17T03:35:31.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.594+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf415c840 con 0x7fbbf40b5c70 2024-09-17T03:35:31.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.598+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fbbdc019020 con 0x7fbbf40b5c70 2024-09-17T03:35:31.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.598+0000 7fbbeaa00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbd0075210 0x7fbbd0077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:31.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.598+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7fbbdc092fa0 con 0x7fbbf40b5c70 2024-09-17T03:35:31.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.598+0000 7fbbf2a00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbd0075210 0x7fbbd0077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:31.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.598+0000 7fbbf2a00640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbd0075210 0x7fbbd0077720 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40af000 tx=0x7fbbe401af90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:31.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.602+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbbdc020030 con 0x7fbbf40b5c70 2024-09-17T03:35:31.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.814+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbf41531a0 con 0x7fbbf40b5c70 2024-09-17T03:35:31.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.814+0000 7fbbeaa00640 1 -- 172.21.15.46:0/3150178669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5241 (secure 0 0 0) 0x7fbbdc05fab0 con 0x7fbbf40b5c70 2024-09-17T03:35:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:31.813 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"btime":"2024-09-17T03:34:26:988878+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:34:18.364785+0000","modified":"2024-09-17T03:34:21.431279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248196":{"gid":248196,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/1168163211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1168163211},{"type":"v1","addr":"172.21.15.46:6837","nonce":1168163211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248196,"qdb_cluster":[248196]},"id":8}]} 2024-09-17T03:35:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.814+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbd0075210 msgr2=0x7fbbd0077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.814+0000 7fbbf9600640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbd0075210 0x7fbbd0077720 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40af000 tx=0x7fbbe401af90 comp rx=0 tx=0).stop 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.814+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 msgr2=0x7fbbf4154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.814+0000 7fbbf9600640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 0x7fbbf4154600 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c4b0 tx=0x7fbbdc00c970 comp rx=0 tx=0).stop 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 shutdown_connections 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbd0075210 0x7fbbd0077720 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf41527b0 0x7fbbf4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf40b5c70 0x7fbbf4154600 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 --2- 172.21.15.46:0/3150178669 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbf40b4de0 0x7fbbf4152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 >> 172.21.15.46:0/3150178669 conn(0x7fbbf40b2040 msgr2=0x7fbbf40b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 shutdown_connections 2024-09-17T03:35:31.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.818+0000 7fbbf9600640 1 -- 172.21.15.46:0/3150178669 wait complete. 2024-09-17T03:35:31.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 -- 172.21.15.46:0/1906970052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 msgr2=0x7f07cc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 --2- 172.21.15.46:0/1906970052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 0x7f07cc0b5a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009870 tx=0x7f07c001c860 comp rx=0 tx=0).stop 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 -- 172.21.15.46:0/1906970052 shutdown_connections 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 --2- 172.21.15.46:0/1906970052 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 0x7f07cc0b5a80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 -- 172.21.15.46:0/1906970052 >> 172.21.15.46:0/1906970052 conn(0x7f07cc0708c0 msgr2=0x7f07cc070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 -- 172.21.15.46:0/1906970052 shutdown_connections 2024-09-17T03:35:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 -- 172.21.15.46:0/1906970052 wait complete. 2024-09-17T03:35:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 Processor -- start 2024-09-17T03:35:31.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:31.998+0000 7f07d3c00640 1 -- start start 2024-09-17T03:35:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 0x7f07cc1fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07cc160d50 0x7f07cc13df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 0x7f07cc14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc16b970 con 0x7f07cc14bfc0 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc16b7f0 con 0x7f07cc160d50 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc16baf0 con 0x7f07cc157030 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 0x7f07cc14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 0x7f07cc14d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42156/0 (socket says 172.21.15.46:42156) 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 -- 172.21.15.46:0/2371054550 learned_addr learned my addr 172.21.15.46:0/2371054550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1200640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 0x7f07cc1fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07cbe00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07cc160d50 0x7f07cc13df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 -- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 msgr2=0x7f07cc1fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 0x7f07cc1fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 -- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07cc160d50 msgr2=0x7f07cc13df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07cc160d50 0x7f07cc13df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 -- 172.21.15.46:0/2371054550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc1fd220 con 0x7f07cc14bfc0 2024-09-17T03:35:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d1c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 0x7f07cc14d200 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f07c400cac0 tx=0x7f07c400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4016020 con 0x7f07cc14bfc0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c400a900 con 0x7f07cc14bfc0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc1fd500 con 0x7f07cc14bfc0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c400ad50 con 0x7f07cc14bfc0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc06d2f0 con 0x7f07cc14bfc0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc0b5000 con 0x7f07cc14bfc0 2024-09-17T03:35:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.002+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f07c4019020 con 0x7f07cc14bfc0 2024-09-17T03:35:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.006+0000 7f07c8c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07ac075210 0x7f07ac077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.006+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7f07c40934b0 con 0x7f07cc14bfc0 2024-09-17T03:35:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.006+0000 7f07d1200640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07ac075210 0x7f07ac077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.006+0000 7f07d1200640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07ac075210 0x7f07ac077720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f07c00af000 tx=0x7f07c001b0a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:32.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.010+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07c4020020 con 0x7f07cc14bfc0 2024-09-17T03:35:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.266+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07cc1fcf70 con 0x7f07cc14bfc0 2024-09-17T03:35:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07c8c00640 1 -- 172.21.15.46:0/2371054550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16915 (secure 0 0 0) 0x7f07c405ff30 con 0x7f07cc14bfc0 2024-09-17T03:35:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:32.268 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:35:26.140430+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:34:16.113350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T03:34:17.334380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07ac075210 msgr2=0x7f07ac077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07ac075210 0x7f07ac077720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f07c00af000 tx=0x7f07c001b0a0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 msgr2=0x7f07cc14d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 0x7f07cc14d200 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f07c400cac0 tx=0x7f07c400cbb0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 shutdown_connections 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07ac075210 0x7f07ac077720 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07cc14bfc0 0x7f07cc14d200 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07cc160d50 0x7f07cc13df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 --2- 172.21.15.46:0/2371054550 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07cc157030 0x7f07cc1fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 >> 172.21.15.46:0/2371054550 conn(0x7f07cc0708c0 msgr2=0x7f07cc15fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 shutdown_connections 2024-09-17T03:35:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.270+0000 7f07d3c00640 1 -- 172.21.15.46:0/2371054550 wait complete. 2024-09-17T03:35:32.292 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:35:32.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 -- 172.21.15.46:0/3884073766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 msgr2=0x7fb9d415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 --2- 172.21.15.46:0/3884073766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d415bb60 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40cca90 tx=0x7fb9c401c740 comp rx=0 tx=0).stop 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 -- 172.21.15.46:0/3884073766 shutdown_connections 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 --2- 172.21.15.46:0/3884073766 >> v2:172.21.15.99:3300/0 conn(0x7fb9d40b4de0 0x7fb9d40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 --2- 172.21.15.46:0/3884073766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d415bb60 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 -- 172.21.15.46:0/3884073766 >> 172.21.15.46:0/3884073766 conn(0x7fb9d40b2040 msgr2=0x7fb9d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 -- 172.21.15.46:0/3884073766 shutdown_connections 2024-09-17T03:35:32.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 -- 172.21.15.46:0/3884073766 wait complete. 2024-09-17T03:35:32.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 Processor -- start 2024-09-17T03:35:32.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.454+0000 7fb9daa00640 1 -- start start 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d40b4de0 0x7fb9d41fa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d4149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9d41527b0 0x7fb9d415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4168910 con 0x7fb9d40b5c70 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4168790 con 0x7fb9d40b4de0 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d4168a90 con 0x7fb9d41527b0 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d4149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9d3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d40b4de0 0x7fb9d41fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d4149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39374/0 (socket says 172.21.15.46:39374) 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 -- 172.21.15.46:0/1834110295 learned_addr learned my addr 172.21.15.46:0/1834110295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9d8a00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9d41527b0 0x7fb9d415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 -- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9d41527b0 msgr2=0x7fb9d415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9d41527b0 0x7fb9d415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 -- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d40b4de0 msgr2=0x7fb9d41fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d40b4de0 0x7fb9d41fa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 -- 172.21.15.46:0/1834110295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d4157bd0 con 0x7fb9d40b5c70 2024-09-17T03:35:32.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9d3e00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d40b4de0 0x7fb9d41fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9cbe00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d4149a40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc001ec0 tx=0x7fb9bc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc017070 con 0x7fb9d40b5c70 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9bc003600 con 0x7fb9d40b5c70 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc003a50 con 0x7fb9d40b5c70 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d41fad50 con 0x7fb9d40b5c70 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d4158190 con 0x7fb9d40b5c70 2024-09-17T03:35:32.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.458+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d40b51f0 con 0x7fb9d40b5c70 2024-09-17T03:35:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.462+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fb9bc01a070 con 0x7fb9d40b5c70 2024-09-17T03:35:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.462+0000 7fb9d0c00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9b0075210 0x7fb9b0077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.462+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7fb9bc0925c0 con 0x7fb9d40b5c70 2024-09-17T03:35:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.462+0000 7fb9d3e00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9b0075210 0x7fb9b0077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.462+0000 7fb9d3e00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9b0075210 0x7fb9b0077720 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40af000 tx=0x7fb9c401af80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:32.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.466+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb9d40b5730 con 0x7fb9d40b5c70 2024-09-17T03:35:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.674+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9d4151770 con 0x7fb9d40b5c70 2024-09-17T03:35:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.674+0000 7fb9d0c00640 1 -- 172.21.15.46:0/1834110295 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5241 (secure 0 0 0) 0x7fb9bc05ea30 con 0x7fb9d40b5c70 2024-09-17T03:35:32.673 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:32.673 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"btime":"2024-09-17T03:34:26:988878+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:34:18.364785+0000","modified":"2024-09-17T03:34:21.431279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248196":{"gid":248196,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/1168163211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1168163211},{"type":"v1","addr":"172.21.15.46:6837","nonce":1168163211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248196,"qdb_cluster":[248196]},"id":8}]} 2024-09-17T03:35:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2024-09-17T03:35:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.674+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9b0075210 msgr2=0x7fb9b0077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.674+0000 7fb9daa00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9b0075210 0x7fb9b0077720 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40af000 tx=0x7fb9c401af80 comp rx=0 tx=0).stop 2024-09-17T03:35:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.674+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 msgr2=0x7fb9d4149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.674+0000 7fb9daa00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d4149a40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc001ec0 tx=0x7fb9bc00c890 comp rx=0 tx=0).stop 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 shutdown_connections 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb9b0075210 0x7fb9b0077720 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb9d41527b0 0x7fb9d415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9d40b5c70 0x7fb9d4149a40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 --2- 172.21.15.46:0/1834110295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d40b4de0 0x7fb9d41fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 >> 172.21.15.46:0/1834110295 conn(0x7fb9d40b2040 msgr2=0x7fb9d415b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 shutdown_connections 2024-09-17T03:35:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.678+0000 7fb9daa00640 1 -- 172.21.15.46:0/1834110295 wait complete. 2024-09-17T03:35:32.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 -- 172.21.15.46:0/2792681603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510074b20 msgr2=0x7fc51015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 --2- 172.21.15.46:0/2792681603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510074b20 0x7fc51015cf90 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc5100713e0 tx=0x7fc50801caf0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 -- 172.21.15.46:0/2792681603 shutdown_connections 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 --2- 172.21.15.46:0/2792681603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510074b20 0x7fc51015cf90 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 --2- 172.21.15.46:0/2792681603 >> v2:172.21.15.99:3300/0 conn(0x7fc510157030 0x7fc5100b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 -- 172.21.15.46:0/2792681603 >> 172.21.15.46:0/2792681603 conn(0x7fc5100708c0 msgr2=0x7fc510070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 -- 172.21.15.46:0/2792681603 shutdown_connections 2024-09-17T03:35:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.854+0000 7fc515a00640 1 -- 172.21.15.46:0/2792681603 wait complete. 2024-09-17T03:35:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 Processor -- start 2024-09-17T03:35:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 -- start start 2024-09-17T03:35:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc510157030 0x7fc5101f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 0x7fc5101fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc51013ec80 0x7fc5100b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc51016a790 con 0x7fc510160d50 2024-09-17T03:35:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc51016a610 con 0x7fc510157030 2024-09-17T03:35:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc515a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc51016a910 con 0x7fc51013ec80 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc506000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 0x7fc5101fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc506000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 0x7fc5101fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39390/0 (socket says 172.21.15.46:39390) 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc506000640 1 -- 172.21.15.46:0/2434644338 learned_addr learned my addr 172.21.15.46:0/2434644338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc50f400640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc51013ec80 0x7fc5100b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.858+0000 7fc50ea00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc510157030 0x7fc5101f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc506000640 1 -- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc51013ec80 msgr2=0x7fc5100b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc506000640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc51013ec80 0x7fc5100b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc506000640 1 -- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc510157030 msgr2=0x7fc5101f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc506000640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc510157030 0x7fc5101f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc506000640 1 -- 172.21.15.46:0/2434644338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101fd4c0 con 0x7fc510160d50 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc50ea00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc510157030 0x7fc5101f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc506000640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 0x7fc5101fcae0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c430 tx=0x7fc4f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8011790 con 0x7fc510160d50 2024-09-17T03:35:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4f8017070 con 0x7fc510160d50 2024-09-17T03:35:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5101fd7a0 con 0x7fc510160d50 2024-09-17T03:35:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8011ce0 con 0x7fc510160d50 2024-09-17T03:35:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5100b2310 con 0x7fc510160d50 2024-09-17T03:35:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5101fd230 con 0x7fc510160d50 2024-09-17T03:35:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.862+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fc4f801c460 con 0x7fc510160d50 2024-09-17T03:35:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.866+0000 7fc507400640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc4ec075210 0x7fc4ec077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.866+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7fc4f8093020 con 0x7fc510160d50 2024-09-17T03:35:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.866+0000 7fc50ea00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc4ec075210 0x7fc4ec077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.866+0000 7fc50ea00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc4ec075210 0x7fc4ec077720 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc500009b90 tx=0x7fc5000091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:32.870+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4f805fb70 con 0x7fc510160d50 2024-09-17T03:35:33.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5101f94f0 con 0x7fc510160d50 2024-09-17T03:35:33.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc507400640 1 -- 172.21.15.46:0/2434644338 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5241 (secure 0 0 0) 0x7fc4f805f470 con 0x7fc510160d50 2024-09-17T03:35:33.077 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:33.077 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"btime":"2024-09-17T03:34:26:988878+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:34:18.364785+0000","modified":"2024-09-17T03:34:21.431279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248196":{"gid":248196,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/1168163211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1168163211},{"type":"v1","addr":"172.21.15.46:6837","nonce":1168163211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248196,"qdb_cluster":[248196]},"id":8}]} 2024-09-17T03:35:33.077 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2024-09-17T03:35:33.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc4ec075210 msgr2=0x7fc4ec077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc4ec075210 0x7fc4ec077720 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc500009b90 tx=0x7fc5000091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 msgr2=0x7fc5101fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 0x7fc5101fcae0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c430 tx=0x7fc4f800c8f0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 shutdown_connections 2024-09-17T03:35:33.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc4ec075210 0x7fc4ec077720 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc51013ec80 0x7fc5100b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510160d50 0x7fc5101fcae0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 --2- 172.21.15.46:0/2434644338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc510157030 0x7fc5101f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 >> 172.21.15.46:0/2434644338 conn(0x7fc5100708c0 msgr2=0x7fc510151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 shutdown_connections 2024-09-17T03:35:33.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.078+0000 7fc515a00640 1 -- 172.21.15.46:0/2434644338 wait complete. 2024-09-17T03:35:33.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 -- 172.21.15.46:0/4196861910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da80744d0 msgr2=0x7f8da806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 --2- 172.21.15.46:0/4196861910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da80744d0 0x7f8da806ea60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009840 tx=0x7f8d9001c7f0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 -- 172.21.15.46:0/4196861910 shutdown_connections 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 --2- 172.21.15.46:0/4196861910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da80744d0 0x7f8da806ea60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 -- 172.21.15.46:0/4196861910 >> 172.21.15.46:0/4196861910 conn(0x7f8da80b2c40 msgr2=0x7f8da80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 -- 172.21.15.46:0/4196861910 shutdown_connections 2024-09-17T03:35:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 -- 172.21.15.46:0/4196861910 wait complete. 2024-09-17T03:35:33.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 Processor -- start 2024-09-17T03:35:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.262+0000 7f8daf000640 1 -- start start 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da806efa0 0x7f8da80ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da815c5d0 0x7f8da814fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 0x7f8da8153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8169f60 con 0x7f8da8150390 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8169de0 con 0x7f8da815c5d0 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8da816a0e0 con 0x7f8da806efa0 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 0x7f8da8153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 0x7f8da8153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39400/0 (socket says 172.21.15.46:39400) 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 -- 172.21.15.46:0/815040293 learned_addr learned my addr 172.21.15.46:0/815040293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8da7e00640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da806efa0 0x7f8da80ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8d9f400640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da815c5d0 0x7f8da814fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 -- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da806efa0 msgr2=0x7f8da80ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da806efa0 0x7f8da80ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 -- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da815c5d0 msgr2=0x7f8da814fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da815c5d0 0x7f8da814fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 -- 172.21.15.46:0/815040293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da81fbce0 con 0x7f8da8150390 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8dad000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 0x7f8da8153310 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8da000c760 tx=0x7f8da000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0003b80 con 0x7f8da8150390 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8da0003d20 con 0x7f8da8150390 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da81fbf50 con 0x7f8da8150390 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da000aab0 con 0x7f8da8150390 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da8150060 con 0x7f8da8150390 2024-09-17T03:35:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.266+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da80ac870 con 0x7f8da8150390 2024-09-17T03:35:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.270+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f8da000ac50 con 0x7f8da8150390 2024-09-17T03:35:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.270+0000 7f8da4c00640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d84075210 0x7f8d84077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.270+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7f8da0092fc0 con 0x7f8da8150390 2024-09-17T03:35:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.270+0000 7f8da7e00640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d84075210 0x7f8d84077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.270+0000 7f8da7e00640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d84075210 0x7f8d84077720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8d980046c0 tx=0x7f8d98007cd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.274+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8da005f3f0 con 0x7f8da8150390 2024-09-17T03:35:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.450+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8da806ea60 con 0x7f8da8150390 2024-09-17T03:35:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.450+0000 7f8da4c00640 1 -- 172.21.15.46:0/815040293 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16915 (secure 0 0 0) 0x7f8da0063e20 con 0x7f8da8150390 2024-09-17T03:35:33.451 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:33.452 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:35:26.140430+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:34:16.113350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T03:34:17.334380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d84075210 msgr2=0x7f8d84077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d84075210 0x7f8d84077720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8d980046c0 tx=0x7f8d98007cd0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 msgr2=0x7f8da8153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 0x7f8da8153310 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8da000c760 tx=0x7f8da000cc20 comp rx=0 tx=0).stop 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 shutdown_connections 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d84075210 0x7f8d84077720 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8da8150390 0x7f8da8153310 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da815c5d0 0x7f8da814fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 --2- 172.21.15.46:0/815040293 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8da806efa0 0x7f8da80ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 >> 172.21.15.46:0/815040293 conn(0x7f8da80b2c40 msgr2=0x7f8da8151840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 shutdown_connections 2024-09-17T03:35:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.454+0000 7f8daf000640 1 -- 172.21.15.46:0/815040293 wait complete. 2024-09-17T03:35:33.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 -- 172.21.15.46:0/740534892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda241527b0 msgr2=0x7fda2415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 --2- 172.21.15.46:0/740534892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda241527b0 0x7fda2415bb60 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fda240cca90 tx=0x7fda0c01c620 comp rx=0 tx=0).stop 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 -- 172.21.15.46:0/740534892 shutdown_connections 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 --2- 172.21.15.46:0/740534892 >> v2:172.21.15.99:3300/0 conn(0x7fda240b4de0 0x7fda240b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 --2- 172.21.15.46:0/740534892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda241527b0 0x7fda2415bb60 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 -- 172.21.15.46:0/740534892 >> 172.21.15.46:0/740534892 conn(0x7fda240b2040 msgr2=0x7fda240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 -- 172.21.15.46:0/740534892 shutdown_connections 2024-09-17T03:35:33.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.634+0000 7fda29e00640 1 -- 172.21.15.46:0/740534892 wait complete. 2024-09-17T03:35:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 Processor -- start 2024-09-17T03:35:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 -- start start 2024-09-17T03:35:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 0x7fda241f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda240b5c70 0x7fda241fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda241527b0 0x7fda24150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda24168910 con 0x7fda240b4de0 2024-09-17T03:35:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda24168790 con 0x7fda240b5c70 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda29e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fda24168a90 con 0x7fda241527b0 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 0x7fda241f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 0x7fda241f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39410/0 (socket says 172.21.15.46:39410) 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda22a00640 1 -- 172.21.15.46:0/3493205420 learned_addr learned my addr 172.21.15.46:0/3493205420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda1a000640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda240b5c70 0x7fda241fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.638+0000 7fda23400640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda241527b0 0x7fda24150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda22a00640 1 -- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda241527b0 msgr2=0x7fda24150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda22a00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda241527b0 0x7fda24150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda22a00640 1 -- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda240b5c70 msgr2=0x7fda241fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda22a00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda240b5c70 0x7fda241fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda22a00640 1 -- 172.21.15.46:0/3493205420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda241fcb60 con 0x7fda240b4de0 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda23400640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda241527b0 0x7fda24150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda1a000640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda240b5c70 0x7fda241fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda22a00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 0x7fda241f9580 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b410 tx=0x7fda1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14015030 con 0x7fda240b4de0 2024-09-17T03:35:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda14004020 con 0x7fda240b4de0 2024-09-17T03:35:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda241fce40 con 0x7fda240b4de0 2024-09-17T03:35:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14004470 con 0x7fda240b4de0 2024-09-17T03:35:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda241f9060 con 0x7fda240b4de0 2024-09-17T03:35:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda240b51f0 con 0x7fda240b4de0 2024-09-17T03:35:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.642+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fda14018070 con 0x7fda240b4de0 2024-09-17T03:35:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.646+0000 7fda1be00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda00075210 0x7fda00077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.646+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7fda14092ea0 con 0x7fda240b4de0 2024-09-17T03:35:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.646+0000 7fda1a000640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda00075210 0x7fda00077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.646+0000 7fda1a000640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda00075210 0x7fda00077720 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0af000 tx=0x7fda0c01ae60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.650+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda1405f920 con 0x7fda240b4de0 2024-09-17T03:35:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.858+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fda241fd010 con 0x7fda240b4de0 2024-09-17T03:35:33.858 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:35:33.858+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:35:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.862+0000 7fda1be00640 1 -- 172.21.15.46:0/3493205420 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7fda1405f220 con 0x7fda240b4de0 2024-09-17T03:35:33.861 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda00075210 msgr2=0x7fda00077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda00075210 0x7fda00077720 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0af000 tx=0x7fda0c01ae60 comp rx=0 tx=0).stop 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 msgr2=0x7fda241f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 0x7fda241f9580 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b410 tx=0x7fda1400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 shutdown_connections 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fda00075210 0x7fda00077720 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fda241527b0 0x7fda24150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda240b5c70 0x7fda241fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 --2- 172.21.15.46:0/3493205420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda240b4de0 0x7fda241f9580 secure :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b410 tx=0x7fda1400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:35:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 >> 172.21.15.46:0/3493205420 conn(0x7fda240b2040 msgr2=0x7fda2415acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 shutdown_connections 2024-09-17T03:35:33.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:33.866+0000 7fda29e00640 1 -- 172.21.15.46:0/3493205420 wait complete. 2024-09-17T03:35:33.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:35:34.007 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:35:34.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 -- 172.21.15.46:0/93599661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 msgr2=0x7f5334074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:34.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 --2- 172.21.15.46:0/93599661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334074950 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f53340713e0 tx=0x7f532c01cd80 comp rx=0 tx=0).stop 2024-09-17T03:35:34.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 -- 172.21.15.46:0/93599661 shutdown_connections 2024-09-17T03:35:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 --2- 172.21.15.46:0/93599661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334074950 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 --2- 172.21.15.46:0/93599661 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5334160d50 0x7f5334074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 --2- 172.21.15.46:0/93599661 >> v2:172.21.15.99:3300/0 conn(0x7f5334157030 0x7f53340b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 -- 172.21.15.46:0/93599661 >> 172.21.15.46:0/93599661 conn(0x7f53340708c0 msgr2=0x7f5334070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 -- 172.21.15.46:0/93599661 shutdown_connections 2024-09-17T03:35:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.054+0000 7f533bc00640 1 -- 172.21.15.46:0/93599661 wait complete. 2024-09-17T03:35:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 Processor -- start 2024-09-17T03:35:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 -- start start 2024-09-17T03:35:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5334157030 0x7f53340b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5334160d50 0x7f53341f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53340722c0 con 0x7f5334161160 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5334072140 con 0x7f5334160d50 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5334072440 con 0x7f5334157030 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39452/0 (socket says 172.21.15.46:39452) 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 -- 172.21.15.46:0/40330649 learned_addr learned my addr 172.21.15.46:0/40330649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5333e00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5334160d50 0x7f53341f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339200640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5334157030 0x7f53340b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 -- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5334157030 msgr2=0x7f53340b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5334157030 0x7f53340b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 -- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5334160d50 msgr2=0x7f53341f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5334160d50 0x7f53341f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 -- 172.21.15.46:0/40330649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533414e840 con 0x7f5334161160 2024-09-17T03:35:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5333e00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5334160d50 0x7f53341f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5339c00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334154a40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f532c009bd0 tx=0x7f532c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0bb070 con 0x7f5334161160 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f532c004d40 con 0x7f5334161160 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533414eb20 con 0x7f5334161160 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.058+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0b2040 con 0x7f5334161160 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.062+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53340b3a90 con 0x7f5334161160 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.062+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53340728f0 con 0x7f5334161160 2024-09-17T03:35:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.062+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f532c0be070 con 0x7f5334161160 2024-09-17T03:35:34.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.062+0000 7f5330c00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5314075210 0x7f5314077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:34.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.062+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5881+0+0 (secure 0 0 0) 0x7f532c12cf70 con 0x7f5334161160 2024-09-17T03:35:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.062+0000 7f5339200640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5314075210 0x7f5314077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.066+0000 7f5339200640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5314075210 0x7f5314077720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f532800bad0 tx=0x7f532800b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.066+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f532c016970 con 0x7f5334161160 2024-09-17T03:35:34.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.286+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f53340d2260 con 0x7f5334161160 2024-09-17T03:35:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.862+0000 7f5330c00640 1 -- 172.21.15.46:0/40330649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7f532c0f9430 con 0x7f5334161160 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5314075210 msgr2=0x7f5314077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5314075210 0x7f5314077720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f532800bad0 tx=0x7f532800b260 comp rx=0 tx=0).stop 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 msgr2=0x7f5334154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334154a40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f532c009bd0 tx=0x7f532c005b00 comp rx=0 tx=0).stop 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 shutdown_connections 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5314075210 0x7f5314077720 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5334161160 0x7f5334154a40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5334160d50 0x7f53341f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 --2- 172.21.15.46:0/40330649 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5334157030 0x7f53340b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 >> 172.21.15.46:0/40330649 conn(0x7f53340708c0 msgr2=0x7f533406e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:34.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 shutdown_connections 2024-09-17T03:35:34.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:34.866+0000 7f533bc00640 1 -- 172.21.15.46:0/40330649 wait complete. 2024-09-17T03:35:34.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:35:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 -- 172.21.15.46:0/2960088211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b5c70 msgr2=0x7f245015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 --2- 172.21.15.46:0/2960088211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b5c70 0x7f245015bb60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2450070710 tx=0x7f244001c800 comp rx=0 tx=0).stop 2024-09-17T03:35:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 -- 172.21.15.46:0/2960088211 shutdown_connections 2024-09-17T03:35:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 --2- 172.21.15.46:0/2960088211 >> v2:172.21.15.99:3300/0 conn(0x7f24501527b0 0x7f2450074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 --2- 172.21.15.46:0/2960088211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b5c70 0x7f245015bb60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 -- 172.21.15.46:0/2960088211 >> 172.21.15.46:0/2960088211 conn(0x7f24500b2040 msgr2=0x7f24500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 -- 172.21.15.46:0/2960088211 shutdown_connections 2024-09-17T03:35:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 -- 172.21.15.46:0/2960088211 wait complete. 2024-09-17T03:35:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 Processor -- start 2024-09-17T03:35:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.058+0000 7f2456800640 1 -- start start 2024-09-17T03:35:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 0x7f245014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24500b5c70 0x7f24501fc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24501527b0 0x7f24501fca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2450168b20 con 0x7f24500b4de0 2024-09-17T03:35:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24501689a0 con 0x7f24501527b0 2024-09-17T03:35:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2450168ca0 con 0x7f24500b5c70 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 0x7f245014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 0x7f245014fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39472/0 (socket says 172.21.15.46:39472) 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 -- 172.21.15.46:0/2333704752 learned_addr learned my addr 172.21.15.46:0/2333704752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2446a00640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24500b5c70 0x7f24501fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244fe00640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24501527b0 0x7f24501fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 -- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24500b5c70 msgr2=0x7f24501fc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24500b5c70 0x7f24501fc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 -- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24501527b0 msgr2=0x7f24501fca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24501527b0 0x7f24501fca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 -- 172.21.15.46:0/2333704752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2450158180 con 0x7f24500b4de0 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244fe00640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24501527b0 0x7f24501fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f244f400640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 0x7f245014fb40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f24400b0040 tx=0x7f24400a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:35.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24400b2040 con 0x7f24500b4de0 2024-09-17T03:35:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24400044b0 con 0x7f24500b4de0 2024-09-17T03:35:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440005280 con 0x7f24500b4de0 2024-09-17T03:35:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24501f8c40 con 0x7f24500b4de0 2024-09-17T03:35:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24501f91f0 con 0x7f24500b4de0 2024-09-17T03:35:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.062+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245015bb60 con 0x7f24500b4de0 2024-09-17T03:35:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.066+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f2440005420 con 0x7f24500b4de0 2024-09-17T03:35:35.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.066+0000 7f2447e00640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f242c075240 0x7f242c077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:35.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.066+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5881+0+0 (secure 0 0 0) 0x7f244012d0c0 con 0x7f24500b4de0 2024-09-17T03:35:35.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.066+0000 7f2446a00640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f242c075240 0x7f242c077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:35.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.066+0000 7f2446a00640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f242c075240 0x7f242c077750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f24380058d0 tx=0x7f2438005860 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:35.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.070+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2450075580 con 0x7f24500b4de0 2024-09-17T03:35:35.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.246+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f24501660e0 con 0x7f24500b4de0 2024-09-17T03:35:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.866+0000 7f2447e00640 1 -- 172.21.15.46:0/2333704752 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f24400164b0 con 0x7f24500b4de0 2024-09-17T03:35:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:35:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.870+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f242c075240 msgr2=0x7f242c077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.870+0000 7f2456800640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f242c075240 0x7f242c077750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f24380058d0 tx=0x7f2438005860 comp rx=0 tx=0).stop 2024-09-17T03:35:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.870+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 msgr2=0x7f245014fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.870+0000 7f2456800640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 0x7f245014fb40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f24400b0040 tx=0x7f24400a5840 comp rx=0 tx=0).stop 2024-09-17T03:35:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 shutdown_connections 2024-09-17T03:35:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f242c075240 0x7f242c077750 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24501527b0 0x7f24501fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f24500b5c70 0x7f24501fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 --2- 172.21.15.46:0/2333704752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24500b4de0 0x7f245014fb40 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 >> 172.21.15.46:0/2333704752 conn(0x7f24500b2040 msgr2=0x7f2450151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 shutdown_connections 2024-09-17T03:35:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:35.874+0000 7f2456800640 1 -- 172.21.15.46:0/2333704752 wait complete. 2024-09-17T03:35:35.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:35:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 -- 172.21.15.46:0/814178384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 msgr2=0x7eff5415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 --2- 172.21.15.46:0/814178384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5415bb60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7eff54074b20 tx=0x7eff4401cae0 comp rx=0 tx=0).stop 2024-09-17T03:35:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 -- 172.21.15.46:0/814178384 shutdown_connections 2024-09-17T03:35:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 --2- 172.21.15.46:0/814178384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5415bb60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 -- 172.21.15.46:0/814178384 >> 172.21.15.46:0/814178384 conn(0x7eff540b2040 msgr2=0x7eff540b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 -- 172.21.15.46:0/814178384 shutdown_connections 2024-09-17T03:35:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.066+0000 7eff58e00640 1 -- 172.21.15.46:0/814178384 wait complete. 2024-09-17T03:35:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 Processor -- start 2024-09-17T03:35:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 -- start start 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff540b4de0 0x7eff5415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff541527b0 0x7eff54151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff54169b30 con 0x7eff540b5c70 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff541699b0 con 0x7eff541527b0 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7eff54169cb0 con 0x7eff540b4de0 2024-09-17T03:35:36.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5414f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39496/0 (socket says 172.21.15.46:39496) 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 -- 172.21.15.46:0/3458518696 learned_addr learned my addr 172.21.15.46:0/3458518696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff51e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff540b4de0 0x7eff5415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff52800640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff541527b0 0x7eff54151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 -- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff540b4de0 msgr2=0x7eff5415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff540b4de0 0x7eff5415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 -- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff541527b0 msgr2=0x7eff54151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff541527b0 0x7eff54151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 -- 172.21.15.46:0/3458518696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff540ae8e0 con 0x7eff540b5c70 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff49400640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5414f180 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c3c0 tx=0x7eff3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c017070 con 0x7eff540b5c70 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff3c003600 con 0x7eff540b5c70 2024-09-17T03:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c003a50 con 0x7eff540b5c70 2024-09-17T03:35:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.070+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5415deb0 con 0x7eff540b5c70 2024-09-17T03:35:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.074+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff540aee80 con 0x7eff540b5c70 2024-09-17T03:35:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.074+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff540af2c0 con 0x7eff540b5c70 2024-09-17T03:35:36.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.074+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7eff3c01a070 con 0x7eff540b5c70 2024-09-17T03:35:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.074+0000 7eff4aa00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30075210 0x7eff30077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.074+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5443+0+0 (secure 0 0 0) 0x7eff3c092500 con 0x7eff540b5c70 2024-09-17T03:35:36.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.074+0000 7eff51e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30075210 0x7eff30077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:36.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.078+0000 7eff51e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30075210 0x7eff30077720 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7eff440af000 tx=0x7eff4401b320 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:36.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.078+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7eff3c05f090 con 0x7eff540b5c70 2024-09-17T03:35:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.254+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7eff5415bb60 con 0x7eff540b5c70 2024-09-17T03:35:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.874+0000 7eff4aa00640 1 -- 172.21.15.46:0/3458518696 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7eff3c05e990 con 0x7eff540b5c70 2024-09-17T03:35:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30075210 msgr2=0x7eff30077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30075210 0x7eff30077720 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7eff440af000 tx=0x7eff4401b320 comp rx=0 tx=0).stop 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 msgr2=0x7eff5414f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5414f180 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c3c0 tx=0x7eff3c00c880 comp rx=0 tx=0).stop 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 shutdown_connections 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff30075210 0x7eff30077720 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7eff440af000 tx=0x7eff4401b320 comp rx=0 tx=0).stop 2024-09-17T03:35:36.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff541527b0 0x7eff54151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:36.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff540b5c70 0x7eff5414f180 secure :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c3c0 tx=0x7eff3c00c880 comp rx=0 tx=0).stop 2024-09-17T03:35:36.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 --2- 172.21.15.46:0/3458518696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7eff540b4de0 0x7eff5415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:36.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 >> 172.21.15.46:0/3458518696 conn(0x7eff540b2040 msgr2=0x7eff540b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:36.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 shutdown_connections 2024-09-17T03:35:36.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:36.878+0000 7eff58e00640 1 -- 172.21.15.46:0/3458518696 wait complete. 2024-09-17T03:35:36.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:35:37.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 -- 172.21.15.46:0/1566222820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12600b5c70 msgr2=0x7f126015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 --2- 172.21.15.46:0/1566222820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12600b5c70 0x7f126015bb60 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1260070710 tx=0x7f125001c800 comp rx=0 tx=0).stop 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 -- 172.21.15.46:0/1566222820 shutdown_connections 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 --2- 172.21.15.46:0/1566222820 >> v2:172.21.15.99:3300/0 conn(0x7f12601527b0 0x7f1260074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 --2- 172.21.15.46:0/1566222820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12600b5c70 0x7f126015bb60 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 -- 172.21.15.46:0/1566222820 >> 172.21.15.46:0/1566222820 conn(0x7f12600b2040 msgr2=0x7f12600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 -- 172.21.15.46:0/1566222820 shutdown_connections 2024-09-17T03:35:37.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 -- 172.21.15.46:0/1566222820 wait complete. 2024-09-17T03:35:37.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 Processor -- start 2024-09-17T03:35:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.078+0000 7f1266800640 1 -- start start 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12600b4de0 0x7f1260152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12600b5c70 0x7f1260140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 0x7f12601fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1260168b20 con 0x7f12601527b0 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12601689a0 con 0x7f12600b4de0 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1260168ca0 con 0x7f12600b5c70 2024-09-17T03:35:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 0x7f12601fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 0x7f12601fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39504/0 (socket says 172.21.15.46:39504) 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 -- 172.21.15.46:0/480295428 learned_addr learned my addr 172.21.15.46:0/480295428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125f400640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12600b4de0 0x7f1260152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1257e00640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12600b5c70 0x7f1260140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 -- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12600b5c70 msgr2=0x7f1260140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12600b5c70 0x7f1260140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 -- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12600b4de0 msgr2=0x7f1260152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12600b4de0 0x7f1260152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 -- 172.21.15.46:0/480295428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1260153500 con 0x7f12601527b0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f125fe00640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 0x7f12601fc160 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f125800be90 tx=0x7f125800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258019070 con 0x7f12601527b0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1258014440 con 0x7f12601527b0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12601537e0 con 0x7f12601527b0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12580148d0 con 0x7f12601527b0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1260153da0 con 0x7f12601527b0 2024-09-17T03:35:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.082+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126015bb60 con 0x7f12601527b0 2024-09-17T03:35:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.086+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f125801c070 con 0x7f12601527b0 2024-09-17T03:35:37.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.086+0000 7f1257400640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f123c075210 0x7f123c077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.086+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7f1258093110 con 0x7f12601527b0 2024-09-17T03:35:37.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.086+0000 7f125f400640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f123c075210 0x7f123c077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.086+0000 7f125f400640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f123c075210 0x7f123c077720 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f12500af000 tx=0x7f12500a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:37.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.090+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f125805f670 con 0x7f12601527b0 2024-09-17T03:35:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.266+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f1260165f10 con 0x7f12601527b0 2024-09-17T03:35:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1257400640 1 -- 172.21.15.46:0/480295428 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v54) ==== 71+0+0 (secure 0 0 0) 0x7f12580640a0 con 0x7f12601527b0 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f123c075210 msgr2=0x7f123c077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f123c075210 0x7f123c077720 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f12500af000 tx=0x7f12500a7040 comp rx=0 tx=0).stop 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 msgr2=0x7f12601fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 0x7f12601fc160 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f125800be90 tx=0x7f125800bf80 comp rx=0 tx=0).stop 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 shutdown_connections 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f123c075210 0x7f123c077720 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12601527b0 0x7f12601fc160 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f12600b5c70 0x7f1260140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 --2- 172.21.15.46:0/480295428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12600b4de0 0x7f1260152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 >> 172.21.15.46:0/480295428 conn(0x7f12600b2040 msgr2=0x7f1260151d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:37.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 shutdown_connections 2024-09-17T03:35:37.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.270+0000 7f1266800640 1 -- 172.21.15.46:0/480295428 wait complete. 2024-09-17T03:35:37.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-17T03:35:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 -- 172.21.15.46:0/2771039038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 msgr2=0x7f74cc0b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 --2- 172.21.15.46:0/2771039038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc0b0730 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f74cc072250 tx=0x7f74bc01c770 comp rx=0 tx=0).stop 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 -- 172.21.15.46:0/2771039038 shutdown_connections 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 --2- 172.21.15.46:0/2771039038 >> v2:172.21.15.99:3300/0 conn(0x7f74cc06dbc0 0x7f74cc06dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 --2- 172.21.15.46:0/2771039038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc0b0730 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 -- 172.21.15.46:0/2771039038 >> 172.21.15.46:0/2771039038 conn(0x7f74cc0b41f0 msgr2=0x7f74cc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 -- 172.21.15.46:0/2771039038 shutdown_connections 2024-09-17T03:35:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.458+0000 7f74d2200640 1 -- 172.21.15.46:0/2771039038 wait complete. 2024-09-17T03:35:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 Processor -- start 2024-09-17T03:35:37.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 -- start start 2024-09-17T03:35:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74cc06dbc0 0x7f74cc0af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc099830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74cc15f5c0 0x7f74cc1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0d0f30 con 0x7f74cc0d1140 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0d0db0 con 0x7f74cc15f5c0 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74d2200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f74cc168c60 con 0x7f74cc06dbc0 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.462+0000 7f74c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc099830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc099830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39530/0 (socket says 172.21.15.46:39530) 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 -- 172.21.15.46:0/3365003166 learned_addr learned my addr 172.21.15.46:0/3365003166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74d0e00640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74cc06dbc0 0x7f74cc0af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74d1800640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74cc15f5c0 0x7f74cc1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 -- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74cc06dbc0 msgr2=0x7f74cc0af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74cc06dbc0 0x7f74cc0af6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 -- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74cc15f5c0 msgr2=0x7f74cc1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74cc15f5c0 0x7f74cc1570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 -- 172.21.15.46:0/3365003166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc1654f0 con 0x7f74cc0d1140 2024-09-17T03:35:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c3e00640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc099830 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b470 tx=0x7f74b400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4015020 con 0x7f74cc0d1140 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74b4011980 con 0x7f74cc0d1140 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc0cfff0 con 0x7f74cc0d1140 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4011dd0 con 0x7f74cc0d1140 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc0d02b0 con 0x7f74cc0d1140 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74cc06dfd0 con 0x7f74cc0d1140 2024-09-17T03:35:37.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.466+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f74b4018020 con 0x7f74cc0d1140 2024-09-17T03:35:37.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.470+0000 7f74c9600640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74a8075240 0x7f74a8077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:37.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.470+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7f74b40934c0 con 0x7f74cc0d1140 2024-09-17T03:35:37.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.470+0000 7f74d0e00640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74a8075240 0x7f74a8077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:37.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.470+0000 7f74d0e00640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74a8075240 0x7f74a8077750 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0af000 tx=0x7f74bc01afb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.474+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74b401f020 con 0x7f74cc0d1140 2024-09-17T03:35:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.714+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f74cc0cb0d0 con 0x7f74cc0d1140 2024-09-17T03:35:37.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.878+0000 7f74c9600640 1 -- 172.21.15.46:0/3365003166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v672) ==== 185+0+0 (secure 0 0 0) 0x7f74b400d040 con 0x7f74cc0d1140 2024-09-17T03:35:37.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.878+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74a8075240 msgr2=0x7f74a8077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.878+0000 7f74d2200640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74a8075240 0x7f74a8077750 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0af000 tx=0x7f74bc01afb0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.878+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 msgr2=0x7f74cc099830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.878+0000 7f74d2200640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc099830 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b470 tx=0x7f74b400b930 comp rx=0 tx=0).stop 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 shutdown_connections 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74a8075240 0x7f74a8077750 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74cc15f5c0 0x7f74cc1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc0d1140 0x7f74cc099830 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 --2- 172.21.15.46:0/3365003166 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74cc06dbc0 0x7f74cc0af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 >> 172.21.15.46:0/3365003166 conn(0x7f74cc0b41f0 msgr2=0x7f74cc15b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 shutdown_connections 2024-09-17T03:35:37.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:37.882+0000 7f74d2200640 1 -- 172.21.15.46:0/3365003166 wait complete. 2024-09-17T03:35:37.900 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-17T03:35:37.900 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-17T03:35:37.900 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:35:37.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 -- 172.21.15.46:0/1305236560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 msgr2=0x7fc92c15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 --2- 172.21.15.46:0/1305236560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c15ecd0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc92000a950 tx=0x7fc92001db70 comp rx=0 tx=0).stop 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 -- 172.21.15.46:0/1305236560 shutdown_connections 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 --2- 172.21.15.46:0/1305236560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c15ecd0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 -- 172.21.15.46:0/1305236560 >> 172.21.15.46:0/1305236560 conn(0x7fc92c0b2040 msgr2=0x7fc92c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 -- 172.21.15.46:0/1305236560 shutdown_connections 2024-09-17T03:35:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 -- 172.21.15.46:0/1305236560 wait complete. 2024-09-17T03:35:38.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 Processor -- start 2024-09-17T03:35:38.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.062+0000 7fc933e00640 1 -- start start 2024-09-17T03:35:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c15b2d0 0x7fc92c143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c15e290 0x7fc92c13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c169210 con 0x7fc92c15e8c0 2024-09-17T03:35:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c169090 con 0x7fc92c15e290 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc92c169390 con 0x7fc92c15b2d0 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39544/0 (socket says 172.21.15.46:39544) 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 -- 172.21.15.46:0/317566179 learned_addr learned my addr 172.21.15.46:0/317566179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931400640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c15b2d0 0x7fc92c143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc930a00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c15e290 0x7fc92c13dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 -- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c15b2d0 msgr2=0x7fc92c143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c15b2d0 0x7fc92c143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 -- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c15e290 msgr2=0x7fc92c13dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c15e290 0x7fc92c13dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 -- 172.21.15.46:0/317566179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c143c20 con 0x7fc92c15e8c0 2024-09-17T03:35:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931400640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c15b2d0 0x7fc92c143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc930a00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c15e290 0x7fc92c13dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc931e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c0b3cd0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c760 tx=0x7fc92400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924003b80 con 0x7fc92c15e8c0 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc924003d20 con 0x7fc92c15e8c0 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92400aab0 con 0x7fc92c15e8c0 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c140da0 con 0x7fc92c15e8c0 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c142900 con 0x7fc92c15e8c0 2024-09-17T03:35:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.066+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92c143990 con 0x7fc92c15e8c0 2024-09-17T03:35:38.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.070+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fc92400ac50 con 0x7fc92c15e8c0 2024-09-17T03:35:38.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.070+0000 7fc929600640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc90c075240 0x7fc90c077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.070+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7fc924093100 con 0x7fc92c15e8c0 2024-09-17T03:35:38.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.070+0000 7fc931400640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc90c075240 0x7fc90c077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.070+0000 7fc931400640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc90c075240 0x7fc90c077750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc9200b0000 tx=0x7fc92001c3b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.074+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc92c15f210 con 0x7fc92c15e8c0 2024-09-17T03:35:38.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.246+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc92c142b60 con 0x7fc92c15e8c0 2024-09-17T03:35:38.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.246+0000 7fc929600640 1 -- 172.21.15.46:0/317566179 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13050 (secure 0 0 0) 0x7fc92405fb00 con 0x7fc92c15e8c0 2024-09-17T03:35:38.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:38.247 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:35:36.868259+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6837/1168163211":"2024-09-18T03:35:33.861235+0000","172.21.15.46:6836/1168163211":"2024-09-18T03:35:33.861235+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:35:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc90c075240 msgr2=0x7fc90c077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc90c075240 0x7fc90c077750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc9200b0000 tx=0x7fc92001c3b0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 msgr2=0x7fc92c0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c0b3cd0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c760 tx=0x7fc92400cc20 comp rx=0 tx=0).stop 2024-09-17T03:35:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 shutdown_connections 2024-09-17T03:35:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc90c075240 0x7fc90c077750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc92c15e8c0 0x7fc92c0b3cd0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc92c15e290 0x7fc92c13dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 --2- 172.21.15.46:0/317566179 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc92c15b2d0 0x7fc92c143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 >> 172.21.15.46:0/317566179 conn(0x7fc92c0b2040 msgr2=0x7fc92c073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 shutdown_connections 2024-09-17T03:35:38.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.250+0000 7fc933e00640 1 -- 172.21.15.46:0/317566179 wait complete. 2024-09-17T03:35:38.270 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:35:38.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:35:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 -- 172.21.15.46:0/832470546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c1536f0 msgr2=0x7fbb2c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 --2- 172.21.15.46:0/832470546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c1536f0 0x7fbb2c15d480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009870 tx=0x7fbb2001c7d0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 -- 172.21.15.46:0/832470546 shutdown_connections 2024-09-17T03:35:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 --2- 172.21.15.46:0/832470546 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c1536f0 0x7fbb2c15d480 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 -- 172.21.15.46:0/832470546 >> 172.21.15.46:0/832470546 conn(0x7fbb2c0b4560 msgr2=0x7fbb2c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 -- 172.21.15.46:0/832470546 shutdown_connections 2024-09-17T03:35:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 -- 172.21.15.46:0/832470546 wait complete. 2024-09-17T03:35:38.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 Processor -- start 2024-09-17T03:35:38.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.434+0000 7fbb32800640 1 -- start start 2024-09-17T03:35:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 0x7fbb2c148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c160060 0x7fbb2c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb2c14a630 0x7fbb2c14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c16b570 con 0x7fbb2c1536f0 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c16b3f0 con 0x7fbb2c14a630 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c16b6f0 con 0x7fbb2c160060 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 0x7fbb2c148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 0x7fbb2c148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39546/0 (socket says 172.21.15.46:39546) 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 -- 172.21.15.46:0/1228687080 learned_addr learned my addr 172.21.15.46:0/1228687080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb30a00640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c160060 0x7fbb2c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31e00640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb2c14a630 0x7fbb2c14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 -- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c160060 msgr2=0x7fbb2c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c160060 0x7fbb2c155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 -- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb2c14a630 msgr2=0x7fbb2c14aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb2c14a630 0x7fbb2c14aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 -- 172.21.15.46:0/1228687080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c14af80 con 0x7fbb2c1536f0 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31e00640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb2c14a630 0x7fbb2c14aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb31400640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 0x7fbb2c148a30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbb2001ccb0 tx=0x7fbb200a6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb200be070 con 0x7fbb2c1536f0 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb20005250 con 0x7fbb2c1536f0 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c0d2610 con 0x7fbb2c1536f0 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb200046d0 con 0x7fbb2c1536f0 2024-09-17T03:35:38.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c0d0600 con 0x7fbb2c1536f0 2024-09-17T03:35:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.438+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c15d480 con 0x7fbb2c1536f0 2024-09-17T03:35:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.442+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fbb20004870 con 0x7fbb2c1536f0 2024-09-17T03:35:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.442+0000 7fbb29600640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb0c075210 0x7fbb0c077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.442+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7fbb2012d260 con 0x7fbb2c1536f0 2024-09-17T03:35:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.442+0000 7fbb30a00640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb0c075210 0x7fbb0c077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.442+0000 7fbb30a00640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb0c075210 0x7fbb0c077720 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0ce680 tx=0x7fbb18007b10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:38.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.446+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb2c14af80 con 0x7fbb2c1536f0 2024-09-17T03:35:38.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.618+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb2c072330 con 0x7fbb2c1536f0 2024-09-17T03:35:38.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb29600640 1 -- 172.21.15.46:0/1228687080 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7fbb200ac030 con 0x7fbb2c1536f0 2024-09-17T03:35:38.619 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:35:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb0c075210 msgr2=0x7fbb0c077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb0c075210 0x7fbb0c077720 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0ce680 tx=0x7fbb18007b10 comp rx=0 tx=0).stop 2024-09-17T03:35:38.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 msgr2=0x7fbb2c148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 0x7fbb2c148a30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbb2001ccb0 tx=0x7fbb200a6690 comp rx=0 tx=0).stop 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 shutdown_connections 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbb0c075210 0x7fbb0c077720 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb2c14a630 0x7fbb2c14aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbb2c160060 0x7fbb2c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 --2- 172.21.15.46:0/1228687080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb2c1536f0 0x7fbb2c148a30 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 >> 172.21.15.46:0/1228687080 conn(0x7fbb2c0b4560 msgr2=0x7fbb2c0b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 shutdown_connections 2024-09-17T03:35:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.622+0000 7fbb32800640 1 -- 172.21.15.46:0/1228687080 wait complete. 2024-09-17T03:35:38.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 -- 172.21.15.46:0/2890243368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b5c70 msgr2=0x7f45b015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 --2- 172.21.15.46:0/2890243368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b5c70 0x7f45b015bb60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f45b0070710 tx=0x7f45a001c800 comp rx=0 tx=0).stop 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 -- 172.21.15.46:0/2890243368 shutdown_connections 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 --2- 172.21.15.46:0/2890243368 >> v2:172.21.15.99:3300/0 conn(0x7f45b01527b0 0x7f45b0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 --2- 172.21.15.46:0/2890243368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b5c70 0x7f45b015bb60 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 -- 172.21.15.46:0/2890243368 >> 172.21.15.46:0/2890243368 conn(0x7f45b00b2040 msgr2=0x7f45b00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 -- 172.21.15.46:0/2890243368 shutdown_connections 2024-09-17T03:35:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 -- 172.21.15.46:0/2890243368 wait complete. 2024-09-17T03:35:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 Processor -- start 2024-09-17T03:35:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.802+0000 7f45b6600640 1 -- start start 2024-09-17T03:35:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 0x7f45b0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b00b5c70 0x7f45b013e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45b01527b0 0x7f45b013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0168b20 con 0x7f45b00b4de0 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01689a0 con 0x7f45b00b5c70 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f45b0168ca0 con 0x7f45b01527b0 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 0x7f45b0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 0x7f45b0159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39568/0 (socket says 172.21.15.46:39568) 2024-09-17T03:35:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 -- 172.21.15.46:0/1693263681 learned_addr learned my addr 172.21.15.46:0/1693263681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 -- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45b01527b0 msgr2=0x7f45b013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45afe00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45b01527b0 0x7f45b013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b00b5c70 0x7f45b013e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45b01527b0 0x7f45b013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 -- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b00b5c70 msgr2=0x7f45b013e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b00b5c70 0x7f45b013e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 -- 172.21.15.46:0/1693263681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b0152d50 con 0x7f45b00b4de0 2024-09-17T03:35:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45afe00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45b01527b0 0x7f45b013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b00b5c70 0x7f45b013e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45af400640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 0x7f45b0159160 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f45a001aeb0 tx=0x7f45a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a00b2040 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45a0003600 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b0152ff0 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0003a30 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01535b0 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.806+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b015bb60 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.810+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f45a00bf060 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.810+0000 7f45a7e00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f458c075210 0x7f458c077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.810+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7f45a012cf20 con 0x7f45b00b4de0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.810+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f458c075210 0x7f458c077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.810+0000 7f45a6a00640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f458c075210 0x7f458c077720 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45b01547a0 tx=0x7f4598005d20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:38.814+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f45b0075580 con 0x7f45b00b4de0 2024-09-17T03:35:39.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.050+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f45b00aeb30 con 0x7f45b00b4de0 2024-09-17T03:35:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.886+0000 7f45a7e00640 1 -- 172.21.15.46:0/1693263681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v674) ==== 189+0+0 (secure 0 0 0) 0x7f45a00164b0 con 0x7f45b00b4de0 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f458c075210 msgr2=0x7f458c077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f458c075210 0x7f458c077720 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f45b01547a0 tx=0x7f4598005d20 comp rx=0 tx=0).stop 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 msgr2=0x7f45b0159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 0x7f45b0159160 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f45a001aeb0 tx=0x7f45a0004040 comp rx=0 tx=0).stop 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 shutdown_connections 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f458c075210 0x7f458c077720 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:39.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f45b01527b0 0x7f45b013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45b00b5c70 0x7f45b013e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 --2- 172.21.15.46:0/1693263681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45b00b4de0 0x7f45b0159160 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 >> 172.21.15.46:0/1693263681 conn(0x7f45b00b2040 msgr2=0x7f45b0151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 shutdown_connections 2024-09-17T03:35:39.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:39.890+0000 7f45b6600640 1 -- 172.21.15.46:0/1693263681 wait complete. 2024-09-17T03:35:39.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:35:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 -- 172.21.15.46:0/294787275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff00cefa0 msgr2=0x7fbff00cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 --2- 172.21.15.46:0/294787275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff00cefa0 0x7fbff00cf3b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fbff00b3040 tx=0x7fbfd801c8c0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 -- 172.21.15.46:0/294787275 shutdown_connections 2024-09-17T03:35:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 --2- 172.21.15.46:0/294787275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff00cefa0 0x7fbff00cf3b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 --2- 172.21.15.46:0/294787275 >> v2:172.21.15.99:3300/0 conn(0x7fbff00749a0 0x7fbff0074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 -- 172.21.15.46:0/294787275 >> 172.21.15.46:0/294787275 conn(0x7fbff00b2040 msgr2=0x7fbff00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 -- 172.21.15.46:0/294787275 shutdown_connections 2024-09-17T03:35:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 -- 172.21.15.46:0/294787275 wait complete. 2024-09-17T03:35:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.077+0000 7fbff5400640 1 Processor -- start 2024-09-17T03:35:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 -- start start 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 0x7fbff01fca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbff00cefa0 0x7fbff006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff0160330 0x7fbff013c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbff016a060 con 0x7fbff0160330 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0169ee0 con 0x7fbff00749a0 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbff016a1e0 con 0x7fbff00cefa0 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 0x7fbff01fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfef200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff0160330 0x7fbff013c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 0x7fbff01fca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:57036/0 (socket says 172.21.15.46:57036) 2024-09-17T03:35:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 -- 172.21.15.46:0/2324654656 learned_addr learned my addr 172.21.15.46:0/2324654656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfe7e00640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbff00cefa0 0x7fbff006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 -- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbff00cefa0 msgr2=0x7fbff006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbff00cefa0 0x7fbff006e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 -- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff0160330 msgr2=0x7fbff013c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff0160330 0x7fbff013c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 -- 172.21.15.46:0/2324654656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff014d770 con 0x7fbff00749a0 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfef200640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff0160330 0x7fbff013c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfe7e00640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbff00cefa0 0x7fbff006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:35:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfee800640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 0x7fbff01fca60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00058d0 tx=0x7fbfe000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0017070 con 0x7fbff00749a0 2024-09-17T03:35:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff014cbf0 con 0x7fbff00749a0 2024-09-17T03:35:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff014d1b0 con 0x7fbff00749a0 2024-09-17T03:35:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfe0009c30 con 0x7fbff00749a0 2024-09-17T03:35:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.081+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0021060 con 0x7fbff00749a0 2024-09-17T03:35:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.085+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff0074db0 con 0x7fbff00749a0 2024-09-17T03:35:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.085+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fbfe001a070 con 0x7fbff00749a0 2024-09-17T03:35:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.085+0000 7fbfe6a00640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbfcc075210 0x7fbfcc077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.085+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7fbfe00936f0 con 0x7fbff00749a0 2024-09-17T03:35:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.085+0000 7fbfe7e00640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbfcc075210 0x7fbfcc077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.085+0000 7fbfe7e00640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbfcc075210 0x7fbfcc077720 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80af000 tx=0x7fbfd801b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:40.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.089+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbfe005fc30 con 0x7fbff00749a0 2024-09-17T03:35:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.265+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbff00b3fc0 con 0x7fbff00749a0 2024-09-17T03:35:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.265+0000 7fbfe6a00640 1 -- 172.21.15.46:0/2324654656 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v54) ==== 59+0+2294 (secure 0 0 0) 0x7fbfe0064660 con 0x7fbff00749a0 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:35:40.267 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:35:40.268 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:35:40.269 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbfcc075210 msgr2=0x7fbfcc077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbfcc075210 0x7fbfcc077720 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80af000 tx=0x7fbfd801b100 comp rx=0 tx=0).stop 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 msgr2=0x7fbff01fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 0x7fbff01fca60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00058d0 tx=0x7fbfe000dde0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 shutdown_connections 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbfcc075210 0x7fbfcc077720 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff0160330 0x7fbff013c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbff00cefa0 0x7fbff006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 --2- 172.21.15.46:0/2324654656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbff00749a0 0x7fbff01fca60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 >> 172.21.15.46:0/2324654656 conn(0x7fbff00b2040 msgr2=0x7fbff0075fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 shutdown_connections 2024-09-17T03:35:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.269+0000 7fbff5400640 1 -- 172.21.15.46:0/2324654656 wait complete. 2024-09-17T03:35:40.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:35:40.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 -- 172.21.15.46:0/1018053570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78381536f0 msgr2=0x7f783815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 --2- 172.21.15.46:0/1018053570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78381536f0 0x7f783815d480 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f7828009840 tx=0x7f782801c790 comp rx=0 tx=0).stop 2024-09-17T03:35:40.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 -- 172.21.15.46:0/1018053570 shutdown_connections 2024-09-17T03:35:40.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 --2- 172.21.15.46:0/1018053570 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7838160060 0x7f7838074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 --2- 172.21.15.46:0/1018053570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78381536f0 0x7f783815d480 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 -- 172.21.15.46:0/1018053570 >> 172.21.15.46:0/1018053570 conn(0x7f78380b4560 msgr2=0x7f78380d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 -- 172.21.15.46:0/1018053570 shutdown_connections 2024-09-17T03:35:40.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.453+0000 7f7837200640 1 -- 172.21.15.46:0/1018053570 wait complete. 2024-09-17T03:35:40.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 Processor -- start 2024-09-17T03:35:40.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 -- start start 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78381536f0 0x7f783806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 0x7f783814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7838160470 0x7f78381432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f783816a440 con 0x7f7838160060 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f783816a2c0 con 0x7f78381536f0 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f783816a5c0 con 0x7f7838160470 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 0x7f783814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 0x7f783814e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39630/0 (socket says 172.21.15.46:39630) 2024-09-17T03:35:40.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 -- 172.21.15.46:0/511148108 learned_addr learned my addr 172.21.15.46:0/511148108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 -- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7838160470 msgr2=0x7f78381432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7838160470 0x7f78381432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 -- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78381536f0 msgr2=0x7f783806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7835e00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78381536f0 0x7f783806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78381536f0 0x7f783806dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 -- 172.21.15.46:0/511148108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78380d05b0 con 0x7f7838160060 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7835e00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78381536f0 0x7f783806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782fe00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 0x7f783814e630 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f782000e7f0 tx=0x7f782000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782000ccc0 con 0x7f7838160060 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f782000ce80 con 0x7f7838160060 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820005280 con 0x7f7838160060 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78380d0890 con 0x7f7838160060 2024-09-17T03:35:40.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.457+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78380d0e50 con 0x7f7838160060 2024-09-17T03:35:40.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.461+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783815d480 con 0x7f7838160060 2024-09-17T03:35:40.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.461+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f7820005420 con 0x7f7838160060 2024-09-17T03:35:40.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.461+0000 7f782e000640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7814075240 0x7f7814077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:40.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.461+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7f7820093cd0 con 0x7f7838160060 2024-09-17T03:35:40.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.461+0000 7f7835e00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7814075240 0x7f7814077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:40.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.461+0000 7f7835e00640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7814075240 0x7f7814077750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f78280ae000 tx=0x7f782801afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:40.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.465+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f782009e020 con 0x7f7838160060 2024-09-17T03:35:40.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.641+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f783815dc40 con 0x7f7838160060 2024-09-17T03:35:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.645+0000 7f782e000640 1 -- 172.21.15.46:0/511148108 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v55)=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7f7820060750 con 0x7f7838160060 2024-09-17T03:35:40.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7814075240 msgr2=0x7f7814077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7814075240 0x7f7814077750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f78280ae000 tx=0x7f782801afd0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 msgr2=0x7f783814e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:40.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 0x7f783814e630 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f782000e7f0 tx=0x7f782000ecb0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 shutdown_connections 2024-09-17T03:35:40.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7814075240 0x7f7814077750 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7838160470 0x7f78381432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7838160060 0x7f783814e630 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 --2- 172.21.15.46:0/511148108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78381536f0 0x7f783806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 >> 172.21.15.46:0/511148108 conn(0x7f78380b4560 msgr2=0x7f78380715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 shutdown_connections 2024-09-17T03:35:40.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:40.649+0000 7f7837200640 1 -- 172.21.15.46:0/511148108 wait complete. 2024-09-17T03:35:40.668 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:35:40.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:35:40.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:35:40.857 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:35:40.860 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:35:40.860 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:40.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:35:40.871 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:35:40.871 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:40.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:35:40.881 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:35:40.881 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:40.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:35:40.919 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:35:40.920 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:40.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:35:40.935 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:35:40.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:41.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 -- 172.21.15.46:0/307163908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 msgr2=0x7ff804003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 --2- 172.21.15.46:0/307163908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff804003ed0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff81000c310 tx=0x7ff81001cf20 comp rx=0 tx=0).stop 2024-09-17T03:35:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 -- 172.21.15.46:0/307163908 shutdown_connections 2024-09-17T03:35:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 --2- 172.21.15.46:0/307163908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff804003ed0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 -- 172.21.15.46:0/307163908 >> 172.21.15.46:0/307163908 conn(0x7ff81c0b2040 msgr2=0x7ff81c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 -- 172.21.15.46:0/307163908 shutdown_connections 2024-09-17T03:35:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.121+0000 7ff823c00640 1 -- 172.21.15.46:0/307163908 wait complete. 2024-09-17T03:35:41.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 Processor -- start 2024-09-17T03:35:41.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 -- start start 2024-09-17T03:35:41.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff81c1fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff81c0cbe60 0x7ff81c15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c0cc270 0x7ff81c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1604b0 con 0x7ff804003a60 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c160330 con 0x7ff81c0cc270 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff823c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c160630 con 0x7ff81c0cbe60 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff81c1fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff81c1fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39644/0 (socket says 172.21.15.46:39644) 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 -- 172.21.15.46:0/1869551266 learned_addr learned my addr 172.21.15.46:0/1869551266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff81be00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff81c0cbe60 0x7ff81c15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c0cc270 0x7ff81c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 -- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff81c0cbe60 msgr2=0x7ff81c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff81c0cbe60 0x7ff81c15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 -- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c0cc270 msgr2=0x7ff81c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c0cc270 0x7ff81c15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 -- 172.21.15.46:0/1869551266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1fc860 con 0x7ff804003a60 2024-09-17T03:35:41.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.125+0000 7ff821200640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff81c1fc320 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff81001b620 tx=0x7ff8100aea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810006c80 con 0x7ff804003a60 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8100aec70 con 0x7ff804003a60 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810001d00 con 0x7ff804003a60 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1f8680 con 0x7ff804003a60 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1f8c40 con 0x7ff804003a60 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c160b90 con 0x7ff804003a60 2024-09-17T03:35:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7ff8100bf060 con 0x7ff804003a60 2024-09-17T03:35:41.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff818c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff7fc075210 0x7ff7fc077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7ff81012d6c0 con 0x7ff804003a60 2024-09-17T03:35:41.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.129+0000 7ff81be00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff7fc075210 0x7ff7fc077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.133+0000 7ff81be00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff7fc075210 0x7ff7fc077720 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff808000c10 tx=0x7ff808006d20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:41.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.137+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff810016cd0 con 0x7ff804003a60 2024-09-17T03:35:41.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff81c15c780 con 0x7ff804003a60 2024-09-17T03:35:41.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff818c00640 1 -- 172.21.15.46:0/1869551266 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3919 (secure 0 0 0) 0x7ff8100fa290 con 0x7ff804003a60 2024-09-17T03:35:41.345 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:41.346 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"btime":"2024-09-17T03:35:34:863598+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/3123362033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3123362033},{"type":"v1","addr":"172.21.15.99:6834","nonce":3123362033}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2618729235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2618729235},{"type":"v1","addr":"172.21.15.46:6835","nonce":2618729235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":248193,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/2939936689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2939936689},{"type":"v1","addr":"172.21.15.99:6835","nonce":2939936689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":248628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2324757369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2324757369},{"type":"v1","addr":"172.21.15.46:6837","nonce":2324757369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[]} 2024-09-17T03:35:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 78 2024-09-17T03:35:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff7fc075210 msgr2=0x7ff7fc077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff7fc075210 0x7ff7fc077720 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff808000c10 tx=0x7ff808006d20 comp rx=0 tx=0).stop 2024-09-17T03:35:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 msgr2=0x7ff81c1fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff81c1fc320 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff81001b620 tx=0x7ff8100aea80 comp rx=0 tx=0).stop 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 shutdown_connections 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff7fc075210 0x7ff7fc077720 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff81c0cc270 0x7ff81c15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff81c0cbe60 0x7ff81c15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 --2- 172.21.15.46:0/1869551266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff804003a60 0x7ff81c1fc320 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.345+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 >> 172.21.15.46:0/1869551266 conn(0x7ff81c0b2040 msgr2=0x7ff81c15a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.349+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 shutdown_connections 2024-09-17T03:35:41.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.349+0000 7ff823c00640 1 -- 172.21.15.46:0/1869551266 wait complete. 2024-09-17T03:35:41.369 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:35:41.369 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:35:41.369 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:35:41.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:35:41.370 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:35:41.370 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:35:41.370 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:35:41.370 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:35:41.370 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:35:41.371 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:35:41.371 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:35:41.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:35:41.371 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:35:41.371 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:35:41.371 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:35:41.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:35:41.386 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:35:41.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:35:41.387 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:35:41.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:35:41.388 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:35:41.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:35:41.394 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:35:41.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:35:41.433 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:35:41.433 DEBUG:teuthology.parallel:result is None 2024-09-17T03:35:41.434 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:35:41.435 DEBUG:teuthology.parallel:result is None 2024-09-17T03:35:41.437 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:35:41.437 DEBUG:teuthology.parallel:result is None 2024-09-17T03:35:41.439 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:35:41.439 DEBUG:teuthology.parallel:result is None 2024-09-17T03:35:41.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:35:41.534 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:35:41.537 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:35:41.540 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:35:41.541 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:35:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 -- 172.21.15.46:0/2584185315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce080744d0 msgr2=0x7fce0806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2584185315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce080744d0 0x7fce0806ea60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009900 tx=0x7fcdf401c770 comp rx=0 tx=0).stop 2024-09-17T03:35:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 -- 172.21.15.46:0/2584185315 shutdown_connections 2024-09-17T03:35:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2584185315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce080744d0 0x7fce0806ea60 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 -- 172.21.15.46:0/2584185315 >> 172.21.15.46:0/2584185315 conn(0x7fce080b2c40 msgr2=0x7fce080b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 -- 172.21.15.46:0/2584185315 shutdown_connections 2024-09-17T03:35:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.605+0000 7fce0fe00640 1 -- 172.21.15.46:0/2584185315 wait complete. 2024-09-17T03:35:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 Processor -- start 2024-09-17T03:35:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 -- start start 2024-09-17T03:35:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce0806efa0 0x7fce080cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 0x7fce0815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0815b2d0 0x7fce08145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce0806fac0 con 0x7fce0815c5d0 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce080b3d50 con 0x7fce0815b2d0 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fce0806fc40 con 0x7fce0806efa0 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 0x7fce0815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 0x7fce0815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39690/0 (socket says 172.21.15.46:39690) 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 -- 172.21.15.46:0/2664146503 learned_addr learned my addr 172.21.15.46:0/2664146503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0de00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0815b2d0 0x7fce08145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0d400640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce0806efa0 0x7fce080cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 -- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce0806efa0 msgr2=0x7fce080cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce0806efa0 0x7fce080cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 -- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0815b2d0 msgr2=0x7fce08145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0815b2d0 0x7fce08145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 -- 172.21.15.46:0/2664146503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce08152860 con 0x7fce0815c5d0 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0ca00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 0x7fce0815ad90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009850 tx=0x7fcdf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40b2040 con 0x7fce0815c5d0 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdf40044e0 con 0x7fce0815c5d0 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4003860 con 0x7fce0815c5d0 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce081f9270 con 0x7fce0815c5d0 2024-09-17T03:35:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.609+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081f9830 con 0x7fce0815c5d0 2024-09-17T03:35:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.613+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0806ea60 con 0x7fce0815c5d0 2024-09-17T03:35:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.613+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fcdf40bf020 con 0x7fce0815c5d0 2024-09-17T03:35:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.613+0000 7fce05600640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcde8075210 0x7fcde8077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.613+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4971+0+0 (secure 0 0 0) 0x7fcdf412d6c0 con 0x7fce0815c5d0 2024-09-17T03:35:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.613+0000 7fce0d400640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcde8075210 0x7fcde8077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.613+0000 7fce0d400640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcde8075210 0x7fcde8077720 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc007fd0 tx=0x7fcdfc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:41.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.617+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcdf40c6020 con 0x7fce0815c5d0 2024-09-17T03:35:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.809+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fce080b12a0 con 0x7fce0815c5d0 2024-09-17T03:35:41.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.905+0000 7fce05600640 1 -- 172.21.15.46:0/2664146503 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) ==== 97+0+0 (secure 0 0 0) 0x7fcdf40163f0 con 0x7fce0815c5d0 2024-09-17T03:35:41.907 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:35:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcde8075210 msgr2=0x7fcde8077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcde8075210 0x7fcde8077720 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc007fd0 tx=0x7fcdfc009040 comp rx=0 tx=0).stop 2024-09-17T03:35:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 msgr2=0x7fce0815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 0x7fce0815ad90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009850 tx=0x7fcdf40042f0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 shutdown_connections 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcde8075210 0x7fcde8077720 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0815b2d0 0x7fce08145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0815c5d0 0x7fce0815ad90 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2664146503 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fce0806efa0 0x7fce080cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 >> 172.21.15.46:0/2664146503 conn(0x7fce080b2c40 msgr2=0x7fce080b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 shutdown_connections 2024-09-17T03:35:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:41.909+0000 7fce0fe00640 1 -- 172.21.15.46:0/2664146503 wait complete. 2024-09-17T03:35:41.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:35:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 -- 172.21.15.46:0/3845034237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 msgr2=0x7f8a280b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 --2- 172.21.15.46:0/3845034237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a280b51f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009bc0 tx=0x7f8a2001cb70 comp rx=0 tx=0).stop 2024-09-17T03:35:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 -- 172.21.15.46:0/3845034237 shutdown_connections 2024-09-17T03:35:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 --2- 172.21.15.46:0/3845034237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a280b51f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 --2- 172.21.15.46:0/3845034237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a280b5c70 0x7f8a2815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 -- 172.21.15.46:0/3845034237 >> 172.21.15.46:0/3845034237 conn(0x7f8a280b2040 msgr2=0x7f8a280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 -- 172.21.15.46:0/3845034237 shutdown_connections 2024-09-17T03:35:42.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.093+0000 7f8a2f400640 1 -- 172.21.15.46:0/3845034237 wait complete. 2024-09-17T03:35:42.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 Processor -- start 2024-09-17T03:35:42.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 -- start start 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a281fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 0x7f8a280cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a281527b0 0x7f8a281f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28168970 con 0x7f8a280b5c70 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a281687f0 con 0x7f8a280b4de0 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8a28168af0 con 0x7f8a281527b0 2024-09-17T03:35:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 0x7f8a280cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 0x7f8a280cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39702/0 (socket says 172.21.15.46:39702) 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a281fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 -- 172.21.15.46:0/151674597 learned_addr learned my addr 172.21.15.46:0/151674597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2d400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a281527b0 0x7f8a281f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 -- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a281527b0 msgr2=0x7f8a281f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a281527b0 0x7f8a281f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 -- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 msgr2=0x7f8a281fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a281fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 -- 172.21.15.46:0/151674597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a280aefe0 con 0x7f8a280b5c70 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2ca00640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a281fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a1fe00640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 0x7f8a280cef90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000c760 tx=0x7f8a1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10006aa0 con 0x7f8a280b5c70 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a10006c40 con 0x7f8a280b5c70 2024-09-17T03:35:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10007290 con 0x7f8a280b5c70 2024-09-17T03:35:42.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a280af2c0 con 0x7f8a280b5c70 2024-09-17T03:35:42.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.097+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281fc440 con 0x7f8a280b5c70 2024-09-17T03:35:42.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.101+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a280b51f0 con 0x7f8a280b5c70 2024-09-17T03:35:42.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.101+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f8a10007430 con 0x7f8a280b5c70 2024-09-17T03:35:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.101+0000 7f8a25600640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8a04075240 0x7f8a04077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.101+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4885+0+0 (secure 0 0 0) 0x7f8a10092c60 con 0x7f8a280b5c70 2024-09-17T03:35:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.101+0000 7f8a2ca00640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8a04075240 0x7f8a04077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.105+0000 7f8a2ca00640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8a04075240 0x7f8a04077750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009f30 tx=0x7f8a180091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:42.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.105+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8a280b5730 con 0x7f8a280b5c70 2024-09-17T03:35:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8a2814c7b0 con 0x7f8a280b5c70 2024-09-17T03:35:42.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a25600640 1 -- 172.21.15.46:0/151674597 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v42) ==== 80+0+5304 (secure 0 0 0) 0x7f8a1005f0a0 con 0x7f8a280b5c70 2024-09-17T03:35:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:35:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:35:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.340 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.341 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:35:42.343 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:35:42.345 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:35:42.347 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:35:42.348 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:35:42.349 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:35:42.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.354 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:35:42.354 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8a04075240 msgr2=0x7f8a04077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8a04075240 0x7f8a04077750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009f30 tx=0x7f8a180091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 msgr2=0x7f8a280cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 0x7f8a280cef90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000c760 tx=0x7f8a1000cc20 comp rx=0 tx=0).stop 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 shutdown_connections 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8a04075240 0x7f8a04077750 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8a281527b0 0x7f8a281f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280b5c70 0x7f8a280cef90 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 --2- 172.21.15.46:0/151674597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a280b4de0 0x7f8a281fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 >> 172.21.15.46:0/151674597 conn(0x7f8a280b2040 msgr2=0x7f8a28154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 shutdown_connections 2024-09-17T03:35:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.341+0000 7f8a2f400640 1 -- 172.21.15.46:0/151674597 wait complete. 2024-09-17T03:35:42.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:35:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 -- 172.21.15.46:0/511357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 msgr2=0x7f5a70075040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 --2- 172.21.15.46:0/511357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a70075040 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5a70075760 tx=0x7f5a5c01c740 comp rx=0 tx=0).stop 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 -- 172.21.15.46:0/511357888 shutdown_connections 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 --2- 172.21.15.46:0/511357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a70075040 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 --2- 172.21.15.46:0/511357888 >> v2:172.21.15.99:3300/0 conn(0x7f5a700746e0 0x7f5a7015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 -- 172.21.15.46:0/511357888 >> 172.21.15.46:0/511357888 conn(0x7f5a700b41f0 msgr2=0x7f5a700b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 -- 172.21.15.46:0/511357888 shutdown_connections 2024-09-17T03:35:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.521+0000 7f5a75600640 1 -- 172.21.15.46:0/511357888 wait complete. 2024-09-17T03:35:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 Processor -- start 2024-09-17T03:35:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 -- start start 2024-09-17T03:35:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a700746e0 0x7f5a701f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a7015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a700d1190 0x7f5a7015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7016a100 con 0x7f5a70074c30 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70169f80 con 0x7f5a700746e0 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a75600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7016a280 con 0x7f5a700d1190 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a7015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a7015def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33394/0 (socket says 172.21.15.46:33394) 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 -- 172.21.15.46:0/446678912 learned_addr learned my addr 172.21.15.46:0/446678912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a74c00640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a700d1190 0x7f5a7015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6fe00640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a700746e0 0x7f5a701f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 -- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a700d1190 msgr2=0x7f5a7015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a700d1190 0x7f5a7015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 -- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a700746e0 msgr2=0x7f5a701f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a700746e0 0x7f5a701f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 -- 172.21.15.46:0/446678912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7015ce90 con 0x7f5a70074c30 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a6f400640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a7015def0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c01cc20 tx=0x7f5a5c0a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.525+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0b2040 con 0x7f5a70074c30 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a5c0bb070 con 0x7f5a70074c30 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003600 con 0x7f5a70074c30 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a701f7e70 con 0x7f5a70074c30 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7015d410 con 0x7f5a70074c30 2024-09-17T03:35:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a700b4f30 con 0x7f5a70074c30 2024-09-17T03:35:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f5a5c0037a0 con 0x7f5a70074c30 2024-09-17T03:35:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a67e00640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5a50075240 0x7f5a50077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4885+0+0 (secure 0 0 0) 0x7f5a5c12e420 con 0x7f5a70074c30 2024-09-17T03:35:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.529+0000 7f5a6fe00640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5a50075240 0x7f5a50077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.533+0000 7f5a6fe00640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5a50075240 0x7f5a50077750 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5a60005fd0 tx=0x7f5a60005d10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:42.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.533+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a70075580 con 0x7f5a70074c30 2024-09-17T03:35:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5a700cbd60 con 0x7f5a70074c30 2024-09-17T03:35:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a67e00640 1 -- 172.21.15.46:0/446678912 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) ==== 84+0+65 (secure 0 0 0) 0x7f5a5c0163f0 con 0x7f5a70074c30 2024-09-17T03:35:42.712 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:42.712 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:35:42.712 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:42.712 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:35:42.712 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:35:42.713 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:42.713 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:35:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5a50075240 msgr2=0x7f5a50077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5a50075240 0x7f5a50077750 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5a60005fd0 tx=0x7f5a60005d10 comp rx=0 tx=0).stop 2024-09-17T03:35:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 msgr2=0x7f5a7015def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a7015def0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c01cc20 tx=0x7f5a5c0a5680 comp rx=0 tx=0).stop 2024-09-17T03:35:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 shutdown_connections 2024-09-17T03:35:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5a50075240 0x7f5a50077750 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5a700d1190 0x7f5a7015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a70074c30 0x7f5a7015def0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 --2- 172.21.15.46:0/446678912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a700746e0 0x7f5a701f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 >> 172.21.15.46:0/446678912 conn(0x7f5a700b41f0 msgr2=0x7f5a700af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 shutdown_connections 2024-09-17T03:35:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.713+0000 7f5a75600640 1 -- 172.21.15.46:0/446678912 wait complete. 2024-09-17T03:35:42.733 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:35:42.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:35:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 -- 172.21.15.46:0/1174076282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 msgr2=0x7f5244074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 --2- 172.21.15.46:0/1174076282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244074950 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f52440713e0 tx=0x7f524001cd80 comp rx=0 tx=0).stop 2024-09-17T03:35:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 -- 172.21.15.46:0/1174076282 shutdown_connections 2024-09-17T03:35:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 --2- 172.21.15.46:0/1174076282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244074950 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 --2- 172.21.15.46:0/1174076282 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5244160d50 0x7f5244074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 --2- 172.21.15.46:0/1174076282 >> v2:172.21.15.99:3300/0 conn(0x7f5244157030 0x7f52440b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 -- 172.21.15.46:0/1174076282 >> 172.21.15.46:0/1174076282 conn(0x7f52440708c0 msgr2=0x7f5244070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 -- 172.21.15.46:0/1174076282 shutdown_connections 2024-09-17T03:35:42.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.897+0000 7f524c800640 1 -- 172.21.15.46:0/1174076282 wait complete. 2024-09-17T03:35:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 Processor -- start 2024-09-17T03:35:42.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 -- start start 2024-09-17T03:35:42.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5244157030 0x7f52440b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5244160d50 0x7f52441f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52440722c0 con 0x7f5244161160 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5244072140 con 0x7f5244160d50 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524c800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5244072440 con 0x7f5244157030 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f5249e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5244157030 0x7f52440b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f5249400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5244160d50 0x7f52441f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f5249400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5244160d50 0x7f52441f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:38332/0 (socket says 172.21.15.46:38332) 2024-09-17T03:35:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f5249400640 1 -- 172.21.15.46:0/1736293709 learned_addr learned my addr 172.21.15.46:0/1736293709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 -- 172.21.15.46:0/1736293709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5244157030 msgr2=0x7f52440b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5244157030 0x7f52440b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 -- 172.21.15.46:0/1736293709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5244160d50 msgr2=0x7f52441f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5244160d50 0x7f52441f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 -- 172.21.15.46:0/1736293709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524414e840 con 0x7f5244161160 2024-09-17T03:35:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.901+0000 7f524a800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244154a40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f52400ac000 tx=0x7f5240009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.905+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52400b2040 con 0x7f5244161160 2024-09-17T03:35:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.905+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524414eb20 con 0x7f5244161160 2024-09-17T03:35:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.905+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52440b3a90 con 0x7f5244161160 2024-09-17T03:35:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.905+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5240004c50 con 0x7f5244161160 2024-09-17T03:35:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.905+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240004200 con 0x7f5244161160 2024-09-17T03:35:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.909+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52200052a0 con 0x7f5244161160 2024-09-17T03:35:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.909+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f52400be060 con 0x7f5244161160 2024-09-17T03:35:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.909+0000 7f523e000640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5228075210 0x7f5228077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.909+0000 7f5249e00640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5228075210 0x7f5228077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:42.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.913+0000 7f5249e00640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5228075210 0x7f5228077720 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5238004610 tx=0x7f523800b260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:42.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.913+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4885+0+0 (secure 0 0 0) 0x7f52400f5550 con 0x7f5244161160 2024-09-17T03:35:42.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:42.917+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5240016970 con 0x7f5244161160 2024-09-17T03:35:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.093+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5220005590 con 0x7f5244161160 2024-09-17T03:35:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.917+0000 7f523e000640 1 -- 172.21.15.46:0/1736293709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) ==== 138+0+0 (secure 0 0 0) 0x7f52400f99e0 con 0x7f5244161160 2024-09-17T03:35:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5228075210 msgr2=0x7f5228077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5228075210 0x7f5228077720 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5238004610 tx=0x7f523800b260 comp rx=0 tx=0).stop 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 msgr2=0x7f5244154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244154a40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f52400ac000 tx=0x7f5240009350 comp rx=0 tx=0).stop 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 shutdown_connections 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5228075210 0x7f5228077720 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5244161160 0x7f5244154a40 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5244160d50 0x7f52441f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 --2- 172.21.15.46:0/1736293709 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5244157030 0x7f52440b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 >> 172.21.15.46:0/1736293709 conn(0x7f52440708c0 msgr2=0x7f524406e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 shutdown_connections 2024-09-17T03:35:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:43.921+0000 7f524c800640 1 -- 172.21.15.46:0/1736293709 wait complete. 2024-09-17T03:35:43.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 -- 172.21.15.46:0/1618684339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 msgr2=0x7f93d4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 --2- 172.21.15.46:0/1618684339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d4074e30 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f93d4071cb0 tx=0x7f93bc01c800 comp rx=0 tx=0).stop 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 -- 172.21.15.46:0/1618684339 shutdown_connections 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 --2- 172.21.15.46:0/1618684339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d4074e30 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 --2- 172.21.15.46:0/1618684339 >> v2:172.21.15.99:3300/0 conn(0x7f93d40b5c70 0x7f93d415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 -- 172.21.15.46:0/1618684339 >> 172.21.15.46:0/1618684339 conn(0x7f93d40b2040 msgr2=0x7f93d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 -- 172.21.15.46:0/1618684339 shutdown_connections 2024-09-17T03:35:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.121+0000 7f93da200640 1 -- 172.21.15.46:0/1618684339 wait complete. 2024-09-17T03:35:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 Processor -- start 2024-09-17T03:35:44.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 -- start start 2024-09-17T03:35:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d40b4de0 0x7f93d40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d40b5c70 0x7f93d40ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4168b20 con 0x7f93d41527b0 2024-09-17T03:35:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41689a0 con 0x7f93d40b5c70 2024-09-17T03:35:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f93d4168ca0 con 0x7f93d40b4de0 2024-09-17T03:35:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33438/0 (socket says 172.21.15.46:33438) 2024-09-17T03:35:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 -- 172.21.15.46:0/3147864710 learned_addr learned my addr 172.21.15.46:0/3147864710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3400640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d40b4de0 0x7f93d40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93cbe00640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d40b5c70 0x7f93d40ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:44.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 -- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d40b4de0 msgr2=0x7f93d40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:44.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d40b4de0 0x7f93d40ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 -- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d40b5c70 msgr2=0x7f93d40ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d40b5c70 0x7f93d40ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 -- 172.21.15.46:0/3147864710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d40cd710 con 0x7f93d41527b0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93d3e00640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d413e370 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c6b0 tx=0x7f93cc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc003b80 con 0x7f93d41527b0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93cc003d20 con 0x7f93d41527b0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d40cd9f0 con 0x7f93d41527b0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc005330 con 0x7f93d41527b0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.125+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d40cdfb0 con 0x7f93d41527b0 2024-09-17T03:35:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.129+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d40cd480 con 0x7f93d41527b0 2024-09-17T03:35:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.129+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f93cc0054d0 con 0x7f93d41527b0 2024-09-17T03:35:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.133+0000 7f93cb400640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b0075240 0x7f93b0077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.133+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5255+0+0 (secure 0 0 0) 0x7f93cc093880 con 0x7f93d41527b0 2024-09-17T03:35:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.133+0000 7f93d3400640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b0075240 0x7f93b0077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.133+0000 7f93d3400640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b0075240 0x7f93b0077750 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009b40 tx=0x7f93c40091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.137+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93cc05fec0 con 0x7f93d41527b0 2024-09-17T03:35:44.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.309+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f93d415bb60 con 0x7f93d41527b0 2024-09-17T03:35:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.925+0000 7f93cb400640 1 -- 172.21.15.46:0/3147864710 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) ==== 170+0+0 (secure 0 0 0) 0x7f93cc0648f0 con 0x7f93d41527b0 2024-09-17T03:35:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:35:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.929+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b0075240 msgr2=0x7f93b0077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.929+0000 7f93da200640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b0075240 0x7f93b0077750 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009b40 tx=0x7f93c40091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.929+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 msgr2=0x7f93d413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.929+0000 7f93da200640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d413e370 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c6b0 tx=0x7f93cc00cb70 comp rx=0 tx=0).stop 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 shutdown_connections 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93b0075240 0x7f93b0077750 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93d41527b0 0x7f93d413e370 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d40b5c70 0x7f93d40ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 --2- 172.21.15.46:0/3147864710 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93d40b4de0 0x7f93d40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 >> 172.21.15.46:0/3147864710 conn(0x7f93d40b2040 msgr2=0x7f93d4151d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 shutdown_connections 2024-09-17T03:35:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:44.933+0000 7f93da200640 1 -- 172.21.15.46:0/3147864710 wait complete. 2024-09-17T03:35:44.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:35:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 -- 172.21.15.46:0/1644586475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c073a90 msgr2=0x7f9f3c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 --2- 172.21.15.46:0/1644586475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c073a90 0x7f9f3c073ea0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009c80 tx=0x7f9f3401cda0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 -- 172.21.15.46:0/1644586475 shutdown_connections 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 --2- 172.21.15.46:0/1644586475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c073a90 0x7f9f3c073ea0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 --2- 172.21.15.46:0/1644586475 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f3c072100 0x7f9f3c155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 -- 172.21.15.46:0/1644586475 >> 172.21.15.46:0/1644586475 conn(0x7f9f3c0b2040 msgr2=0x7f9f3c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 -- 172.21.15.46:0/1644586475 shutdown_connections 2024-09-17T03:35:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.141+0000 7f9f41800640 1 -- 172.21.15.46:0/1644586475 wait complete. 2024-09-17T03:35:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 Processor -- start 2024-09-17T03:35:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 -- start start 2024-09-17T03:35:45.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 0x7f9f3c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:45.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f3c073a90 0x7f9f3c1fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0b4de0 0x7f9f3c1fc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c168f30 con 0x7f9f3c072100 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c168db0 con 0x7f9f3c0b4de0 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c1690b0 con 0x7f9f3c073a90 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 0x7f9f3c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 0x7f9f3c152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33450/0 (socket says 172.21.15.46:33450) 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 -- 172.21.15.46:0/2261628836 learned_addr learned my addr 172.21.15.46:0/2261628836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3b400640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0b4de0 0x7f9f3c1fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f3c073a90 0x7f9f3c1fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 -- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f3c073a90 msgr2=0x7f9f3c1fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f3c073a90 0x7f9f3c1fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 -- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0b4de0 msgr2=0x7f9f3c1fc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0b4de0 0x7f9f3c1fc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 -- 172.21.15.46:0/2261628836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c163790 con 0x7f9f3c072100 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f3aa00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 0x7f9f3c152cf0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e7f0 tx=0x7f9f2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00ccc0 con 0x7f9f3c072100 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f2c00ce80 con 0x7f9f3c072100 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c005280 con 0x7f9f3c072100 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c163a70 con 0x7f9f3c072100 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.145+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c0aad30 con 0x7f9f3c072100 2024-09-17T03:35:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.149+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c0aaf90 con 0x7f9f3c072100 2024-09-17T03:35:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.149+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f9f2c005420 con 0x7f9f3c072100 2024-09-17T03:35:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.149+0000 7f9f32a00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f18075240 0x7f9f18077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.149+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5637+0+0 (secure 0 0 0) 0x7f9f2c093fd0 con 0x7f9f3c072100 2024-09-17T03:35:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.149+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f18075240 0x7f9f18077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.153+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f18075240 0x7f9f18077750 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0b2760 tx=0x7f9f24009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.153+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9f2c060490 con 0x7f9f3c072100 2024-09-17T03:35:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.365+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9f3c155b50 con 0x7f9f3c072100 2024-09-17T03:35:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.937+0000 7f9f32a00640 1 -- 172.21.15.46:0/2261628836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v81) ==== 174+0+0 (secure 0 0 0) 0x7f9f2c069020 con 0x7f9f3c072100 2024-09-17T03:35:45.941 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 18 and data pool 19 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f18075240 msgr2=0x7f9f18077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f18075240 0x7f9f18077750 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0b2760 tx=0x7f9f24009040 comp rx=0 tx=0).stop 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 msgr2=0x7f9f3c152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 0x7f9f3c152cf0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e7f0 tx=0x7f9f2c00ecb0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 shutdown_connections 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9f18075240 0x7f9f18077750 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0b2760 tx=0x7f9f24009040 comp rx=0 tx=0).stop 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0b4de0 0x7f9f3c1fc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9f3c073a90 0x7f9f3c1fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 --2- 172.21.15.46:0/2261628836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3c072100 0x7f9f3c152cf0 secure :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e7f0 tx=0x7f9f2c00ecb0 comp rx=0 tx=0).stop 2024-09-17T03:35:45.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.941+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 >> 172.21.15.46:0/2261628836 conn(0x7f9f3c0b2040 msgr2=0x7f9f3c15dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.949+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 shutdown_connections 2024-09-17T03:35:45.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:45.949+0000 7f9f41800640 1 -- 172.21.15.46:0/2261628836 wait complete. 2024-09-17T03:35:45.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 -- 172.21.15.46:0/2053400721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b415f5c0 msgr2=0x7fc1b415f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 --2- 172.21.15.46:0/2053400721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b415f5c0 0x7fc1b415f9d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009bc0 tx=0x7fc1ac01cb10 comp rx=0 tx=0).stop 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 -- 172.21.15.46:0/2053400721 shutdown_connections 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 --2- 172.21.15.46:0/2053400721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b415f5c0 0x7fc1b415f9d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 -- 172.21.15.46:0/2053400721 >> 172.21.15.46:0/2053400721 conn(0x7fc1b40b41f0 msgr2=0x7fc1b40b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 -- 172.21.15.46:0/2053400721 shutdown_connections 2024-09-17T03:35:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 -- 172.21.15.46:0/2053400721 wait complete. 2024-09-17T03:35:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 Processor -- start 2024-09-17T03:35:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.149+0000 7fc1ba000640 1 -- start start 2024-09-17T03:35:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 0x7fc1b41f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b40d1140 0x7fc1b40998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1b415f5c0 0x7fc1b415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b416a160 con 0x7fc1b406dbc0 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4169fe0 con 0x7fc1b40d1140 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b416a2e0 con 0x7fc1b415f5c0 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 0x7fc1b41f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 0x7fc1b41f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33456/0 (socket says 172.21.15.46:33456) 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 -- 172.21.15.46:0/257488802 learned_addr learned my addr 172.21.15.46:0/257488802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b3e00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b40d1140 0x7fc1b40998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b9600640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1b415f5c0 0x7fc1b415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 -- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1b415f5c0 msgr2=0x7fc1b415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1b415f5c0 0x7fc1b415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 -- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b40d1140 msgr2=0x7fc1b40998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b40d1140 0x7fc1b40998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 -- 172.21.15.46:0/257488802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b4099e00 con 0x7fc1b406dbc0 2024-09-17T03:35:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b8c00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 0x7fc1b41f7620 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800c3f0 tx=0x7fc1a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8017070 con 0x7fc1b406dbc0 2024-09-17T03:35:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1a80053c0 con 0x7fc1b406dbc0 2024-09-17T03:35:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b409abb0 con 0x7fc1b406dbc0 2024-09-17T03:35:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8003860 con 0x7fc1b406dbc0 2024-09-17T03:35:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b409a5f0 con 0x7fc1b406dbc0 2024-09-17T03:35:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.153+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b415f9d0 con 0x7fc1b406dbc0 2024-09-17T03:35:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.157+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fc1a801a070 con 0x7fc1b406dbc0 2024-09-17T03:35:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.157+0000 7fc1b0c00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc194075210 0x7fc194077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.157+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5737+0+0 (secure 0 0 0) 0x7fc1a8092580 con 0x7fc1b406dbc0 2024-09-17T03:35:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.157+0000 7fc1b3e00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc194075210 0x7fc194077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.157+0000 7fc1b3e00640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc194075210 0x7fc194077720 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc1b41fb1a0 tx=0x7fc1a0007ea0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:46.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.161+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc1a805ea40 con 0x7fc1b406dbc0 2024-09-17T03:35:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1b40cf9e0 con 0x7fc1b406dbc0 2024-09-17T03:35:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1b0c00640 1 -- 172.21.15.46:0/257488802 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16760 (secure 0 0 0) 0x7fc1a8063470 con 0x7fc1b406dbc0 2024-09-17T03:35:46.337 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:46.337 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":128,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:35:45.367395+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:35:43.106524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T03:35:44.323553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc194075210 msgr2=0x7fc194077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc194075210 0x7fc194077720 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc1b41fb1a0 tx=0x7fc1a0007ea0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 msgr2=0x7fc1b41f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 0x7fc1b41f7620 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800c3f0 tx=0x7fc1a800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 shutdown_connections 2024-09-17T03:35:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc194075210 0x7fc194077720 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1b415f5c0 0x7fc1b415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b40d1140 0x7fc1b40998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 --2- 172.21.15.46:0/257488802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1b406dbc0 0x7fc1b41f7620 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 >> 172.21.15.46:0/257488802 conn(0x7fc1b40b41f0 msgr2=0x7fc1b406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 shutdown_connections 2024-09-17T03:35:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.337+0000 7fc1ba000640 1 -- 172.21.15.46:0/257488802 wait complete. 2024-09-17T03:35:46.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 -- 172.21.15.46:0/109967256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 msgr2=0x7fe8880b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 --2- 172.21.15.46:0/109967256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8880b51f0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fe880009c80 tx=0x7fe88001cb00 comp rx=0 tx=0).stop 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 -- 172.21.15.46:0/109967256 shutdown_connections 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 --2- 172.21.15.46:0/109967256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b4de0 0x7fe8880b51f0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 --2- 172.21.15.46:0/109967256 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8880b5c70 0x7fe88815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 -- 172.21.15.46:0/109967256 >> 172.21.15.46:0/109967256 conn(0x7fe8880b2040 msgr2=0x7fe8880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:46.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 -- 172.21.15.46:0/109967256 shutdown_connections 2024-09-17T03:35:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 -- 172.21.15.46:0/109967256 wait complete. 2024-09-17T03:35:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 Processor -- start 2024-09-17T03:35:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.521+0000 7fe88ce00640 1 -- start start 2024-09-17T03:35:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8880b4de0 0x7fe8881fc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe888157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8881527b0 0x7fe8880aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe888168a00 con 0x7fe8880b5c70 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe888168880 con 0x7fe8881527b0 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe888168b80 con 0x7fe8880b4de0 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe888157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe888157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33470/0 (socket says 172.21.15.46:33470) 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 -- 172.21.15.46:0/256371583 learned_addr learned my addr 172.21.15.46:0/256371583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe885e00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8880b4de0 0x7fe8881fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe886800640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8881527b0 0x7fe8880aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 -- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8880b4de0 msgr2=0x7fe8881fc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8880b4de0 0x7fe8881fc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 -- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8881527b0 msgr2=0x7fe8880aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8881527b0 0x7fe8880aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 -- 172.21.15.46:0/256371583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe888157f40 con 0x7fe8880b5c70 2024-09-17T03:35:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87fe00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe888157a00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe87000b3f0 tx=0x7fe87000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870015020 con 0x7fe8880b5c70 2024-09-17T03:35:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe870011920 con 0x7fe8880b5c70 2024-09-17T03:35:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870011d50 con 0x7fe8880b5c70 2024-09-17T03:35:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8881f8780 con 0x7fe8880b5c70 2024-09-17T03:35:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881f8d40 con 0x7fe8880b5c70 2024-09-17T03:35:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.525+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8880b51f0 con 0x7fe8880b5c70 2024-09-17T03:35:46.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.529+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fe870018020 con 0x7fe8880b5c70 2024-09-17T03:35:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.529+0000 7fe87e000640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe864075210 0x7fe864077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.529+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5737+0+0 (secure 0 0 0) 0x7fe8700931e0 con 0x7fe8880b5c70 2024-09-17T03:35:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.529+0000 7fe885e00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe864075210 0x7fe864077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.529+0000 7fe885e00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe864075210 0x7fe864077720 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe87800ab40 tx=0x7fe87800a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.533+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe87001f030 con 0x7fe8880b5c70 2024-09-17T03:35:46.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.705+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8881f9140 con 0x7fe8880b5c70 2024-09-17T03:35:46.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe87e000640 1 -- 172.21.15.46:0/256371583 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16760 (secure 0 0 0) 0x7fe87005f720 con 0x7fe8880b5c70 2024-09-17T03:35:46.708 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:46.709 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":128,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:35:45.367395+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:35:43.106524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T03:35:44.323553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe864075210 msgr2=0x7fe864077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe864075210 0x7fe864077720 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe87800ab40 tx=0x7fe87800a250 comp rx=0 tx=0).stop 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 msgr2=0x7fe888157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe888157a00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe87000b3f0 tx=0x7fe87000b8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 shutdown_connections 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe864075210 0x7fe864077720 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8881527b0 0x7fe8880aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8880b5c70 0x7fe888157a00 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 --2- 172.21.15.46:0/256371583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe8880b4de0 0x7fe8881fc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 >> 172.21.15.46:0/256371583 conn(0x7fe8880b2040 msgr2=0x7fe88815b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:46.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 shutdown_connections 2024-09-17T03:35:46.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.709+0000 7fe88ce00640 1 -- 172.21.15.46:0/256371583 wait complete. 2024-09-17T03:35:46.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:35:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 -- 172.21.15.46:0/1046783432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f40b4de0 msgr2=0x7fc1f40b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1046783432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f40b4de0 0x7fc1f40b51f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4071cb0 tx=0x7fc1f001ccc0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 -- 172.21.15.46:0/1046783432 shutdown_connections 2024-09-17T03:35:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1046783432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f40b4de0 0x7fc1f40b51f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1046783432 >> v2:172.21.15.99:3300/0 conn(0x7fc1f40b5c70 0x7fc1f415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 -- 172.21.15.46:0/1046783432 >> 172.21.15.46:0/1046783432 conn(0x7fc1f40b2040 msgr2=0x7fc1f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 -- 172.21.15.46:0/1046783432 shutdown_connections 2024-09-17T03:35:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.893+0000 7fc1fc800640 1 -- 172.21.15.46:0/1046783432 wait complete. 2024-09-17T03:35:46.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 Processor -- start 2024-09-17T03:35:46.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 -- start start 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1f40b4de0 0x7fc1f41498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1f40b5c70 0x7fc1f40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 0x7fc1f40aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41688b0 con 0x7fc1f41527b0 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f4168730 con 0x7fc1f40b4de0 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fc800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f4168a30 con 0x7fc1f40b5c70 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 0x7fc1f40aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 0x7fc1f40aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33480/0 (socket says 172.21.15.46:33480) 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 -- 172.21.15.46:0/1870781944 learned_addr learned my addr 172.21.15.46:0/1870781944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:46.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1f9e00640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1f40b4de0 0x7fc1f41498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1f9400640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1f40b5c70 0x7fc1f40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 -- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1f40b5c70 msgr2=0x7fc1f40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1f40b5c70 0x7fc1f40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 -- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1f40b4de0 msgr2=0x7fc1f41498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1f40b4de0 0x7fc1f41498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 -- 172.21.15.46:0/1870781944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f40764c0 con 0x7fc1f41527b0 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1f9400640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1f40b5c70 0x7fc1f40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:46.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1fa800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 0x7fc1f40aee20 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009bd0 tx=0x7fc1f0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:46.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.897+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00bc070 con 0x7fc1f41527b0 2024-09-17T03:35:46.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f00ad8e0 con 0x7fc1f41527b0 2024-09-17T03:35:46.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00b2040 con 0x7fc1f41527b0 2024-09-17T03:35:46.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f4153240 con 0x7fc1f41527b0 2024-09-17T03:35:46.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f41f89f0 con 0x7fc1f41527b0 2024-09-17T03:35:46.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f40b51f0 con 0x7fc1f41527b0 2024-09-17T03:35:46.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fc1f00bf070 con 0x7fc1f41527b0 2024-09-17T03:35:46.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.901+0000 7fc1ee000640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1d8075240 0x7fc1d8077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:46.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.905+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5737+0+0 (secure 0 0 0) 0x7fc1f012d600 con 0x7fc1f41527b0 2024-09-17T03:35:46.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.905+0000 7fc1f9e00640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1d8075240 0x7fc1d8077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:46.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.905+0000 7fc1f9e00640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1d8075240 0x7fc1d8077750 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005fd0 tx=0x7fc1e8005e40 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:46.905+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc1f00c6080 con 0x7fc1f41527b0 2024-09-17T03:35:47.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.121+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc1f415c0f0 con 0x7fc1f41527b0 2024-09-17T03:35:47.913 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-09-17T03:35:47.913+0000 7f5772000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:35:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.953+0000 7fc1ee000640 1 -- 172.21.15.46:0/1870781944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7fc1f0016970 con 0x7fc1f41527b0 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1d8075240 msgr2=0x7fc1d8077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1d8075240 0x7fc1d8077750 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005fd0 tx=0x7fc1e8005e40 comp rx=0 tx=0).stop 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 msgr2=0x7fc1f40aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 0x7fc1f40aee20 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009bd0 tx=0x7fc1f0005b10 comp rx=0 tx=0).stop 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 shutdown_connections 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc1d8075240 0x7fc1d8077750 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1f41527b0 0x7fc1f40aee20 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc1f40b5c70 0x7fc1f40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 --2- 172.21.15.46:0/1870781944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1f40b4de0 0x7fc1f41498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 >> 172.21.15.46:0/1870781944 conn(0x7fc1f40b2040 msgr2=0x7fc1f414d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 shutdown_connections 2024-09-17T03:35:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:47.957+0000 7fc1fc800640 1 -- 172.21.15.46:0/1870781944 wait complete. 2024-09-17T03:35:47.980 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:35:47.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:35:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 -- 172.21.15.46:0/3000942024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 msgr2=0x7fd3d8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/3000942024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3d8004c20 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009b60 tx=0x7fd3dc01d240 comp rx=0 tx=0).stop 2024-09-17T03:35:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 -- 172.21.15.46:0/3000942024 shutdown_connections 2024-09-17T03:35:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/3000942024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3d8004c20 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 -- 172.21.15.46:0/3000942024 >> 172.21.15.46:0/3000942024 conn(0x7fd3f40b2040 msgr2=0x7fd3f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 -- 172.21.15.46:0/3000942024 shutdown_connections 2024-09-17T03:35:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 -- 172.21.15.46:0/3000942024 wait complete. 2024-09-17T03:35:48.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.149+0000 7fd3f8c00640 1 Processor -- start 2024-09-17T03:35:48.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- start start 2024-09-17T03:35:48.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3f41ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:48.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f4157600 0x7fd3f40cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:48.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3f40cd870 0x7fd3f4154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:48.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f40b5690 con 0x7fd3d8004810 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f40b5510 con 0x7fd3f4157600 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f415f180 con 0x7fd3f40cd870 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3f41ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3f41ea6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33496/0 (socket says 172.21.15.46:33496) 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 -- 172.21.15.46:0/761627907 learned_addr learned my addr 172.21.15.46:0/761627907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 -- 172.21.15.46:0/761627907 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3f40cd870 msgr2=0x7fd3f4154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3f40cd870 0x7fd3f4154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 -- 172.21.15.46:0/761627907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f4157600 msgr2=0x7fd3f40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f4157600 0x7fd3f40cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 -- 172.21.15.46:0/761627907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41fcde0 con 0x7fd3d8004810 2024-09-17T03:35:48.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f1e00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3f41ea6e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4004dd0 tx=0x7fd3e4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4001d50 con 0x7fd3d8004810 2024-09-17T03:35:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3e4016070 con 0x7fd3d8004810 2024-09-17T03:35:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4010b40 con 0x7fd3d8004810 2024-09-17T03:35:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41fd0c0 con 0x7fd3d8004810 2024-09-17T03:35:48.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41fd680 con 0x7fd3d8004810 2024-09-17T03:35:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.153+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f41fd880 con 0x7fd3d8004810 2024-09-17T03:35:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.157+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fd3e4010ce0 con 0x7fd3d8004810 2024-09-17T03:35:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.157+0000 7fd3eaa00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3d0075210 0x7fd3d0077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.157+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fd3e4093a80 con 0x7fd3d8004810 2024-09-17T03:35:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.157+0000 7fd3e9400640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3d0075210 0x7fd3d0077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:48.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.157+0000 7fd3e9400640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3d0075210 0x7fd3d0077720 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0ae000 tx=0x7fd3d8005840 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:48.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.161+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd3f41fd880 con 0x7fd3d8004810 2024-09-17T03:35:48.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.373+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd3f415f670 con 0x7fd3d8004810 2024-09-17T03:35:48.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.953+0000 7fd3eaa00640 1 -- 172.21.15.46:0/761627907 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7fd3e40605a0 con 0x7fd3d8004810 2024-09-17T03:35:48.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3d0075210 msgr2=0x7fd3d0077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3d0075210 0x7fd3d0077720 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0ae000 tx=0x7fd3d8005840 comp rx=0 tx=0).stop 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 msgr2=0x7fd3f41ea6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3f41ea6e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4004dd0 tx=0x7fd3e4001640 comp rx=0 tx=0).stop 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 shutdown_connections 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd3d0075210 0x7fd3d0077720 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3f40cd870 0x7fd3f4154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f4157600 0x7fd3f40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 --2- 172.21.15.46:0/761627907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3d8004810 0x7fd3f41ea6e0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 >> 172.21.15.46:0/761627907 conn(0x7fd3f40b2040 msgr2=0x7fd3f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:48.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 shutdown_connections 2024-09-17T03:35:48.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:48.957+0000 7fd3f8c00640 1 -- 172.21.15.46:0/761627907 wait complete. 2024-09-17T03:35:48.984 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:35:48.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f9444c00640 1 -- 172.21.15.46:0/1852307167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9424004810 msgr2=0x7f9424004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f9444c00640 1 --2- 172.21.15.46:0/1852307167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9424004810 0x7f9424004c20 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f942800c470 tx=0x7f942801f700 comp rx=0 tx=0).stop 2024-09-17T03:35:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f943e800640 1 -- 172.21.15.46:0/1852307167 reap_dead start 2024-09-17T03:35:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f9444c00640 1 -- 172.21.15.46:0/1852307167 shutdown_connections 2024-09-17T03:35:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f9444c00640 1 -- 172.21.15.46:0/1852307167 >> 172.21.15.46:0/1852307167 conn(0x7f94400b2040 msgr2=0x7f94400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f9444c00640 1 -- 172.21.15.46:0/1852307167 shutdown_connections 2024-09-17T03:35:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.149+0000 7f9444c00640 1 -- 172.21.15.46:0/1852307167 wait complete. 2024-09-17T03:35:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 Processor -- start 2024-09-17T03:35:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 -- start start 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9424004810 0x7f944014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 0x7f94401e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401e60b0 0x7f9440158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f944015cf30 con 0x7f9440150180 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f944015cdb0 con 0x7f94401e60b0 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9444c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f944015d0b0 con 0x7f9424004810 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 0x7f94401e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 0x7f94401e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33516/0 (socket says 172.21.15.46:33516) 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 -- 172.21.15.46:0/459547716 learned_addr learned my addr 172.21.15.46:0/459547716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 -- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9424004810 msgr2=0x7f944014fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f943de00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9424004810 0x7f944014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f943e800640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401e60b0 0x7f9440158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9424004810 0x7f944014fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 -- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401e60b0 msgr2=0x7f9440158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401e60b0 0x7f9440158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f9435400640 1 -- 172.21.15.46:0/459547716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94400cdc50 con 0x7f9440150180 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f943de00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9424004810 0x7f944014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.153+0000 7f943e800640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401e60b0 0x7f9440158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9435400640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 0x7f94401e5b70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f942809c000 tx=0x7f9428006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94280c1070 con 0x7f9440150180 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9428001690 con 0x7f9440150180 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94280b3030 con 0x7f9440150180 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94401e68e0 con 0x7f9440150180 2024-09-17T03:35:49.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94400cdec0 con 0x7f9440150180 2024-09-17T03:35:49.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94401e6650 con 0x7f9440150180 2024-09-17T03:35:49.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f94280c4070 con 0x7f9440150180 2024-09-17T03:35:49.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.157+0000 7f9436a00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f941c075210 0x7f941c077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.161+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f9428130770 con 0x7f9440150180 2024-09-17T03:35:49.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.161+0000 7f943de00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f941c075210 0x7f941c077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.161+0000 7f943de00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f941c075210 0x7f941c077720 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9430009b50 tx=0x7f94300091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:49.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.165+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f94400b5730 con 0x7f9440150180 2024-09-17T03:35:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.369+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94400768c0 con 0x7f9440150180 2024-09-17T03:35:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9436a00640 1 -- 172.21.15.46:0/459547716 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5241 (secure 0 0 0) 0x7f94280fcc20 con 0x7f9440150180 2024-09-17T03:35:49.372 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:49.372 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"btime":"2024-09-17T03:35:48:955180+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:35:45.367356+0000","modified":"2024-09-17T03:35:48.376364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248335":{"gid":248335,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2479724490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2479724490},{"type":"v1","addr":"172.21.15.46:6836","nonce":2479724490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248335,"qdb_cluster":[248335]},"id":9}]} 2024-09-17T03:35:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 84 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f941c075210 msgr2=0x7f941c077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f941c075210 0x7f941c077720 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9430009b50 tx=0x7f94300091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 msgr2=0x7f94401e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 0x7f94401e5b70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f942809c000 tx=0x7f9428006f40 comp rx=0 tx=0).stop 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 shutdown_connections 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f941c075210 0x7f941c077720 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94401e60b0 0x7f9440158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9440150180 0x7f94401e5b70 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 --2- 172.21.15.46:0/459547716 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9424004810 0x7f944014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 >> 172.21.15.46:0/459547716 conn(0x7f94400b2040 msgr2=0x7f94400b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 shutdown_connections 2024-09-17T03:35:49.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.373+0000 7f9444c00640 1 -- 172.21.15.46:0/459547716 wait complete. 2024-09-17T03:35:49.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 -- 172.21.15.46:0/2971376949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd015c5d0 msgr2=0x7f4cd0073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/2971376949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd015c5d0 0x7f4cd0073dc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009900 tx=0x7f4cc001c8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 -- 172.21.15.46:0/2971376949 shutdown_connections 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/2971376949 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd006fa70 0x7f4cd015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/2971376949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd015c5d0 0x7f4cd0073dc0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 -- 172.21.15.46:0/2971376949 >> 172.21.15.46:0/2971376949 conn(0x7f4cd00b2c40 msgr2=0x7f4cd00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:49.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 -- 172.21.15.46:0/2971376949 shutdown_connections 2024-09-17T03:35:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.553+0000 7f4cd4c00640 1 -- 172.21.15.46:0/2971376949 wait complete. 2024-09-17T03:35:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 Processor -- start 2024-09-17T03:35:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 -- start start 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd006fa70 0x7f4cd014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 0x7f4cd0150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd015c5d0 0x7f4cd01509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd00b3d30 con 0x7f4cd0074300 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd015ffd0 con 0x7f4cd006fa70 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd0168b10 con 0x7f4cd015c5d0 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 0x7f4cd0150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 0x7f4cd0150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33522/0 (socket says 172.21.15.46:33522) 2024-09-17T03:35:49.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 -- 172.21.15.46:0/3008257744 learned_addr learned my addr 172.21.15.46:0/3008257744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4ccde00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd006fa70 0x7f4cd014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cce800640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd015c5d0 0x7f4cd01509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 -- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd015c5d0 msgr2=0x7f4cd01509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd015c5d0 0x7f4cd01509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 -- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd006fa70 msgr2=0x7f4cd014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd006fa70 0x7f4cd014b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 -- 172.21.15.46:0/3008257744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd0076900 con 0x7f4cd0074300 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cce800640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd015c5d0 0x7f4cd01509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc5400640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 0x7f4cd0150490 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800c390 tx=0x7f4cb800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8017070 con 0x7f4cd0074300 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cb80053c0 con 0x7f4cd0074300 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8003840 con 0x7f4cd0074300 2024-09-17T03:35:49.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd00b0910 con 0x7f4cd0074300 2024-09-17T03:35:49.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.557+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd015c400 con 0x7f4cd0074300 2024-09-17T03:35:49.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.561+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd014c130 con 0x7f4cd0074300 2024-09-17T03:35:49.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.561+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f4cb801a070 con 0x7f4cd0074300 2024-09-17T03:35:49.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.561+0000 7f4cc6a00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cac075210 0x7f4cac077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.561+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f4cb80925b0 con 0x7f4cd0074300 2024-09-17T03:35:49.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.561+0000 7f4ccde00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cac075210 0x7f4cac077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.565+0000 7f4ccde00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cac075210 0x7f4cac077720 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00af000 tx=0x7f4cc001b0f0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.565+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4cb805ea70 con 0x7f4cd0074300 2024-09-17T03:35:49.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.741+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cd014cf10 con 0x7f4cd0074300 2024-09-17T03:35:49.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cc6a00640 1 -- 172.21.15.46:0/3008257744 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16747 (secure 0 0 0) 0x7f4cb80634a0 con 0x7f4cd0074300 2024-09-17T03:35:49.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:49.744 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":129,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:35:46.938490+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:35:43.106524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T03:35:44.323553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:35:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cac075210 msgr2=0x7f4cac077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cac075210 0x7f4cac077720 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00af000 tx=0x7f4cc001b0f0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 msgr2=0x7f4cd0150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 0x7f4cd0150490 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800c390 tx=0x7f4cb800c850 comp rx=0 tx=0).stop 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 shutdown_connections 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4cac075210 0x7f4cac077720 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4cd015c5d0 0x7f4cd01509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0074300 0x7f4cd0150490 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 --2- 172.21.15.46:0/3008257744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cd006fa70 0x7f4cd014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 >> 172.21.15.46:0/3008257744 conn(0x7f4cd00b2c40 msgr2=0x7f4cd0153280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 shutdown_connections 2024-09-17T03:35:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.745+0000 7f4cd4c00640 1 -- 172.21.15.46:0/3008257744 wait complete. 2024-09-17T03:35:49.767 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:35:49.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 -- 172.21.15.46:0/2418803186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90160d50 msgr2=0x7f8d90074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 --2- 172.21.15.46:0/2418803186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90160d50 0x7f8d90074410 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f8d900713e0 tx=0x7f8d7c01c770 comp rx=0 tx=0).stop 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 -- 172.21.15.46:0/2418803186 shutdown_connections 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 --2- 172.21.15.46:0/2418803186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90160d50 0x7f8d90074410 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 --2- 172.21.15.46:0/2418803186 >> v2:172.21.15.99:3300/0 conn(0x7f8d90157030 0x7f8d900b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 -- 172.21.15.46:0/2418803186 >> 172.21.15.46:0/2418803186 conn(0x7f8d900708c0 msgr2=0x7f8d90070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 -- 172.21.15.46:0/2418803186 shutdown_connections 2024-09-17T03:35:49.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.925+0000 7f8d98000640 1 -- 172.21.15.46:0/2418803186 wait complete. 2024-09-17T03:35:49.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 Processor -- start 2024-09-17T03:35:49.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 -- start start 2024-09-17T03:35:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 0x7f8d9014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d90160d50 0x7f8d901548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d90161160 0x7f8d9015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9016a790 con 0x7f8d90157030 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9016a610 con 0x7f8d90161160 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d98000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9016a910 con 0x7f8d90160d50 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d95600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 0x7f8d9014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d95600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 0x7f8d9014f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33544/0 (socket says 172.21.15.46:33544) 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d95600640 1 -- 172.21.15.46:0/3043170982 learned_addr learned my addr 172.21.15.46:0/3043170982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d94c00640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d90160d50 0x7f8d901548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d96000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d90161160 0x7f8d9015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d95600640 1 -- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d90160d50 msgr2=0x7f8d901548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.929+0000 7f8d95600640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d90160d50 0x7f8d901548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d95600640 1 -- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d90161160 msgr2=0x7f8d9015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d95600640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d90161160 0x7f8d9015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d95600640 1 -- 172.21.15.46:0/3043170982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9014f6a0 con 0x7f8d90157030 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d96000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d90161160 0x7f8d9015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d95600640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 0x7f8d9014f160 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400d720 tx=0x7f8d8400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84017070 con 0x7f8d90157030 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d84009cc0 con 0x7f8d90157030 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84005280 con 0x7f8d90157030 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d901fa520 con 0x7f8d90157030 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901f9760 con 0x7f8d90157030 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d900d2260 con 0x7f8d90157030 2024-09-17T03:35:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f8d84005420 con 0x7f8d90157030 2024-09-17T03:35:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.933+0000 7f8d8d600640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d70075210 0x7f8d70077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.937+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f8d840933e0 con 0x7f8d90157030 2024-09-17T03:35:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.937+0000 7f8d94c00640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d70075210 0x7f8d70077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.937+0000 7f8d94c00640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d70075210 0x7f8d70077720 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0af000 tx=0x7f8d7c01afb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:49.941+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d9014f6a0 con 0x7f8d90157030 2024-09-17T03:35:50.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.113+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d9006b540 con 0x7f8d70075210 2024-09-17T03:35:50.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.117+0000 7f8d8d600640 1 -- 172.21.15.46:0/3043170982 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252337 (secure 0 0 0) 0x7f8d7c0a68d0 con 0x7f8d70075210 2024-09-17T03:35:50.123 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:50.126 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:35:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d70075210 msgr2=0x7f8d70077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d70075210 0x7f8d70077720 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0af000 tx=0x7f8d7c01afb0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 msgr2=0x7f8d9014f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 0x7f8d9014f160 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400d720 tx=0x7f8d8400dbe0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 shutdown_connections 2024-09-17T03:35:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d70075210 0x7f8d70077720 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d90161160 0x7f8d9015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d90160d50 0x7f8d901548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 --2- 172.21.15.46:0/3043170982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90157030 0x7f8d9014f160 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.121+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 >> 172.21.15.46:0/3043170982 conn(0x7f8d900708c0 msgr2=0x7f8d900b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:50.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.125+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 shutdown_connections 2024-09-17T03:35:50.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.125+0000 7f8d98000640 1 -- 172.21.15.46:0/3043170982 wait complete. 2024-09-17T03:35:50.145 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":569,"stamp":"2024-09-17T03:35:49.561285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":140,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1152484,"kb_used_data":13620,"kb_used_omap":12,"kb_used_meta":1138675,"kb_avail":753822236,"statfs":{"total":773094113280,"available":771913969664,"internally_reserved":0,"allocated":13946880,"data_stored":6730585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1166003760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.642018"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116128+0000","last_change":"2024-09-17T03:35:45.955819+0000","last_active":"2024-09-17T03:35:47.116128+0000","last_peered":"2024-09-17T03:35:47.116128+0000","last_clean":"2024-09-17T03:35:47.116128+0000","last_became_active":"2024-09-17T03:35:45.954935+0000","last_became_peered":"2024-09-17T03:35:45.954935+0000","last_unstale":"2024-09-17T03:35:47.116128+0000","last_undegraded":"2024-09-17T03:35:47.116128+0000","last_fullsized":"2024-09-17T03:35:47.116128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.363688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116488+0000","last_change":"2024-09-17T03:35:45.945240+0000","last_active":"2024-09-17T03:35:47.116488+0000","last_peered":"2024-09-17T03:35:47.116488+0000","last_clean":"2024-09-17T03:35:47.116488+0000","last_became_active":"2024-09-17T03:35:45.944788+0000","last_became_peered":"2024-09-17T03:35:45.944788+0000","last_unstale":"2024-09-17T03:35:47.116488+0000","last_undegraded":"2024-09-17T03:35:47.116488+0000","last_fullsized":"2024-09-17T03:35:47.116488+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:02.551861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.973644+0000","last_change":"2024-09-17T03:35:45.965545+0000","last_active":"2024-09-17T03:35:45.973644+0000","last_peered":"2024-09-17T03:35:45.973644+0000","last_clean":"2024-09-17T03:35:45.973644+0000","last_became_active":"2024-09-17T03:35:45.961480+0000","last_became_peered":"2024-09-17T03:35:45.961480+0000","last_unstale":"2024-09-17T03:35:45.973644+0000","last_undegraded":"2024-09-17T03:35:45.973644+0000","last_fullsized":"2024-09-17T03:35:45.973644+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:35.059698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115130+0000","last_change":"2024-09-17T03:35:45.960750+0000","last_active":"2024-09-17T03:35:47.115130+0000","last_peered":"2024-09-17T03:35:47.115130+0000","last_clean":"2024-09-17T03:35:47.115130+0000","last_became_active":"2024-09-17T03:35:45.960431+0000","last_became_peered":"2024-09-17T03:35:45.960431+0000","last_unstale":"2024-09-17T03:35:47.115130+0000","last_undegraded":"2024-09-17T03:35:47.115130+0000","last_fullsized":"2024-09-17T03:35:47.115130+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:59.838096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963365+0000","last_change":"2024-09-17T03:35:45.971487+0000","last_active":"2024-09-17T03:35:46.963365+0000","last_peered":"2024-09-17T03:35:46.963365+0000","last_clean":"2024-09-17T03:35:46.963365+0000","last_became_active":"2024-09-17T03:35:45.970309+0000","last_became_peered":"2024-09-17T03:35:45.970309+0000","last_unstale":"2024-09-17T03:35:46.963365+0000","last_undegraded":"2024-09-17T03:35:46.963365+0000","last_fullsized":"2024-09-17T03:35:46.963365+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953008+0000","last_change":"2024-09-17T03:35:45.961513+0000","last_active":"2024-09-17T03:35:46.953008+0000","last_peered":"2024-09-17T03:35:46.953008+0000","last_clean":"2024-09-17T03:35:46.953008+0000","last_became_active":"2024-09-17T03:35:45.961192+0000","last_became_peered":"2024-09-17T03:35:45.961192+0000","last_unstale":"2024-09-17T03:35:46.953008+0000","last_undegraded":"2024-09-17T03:35:46.953008+0000","last_fullsized":"2024-09-17T03:35:46.953008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:45.692040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952824+0000","last_change":"2024-09-17T03:35:45.955196+0000","last_active":"2024-09-17T03:35:46.952824+0000","last_peered":"2024-09-17T03:35:46.952824+0000","last_clean":"2024-09-17T03:35:46.952824+0000","last_became_active":"2024-09-17T03:35:45.954869+0000","last_became_peered":"2024-09-17T03:35:45.954869+0000","last_unstale":"2024-09-17T03:35:46.952824+0000","last_undegraded":"2024-09-17T03:35:46.952824+0000","last_fullsized":"2024-09-17T03:35:46.952824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:27.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978568+0000","last_change":"2024-09-17T03:35:45.968695+0000","last_active":"2024-09-17T03:35:45.978568+0000","last_peered":"2024-09-17T03:35:45.978568+0000","last_clean":"2024-09-17T03:35:45.978568+0000","last_became_active":"2024-09-17T03:35:45.966016+0000","last_became_peered":"2024-09-17T03:35:45.966016+0000","last_unstale":"2024-09-17T03:35:45.978568+0000","last_undegraded":"2024-09-17T03:35:45.978568+0000","last_fullsized":"2024-09-17T03:35:45.978568+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:29.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975084+0000","last_change":"2024-09-17T03:35:45.969067+0000","last_active":"2024-09-17T03:35:45.975084+0000","last_peered":"2024-09-17T03:35:45.975084+0000","last_clean":"2024-09-17T03:35:45.975084+0000","last_became_active":"2024-09-17T03:35:45.968667+0000","last_became_peered":"2024-09-17T03:35:45.968667+0000","last_unstale":"2024-09-17T03:35:45.975084+0000","last_undegraded":"2024-09-17T03:35:45.975084+0000","last_fullsized":"2024-09-17T03:35:45.975084+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:05:08.235660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116451+0000","last_change":"2024-09-17T03:35:45.945553+0000","last_active":"2024-09-17T03:35:47.116451+0000","last_peered":"2024-09-17T03:35:47.116451+0000","last_clean":"2024-09-17T03:35:47.116451+0000","last_became_active":"2024-09-17T03:35:45.945233+0000","last_became_peered":"2024-09-17T03:35:45.945233+0000","last_unstale":"2024-09-17T03:35:47.116451+0000","last_undegraded":"2024-09-17T03:35:47.116451+0000","last_fullsized":"2024-09-17T03:35:47.116451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:52.862294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942689+0000","last_change":"2024-09-17T03:35:45.972155+0000","last_active":"2024-09-17T03:35:46.942689+0000","last_peered":"2024-09-17T03:35:46.942689+0000","last_clean":"2024-09-17T03:35:46.942689+0000","last_became_active":"2024-09-17T03:35:45.971845+0000","last_became_peered":"2024-09-17T03:35:45.971845+0000","last_unstale":"2024-09-17T03:35:46.942689+0000","last_undegraded":"2024-09-17T03:35:46.942689+0000","last_fullsized":"2024-09-17T03:35:46.942689+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:13.103673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977386+0000","last_change":"2024-09-17T03:35:45.965493+0000","last_active":"2024-09-17T03:35:45.977386+0000","last_peered":"2024-09-17T03:35:45.977386+0000","last_clean":"2024-09-17T03:35:45.977386+0000","last_became_active":"2024-09-17T03:35:45.962965+0000","last_became_peered":"2024-09-17T03:35:45.962965+0000","last_unstale":"2024-09-17T03:35:45.977386+0000","last_undegraded":"2024-09-17T03:35:45.977386+0000","last_fullsized":"2024-09-17T03:35:45.977386+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:59.517858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115407+0000","last_change":"2024-09-17T03:35:45.957732+0000","last_active":"2024-09-17T03:35:47.115407+0000","last_peered":"2024-09-17T03:35:47.115407+0000","last_clean":"2024-09-17T03:35:47.115407+0000","last_became_active":"2024-09-17T03:35:45.957405+0000","last_became_peered":"2024-09-17T03:35:45.957405+0000","last_unstale":"2024-09-17T03:35:47.115407+0000","last_undegraded":"2024-09-17T03:35:47.115407+0000","last_fullsized":"2024-09-17T03:35:47.115407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:23.451219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115934+0000","last_change":"2024-09-17T03:35:45.952592+0000","last_active":"2024-09-17T03:35:47.115934+0000","last_peered":"2024-09-17T03:35:47.115934+0000","last_clean":"2024-09-17T03:35:47.115934+0000","last_became_active":"2024-09-17T03:35:45.949816+0000","last_became_peered":"2024-09-17T03:35:45.949816+0000","last_unstale":"2024-09-17T03:35:47.115934+0000","last_undegraded":"2024-09-17T03:35:47.115934+0000","last_fullsized":"2024-09-17T03:35:47.115934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:16.316918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114925+0000","last_change":"2024-09-17T03:35:45.958139+0000","last_active":"2024-09-17T03:35:47.114925+0000","last_peered":"2024-09-17T03:35:47.114925+0000","last_clean":"2024-09-17T03:35:47.114925+0000","last_became_active":"2024-09-17T03:35:45.957813+0000","last_became_peered":"2024-09-17T03:35:45.957813+0000","last_unstale":"2024-09-17T03:35:47.114925+0000","last_undegraded":"2024-09-17T03:35:47.114925+0000","last_fullsized":"2024-09-17T03:35:47.114925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:28.891920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945605+0000","last_change":"2024-09-17T03:35:45.950827+0000","last_active":"2024-09-17T03:35:46.945605+0000","last_peered":"2024-09-17T03:35:46.945605+0000","last_clean":"2024-09-17T03:35:46.945605+0000","last_became_active":"2024-09-17T03:35:45.950514+0000","last_became_peered":"2024-09-17T03:35:45.950514+0000","last_unstale":"2024-09-17T03:35:46.945605+0000","last_undegraded":"2024-09-17T03:35:46.945605+0000","last_fullsized":"2024-09-17T03:35:46.945605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:52.381399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953466+0000","last_change":"2024-09-17T03:35:45.964355+0000","last_active":"2024-09-17T03:35:46.953466+0000","last_peered":"2024-09-17T03:35:46.953466+0000","last_clean":"2024-09-17T03:35:46.953466+0000","last_became_active":"2024-09-17T03:35:45.962165+0000","last_became_peered":"2024-09-17T03:35:45.962165+0000","last_unstale":"2024-09-17T03:35:46.953466+0000","last_undegraded":"2024-09-17T03:35:46.953466+0000","last_fullsized":"2024-09-17T03:35:46.953466+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:26.092966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946215+0000","last_change":"2024-09-17T03:35:45.955274+0000","last_active":"2024-09-17T03:35:46.946215+0000","last_peered":"2024-09-17T03:35:46.946215+0000","last_clean":"2024-09-17T03:35:46.946215+0000","last_became_active":"2024-09-17T03:35:45.953802+0000","last_became_peered":"2024-09-17T03:35:45.953802+0000","last_unstale":"2024-09-17T03:35:46.946215+0000","last_undegraded":"2024-09-17T03:35:46.946215+0000","last_fullsized":"2024-09-17T03:35:46.946215+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942937+0000","last_change":"2024-09-17T03:35:45.963307+0000","last_active":"2024-09-17T03:35:46.942937+0000","last_peered":"2024-09-17T03:35:46.942937+0000","last_clean":"2024-09-17T03:35:46.942937+0000","last_became_active":"2024-09-17T03:35:45.963007+0000","last_became_peered":"2024-09-17T03:35:45.963007+0000","last_unstale":"2024-09-17T03:35:46.942937+0000","last_undegraded":"2024-09-17T03:35:46.942937+0000","last_fullsized":"2024-09-17T03:35:46.942937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:09.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945919+0000","last_change":"2024-09-17T03:35:45.950245+0000","last_active":"2024-09-17T03:35:46.945919+0000","last_peered":"2024-09-17T03:35:46.945919+0000","last_clean":"2024-09-17T03:35:46.945919+0000","last_became_active":"2024-09-17T03:35:45.949852+0000","last_became_peered":"2024-09-17T03:35:45.949852+0000","last_unstale":"2024-09-17T03:35:46.945919+0000","last_undegraded":"2024-09-17T03:35:46.945919+0000","last_fullsized":"2024-09-17T03:35:46.945919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:09.687961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963523+0000","last_change":"2024-09-17T03:35:45.967983+0000","last_active":"2024-09-17T03:35:46.963523+0000","last_peered":"2024-09-17T03:35:46.963523+0000","last_clean":"2024-09-17T03:35:46.963523+0000","last_became_active":"2024-09-17T03:35:45.964201+0000","last_became_peered":"2024-09-17T03:35:45.964201+0000","last_unstale":"2024-09-17T03:35:46.963523+0000","last_undegraded":"2024-09-17T03:35:46.963523+0000","last_fullsized":"2024-09-17T03:35:46.963523+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:26.825390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942483+0000","last_change":"2024-09-17T03:35:45.969671+0000","last_active":"2024-09-17T03:35:46.942483+0000","last_peered":"2024-09-17T03:35:46.942483+0000","last_clean":"2024-09-17T03:35:46.942483+0000","last_became_active":"2024-09-17T03:35:45.969304+0000","last_became_peered":"2024-09-17T03:35:45.969304+0000","last_unstale":"2024-09-17T03:35:46.942483+0000","last_undegraded":"2024-09-17T03:35:46.942483+0000","last_fullsized":"2024-09-17T03:35:46.942483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:30.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.972932+0000","last_change":"2024-09-17T03:35:45.962436+0000","last_active":"2024-09-17T03:35:45.972932+0000","last_peered":"2024-09-17T03:35:45.972932+0000","last_clean":"2024-09-17T03:35:45.972932+0000","last_became_active":"2024-09-17T03:35:45.962092+0000","last_became_peered":"2024-09-17T03:35:45.962092+0000","last_unstale":"2024-09-17T03:35:45.972932+0000","last_undegraded":"2024-09-17T03:35:45.972932+0000","last_fullsized":"2024-09-17T03:35:45.972932+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:33.151389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942550+0000","last_change":"2024-09-17T03:35:45.968660+0000","last_active":"2024-09-17T03:35:46.942550+0000","last_peered":"2024-09-17T03:35:46.942550+0000","last_clean":"2024-09-17T03:35:46.942550+0000","last_became_active":"2024-09-17T03:35:45.968290+0000","last_became_peered":"2024-09-17T03:35:45.968290+0000","last_unstale":"2024-09-17T03:35:46.942550+0000","last_undegraded":"2024-09-17T03:35:46.942550+0000","last_fullsized":"2024-09-17T03:35:46.942550+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:23.605758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116395+0000","last_change":"2024-09-17T03:35:45.950767+0000","last_active":"2024-09-17T03:35:47.116395+0000","last_peered":"2024-09-17T03:35:47.116395+0000","last_clean":"2024-09-17T03:35:47.116395+0000","last_became_active":"2024-09-17T03:35:45.950459+0000","last_became_peered":"2024-09-17T03:35:45.950459+0000","last_unstale":"2024-09-17T03:35:47.116395+0000","last_undegraded":"2024-09-17T03:35:47.116395+0000","last_fullsized":"2024-09-17T03:35:47.116395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:00.727158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946018+0000","last_change":"2024-09-17T03:35:45.951322+0000","last_active":"2024-09-17T03:35:46.946018+0000","last_peered":"2024-09-17T03:35:46.946018+0000","last_clean":"2024-09-17T03:35:46.946018+0000","last_became_active":"2024-09-17T03:35:45.950888+0000","last_became_peered":"2024-09-17T03:35:45.950888+0000","last_unstale":"2024-09-17T03:35:46.946018+0000","last_undegraded":"2024-09-17T03:35:46.946018+0000","last_fullsized":"2024-09-17T03:35:46.946018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115304+0000","last_change":"2024-09-17T03:35:45.958772+0000","last_active":"2024-09-17T03:35:47.115304+0000","last_peered":"2024-09-17T03:35:47.115304+0000","last_clean":"2024-09-17T03:35:47.115304+0000","last_became_active":"2024-09-17T03:35:45.958227+0000","last_became_peered":"2024-09-17T03:35:45.958227+0000","last_unstale":"2024-09-17T03:35:47.115304+0000","last_undegraded":"2024-09-17T03:35:47.115304+0000","last_fullsized":"2024-09-17T03:35:47.115304+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:36.638860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":3,"reported_epoch":128,"state":"creating+activating","last_fresh":"2024-09-17T03:35:45.948224+0000","last_change":"2024-09-17T03:35:45.948224+0000","last_active":"2024-09-17T03:35:44.919052+0000","last_peered":"2024-09-17T03:35:44.919052+0000","last_clean":"2024-09-17T03:35:44.919052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:35:45.948224+0000","last_undegraded":"2024-09-17T03:35:45.948224+0000","last_fullsized":"2024-09-17T03:35:45.948224+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115694+0000","last_change":"2024-09-17T03:35:45.957427+0000","last_active":"2024-09-17T03:35:47.115694+0000","last_peered":"2024-09-17T03:35:47.115694+0000","last_clean":"2024-09-17T03:35:47.115694+0000","last_became_active":"2024-09-17T03:35:45.956963+0000","last_became_peered":"2024-09-17T03:35:45.956963+0000","last_unstale":"2024-09-17T03:35:47.115694+0000","last_undegraded":"2024-09-17T03:35:47.115694+0000","last_fullsized":"2024-09-17T03:35:47.115694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.736866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115434+0000","last_change":"2024-09-17T03:35:45.959532+0000","last_active":"2024-09-17T03:35:47.115434+0000","last_peered":"2024-09-17T03:35:47.115434+0000","last_clean":"2024-09-17T03:35:47.115434+0000","last_became_active":"2024-09-17T03:35:45.952528+0000","last_became_peered":"2024-09-17T03:35:45.952528+0000","last_unstale":"2024-09-17T03:35:47.115434+0000","last_undegraded":"2024-09-17T03:35:47.115434+0000","last_fullsized":"2024-09-17T03:35:47.115434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:49.407151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963303+0000","last_change":"2024-09-17T03:35:45.975662+0000","last_active":"2024-09-17T03:35:46.963303+0000","last_peered":"2024-09-17T03:35:46.963303+0000","last_clean":"2024-09-17T03:35:46.963303+0000","last_became_active":"2024-09-17T03:35:45.975295+0000","last_became_peered":"2024-09-17T03:35:45.975295+0000","last_unstale":"2024-09-17T03:35:46.963303+0000","last_undegraded":"2024-09-17T03:35:46.963303+0000","last_fullsized":"2024-09-17T03:35:46.963303+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952833+0000","last_change":"2024-09-17T03:35:45.947162+0000","last_active":"2024-09-17T03:35:46.952833+0000","last_peered":"2024-09-17T03:35:46.952833+0000","last_clean":"2024-09-17T03:35:46.952833+0000","last_became_active":"2024-09-17T03:35:45.946769+0000","last_became_peered":"2024-09-17T03:35:45.946769+0000","last_unstale":"2024-09-17T03:35:46.952833+0000","last_undegraded":"2024-09-17T03:35:46.952833+0000","last_fullsized":"2024-09-17T03:35:46.952833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:12.153774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946273+0000","last_change":"2024-09-17T03:35:45.951693+0000","last_active":"2024-09-17T03:35:46.946273+0000","last_peered":"2024-09-17T03:35:46.946273+0000","last_clean":"2024-09-17T03:35:46.946273+0000","last_became_active":"2024-09-17T03:35:45.951350+0000","last_became_peered":"2024-09-17T03:35:45.951350+0000","last_unstale":"2024-09-17T03:35:46.946273+0000","last_undegraded":"2024-09-17T03:35:46.946273+0000","last_fullsized":"2024-09-17T03:35:46.946273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963772+0000","last_change":"2024-09-17T03:35:44.961465+0000","last_active":"2024-09-17T03:35:46.963772+0000","last_peered":"2024-09-17T03:35:46.963772+0000","last_clean":"2024-09-17T03:35:46.963772+0000","last_became_active":"2024-09-17T03:35:44.961034+0000","last_became_peered":"2024-09-17T03:35:44.961034+0000","last_unstale":"2024-09-17T03:35:46.963772+0000","last_undegraded":"2024-09-17T03:35:46.963772+0000","last_fullsized":"2024-09-17T03:35:46.963772+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.962934+0000","last_change":"2024-09-17T03:35:45.959757+0000","last_active":"2024-09-17T03:35:45.962934+0000","last_peered":"2024-09-17T03:35:45.962934+0000","last_clean":"2024-09-17T03:35:45.962934+0000","last_became_active":"2024-09-17T03:35:45.959088+0000","last_became_peered":"2024-09-17T03:35:45.959088+0000","last_unstale":"2024-09-17T03:35:45.962934+0000","last_undegraded":"2024-09-17T03:35:45.962934+0000","last_fullsized":"2024-09-17T03:35:45.962934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:09.348110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950367+0000","last_change":"2024-09-17T03:35:44.962725+0000","last_active":"2024-09-17T03:35:45.950367+0000","last_peered":"2024-09-17T03:35:45.950367+0000","last_clean":"2024-09-17T03:35:45.950367+0000","last_became_active":"2024-09-17T03:35:44.962372+0000","last_became_peered":"2024-09-17T03:35:44.962372+0000","last_unstale":"2024-09-17T03:35:45.950367+0000","last_undegraded":"2024-09-17T03:35:45.950367+0000","last_fullsized":"2024-09-17T03:35:45.950367+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:27.452690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963667+0000","last_change":"2024-09-17T03:35:45.971272+0000","last_active":"2024-09-17T03:35:46.963667+0000","last_peered":"2024-09-17T03:35:46.963667+0000","last_clean":"2024-09-17T03:35:46.963667+0000","last_became_active":"2024-09-17T03:35:45.967588+0000","last_became_peered":"2024-09-17T03:35:45.967588+0000","last_unstale":"2024-09-17T03:35:46.963667+0000","last_undegraded":"2024-09-17T03:35:46.963667+0000","last_fullsized":"2024-09-17T03:35:46.963667+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:57.132363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942929+0000","last_change":"2024-09-17T03:35:44.955317+0000","last_active":"2024-09-17T03:35:46.942929+0000","last_peered":"2024-09-17T03:35:46.942929+0000","last_clean":"2024-09-17T03:35:46.942929+0000","last_became_active":"2024-09-17T03:35:44.952983+0000","last_became_peered":"2024-09-17T03:35:44.952983+0000","last_unstale":"2024-09-17T03:35:46.942929+0000","last_undegraded":"2024-09-17T03:35:46.942929+0000","last_fullsized":"2024-09-17T03:35:46.942929+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:52.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963619+0000","last_change":"2024-09-17T03:35:45.976105+0000","last_active":"2024-09-17T03:35:46.963619+0000","last_peered":"2024-09-17T03:35:46.963619+0000","last_clean":"2024-09-17T03:35:46.963619+0000","last_became_active":"2024-09-17T03:35:45.975807+0000","last_became_peered":"2024-09-17T03:35:45.975807+0000","last_unstale":"2024-09-17T03:35:46.963619+0000","last_undegraded":"2024-09-17T03:35:46.963619+0000","last_fullsized":"2024-09-17T03:35:46.963619+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:26.601742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116356+0000","last_change":"2024-09-17T03:35:44.959733+0000","last_active":"2024-09-17T03:35:47.116356+0000","last_peered":"2024-09-17T03:35:47.116356+0000","last_clean":"2024-09-17T03:35:47.116356+0000","last_became_active":"2024-09-17T03:35:44.959330+0000","last_became_peered":"2024-09-17T03:35:44.959330+0000","last_unstale":"2024-09-17T03:35:47.116356+0000","last_undegraded":"2024-09-17T03:35:47.116356+0000","last_fullsized":"2024-09-17T03:35:47.116356+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:34.223217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945544+0000","last_change":"2024-09-17T03:35:45.945250+0000","last_active":"2024-09-17T03:35:46.945544+0000","last_peered":"2024-09-17T03:35:46.945544+0000","last_clean":"2024-09-17T03:35:46.945544+0000","last_became_active":"2024-09-17T03:35:45.944882+0000","last_became_peered":"2024-09-17T03:35:45.944882+0000","last_unstale":"2024-09-17T03:35:46.945544+0000","last_undegraded":"2024-09-17T03:35:46.945544+0000","last_fullsized":"2024-09-17T03:35:46.945544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:03.455251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115803+0000","last_change":"2024-09-17T03:35:44.957196+0000","last_active":"2024-09-17T03:35:47.115803+0000","last_peered":"2024-09-17T03:35:47.115803+0000","last_clean":"2024-09-17T03:35:47.115803+0000","last_became_active":"2024-09-17T03:35:44.956846+0000","last_became_peered":"2024-09-17T03:35:44.956846+0000","last_unstale":"2024-09-17T03:35:47.115803+0000","last_undegraded":"2024-09-17T03:35:47.115803+0000","last_fullsized":"2024-09-17T03:35:47.115803+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:28.310407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953166+0000","last_change":"2024-09-17T03:35:45.975695+0000","last_active":"2024-09-17T03:35:46.953166+0000","last_peered":"2024-09-17T03:35:46.953166+0000","last_clean":"2024-09-17T03:35:46.953166+0000","last_became_active":"2024-09-17T03:35:45.975349+0000","last_became_peered":"2024-09-17T03:35:45.975349+0000","last_unstale":"2024-09-17T03:35:46.953166+0000","last_undegraded":"2024-09-17T03:35:46.953166+0000","last_fullsized":"2024-09-17T03:35:46.953166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:24.952949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945624+0000","last_change":"2024-09-17T03:35:44.952496+0000","last_active":"2024-09-17T03:35:46.945624+0000","last_peered":"2024-09-17T03:35:46.945624+0000","last_clean":"2024-09-17T03:35:46.945624+0000","last_became_active":"2024-09-17T03:35:44.952172+0000","last_became_peered":"2024-09-17T03:35:44.952172+0000","last_unstale":"2024-09-17T03:35:46.945624+0000","last_undegraded":"2024-09-17T03:35:46.945624+0000","last_fullsized":"2024-09-17T03:35:46.945624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.251927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978605+0000","last_change":"2024-09-17T03:35:45.970502+0000","last_active":"2024-09-17T03:35:45.978605+0000","last_peered":"2024-09-17T03:35:45.978605+0000","last_clean":"2024-09-17T03:35:45.978605+0000","last_became_active":"2024-09-17T03:35:45.967366+0000","last_became_peered":"2024-09-17T03:35:45.967366+0000","last_unstale":"2024-09-17T03:35:45.978605+0000","last_undegraded":"2024-09-17T03:35:45.978605+0000","last_fullsized":"2024-09-17T03:35:45.978605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:38.209115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952670+0000","last_change":"2024-09-17T03:35:44.958325+0000","last_active":"2024-09-17T03:35:46.952670+0000","last_peered":"2024-09-17T03:35:46.952670+0000","last_clean":"2024-09-17T03:35:46.952670+0000","last_became_active":"2024-09-17T03:35:44.958015+0000","last_became_peered":"2024-09-17T03:35:44.958015+0000","last_unstale":"2024-09-17T03:35:46.952670+0000","last_undegraded":"2024-09-17T03:35:46.952670+0000","last_fullsized":"2024-09-17T03:35:46.952670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:32.736499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114786+0000","last_change":"2024-09-17T03:35:45.955593+0000","last_active":"2024-09-17T03:35:47.114786+0000","last_peered":"2024-09-17T03:35:47.114786+0000","last_clean":"2024-09-17T03:35:47.114786+0000","last_became_active":"2024-09-17T03:35:45.954658+0000","last_became_peered":"2024-09-17T03:35:45.954658+0000","last_unstale":"2024-09-17T03:35:47.114786+0000","last_undegraded":"2024-09-17T03:35:47.114786+0000","last_fullsized":"2024-09-17T03:35:47.114786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114901+0000","last_change":"2024-09-17T03:35:44.951328+0000","last_active":"2024-09-17T03:35:47.114901+0000","last_peered":"2024-09-17T03:35:47.114901+0000","last_clean":"2024-09-17T03:35:47.114901+0000","last_became_active":"2024-09-17T03:35:44.951004+0000","last_became_peered":"2024-09-17T03:35:44.951004+0000","last_unstale":"2024-09-17T03:35:47.114901+0000","last_undegraded":"2024-09-17T03:35:47.114901+0000","last_fullsized":"2024-09-17T03:35:47.114901+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:57.623042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115774+0000","last_change":"2024-09-17T03:35:45.963042+0000","last_active":"2024-09-17T03:35:47.115774+0000","last_peered":"2024-09-17T03:35:47.115774+0000","last_clean":"2024-09-17T03:35:47.115774+0000","last_became_active":"2024-09-17T03:35:45.962752+0000","last_became_peered":"2024-09-17T03:35:45.962752+0000","last_unstale":"2024-09-17T03:35:47.115774+0000","last_undegraded":"2024-09-17T03:35:47.115774+0000","last_fullsized":"2024-09-17T03:35:47.115774+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:51.335684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950811+0000","last_change":"2024-09-17T03:35:44.962808+0000","last_active":"2024-09-17T03:35:45.950811+0000","last_peered":"2024-09-17T03:35:45.950811+0000","last_clean":"2024-09-17T03:35:45.950811+0000","last_became_active":"2024-09-17T03:35:44.962473+0000","last_became_peered":"2024-09-17T03:35:44.962473+0000","last_unstale":"2024-09-17T03:35:45.950811+0000","last_undegraded":"2024-09-17T03:35:45.950811+0000","last_fullsized":"2024-09-17T03:35:45.950811+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:59.547888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.967805+0000","last_change":"2024-09-17T03:35:45.963711+0000","last_active":"2024-09-17T03:35:45.967805+0000","last_peered":"2024-09-17T03:35:45.967805+0000","last_clean":"2024-09-17T03:35:45.967805+0000","last_became_active":"2024-09-17T03:35:45.963372+0000","last_became_peered":"2024-09-17T03:35:45.963372+0000","last_unstale":"2024-09-17T03:35:45.967805+0000","last_undegraded":"2024-09-17T03:35:45.967805+0000","last_fullsized":"2024-09-17T03:35:45.967805+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:56.111252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946333+0000","last_change":"2024-09-17T03:35:44.946163+0000","last_active":"2024-09-17T03:35:46.946333+0000","last_peered":"2024-09-17T03:35:46.946333+0000","last_clean":"2024-09-17T03:35:46.946333+0000","last_became_active":"2024-09-17T03:35:44.945021+0000","last_became_peered":"2024-09-17T03:35:44.945021+0000","last_unstale":"2024-09-17T03:35:46.946333+0000","last_undegraded":"2024-09-17T03:35:46.946333+0000","last_fullsized":"2024-09-17T03:35:46.946333+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963634+0000","last_change":"2024-09-17T03:35:45.972309+0000","last_active":"2024-09-17T03:35:46.963634+0000","last_peered":"2024-09-17T03:35:46.963634+0000","last_clean":"2024-09-17T03:35:46.963634+0000","last_became_active":"2024-09-17T03:35:45.971924+0000","last_became_peered":"2024-09-17T03:35:45.971924+0000","last_unstale":"2024-09-17T03:35:46.963634+0000","last_undegraded":"2024-09-17T03:35:46.963634+0000","last_fullsized":"2024-09-17T03:35:46.963634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:53.929773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.949681+0000","last_change":"2024-09-17T03:35:44.961523+0000","last_active":"2024-09-17T03:35:45.949681+0000","last_peered":"2024-09-17T03:35:45.949681+0000","last_clean":"2024-09-17T03:35:45.949681+0000","last_became_active":"2024-09-17T03:35:44.960356+0000","last_became_peered":"2024-09-17T03:35:44.960356+0000","last_unstale":"2024-09-17T03:35:45.949681+0000","last_undegraded":"2024-09-17T03:35:45.949681+0000","last_fullsized":"2024-09-17T03:35:45.949681+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.980673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963629+0000","last_change":"2024-09-17T03:35:45.964975+0000","last_active":"2024-09-17T03:35:46.963629+0000","last_peered":"2024-09-17T03:35:46.963629+0000","last_clean":"2024-09-17T03:35:46.963629+0000","last_became_active":"2024-09-17T03:35:45.964625+0000","last_became_peered":"2024-09-17T03:35:45.964625+0000","last_unstale":"2024-09-17T03:35:46.963629+0000","last_undegraded":"2024-09-17T03:35:46.963629+0000","last_fullsized":"2024-09-17T03:35:46.963629+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:14.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"128'6","reported_seq":19,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.988197+0000","last_change":"2024-09-17T03:35:44.957186+0000","last_active":"2024-09-17T03:35:45.988197+0000","last_peered":"2024-09-17T03:35:45.988197+0000","last_clean":"2024-09-17T03:35:45.988197+0000","last_became_active":"2024-09-17T03:35:44.955711+0000","last_became_peered":"2024-09-17T03:35:44.955711+0000","last_unstale":"2024-09-17T03:35:45.988197+0000","last_undegraded":"2024-09-17T03:35:45.988197+0000","last_fullsized":"2024-09-17T03:35:45.988197+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:14.988765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115670+0000","last_change":"2024-09-17T03:35:45.966154+0000","last_active":"2024-09-17T03:35:47.115670+0000","last_peered":"2024-09-17T03:35:47.115670+0000","last_clean":"2024-09-17T03:35:47.115670+0000","last_became_active":"2024-09-17T03:35:45.965306+0000","last_became_peered":"2024-09-17T03:35:45.965306+0000","last_unstale":"2024-09-17T03:35:47.115670+0000","last_undegraded":"2024-09-17T03:35:47.115670+0000","last_fullsized":"2024-09-17T03:35:47.115670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:33.592156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115553+0000","last_change":"2024-09-17T03:35:44.943415+0000","last_active":"2024-09-17T03:35:47.115553+0000","last_peered":"2024-09-17T03:35:47.115553+0000","last_clean":"2024-09-17T03:35:47.115553+0000","last_became_active":"2024-09-17T03:35:44.943067+0000","last_became_peered":"2024-09-17T03:35:44.943067+0000","last_unstale":"2024-09-17T03:35:47.115553+0000","last_undegraded":"2024-09-17T03:35:47.115553+0000","last_fullsized":"2024-09-17T03:35:47.115553+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:05.563987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114777+0000","last_change":"2024-09-17T03:35:45.952030+0000","last_active":"2024-09-17T03:35:47.114777+0000","last_peered":"2024-09-17T03:35:47.114777+0000","last_clean":"2024-09-17T03:35:47.114777+0000","last_became_active":"2024-09-17T03:35:45.951717+0000","last_became_peered":"2024-09-17T03:35:45.951717+0000","last_unstale":"2024-09-17T03:35:47.114777+0000","last_undegraded":"2024-09-17T03:35:47.114777+0000","last_fullsized":"2024-09-17T03:35:47.114777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:58.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115584+0000","last_change":"2024-09-17T03:35:44.952448+0000","last_active":"2024-09-17T03:35:47.115584+0000","last_peered":"2024-09-17T03:35:47.115584+0000","last_clean":"2024-09-17T03:35:47.115584+0000","last_became_active":"2024-09-17T03:35:44.952088+0000","last_became_peered":"2024-09-17T03:35:44.952088+0000","last_unstale":"2024-09-17T03:35:47.115584+0000","last_undegraded":"2024-09-17T03:35:47.115584+0000","last_fullsized":"2024-09-17T03:35:47.115584+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.522764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975135+0000","last_change":"2024-09-17T03:35:45.967590+0000","last_active":"2024-09-17T03:35:45.975135+0000","last_peered":"2024-09-17T03:35:45.975135+0000","last_clean":"2024-09-17T03:35:45.975135+0000","last_became_active":"2024-09-17T03:35:45.966946+0000","last_became_peered":"2024-09-17T03:35:45.966946+0000","last_unstale":"2024-09-17T03:35:45.975135+0000","last_undegraded":"2024-09-17T03:35:45.975135+0000","last_fullsized":"2024-09-17T03:35:45.975135+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:22.418085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"128'8","reported_seq":21,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.982528+0000","last_change":"2024-09-17T03:35:44.957549+0000","last_active":"2024-09-17T03:35:45.982528+0000","last_peered":"2024-09-17T03:35:45.982528+0000","last_clean":"2024-09-17T03:35:45.982528+0000","last_became_active":"2024-09-17T03:35:44.957170+0000","last_became_peered":"2024-09-17T03:35:44.957170+0000","last_unstale":"2024-09-17T03:35:45.982528+0000","last_undegraded":"2024-09-17T03:35:45.982528+0000","last_fullsized":"2024-09-17T03:35:45.982528+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:17.448836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977529+0000","last_change":"2024-09-17T03:35:45.972995+0000","last_active":"2024-09-17T03:35:45.977529+0000","last_peered":"2024-09-17T03:35:45.977529+0000","last_clean":"2024-09-17T03:35:45.977529+0000","last_became_active":"2024-09-17T03:35:45.972604+0000","last_became_peered":"2024-09-17T03:35:45.972604+0000","last_unstale":"2024-09-17T03:35:45.977529+0000","last_undegraded":"2024-09-17T03:35:45.977529+0000","last_fullsized":"2024-09-17T03:35:45.977529+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:25.307232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950936+0000","last_change":"2024-09-17T03:35:44.958004+0000","last_active":"2024-09-17T03:35:45.950936+0000","last_peered":"2024-09-17T03:35:45.950936+0000","last_clean":"2024-09-17T03:35:45.950936+0000","last_became_active":"2024-09-17T03:35:44.957642+0000","last_became_peered":"2024-09-17T03:35:44.957642+0000","last_unstale":"2024-09-17T03:35:45.950936+0000","last_undegraded":"2024-09-17T03:35:45.950936+0000","last_fullsized":"2024-09-17T03:35:45.950936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:31.183310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":3,"reported_epoch":128,"state":"creating+activating","last_fresh":"2024-09-17T03:35:45.947281+0000","last_change":"2024-09-17T03:35:45.947281+0000","last_active":"2024-09-17T03:35:44.919052+0000","last_peered":"2024-09-17T03:35:44.919052+0000","last_clean":"2024-09-17T03:35:44.919052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:35:45.947281+0000","last_undegraded":"2024-09-17T03:35:45.947281+0000","last_fullsized":"2024-09-17T03:35:45.947281+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942821+0000","last_change":"2024-09-17T03:35:44.954770+0000","last_active":"2024-09-17T03:35:46.942821+0000","last_peered":"2024-09-17T03:35:46.942821+0000","last_clean":"2024-09-17T03:35:46.942821+0000","last_became_active":"2024-09-17T03:35:44.953523+0000","last_became_peered":"2024-09-17T03:35:44.953523+0000","last_unstale":"2024-09-17T03:35:46.942821+0000","last_undegraded":"2024-09-17T03:35:46.942821+0000","last_fullsized":"2024-09-17T03:35:46.942821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:40.501189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945946+0000","last_change":"2024-09-17T03:35:45.950548+0000","last_active":"2024-09-17T03:35:46.945946+0000","last_peered":"2024-09-17T03:35:46.945946+0000","last_clean":"2024-09-17T03:35:46.945946+0000","last_became_active":"2024-09-17T03:35:45.949460+0000","last_became_peered":"2024-09-17T03:35:45.949460+0000","last_unstale":"2024-09-17T03:35:46.945946+0000","last_undegraded":"2024-09-17T03:35:46.945946+0000","last_fullsized":"2024-09-17T03:35:46.945946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953125+0000","last_change":"2024-09-17T03:35:44.961601+0000","last_active":"2024-09-17T03:35:46.953125+0000","last_peered":"2024-09-17T03:35:46.953125+0000","last_clean":"2024-09-17T03:35:46.953125+0000","last_became_active":"2024-09-17T03:35:44.961301+0000","last_became_peered":"2024-09-17T03:35:44.961301+0000","last_unstale":"2024-09-17T03:35:46.953125+0000","last_undegraded":"2024-09-17T03:35:46.953125+0000","last_fullsized":"2024-09-17T03:35:46.953125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:55.987785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116185+0000","last_change":"2024-09-17T03:35:45.958025+0000","last_active":"2024-09-17T03:35:47.116185+0000","last_peered":"2024-09-17T03:35:47.116185+0000","last_clean":"2024-09-17T03:35:47.116185+0000","last_became_active":"2024-09-17T03:35:45.957707+0000","last_became_peered":"2024-09-17T03:35:45.957707+0000","last_unstale":"2024-09-17T03:35:47.116185+0000","last_undegraded":"2024-09-17T03:35:47.116185+0000","last_fullsized":"2024-09-17T03:35:47.116185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.185058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950067+0000","last_change":"2024-09-17T03:35:44.961854+0000","last_active":"2024-09-17T03:35:45.950067+0000","last_peered":"2024-09-17T03:35:45.950067+0000","last_clean":"2024-09-17T03:35:45.950067+0000","last_became_active":"2024-09-17T03:35:44.961082+0000","last_became_peered":"2024-09-17T03:35:44.961082+0000","last_unstale":"2024-09-17T03:35:45.950067+0000","last_undegraded":"2024-09-17T03:35:45.950067+0000","last_fullsized":"2024-09-17T03:35:45.950067+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:42.433406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115537+0000","last_change":"2024-09-17T03:35:45.960079+0000","last_active":"2024-09-17T03:35:47.115537+0000","last_peered":"2024-09-17T03:35:47.115537+0000","last_clean":"2024-09-17T03:35:47.115537+0000","last_became_active":"2024-09-17T03:35:45.959680+0000","last_became_peered":"2024-09-17T03:35:45.959680+0000","last_unstale":"2024-09-17T03:35:47.115537+0000","last_undegraded":"2024-09-17T03:35:47.115537+0000","last_fullsized":"2024-09-17T03:35:47.115537+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:24.060327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.949362+0000","last_change":"2024-09-17T03:35:44.958405+0000","last_active":"2024-09-17T03:35:45.949362+0000","last_peered":"2024-09-17T03:35:45.949362+0000","last_clean":"2024-09-17T03:35:45.949362+0000","last_became_active":"2024-09-17T03:35:44.958102+0000","last_became_peered":"2024-09-17T03:35:44.958102+0000","last_unstale":"2024-09-17T03:35:45.949362+0000","last_undegraded":"2024-09-17T03:35:45.949362+0000","last_fullsized":"2024-09-17T03:35:45.949362+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.755778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953353+0000","last_change":"2024-09-17T03:35:45.955137+0000","last_active":"2024-09-17T03:35:46.953353+0000","last_peered":"2024-09-17T03:35:46.953353+0000","last_clean":"2024-09-17T03:35:46.953353+0000","last_became_active":"2024-09-17T03:35:45.954787+0000","last_became_peered":"2024-09-17T03:35:45.954787+0000","last_unstale":"2024-09-17T03:35:46.953353+0000","last_undegraded":"2024-09-17T03:35:46.953353+0000","last_fullsized":"2024-09-17T03:35:46.953353+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:10.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945674+0000","last_change":"2024-09-17T03:35:44.950660+0000","last_active":"2024-09-17T03:35:46.945674+0000","last_peered":"2024-09-17T03:35:46.945674+0000","last_clean":"2024-09-17T03:35:46.945674+0000","last_became_active":"2024-09-17T03:35:44.949123+0000","last_became_peered":"2024-09-17T03:35:44.949123+0000","last_unstale":"2024-09-17T03:35:46.945674+0000","last_undegraded":"2024-09-17T03:35:46.945674+0000","last_fullsized":"2024-09-17T03:35:46.945674+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114873+0000","last_change":"2024-09-17T03:35:45.956952+0000","last_active":"2024-09-17T03:35:47.114873+0000","last_peered":"2024-09-17T03:35:47.114873+0000","last_clean":"2024-09-17T03:35:47.114873+0000","last_became_active":"2024-09-17T03:35:45.956620+0000","last_became_peered":"2024-09-17T03:35:45.956620+0000","last_unstale":"2024-09-17T03:35:47.114873+0000","last_undegraded":"2024-09-17T03:35:47.114873+0000","last_fullsized":"2024-09-17T03:35:47.114873+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:11.263633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945421+0000","last_change":"2024-09-17T03:35:44.947967+0000","last_active":"2024-09-17T03:35:46.945421+0000","last_peered":"2024-09-17T03:35:46.945421+0000","last_clean":"2024-09-17T03:35:46.945421+0000","last_became_active":"2024-09-17T03:35:44.947694+0000","last_became_peered":"2024-09-17T03:35:44.947694+0000","last_unstale":"2024-09-17T03:35:46.945421+0000","last_undegraded":"2024-09-17T03:35:46.945421+0000","last_fullsized":"2024-09-17T03:35:46.945421+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:00.392947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963284+0000","last_change":"2024-09-17T03:35:45.973217+0000","last_active":"2024-09-17T03:35:46.963284+0000","last_peered":"2024-09-17T03:35:46.963284+0000","last_clean":"2024-09-17T03:35:46.963284+0000","last_became_active":"2024-09-17T03:35:45.972910+0000","last_became_peered":"2024-09-17T03:35:45.972910+0000","last_unstale":"2024-09-17T03:35:46.963284+0000","last_undegraded":"2024-09-17T03:35:46.963284+0000","last_fullsized":"2024-09-17T03:35:46.963284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942268+0000","last_change":"2024-09-17T03:35:44.947744+0000","last_active":"2024-09-17T03:35:46.942268+0000","last_peered":"2024-09-17T03:35:46.942268+0000","last_clean":"2024-09-17T03:35:46.942268+0000","last_became_active":"2024-09-17T03:35:44.947045+0000","last_became_peered":"2024-09-17T03:35:44.947045+0000","last_unstale":"2024-09-17T03:35:46.942268+0000","last_undegraded":"2024-09-17T03:35:46.942268+0000","last_fullsized":"2024-09-17T03:35:46.942268+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:56.123881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953525+0000","last_change":"2024-09-17T03:35:45.963241+0000","last_active":"2024-09-17T03:35:46.953525+0000","last_peered":"2024-09-17T03:35:46.953525+0000","last_clean":"2024-09-17T03:35:46.953525+0000","last_became_active":"2024-09-17T03:35:45.962942+0000","last_became_peered":"2024-09-17T03:35:45.962942+0000","last_unstale":"2024-09-17T03:35:46.953525+0000","last_undegraded":"2024-09-17T03:35:46.953525+0000","last_fullsized":"2024-09-17T03:35:46.953525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:47.099040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115936+0000","last_change":"2024-09-17T03:35:44.950227+0000","last_active":"2024-09-17T03:35:47.115936+0000","last_peered":"2024-09-17T03:35:47.115936+0000","last_clean":"2024-09-17T03:35:47.115936+0000","last_became_active":"2024-09-17T03:35:44.949838+0000","last_became_peered":"2024-09-17T03:35:44.949838+0000","last_unstale":"2024-09-17T03:35:47.115936+0000","last_undegraded":"2024-09-17T03:35:47.115936+0000","last_fullsized":"2024-09-17T03:35:47.115936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:38.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115830+0000","last_change":"2024-09-17T03:35:45.952300+0000","last_active":"2024-09-17T03:35:47.115830+0000","last_peered":"2024-09-17T03:35:47.115830+0000","last_clean":"2024-09-17T03:35:47.115830+0000","last_became_active":"2024-09-17T03:35:45.951940+0000","last_became_peered":"2024-09-17T03:35:45.951940+0000","last_unstale":"2024-09-17T03:35:47.115830+0000","last_undegraded":"2024-09-17T03:35:47.115830+0000","last_fullsized":"2024-09-17T03:35:47.115830+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.804587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.965242+0000","last_change":"2024-09-17T03:35:44.960044+0000","last_active":"2024-09-17T03:35:46.965242+0000","last_peered":"2024-09-17T03:35:46.965242+0000","last_clean":"2024-09-17T03:35:46.965242+0000","last_became_active":"2024-09-17T03:35:44.959679+0000","last_became_peered":"2024-09-17T03:35:44.959679+0000","last_unstale":"2024-09-17T03:35:46.965242+0000","last_undegraded":"2024-09-17T03:35:46.965242+0000","last_fullsized":"2024-09-17T03:35:46.965242+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:26.474116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116521+0000","last_change":"2024-09-17T03:35:45.957578+0000","last_active":"2024-09-17T03:35:47.116521+0000","last_peered":"2024-09-17T03:35:47.116521+0000","last_clean":"2024-09-17T03:35:47.116521+0000","last_became_active":"2024-09-17T03:35:45.956972+0000","last_became_peered":"2024-09-17T03:35:45.956972+0000","last_unstale":"2024-09-17T03:35:47.116521+0000","last_undegraded":"2024-09-17T03:35:47.116521+0000","last_fullsized":"2024-09-17T03:35:47.116521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:37.224976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116267+0000","last_change":"2024-09-17T03:35:44.953819+0000","last_active":"2024-09-17T03:35:47.116267+0000","last_peered":"2024-09-17T03:35:47.116267+0000","last_clean":"2024-09-17T03:35:47.116267+0000","last_became_active":"2024-09-17T03:35:44.951031+0000","last_became_peered":"2024-09-17T03:35:44.951031+0000","last_unstale":"2024-09-17T03:35:47.116267+0000","last_undegraded":"2024-09-17T03:35:47.116267+0000","last_fullsized":"2024-09-17T03:35:47.116267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:41.554774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953195+0000","last_change":"2024-09-17T03:35:45.962505+0000","last_active":"2024-09-17T03:35:46.953195+0000","last_peered":"2024-09-17T03:35:46.953195+0000","last_clean":"2024-09-17T03:35:46.953195+0000","last_became_active":"2024-09-17T03:35:45.962169+0000","last_became_peered":"2024-09-17T03:35:45.962169+0000","last_unstale":"2024-09-17T03:35:46.953195+0000","last_undegraded":"2024-09-17T03:35:46.953195+0000","last_fullsized":"2024-09-17T03:35:46.953195+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:20.774078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942572+0000","last_change":"2024-09-17T03:35:44.956080+0000","last_active":"2024-09-17T03:35:46.942572+0000","last_peered":"2024-09-17T03:35:46.942572+0000","last_clean":"2024-09-17T03:35:46.942572+0000","last_became_active":"2024-09-17T03:35:44.955748+0000","last_became_peered":"2024-09-17T03:35:44.955748+0000","last_unstale":"2024-09-17T03:35:46.942572+0000","last_undegraded":"2024-09-17T03:35:46.942572+0000","last_fullsized":"2024-09-17T03:35:46.942572+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:04.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963292+0000","last_change":"2024-09-17T03:35:45.963696+0000","last_active":"2024-09-17T03:35:46.963292+0000","last_peered":"2024-09-17T03:35:46.963292+0000","last_clean":"2024-09-17T03:35:46.963292+0000","last_became_active":"2024-09-17T03:35:45.963323+0000","last_became_peered":"2024-09-17T03:35:45.963323+0000","last_unstale":"2024-09-17T03:35:46.963292+0000","last_undegraded":"2024-09-17T03:35:46.963292+0000","last_fullsized":"2024-09-17T03:35:46.963292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:59.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115974+0000","last_change":"2024-09-17T03:35:44.953364+0000","last_active":"2024-09-17T03:35:47.115974+0000","last_peered":"2024-09-17T03:35:47.115974+0000","last_clean":"2024-09-17T03:35:47.115974+0000","last_became_active":"2024-09-17T03:35:44.953034+0000","last_became_peered":"2024-09-17T03:35:44.953034+0000","last_unstale":"2024-09-17T03:35:47.115974+0000","last_undegraded":"2024-09-17T03:35:47.115974+0000","last_fullsized":"2024-09-17T03:35:47.115974+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:23.777269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.966928+0000","last_change":"2024-09-17T03:35:45.963406+0000","last_active":"2024-09-17T03:35:45.966928+0000","last_peered":"2024-09-17T03:35:45.966928+0000","last_clean":"2024-09-17T03:35:45.966928+0000","last_became_active":"2024-09-17T03:35:45.963028+0000","last_became_peered":"2024-09-17T03:35:45.963028+0000","last_unstale":"2024-09-17T03:35:45.966928+0000","last_undegraded":"2024-09-17T03:35:45.966928+0000","last_fullsized":"2024-09-17T03:35:45.966928+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:26.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963480+0000","last_change":"2024-09-17T03:35:44.961829+0000","last_active":"2024-09-17T03:35:46.963480+0000","last_peered":"2024-09-17T03:35:46.963480+0000","last_clean":"2024-09-17T03:35:46.963480+0000","last_became_active":"2024-09-17T03:35:44.961479+0000","last_became_peered":"2024-09-17T03:35:44.961479+0000","last_unstale":"2024-09-17T03:35:46.963480+0000","last_undegraded":"2024-09-17T03:35:46.963480+0000","last_fullsized":"2024-09-17T03:35:46.963480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945631+0000","last_change":"2024-09-17T03:35:45.954094+0000","last_active":"2024-09-17T03:35:46.945631+0000","last_peered":"2024-09-17T03:35:46.945631+0000","last_clean":"2024-09-17T03:35:46.945631+0000","last_became_active":"2024-09-17T03:35:45.952010+0000","last_became_peered":"2024-09-17T03:35:45.952010+0000","last_unstale":"2024-09-17T03:35:46.945631+0000","last_undegraded":"2024-09-17T03:35:46.945631+0000","last_fullsized":"2024-09-17T03:35:46.945631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.979607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945521+0000","last_change":"2024-09-17T03:35:44.944467+0000","last_active":"2024-09-17T03:35:46.945521+0000","last_peered":"2024-09-17T03:35:46.945521+0000","last_clean":"2024-09-17T03:35:46.945521+0000","last_became_active":"2024-09-17T03:35:44.944155+0000","last_became_peered":"2024-09-17T03:35:44.944155+0000","last_unstale":"2024-09-17T03:35:46.945521+0000","last_undegraded":"2024-09-17T03:35:46.945521+0000","last_fullsized":"2024-09-17T03:35:46.945521+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:00.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115398+0000","last_change":"2024-09-17T03:35:45.954997+0000","last_active":"2024-09-17T03:35:47.115398+0000","last_peered":"2024-09-17T03:35:47.115398+0000","last_clean":"2024-09-17T03:35:47.115398+0000","last_became_active":"2024-09-17T03:35:45.954184+0000","last_became_peered":"2024-09-17T03:35:45.954184+0000","last_unstale":"2024-09-17T03:35:47.115398+0000","last_undegraded":"2024-09-17T03:35:47.115398+0000","last_fullsized":"2024-09-17T03:35:47.115398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.149975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942555+0000","last_change":"2024-09-17T03:35:44.948114+0000","last_active":"2024-09-17T03:35:46.942555+0000","last_peered":"2024-09-17T03:35:46.942555+0000","last_clean":"2024-09-17T03:35:46.942555+0000","last_became_active":"2024-09-17T03:35:44.947433+0000","last_became_peered":"2024-09-17T03:35:44.947433+0000","last_unstale":"2024-09-17T03:35:46.942555+0000","last_undegraded":"2024-09-17T03:35:46.942555+0000","last_fullsized":"2024-09-17T03:35:46.942555+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:22.126761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963403+0000","last_change":"2024-09-17T03:35:45.968401+0000","last_active":"2024-09-17T03:35:46.963403+0000","last_peered":"2024-09-17T03:35:46.963403+0000","last_clean":"2024-09-17T03:35:46.963403+0000","last_became_active":"2024-09-17T03:35:45.968050+0000","last_became_peered":"2024-09-17T03:35:45.968050+0000","last_unstale":"2024-09-17T03:35:46.963403+0000","last_undegraded":"2024-09-17T03:35:46.963403+0000","last_fullsized":"2024-09-17T03:35:46.963403+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963509+0000","last_change":"2024-09-17T03:35:44.963161+0000","last_active":"2024-09-17T03:35:46.963509+0000","last_peered":"2024-09-17T03:35:46.963509+0000","last_clean":"2024-09-17T03:35:46.963509+0000","last_became_active":"2024-09-17T03:35:44.962822+0000","last_became_peered":"2024-09-17T03:35:44.962822+0000","last_unstale":"2024-09-17T03:35:46.963509+0000","last_undegraded":"2024-09-17T03:35:46.963509+0000","last_fullsized":"2024-09-17T03:35:46.963509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:12.300948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":309,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.962781+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:35:46.962781+0000","last_peered":"2024-09-17T03:35:46.962781+0000","last_clean":"2024-09-17T03:35:46.962781+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:35:46.962781+0000","last_undegraded":"2024-09-17T03:35:46.962781+0000","last_fullsized":"2024-09-17T03:35:46.962781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509621,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136000,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235840,"statfs":{"total":96636764160,"available":96497500160,"internally_reserved":0,"allocated":2207744,"data_stored":1286611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145140,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226700,"statfs":{"total":96636764160,"available":96488140800,"internally_reserved":0,"allocated":1601536,"data_stored":695605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140968,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230872,"statfs":{"total":96636764160,"available":96492412928,"internally_reserved":0,"allocated":1593344,"data_stored":694063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127084,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244756,"statfs":{"total":96636764160,"available":96506630144,"internally_reserved":0,"allocated":1597440,"data_stored":694153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163180,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208660,"statfs":{"total":96636764160,"available":96469667840,"internally_reserved":0,"allocated":1597440,"data_stored":694609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145704,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226136,"statfs":{"total":96636764160,"available":96487563264,"internally_reserved":0,"allocated":2183168,"data_stored":1284361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542325,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127068,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244772,"statfs":{"total":96636764160,"available":96506646528,"internally_reserved":0,"allocated":1576960,"data_stored":687190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167340,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204500,"statfs":{"total":96636764160,"available":96465408000,"internally_reserved":0,"allocated":1589248,"data_stored":693993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:35:50.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:35:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 -- 172.21.15.46:0/3206034703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b4de0 msgr2=0x7f95940b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 --2- 172.21.15.46:0/3206034703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b4de0 0x7f95940b51f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f95940703e0 tx=0x7f958c01ccc0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 -- 172.21.15.46:0/3206034703 shutdown_connections 2024-09-17T03:35:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 --2- 172.21.15.46:0/3206034703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b4de0 0x7f95940b51f0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 --2- 172.21.15.46:0/3206034703 >> v2:172.21.15.99:3300/0 conn(0x7f95941527b0 0x7f9594074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 --2- 172.21.15.46:0/3206034703 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95940b5c70 0x7f959415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 -- 172.21.15.46:0/3206034703 >> 172.21.15.46:0/3206034703 conn(0x7f95940b2040 msgr2=0x7f95940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 -- 172.21.15.46:0/3206034703 shutdown_connections 2024-09-17T03:35:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 -- 172.21.15.46:0/3206034703 wait complete. 2024-09-17T03:35:50.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 Processor -- start 2024-09-17T03:35:50.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.345+0000 7f9599e00640 1 -- start start 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940b4de0 0x7f9594142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 0x7f959415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95941527b0 0x7f95940af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f959415f730 con 0x7f95940b5c70 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f959415f5b0 con 0x7f95940b4de0 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f959415f8b0 con 0x7f95941527b0 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 0x7f959415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9592a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940b4de0 0x7f9594142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9592a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940b4de0 0x7f9594142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:38514/0 (socket says 172.21.15.46:38514) 2024-09-17T03:35:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9592a00640 1 -- 172.21.15.46:0/3091378746 learned_addr learned my addr 172.21.15.46:0/3091378746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 0x7f959415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33560/0 (socket says 172.21.15.46:33560) 2024-09-17T03:35:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9593400640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95941527b0 0x7f95940af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 -- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95941527b0 msgr2=0x7f95940af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95941527b0 0x7f95940af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 -- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940b4de0 msgr2=0x7f9594142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940b4de0 0x7f9594142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 -- 172.21.15.46:0/3091378746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95941523f0 con 0x7f95940b5c70 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958be00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 0x7f959415a510 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9594071fd0 tx=0x7f957c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c007bb0 con 0x7f95940b5c70 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f957c007e40 con 0x7f95940b5c70 2024-09-17T03:35:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c00f6b0 con 0x7f95940b5c70 2024-09-17T03:35:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95940aa4d0 con 0x7f95940b5c70 2024-09-17T03:35:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.349+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9594158700 con 0x7f95940b5c70 2024-09-17T03:35:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.353+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9594158450 con 0x7f95940b5c70 2024-09-17T03:35:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.353+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f957c00f850 con 0x7f95940b5c70 2024-09-17T03:35:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.353+0000 7f958b400640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9570075240 0x7f9570077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.353+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f957c0938b0 con 0x7f95940b5c70 2024-09-17T03:35:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.353+0000 7f9592a00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9570075240 0x7f9570077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.353+0000 7f9592a00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9570075240 0x7f9570077750 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f958400bb90 tx=0x7f958400b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:50.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.357+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f957c014030 con 0x7f95940b5c70 2024-09-17T03:35:50.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.529+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f959406b540 con 0x7f9570075240 2024-09-17T03:35:50.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.537+0000 7f958b400640 1 -- 172.21.15.46:0/3091378746 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252337 (secure 0 0 0) 0x7f9584010bd0 con 0x7f9570075240 2024-09-17T03:35:50.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:50.539 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:35:50.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.537+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9570075240 msgr2=0x7f9570077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.537+0000 7f9599e00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9570075240 0x7f9570077750 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f958400bb90 tx=0x7f958400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.537+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 msgr2=0x7f959415a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.537+0000 7f9599e00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 0x7f959415a510 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9594071fd0 tx=0x7f957c00cdd0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 shutdown_connections 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9570075240 0x7f9570077750 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f95941527b0 0x7f95940af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940b5c70 0x7f959415a510 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 --2- 172.21.15.46:0/3091378746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940b4de0 0x7f9594142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 >> 172.21.15.46:0/3091378746 conn(0x7f95940b2040 msgr2=0x7f959406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 shutdown_connections 2024-09-17T03:35:50.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.541+0000 7f9599e00640 1 -- 172.21.15.46:0/3091378746 wait complete. 2024-09-17T03:35:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":569,"stamp":"2024-09-17T03:35:49.561285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":140,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1152484,"kb_used_data":13620,"kb_used_omap":12,"kb_used_meta":1138675,"kb_avail":753822236,"statfs":{"total":773094113280,"available":771913969664,"internally_reserved":0,"allocated":13946880,"data_stored":6730585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1166003760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.642018"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116128+0000","last_change":"2024-09-17T03:35:45.955819+0000","last_active":"2024-09-17T03:35:47.116128+0000","last_peered":"2024-09-17T03:35:47.116128+0000","last_clean":"2024-09-17T03:35:47.116128+0000","last_became_active":"2024-09-17T03:35:45.954935+0000","last_became_peered":"2024-09-17T03:35:45.954935+0000","last_unstale":"2024-09-17T03:35:47.116128+0000","last_undegraded":"2024-09-17T03:35:47.116128+0000","last_fullsized":"2024-09-17T03:35:47.116128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.363688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116488+0000","last_change":"2024-09-17T03:35:45.945240+0000","last_active":"2024-09-17T03:35:47.116488+0000","last_peered":"2024-09-17T03:35:47.116488+0000","last_clean":"2024-09-17T03:35:47.116488+0000","last_became_active":"2024-09-17T03:35:45.944788+0000","last_became_peered":"2024-09-17T03:35:45.944788+0000","last_unstale":"2024-09-17T03:35:47.116488+0000","last_undegraded":"2024-09-17T03:35:47.116488+0000","last_fullsized":"2024-09-17T03:35:47.116488+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:02.551861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.973644+0000","last_change":"2024-09-17T03:35:45.965545+0000","last_active":"2024-09-17T03:35:45.973644+0000","last_peered":"2024-09-17T03:35:45.973644+0000","last_clean":"2024-09-17T03:35:45.973644+0000","last_became_active":"2024-09-17T03:35:45.961480+0000","last_became_peered":"2024-09-17T03:35:45.961480+0000","last_unstale":"2024-09-17T03:35:45.973644+0000","last_undegraded":"2024-09-17T03:35:45.973644+0000","last_fullsized":"2024-09-17T03:35:45.973644+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:35.059698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115130+0000","last_change":"2024-09-17T03:35:45.960750+0000","last_active":"2024-09-17T03:35:47.115130+0000","last_peered":"2024-09-17T03:35:47.115130+0000","last_clean":"2024-09-17T03:35:47.115130+0000","last_became_active":"2024-09-17T03:35:45.960431+0000","last_became_peered":"2024-09-17T03:35:45.960431+0000","last_unstale":"2024-09-17T03:35:47.115130+0000","last_undegraded":"2024-09-17T03:35:47.115130+0000","last_fullsized":"2024-09-17T03:35:47.115130+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:59.838096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963365+0000","last_change":"2024-09-17T03:35:45.971487+0000","last_active":"2024-09-17T03:35:46.963365+0000","last_peered":"2024-09-17T03:35:46.963365+0000","last_clean":"2024-09-17T03:35:46.963365+0000","last_became_active":"2024-09-17T03:35:45.970309+0000","last_became_peered":"2024-09-17T03:35:45.970309+0000","last_unstale":"2024-09-17T03:35:46.963365+0000","last_undegraded":"2024-09-17T03:35:46.963365+0000","last_fullsized":"2024-09-17T03:35:46.963365+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953008+0000","last_change":"2024-09-17T03:35:45.961513+0000","last_active":"2024-09-17T03:35:46.953008+0000","last_peered":"2024-09-17T03:35:46.953008+0000","last_clean":"2024-09-17T03:35:46.953008+0000","last_became_active":"2024-09-17T03:35:45.961192+0000","last_became_peered":"2024-09-17T03:35:45.961192+0000","last_unstale":"2024-09-17T03:35:46.953008+0000","last_undegraded":"2024-09-17T03:35:46.953008+0000","last_fullsized":"2024-09-17T03:35:46.953008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:45.692040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952824+0000","last_change":"2024-09-17T03:35:45.955196+0000","last_active":"2024-09-17T03:35:46.952824+0000","last_peered":"2024-09-17T03:35:46.952824+0000","last_clean":"2024-09-17T03:35:46.952824+0000","last_became_active":"2024-09-17T03:35:45.954869+0000","last_became_peered":"2024-09-17T03:35:45.954869+0000","last_unstale":"2024-09-17T03:35:46.952824+0000","last_undegraded":"2024-09-17T03:35:46.952824+0000","last_fullsized":"2024-09-17T03:35:46.952824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:27.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978568+0000","last_change":"2024-09-17T03:35:45.968695+0000","last_active":"2024-09-17T03:35:45.978568+0000","last_peered":"2024-09-17T03:35:45.978568+0000","last_clean":"2024-09-17T03:35:45.978568+0000","last_became_active":"2024-09-17T03:35:45.966016+0000","last_became_peered":"2024-09-17T03:35:45.966016+0000","last_unstale":"2024-09-17T03:35:45.978568+0000","last_undegraded":"2024-09-17T03:35:45.978568+0000","last_fullsized":"2024-09-17T03:35:45.978568+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:29.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975084+0000","last_change":"2024-09-17T03:35:45.969067+0000","last_active":"2024-09-17T03:35:45.975084+0000","last_peered":"2024-09-17T03:35:45.975084+0000","last_clean":"2024-09-17T03:35:45.975084+0000","last_became_active":"2024-09-17T03:35:45.968667+0000","last_became_peered":"2024-09-17T03:35:45.968667+0000","last_unstale":"2024-09-17T03:35:45.975084+0000","last_undegraded":"2024-09-17T03:35:45.975084+0000","last_fullsized":"2024-09-17T03:35:45.975084+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:05:08.235660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116451+0000","last_change":"2024-09-17T03:35:45.945553+0000","last_active":"2024-09-17T03:35:47.116451+0000","last_peered":"2024-09-17T03:35:47.116451+0000","last_clean":"2024-09-17T03:35:47.116451+0000","last_became_active":"2024-09-17T03:35:45.945233+0000","last_became_peered":"2024-09-17T03:35:45.945233+0000","last_unstale":"2024-09-17T03:35:47.116451+0000","last_undegraded":"2024-09-17T03:35:47.116451+0000","last_fullsized":"2024-09-17T03:35:47.116451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:52.862294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942689+0000","last_change":"2024-09-17T03:35:45.972155+0000","last_active":"2024-09-17T03:35:46.942689+0000","last_peered":"2024-09-17T03:35:46.942689+0000","last_clean":"2024-09-17T03:35:46.942689+0000","last_became_active":"2024-09-17T03:35:45.971845+0000","last_became_peered":"2024-09-17T03:35:45.971845+0000","last_unstale":"2024-09-17T03:35:46.942689+0000","last_undegraded":"2024-09-17T03:35:46.942689+0000","last_fullsized":"2024-09-17T03:35:46.942689+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:13.103673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977386+0000","last_change":"2024-09-17T03:35:45.965493+0000","last_active":"2024-09-17T03:35:45.977386+0000","last_peered":"2024-09-17T03:35:45.977386+0000","last_clean":"2024-09-17T03:35:45.977386+0000","last_became_active":"2024-09-17T03:35:45.962965+0000","last_became_peered":"2024-09-17T03:35:45.962965+0000","last_unstale":"2024-09-17T03:35:45.977386+0000","last_undegraded":"2024-09-17T03:35:45.977386+0000","last_fullsized":"2024-09-17T03:35:45.977386+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:59.517858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115407+0000","last_change":"2024-09-17T03:35:45.957732+0000","last_active":"2024-09-17T03:35:47.115407+0000","last_peered":"2024-09-17T03:35:47.115407+0000","last_clean":"2024-09-17T03:35:47.115407+0000","last_became_active":"2024-09-17T03:35:45.957405+0000","last_became_peered":"2024-09-17T03:35:45.957405+0000","last_unstale":"2024-09-17T03:35:47.115407+0000","last_undegraded":"2024-09-17T03:35:47.115407+0000","last_fullsized":"2024-09-17T03:35:47.115407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:23.451219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115934+0000","last_change":"2024-09-17T03:35:45.952592+0000","last_active":"2024-09-17T03:35:47.115934+0000","last_peered":"2024-09-17T03:35:47.115934+0000","last_clean":"2024-09-17T03:35:47.115934+0000","last_became_active":"2024-09-17T03:35:45.949816+0000","last_became_peered":"2024-09-17T03:35:45.949816+0000","last_unstale":"2024-09-17T03:35:47.115934+0000","last_undegraded":"2024-09-17T03:35:47.115934+0000","last_fullsized":"2024-09-17T03:35:47.115934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:16.316918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114925+0000","last_change":"2024-09-17T03:35:45.958139+0000","last_active":"2024-09-17T03:35:47.114925+0000","last_peered":"2024-09-17T03:35:47.114925+0000","last_clean":"2024-09-17T03:35:47.114925+0000","last_became_active":"2024-09-17T03:35:45.957813+0000","last_became_peered":"2024-09-17T03:35:45.957813+0000","last_unstale":"2024-09-17T03:35:47.114925+0000","last_undegraded":"2024-09-17T03:35:47.114925+0000","last_fullsized":"2024-09-17T03:35:47.114925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:28.891920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945605+0000","last_change":"2024-09-17T03:35:45.950827+0000","last_active":"2024-09-17T03:35:46.945605+0000","last_peered":"2024-09-17T03:35:46.945605+0000","last_clean":"2024-09-17T03:35:46.945605+0000","last_became_active":"2024-09-17T03:35:45.950514+0000","last_became_peered":"2024-09-17T03:35:45.950514+0000","last_unstale":"2024-09-17T03:35:46.945605+0000","last_undegraded":"2024-09-17T03:35:46.945605+0000","last_fullsized":"2024-09-17T03:35:46.945605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:52.381399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953466+0000","last_change":"2024-09-17T03:35:45.964355+0000","last_active":"2024-09-17T03:35:46.953466+0000","last_peered":"2024-09-17T03:35:46.953466+0000","last_clean":"2024-09-17T03:35:46.953466+0000","last_became_active":"2024-09-17T03:35:45.962165+0000","last_became_peered":"2024-09-17T03:35:45.962165+0000","last_unstale":"2024-09-17T03:35:46.953466+0000","last_undegraded":"2024-09-17T03:35:46.953466+0000","last_fullsized":"2024-09-17T03:35:46.953466+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:26.092966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946215+0000","last_change":"2024-09-17T03:35:45.955274+0000","last_active":"2024-09-17T03:35:46.946215+0000","last_peered":"2024-09-17T03:35:46.946215+0000","last_clean":"2024-09-17T03:35:46.946215+0000","last_became_active":"2024-09-17T03:35:45.953802+0000","last_became_peered":"2024-09-17T03:35:45.953802+0000","last_unstale":"2024-09-17T03:35:46.946215+0000","last_undegraded":"2024-09-17T03:35:46.946215+0000","last_fullsized":"2024-09-17T03:35:46.946215+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942937+0000","last_change":"2024-09-17T03:35:45.963307+0000","last_active":"2024-09-17T03:35:46.942937+0000","last_peered":"2024-09-17T03:35:46.942937+0000","last_clean":"2024-09-17T03:35:46.942937+0000","last_became_active":"2024-09-17T03:35:45.963007+0000","last_became_peered":"2024-09-17T03:35:45.963007+0000","last_unstale":"2024-09-17T03:35:46.942937+0000","last_undegraded":"2024-09-17T03:35:46.942937+0000","last_fullsized":"2024-09-17T03:35:46.942937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:09.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945919+0000","last_change":"2024-09-17T03:35:45.950245+0000","last_active":"2024-09-17T03:35:46.945919+0000","last_peered":"2024-09-17T03:35:46.945919+0000","last_clean":"2024-09-17T03:35:46.945919+0000","last_became_active":"2024-09-17T03:35:45.949852+0000","last_became_peered":"2024-09-17T03:35:45.949852+0000","last_unstale":"2024-09-17T03:35:46.945919+0000","last_undegraded":"2024-09-17T03:35:46.945919+0000","last_fullsized":"2024-09-17T03:35:46.945919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:09.687961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963523+0000","last_change":"2024-09-17T03:35:45.967983+0000","last_active":"2024-09-17T03:35:46.963523+0000","last_peered":"2024-09-17T03:35:46.963523+0000","last_clean":"2024-09-17T03:35:46.963523+0000","last_became_active":"2024-09-17T03:35:45.964201+0000","last_became_peered":"2024-09-17T03:35:45.964201+0000","last_unstale":"2024-09-17T03:35:46.963523+0000","last_undegraded":"2024-09-17T03:35:46.963523+0000","last_fullsized":"2024-09-17T03:35:46.963523+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:26.825390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942483+0000","last_change":"2024-09-17T03:35:45.969671+0000","last_active":"2024-09-17T03:35:46.942483+0000","last_peered":"2024-09-17T03:35:46.942483+0000","last_clean":"2024-09-17T03:35:46.942483+0000","last_became_active":"2024-09-17T03:35:45.969304+0000","last_became_peered":"2024-09-17T03:35:45.969304+0000","last_unstale":"2024-09-17T03:35:46.942483+0000","last_undegraded":"2024-09-17T03:35:46.942483+0000","last_fullsized":"2024-09-17T03:35:46.942483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:30.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.972932+0000","last_change":"2024-09-17T03:35:45.962436+0000","last_active":"2024-09-17T03:35:45.972932+0000","last_peered":"2024-09-17T03:35:45.972932+0000","last_clean":"2024-09-17T03:35:45.972932+0000","last_became_active":"2024-09-17T03:35:45.962092+0000","last_became_peered":"2024-09-17T03:35:45.962092+0000","last_unstale":"2024-09-17T03:35:45.972932+0000","last_undegraded":"2024-09-17T03:35:45.972932+0000","last_fullsized":"2024-09-17T03:35:45.972932+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:33.151389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942550+0000","last_change":"2024-09-17T03:35:45.968660+0000","last_active":"2024-09-17T03:35:46.942550+0000","last_peered":"2024-09-17T03:35:46.942550+0000","last_clean":"2024-09-17T03:35:46.942550+0000","last_became_active":"2024-09-17T03:35:45.968290+0000","last_became_peered":"2024-09-17T03:35:45.968290+0000","last_unstale":"2024-09-17T03:35:46.942550+0000","last_undegraded":"2024-09-17T03:35:46.942550+0000","last_fullsized":"2024-09-17T03:35:46.942550+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:23.605758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116395+0000","last_change":"2024-09-17T03:35:45.950767+0000","last_active":"2024-09-17T03:35:47.116395+0000","last_peered":"2024-09-17T03:35:47.116395+0000","last_clean":"2024-09-17T03:35:47.116395+0000","last_became_active":"2024-09-17T03:35:45.950459+0000","last_became_peered":"2024-09-17T03:35:45.950459+0000","last_unstale":"2024-09-17T03:35:47.116395+0000","last_undegraded":"2024-09-17T03:35:47.116395+0000","last_fullsized":"2024-09-17T03:35:47.116395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:00.727158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946018+0000","last_change":"2024-09-17T03:35:45.951322+0000","last_active":"2024-09-17T03:35:46.946018+0000","last_peered":"2024-09-17T03:35:46.946018+0000","last_clean":"2024-09-17T03:35:46.946018+0000","last_became_active":"2024-09-17T03:35:45.950888+0000","last_became_peered":"2024-09-17T03:35:45.950888+0000","last_unstale":"2024-09-17T03:35:46.946018+0000","last_undegraded":"2024-09-17T03:35:46.946018+0000","last_fullsized":"2024-09-17T03:35:46.946018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115304+0000","last_change":"2024-09-17T03:35:45.958772+0000","last_active":"2024-09-17T03:35:47.115304+0000","last_peered":"2024-09-17T03:35:47.115304+0000","last_clean":"2024-09-17T03:35:47.115304+0000","last_became_active":"2024-09-17T03:35:45.958227+0000","last_became_peered":"2024-09-17T03:35:45.958227+0000","last_unstale":"2024-09-17T03:35:47.115304+0000","last_undegraded":"2024-09-17T03:35:47.115304+0000","last_fullsized":"2024-09-17T03:35:47.115304+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:36.638860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":3,"reported_epoch":128,"state":"creating+activating","last_fresh":"2024-09-17T03:35:45.948224+0000","last_change":"2024-09-17T03:35:45.948224+0000","last_active":"2024-09-17T03:35:44.919052+0000","last_peered":"2024-09-17T03:35:44.919052+0000","last_clean":"2024-09-17T03:35:44.919052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:35:45.948224+0000","last_undegraded":"2024-09-17T03:35:45.948224+0000","last_fullsized":"2024-09-17T03:35:45.948224+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115694+0000","last_change":"2024-09-17T03:35:45.957427+0000","last_active":"2024-09-17T03:35:47.115694+0000","last_peered":"2024-09-17T03:35:47.115694+0000","last_clean":"2024-09-17T03:35:47.115694+0000","last_became_active":"2024-09-17T03:35:45.956963+0000","last_became_peered":"2024-09-17T03:35:45.956963+0000","last_unstale":"2024-09-17T03:35:47.115694+0000","last_undegraded":"2024-09-17T03:35:47.115694+0000","last_fullsized":"2024-09-17T03:35:47.115694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.736866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115434+0000","last_change":"2024-09-17T03:35:45.959532+0000","last_active":"2024-09-17T03:35:47.115434+0000","last_peered":"2024-09-17T03:35:47.115434+0000","last_clean":"2024-09-17T03:35:47.115434+0000","last_became_active":"2024-09-17T03:35:45.952528+0000","last_became_peered":"2024-09-17T03:35:45.952528+0000","last_unstale":"2024-09-17T03:35:47.115434+0000","last_undegraded":"2024-09-17T03:35:47.115434+0000","last_fullsized":"2024-09-17T03:35:47.115434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:49.407151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963303+0000","last_change":"2024-09-17T03:35:45.975662+0000","last_active":"2024-09-17T03:35:46.963303+0000","last_peered":"2024-09-17T03:35:46.963303+0000","last_clean":"2024-09-17T03:35:46.963303+0000","last_became_active":"2024-09-17T03:35:45.975295+0000","last_became_peered":"2024-09-17T03:35:45.975295+0000","last_unstale":"2024-09-17T03:35:46.963303+0000","last_undegraded":"2024-09-17T03:35:46.963303+0000","last_fullsized":"2024-09-17T03:35:46.963303+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952833+0000","last_change":"2024-09-17T03:35:45.947162+0000","last_active":"2024-09-17T03:35:46.952833+0000","last_peered":"2024-09-17T03:35:46.952833+0000","last_clean":"2024-09-17T03:35:46.952833+0000","last_became_active":"2024-09-17T03:35:45.946769+0000","last_became_peered":"2024-09-17T03:35:45.946769+0000","last_unstale":"2024-09-17T03:35:46.952833+0000","last_undegraded":"2024-09-17T03:35:46.952833+0000","last_fullsized":"2024-09-17T03:35:46.952833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:12.153774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946273+0000","last_change":"2024-09-17T03:35:45.951693+0000","last_active":"2024-09-17T03:35:46.946273+0000","last_peered":"2024-09-17T03:35:46.946273+0000","last_clean":"2024-09-17T03:35:46.946273+0000","last_became_active":"2024-09-17T03:35:45.951350+0000","last_became_peered":"2024-09-17T03:35:45.951350+0000","last_unstale":"2024-09-17T03:35:46.946273+0000","last_undegraded":"2024-09-17T03:35:46.946273+0000","last_fullsized":"2024-09-17T03:35:46.946273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963772+0000","last_change":"2024-09-17T03:35:44.961465+0000","last_active":"2024-09-17T03:35:46.963772+0000","last_peered":"2024-09-17T03:35:46.963772+0000","last_clean":"2024-09-17T03:35:46.963772+0000","last_became_active":"2024-09-17T03:35:44.961034+0000","last_became_peered":"2024-09-17T03:35:44.961034+0000","last_unstale":"2024-09-17T03:35:46.963772+0000","last_undegraded":"2024-09-17T03:35:46.963772+0000","last_fullsized":"2024-09-17T03:35:46.963772+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.962934+0000","last_change":"2024-09-17T03:35:45.959757+0000","last_active":"2024-09-17T03:35:45.962934+0000","last_peered":"2024-09-17T03:35:45.962934+0000","last_clean":"2024-09-17T03:35:45.962934+0000","last_became_active":"2024-09-17T03:35:45.959088+0000","last_became_peered":"2024-09-17T03:35:45.959088+0000","last_unstale":"2024-09-17T03:35:45.962934+0000","last_undegraded":"2024-09-17T03:35:45.962934+0000","last_fullsized":"2024-09-17T03:35:45.962934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:09.348110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950367+0000","last_change":"2024-09-17T03:35:44.962725+0000","last_active":"2024-09-17T03:35:45.950367+0000","last_peered":"2024-09-17T03:35:45.950367+0000","last_clean":"2024-09-17T03:35:45.950367+0000","last_became_active":"2024-09-17T03:35:44.962372+0000","last_became_peered":"2024-09-17T03:35:44.962372+0000","last_unstale":"2024-09-17T03:35:45.950367+0000","last_undegraded":"2024-09-17T03:35:45.950367+0000","last_fullsized":"2024-09-17T03:35:45.950367+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:27.452690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963667+0000","last_change":"2024-09-17T03:35:45.971272+0000","last_active":"2024-09-17T03:35:46.963667+0000","last_peered":"2024-09-17T03:35:46.963667+0000","last_clean":"2024-09-17T03:35:46.963667+0000","last_became_active":"2024-09-17T03:35:45.967588+0000","last_became_peered":"2024-09-17T03:35:45.967588+0000","last_unstale":"2024-09-17T03:35:46.963667+0000","last_undegraded":"2024-09-17T03:35:46.963667+0000","last_fullsized":"2024-09-17T03:35:46.963667+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:57.132363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942929+0000","last_change":"2024-09-17T03:35:44.955317+0000","last_active":"2024-09-17T03:35:46.942929+0000","last_peered":"2024-09-17T03:35:46.942929+0000","last_clean":"2024-09-17T03:35:46.942929+0000","last_became_active":"2024-09-17T03:35:44.952983+0000","last_became_peered":"2024-09-17T03:35:44.952983+0000","last_unstale":"2024-09-17T03:35:46.942929+0000","last_undegraded":"2024-09-17T03:35:46.942929+0000","last_fullsized":"2024-09-17T03:35:46.942929+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:52.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963619+0000","last_change":"2024-09-17T03:35:45.976105+0000","last_active":"2024-09-17T03:35:46.963619+0000","last_peered":"2024-09-17T03:35:46.963619+0000","last_clean":"2024-09-17T03:35:46.963619+0000","last_became_active":"2024-09-17T03:35:45.975807+0000","last_became_peered":"2024-09-17T03:35:45.975807+0000","last_unstale":"2024-09-17T03:35:46.963619+0000","last_undegraded":"2024-09-17T03:35:46.963619+0000","last_fullsized":"2024-09-17T03:35:46.963619+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:26.601742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116356+0000","last_change":"2024-09-17T03:35:44.959733+0000","last_active":"2024-09-17T03:35:47.116356+0000","last_peered":"2024-09-17T03:35:47.116356+0000","last_clean":"2024-09-17T03:35:47.116356+0000","last_became_active":"2024-09-17T03:35:44.959330+0000","last_became_peered":"2024-09-17T03:35:44.959330+0000","last_unstale":"2024-09-17T03:35:47.116356+0000","last_undegraded":"2024-09-17T03:35:47.116356+0000","last_fullsized":"2024-09-17T03:35:47.116356+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:34.223217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945544+0000","last_change":"2024-09-17T03:35:45.945250+0000","last_active":"2024-09-17T03:35:46.945544+0000","last_peered":"2024-09-17T03:35:46.945544+0000","last_clean":"2024-09-17T03:35:46.945544+0000","last_became_active":"2024-09-17T03:35:45.944882+0000","last_became_peered":"2024-09-17T03:35:45.944882+0000","last_unstale":"2024-09-17T03:35:46.945544+0000","last_undegraded":"2024-09-17T03:35:46.945544+0000","last_fullsized":"2024-09-17T03:35:46.945544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:03.455251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115803+0000","last_change":"2024-09-17T03:35:44.957196+0000","last_active":"2024-09-17T03:35:47.115803+0000","last_peered":"2024-09-17T03:35:47.115803+0000","last_clean":"2024-09-17T03:35:47.115803+0000","last_became_active":"2024-09-17T03:35:44.956846+0000","last_became_peered":"2024-09-17T03:35:44.956846+0000","last_unstale":"2024-09-17T03:35:47.115803+0000","last_undegraded":"2024-09-17T03:35:47.115803+0000","last_fullsized":"2024-09-17T03:35:47.115803+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:28.310407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953166+0000","last_change":"2024-09-17T03:35:45.975695+0000","last_active":"2024-09-17T03:35:46.953166+0000","last_peered":"2024-09-17T03:35:46.953166+0000","last_clean":"2024-09-17T03:35:46.953166+0000","last_became_active":"2024-09-17T03:35:45.975349+0000","last_became_peered":"2024-09-17T03:35:45.975349+0000","last_unstale":"2024-09-17T03:35:46.953166+0000","last_undegraded":"2024-09-17T03:35:46.953166+0000","last_fullsized":"2024-09-17T03:35:46.953166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:24.952949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945624+0000","last_change":"2024-09-17T03:35:44.952496+0000","last_active":"2024-09-17T03:35:46.945624+0000","last_peered":"2024-09-17T03:35:46.945624+0000","last_clean":"2024-09-17T03:35:46.945624+0000","last_became_active":"2024-09-17T03:35:44.952172+0000","last_became_peered":"2024-09-17T03:35:44.952172+0000","last_unstale":"2024-09-17T03:35:46.945624+0000","last_undegraded":"2024-09-17T03:35:46.945624+0000","last_fullsized":"2024-09-17T03:35:46.945624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.251927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978605+0000","last_change":"2024-09-17T03:35:45.970502+0000","last_active":"2024-09-17T03:35:45.978605+0000","last_peered":"2024-09-17T03:35:45.978605+0000","last_clean":"2024-09-17T03:35:45.978605+0000","last_became_active":"2024-09-17T03:35:45.967366+0000","last_became_peered":"2024-09-17T03:35:45.967366+0000","last_unstale":"2024-09-17T03:35:45.978605+0000","last_undegraded":"2024-09-17T03:35:45.978605+0000","last_fullsized":"2024-09-17T03:35:45.978605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:38.209115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952670+0000","last_change":"2024-09-17T03:35:44.958325+0000","last_active":"2024-09-17T03:35:46.952670+0000","last_peered":"2024-09-17T03:35:46.952670+0000","last_clean":"2024-09-17T03:35:46.952670+0000","last_became_active":"2024-09-17T03:35:44.958015+0000","last_became_peered":"2024-09-17T03:35:44.958015+0000","last_unstale":"2024-09-17T03:35:46.952670+0000","last_undegraded":"2024-09-17T03:35:46.952670+0000","last_fullsized":"2024-09-17T03:35:46.952670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:32.736499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114786+0000","last_change":"2024-09-17T03:35:45.955593+0000","last_active":"2024-09-17T03:35:47.114786+0000","last_peered":"2024-09-17T03:35:47.114786+0000","last_clean":"2024-09-17T03:35:47.114786+0000","last_became_active":"2024-09-17T03:35:45.954658+0000","last_became_peered":"2024-09-17T03:35:45.954658+0000","last_unstale":"2024-09-17T03:35:47.114786+0000","last_undegraded":"2024-09-17T03:35:47.114786+0000","last_fullsized":"2024-09-17T03:35:47.114786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114901+0000","last_change":"2024-09-17T03:35:44.951328+0000","last_active":"2024-09-17T03:35:47.114901+0000","last_peered":"2024-09-17T03:35:47.114901+0000","last_clean":"2024-09-17T03:35:47.114901+0000","last_became_active":"2024-09-17T03:35:44.951004+0000","last_became_peered":"2024-09-17T03:35:44.951004+0000","last_unstale":"2024-09-17T03:35:47.114901+0000","last_undegraded":"2024-09-17T03:35:47.114901+0000","last_fullsized":"2024-09-17T03:35:47.114901+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:57.623042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115774+0000","last_change":"2024-09-17T03:35:45.963042+0000","last_active":"2024-09-17T03:35:47.115774+0000","last_peered":"2024-09-17T03:35:47.115774+0000","last_clean":"2024-09-17T03:35:47.115774+0000","last_became_active":"2024-09-17T03:35:45.962752+0000","last_became_peered":"2024-09-17T03:35:45.962752+0000","last_unstale":"2024-09-17T03:35:47.115774+0000","last_undegraded":"2024-09-17T03:35:47.115774+0000","last_fullsized":"2024-09-17T03:35:47.115774+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:51.335684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950811+0000","last_change":"2024-09-17T03:35:44.962808+0000","last_active":"2024-09-17T03:35:45.950811+0000","last_peered":"2024-09-17T03:35:45.950811+0000","last_clean":"2024-09-17T03:35:45.950811+0000","last_became_active":"2024-09-17T03:35:44.962473+0000","last_became_peered":"2024-09-17T03:35:44.962473+0000","last_unstale":"2024-09-17T03:35:45.950811+0000","last_undegraded":"2024-09-17T03:35:45.950811+0000","last_fullsized":"2024-09-17T03:35:45.950811+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:59.547888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.967805+0000","last_change":"2024-09-17T03:35:45.963711+0000","last_active":"2024-09-17T03:35:45.967805+0000","last_peered":"2024-09-17T03:35:45.967805+0000","last_clean":"2024-09-17T03:35:45.967805+0000","last_became_active":"2024-09-17T03:35:45.963372+0000","last_became_peered":"2024-09-17T03:35:45.963372+0000","last_unstale":"2024-09-17T03:35:45.967805+0000","last_undegraded":"2024-09-17T03:35:45.967805+0000","last_fullsized":"2024-09-17T03:35:45.967805+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:56.111252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946333+0000","last_change":"2024-09-17T03:35:44.946163+0000","last_active":"2024-09-17T03:35:46.946333+0000","last_peered":"2024-09-17T03:35:46.946333+0000","last_clean":"2024-09-17T03:35:46.946333+0000","last_became_active":"2024-09-17T03:35:44.945021+0000","last_became_peered":"2024-09-17T03:35:44.945021+0000","last_unstale":"2024-09-17T03:35:46.946333+0000","last_undegraded":"2024-09-17T03:35:46.946333+0000","last_fullsized":"2024-09-17T03:35:46.946333+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963634+0000","last_change":"2024-09-17T03:35:45.972309+0000","last_active":"2024-09-17T03:35:46.963634+0000","last_peered":"2024-09-17T03:35:46.963634+0000","last_clean":"2024-09-17T03:35:46.963634+0000","last_became_active":"2024-09-17T03:35:45.971924+0000","last_became_peered":"2024-09-17T03:35:45.971924+0000","last_unstale":"2024-09-17T03:35:46.963634+0000","last_undegraded":"2024-09-17T03:35:46.963634+0000","last_fullsized":"2024-09-17T03:35:46.963634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:53.929773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.949681+0000","last_change":"2024-09-17T03:35:44.961523+0000","last_active":"2024-09-17T03:35:45.949681+0000","last_peered":"2024-09-17T03:35:45.949681+0000","last_clean":"2024-09-17T03:35:45.949681+0000","last_became_active":"2024-09-17T03:35:44.960356+0000","last_became_peered":"2024-09-17T03:35:44.960356+0000","last_unstale":"2024-09-17T03:35:45.949681+0000","last_undegraded":"2024-09-17T03:35:45.949681+0000","last_fullsized":"2024-09-17T03:35:45.949681+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.980673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963629+0000","last_change":"2024-09-17T03:35:45.964975+0000","last_active":"2024-09-17T03:35:46.963629+0000","last_peered":"2024-09-17T03:35:46.963629+0000","last_clean":"2024-09-17T03:35:46.963629+0000","last_became_active":"2024-09-17T03:35:45.964625+0000","last_became_peered":"2024-09-17T03:35:45.964625+0000","last_unstale":"2024-09-17T03:35:46.963629+0000","last_undegraded":"2024-09-17T03:35:46.963629+0000","last_fullsized":"2024-09-17T03:35:46.963629+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:14.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"128'6","reported_seq":19,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.988197+0000","last_change":"2024-09-17T03:35:44.957186+0000","last_active":"2024-09-17T03:35:45.988197+0000","last_peered":"2024-09-17T03:35:45.988197+0000","last_clean":"2024-09-17T03:35:45.988197+0000","last_became_active":"2024-09-17T03:35:44.955711+0000","last_became_peered":"2024-09-17T03:35:44.955711+0000","last_unstale":"2024-09-17T03:35:45.988197+0000","last_undegraded":"2024-09-17T03:35:45.988197+0000","last_fullsized":"2024-09-17T03:35:45.988197+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:14.988765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115670+0000","last_change":"2024-09-17T03:35:45.966154+0000","last_active":"2024-09-17T03:35:47.115670+0000","last_peered":"2024-09-17T03:35:47.115670+0000","last_clean":"2024-09-17T03:35:47.115670+0000","last_became_active":"2024-09-17T03:35:45.965306+0000","last_became_peered":"2024-09-17T03:35:45.965306+0000","last_unstale":"2024-09-17T03:35:47.115670+0000","last_undegraded":"2024-09-17T03:35:47.115670+0000","last_fullsized":"2024-09-17T03:35:47.115670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:33.592156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115553+0000","last_change":"2024-09-17T03:35:44.943415+0000","last_active":"2024-09-17T03:35:47.115553+0000","last_peered":"2024-09-17T03:35:47.115553+0000","last_clean":"2024-09-17T03:35:47.115553+0000","last_became_active":"2024-09-17T03:35:44.943067+0000","last_became_peered":"2024-09-17T03:35:44.943067+0000","last_unstale":"2024-09-17T03:35:47.115553+0000","last_undegraded":"2024-09-17T03:35:47.115553+0000","last_fullsized":"2024-09-17T03:35:47.115553+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:05.563987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114777+0000","last_change":"2024-09-17T03:35:45.952030+0000","last_active":"2024-09-17T03:35:47.114777+0000","last_peered":"2024-09-17T03:35:47.114777+0000","last_clean":"2024-09-17T03:35:47.114777+0000","last_became_active":"2024-09-17T03:35:45.951717+0000","last_became_peered":"2024-09-17T03:35:45.951717+0000","last_unstale":"2024-09-17T03:35:47.114777+0000","last_undegraded":"2024-09-17T03:35:47.114777+0000","last_fullsized":"2024-09-17T03:35:47.114777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:58.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115584+0000","last_change":"2024-09-17T03:35:44.952448+0000","last_active":"2024-09-17T03:35:47.115584+0000","last_peered":"2024-09-17T03:35:47.115584+0000","last_clean":"2024-09-17T03:35:47.115584+0000","last_became_active":"2024-09-17T03:35:44.952088+0000","last_became_peered":"2024-09-17T03:35:44.952088+0000","last_unstale":"2024-09-17T03:35:47.115584+0000","last_undegraded":"2024-09-17T03:35:47.115584+0000","last_fullsized":"2024-09-17T03:35:47.115584+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.522764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975135+0000","last_change":"2024-09-17T03:35:45.967590+0000","last_active":"2024-09-17T03:35:45.975135+0000","last_peered":"2024-09-17T03:35:45.975135+0000","last_clean":"2024-09-17T03:35:45.975135+0000","last_became_active":"2024-09-17T03:35:45.966946+0000","last_became_peered":"2024-09-17T03:35:45.966946+0000","last_unstale":"2024-09-17T03:35:45.975135+0000","last_undegraded":"2024-09-17T03:35:45.975135+0000","last_fullsized":"2024-09-17T03:35:45.975135+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:22.418085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"128'8","reported_seq":21,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.982528+0000","last_change":"2024-09-17T03:35:44.957549+0000","last_active":"2024-09-17T03:35:45.982528+0000","last_peered":"2024-09-17T03:35:45.982528+0000","last_clean":"2024-09-17T03:35:45.982528+0000","last_became_active":"2024-09-17T03:35:44.957170+0000","last_became_peered":"2024-09-17T03:35:44.957170+0000","last_unstale":"2024-09-17T03:35:45.982528+0000","last_undegraded":"2024-09-17T03:35:45.982528+0000","last_fullsized":"2024-09-17T03:35:45.982528+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:17.448836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977529+0000","last_change":"2024-09-17T03:35:45.972995+0000","last_active":"2024-09-17T03:35:45.977529+0000","last_peered":"2024-09-17T03:35:45.977529+0000","last_clean":"2024-09-17T03:35:45.977529+0000","last_became_active":"2024-09-17T03:35:45.972604+0000","last_became_peered":"2024-09-17T03:35:45.972604+0000","last_unstale":"2024-09-17T03:35:45.977529+0000","last_undegraded":"2024-09-17T03:35:45.977529+0000","last_fullsized":"2024-09-17T03:35:45.977529+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:25.307232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950936+0000","last_change":"2024-09-17T03:35:44.958004+0000","last_active":"2024-09-17T03:35:45.950936+0000","last_peered":"2024-09-17T03:35:45.950936+0000","last_clean":"2024-09-17T03:35:45.950936+0000","last_became_active":"2024-09-17T03:35:44.957642+0000","last_became_peered":"2024-09-17T03:35:44.957642+0000","last_unstale":"2024-09-17T03:35:45.950936+0000","last_undegraded":"2024-09-17T03:35:45.950936+0000","last_fullsized":"2024-09-17T03:35:45.950936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:31.183310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":3,"reported_epoch":128,"state":"creating+activating","last_fresh":"2024-09-17T03:35:45.947281+0000","last_change":"2024-09-17T03:35:45.947281+0000","last_active":"2024-09-17T03:35:44.919052+0000","last_peered":"2024-09-17T03:35:44.919052+0000","last_clean":"2024-09-17T03:35:44.919052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:35:45.947281+0000","last_undegraded":"2024-09-17T03:35:45.947281+0000","last_fullsized":"2024-09-17T03:35:45.947281+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942821+0000","last_change":"2024-09-17T03:35:44.954770+0000","last_active":"2024-09-17T03:35:46.942821+0000","last_peered":"2024-09-17T03:35:46.942821+0000","last_clean":"2024-09-17T03:35:46.942821+0000","last_became_active":"2024-09-17T03:35:44.953523+0000","last_became_peered":"2024-09-17T03:35:44.953523+0000","last_unstale":"2024-09-17T03:35:46.942821+0000","last_undegraded":"2024-09-17T03:35:46.942821+0000","last_fullsized":"2024-09-17T03:35:46.942821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:40.501189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945946+0000","last_change":"2024-09-17T03:35:45.950548+0000","last_active":"2024-09-17T03:35:46.945946+0000","last_peered":"2024-09-17T03:35:46.945946+0000","last_clean":"2024-09-17T03:35:46.945946+0000","last_became_active":"2024-09-17T03:35:45.949460+0000","last_became_peered":"2024-09-17T03:35:45.949460+0000","last_unstale":"2024-09-17T03:35:46.945946+0000","last_undegraded":"2024-09-17T03:35:46.945946+0000","last_fullsized":"2024-09-17T03:35:46.945946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953125+0000","last_change":"2024-09-17T03:35:44.961601+0000","last_active":"2024-09-17T03:35:46.953125+0000","last_peered":"2024-09-17T03:35:46.953125+0000","last_clean":"2024-09-17T03:35:46.953125+0000","last_became_active":"2024-09-17T03:35:44.961301+0000","last_became_peered":"2024-09-17T03:35:44.961301+0000","last_unstale":"2024-09-17T03:35:46.953125+0000","last_undegraded":"2024-09-17T03:35:46.953125+0000","last_fullsized":"2024-09-17T03:35:46.953125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:55.987785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116185+0000","last_change":"2024-09-17T03:35:45.958025+0000","last_active":"2024-09-17T03:35:47.116185+0000","last_peered":"2024-09-17T03:35:47.116185+0000","last_clean":"2024-09-17T03:35:47.116185+0000","last_became_active":"2024-09-17T03:35:45.957707+0000","last_became_peered":"2024-09-17T03:35:45.957707+0000","last_unstale":"2024-09-17T03:35:47.116185+0000","last_undegraded":"2024-09-17T03:35:47.116185+0000","last_fullsized":"2024-09-17T03:35:47.116185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.185058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950067+0000","last_change":"2024-09-17T03:35:44.961854+0000","last_active":"2024-09-17T03:35:45.950067+0000","last_peered":"2024-09-17T03:35:45.950067+0000","last_clean":"2024-09-17T03:35:45.950067+0000","last_became_active":"2024-09-17T03:35:44.961082+0000","last_became_peered":"2024-09-17T03:35:44.961082+0000","last_unstale":"2024-09-17T03:35:45.950067+0000","last_undegraded":"2024-09-17T03:35:45.950067+0000","last_fullsized":"2024-09-17T03:35:45.950067+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:42.433406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115537+0000","last_change":"2024-09-17T03:35:45.960079+0000","last_active":"2024-09-17T03:35:47.115537+0000","last_peered":"2024-09-17T03:35:47.115537+0000","last_clean":"2024-09-17T03:35:47.115537+0000","last_became_active":"2024-09-17T03:35:45.959680+0000","last_became_peered":"2024-09-17T03:35:45.959680+0000","last_unstale":"2024-09-17T03:35:47.115537+0000","last_undegraded":"2024-09-17T03:35:47.115537+0000","last_fullsized":"2024-09-17T03:35:47.115537+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:24.060327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.949362+0000","last_change":"2024-09-17T03:35:44.958405+0000","last_active":"2024-09-17T03:35:45.949362+0000","last_peered":"2024-09-17T03:35:45.949362+0000","last_clean":"2024-09-17T03:35:45.949362+0000","last_became_active":"2024-09-17T03:35:44.958102+0000","last_became_peered":"2024-09-17T03:35:44.958102+0000","last_unstale":"2024-09-17T03:35:45.949362+0000","last_undegraded":"2024-09-17T03:35:45.949362+0000","last_fullsized":"2024-09-17T03:35:45.949362+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.755778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953353+0000","last_change":"2024-09-17T03:35:45.955137+0000","last_active":"2024-09-17T03:35:46.953353+0000","last_peered":"2024-09-17T03:35:46.953353+0000","last_clean":"2024-09-17T03:35:46.953353+0000","last_became_active":"2024-09-17T03:35:45.954787+0000","last_became_peered":"2024-09-17T03:35:45.954787+0000","last_unstale":"2024-09-17T03:35:46.953353+0000","last_undegraded":"2024-09-17T03:35:46.953353+0000","last_fullsized":"2024-09-17T03:35:46.953353+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:10.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945674+0000","last_change":"2024-09-17T03:35:44.950660+0000","last_active":"2024-09-17T03:35:46.945674+0000","last_peered":"2024-09-17T03:35:46.945674+0000","last_clean":"2024-09-17T03:35:46.945674+0000","last_became_active":"2024-09-17T03:35:44.949123+0000","last_became_peered":"2024-09-17T03:35:44.949123+0000","last_unstale":"2024-09-17T03:35:46.945674+0000","last_undegraded":"2024-09-17T03:35:46.945674+0000","last_fullsized":"2024-09-17T03:35:46.945674+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114873+0000","last_change":"2024-09-17T03:35:45.956952+0000","last_active":"2024-09-17T03:35:47.114873+0000","last_peered":"2024-09-17T03:35:47.114873+0000","last_clean":"2024-09-17T03:35:47.114873+0000","last_became_active":"2024-09-17T03:35:45.956620+0000","last_became_peered":"2024-09-17T03:35:45.956620+0000","last_unstale":"2024-09-17T03:35:47.114873+0000","last_undegraded":"2024-09-17T03:35:47.114873+0000","last_fullsized":"2024-09-17T03:35:47.114873+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:11.263633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945421+0000","last_change":"2024-09-17T03:35:44.947967+0000","last_active":"2024-09-17T03:35:46.945421+0000","last_peered":"2024-09-17T03:35:46.945421+0000","last_clean":"2024-09-17T03:35:46.945421+0000","last_became_active":"2024-09-17T03:35:44.947694+0000","last_became_peered":"2024-09-17T03:35:44.947694+0000","last_unstale":"2024-09-17T03:35:46.945421+0000","last_undegraded":"2024-09-17T03:35:46.945421+0000","last_fullsized":"2024-09-17T03:35:46.945421+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:00.392947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963284+0000","last_change":"2024-09-17T03:35:45.973217+0000","last_active":"2024-09-17T03:35:46.963284+0000","last_peered":"2024-09-17T03:35:46.963284+0000","last_clean":"2024-09-17T03:35:46.963284+0000","last_became_active":"2024-09-17T03:35:45.972910+0000","last_became_peered":"2024-09-17T03:35:45.972910+0000","last_unstale":"2024-09-17T03:35:46.963284+0000","last_undegraded":"2024-09-17T03:35:46.963284+0000","last_fullsized":"2024-09-17T03:35:46.963284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942268+0000","last_change":"2024-09-17T03:35:44.947744+0000","last_active":"2024-09-17T03:35:46.942268+0000","last_peered":"2024-09-17T03:35:46.942268+0000","last_clean":"2024-09-17T03:35:46.942268+0000","last_became_active":"2024-09-17T03:35:44.947045+0000","last_became_peered":"2024-09-17T03:35:44.947045+0000","last_unstale":"2024-09-17T03:35:46.942268+0000","last_undegraded":"2024-09-17T03:35:46.942268+0000","last_fullsized":"2024-09-17T03:35:46.942268+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:56.123881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953525+0000","last_change":"2024-09-17T03:35:45.963241+0000","last_active":"2024-09-17T03:35:46.953525+0000","last_peered":"2024-09-17T03:35:46.953525+0000","last_clean":"2024-09-17T03:35:46.953525+0000","last_became_active":"2024-09-17T03:35:45.962942+0000","last_became_peered":"2024-09-17T03:35:45.962942+0000","last_unstale":"2024-09-17T03:35:46.953525+0000","last_undegraded":"2024-09-17T03:35:46.953525+0000","last_fullsized":"2024-09-17T03:35:46.953525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:47.099040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115936+0000","last_change":"2024-09-17T03:35:44.950227+0000","last_active":"2024-09-17T03:35:47.115936+0000","last_peered":"2024-09-17T03:35:47.115936+0000","last_clean":"2024-09-17T03:35:47.115936+0000","last_became_active":"2024-09-17T03:35:44.949838+0000","last_became_peered":"2024-09-17T03:35:44.949838+0000","last_unstale":"2024-09-17T03:35:47.115936+0000","last_undegraded":"2024-09-17T03:35:47.115936+0000","last_fullsized":"2024-09-17T03:35:47.115936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:38.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115830+0000","last_change":"2024-09-17T03:35:45.952300+0000","last_active":"2024-09-17T03:35:47.115830+0000","last_peered":"2024-09-17T03:35:47.115830+0000","last_clean":"2024-09-17T03:35:47.115830+0000","last_became_active":"2024-09-17T03:35:45.951940+0000","last_became_peered":"2024-09-17T03:35:45.951940+0000","last_unstale":"2024-09-17T03:35:47.115830+0000","last_undegraded":"2024-09-17T03:35:47.115830+0000","last_fullsized":"2024-09-17T03:35:47.115830+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.804587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.965242+0000","last_change":"2024-09-17T03:35:44.960044+0000","last_active":"2024-09-17T03:35:46.965242+0000","last_peered":"2024-09-17T03:35:46.965242+0000","last_clean":"2024-09-17T03:35:46.965242+0000","last_became_active":"2024-09-17T03:35:44.959679+0000","last_became_peered":"2024-09-17T03:35:44.959679+0000","last_unstale":"2024-09-17T03:35:46.965242+0000","last_undegraded":"2024-09-17T03:35:46.965242+0000","last_fullsized":"2024-09-17T03:35:46.965242+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:26.474116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116521+0000","last_change":"2024-09-17T03:35:45.957578+0000","last_active":"2024-09-17T03:35:47.116521+0000","last_peered":"2024-09-17T03:35:47.116521+0000","last_clean":"2024-09-17T03:35:47.116521+0000","last_became_active":"2024-09-17T03:35:45.956972+0000","last_became_peered":"2024-09-17T03:35:45.956972+0000","last_unstale":"2024-09-17T03:35:47.116521+0000","last_undegraded":"2024-09-17T03:35:47.116521+0000","last_fullsized":"2024-09-17T03:35:47.116521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:37.224976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116267+0000","last_change":"2024-09-17T03:35:44.953819+0000","last_active":"2024-09-17T03:35:47.116267+0000","last_peered":"2024-09-17T03:35:47.116267+0000","last_clean":"2024-09-17T03:35:47.116267+0000","last_became_active":"2024-09-17T03:35:44.951031+0000","last_became_peered":"2024-09-17T03:35:44.951031+0000","last_unstale":"2024-09-17T03:35:47.116267+0000","last_undegraded":"2024-09-17T03:35:47.116267+0000","last_fullsized":"2024-09-17T03:35:47.116267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:41.554774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953195+0000","last_change":"2024-09-17T03:35:45.962505+0000","last_active":"2024-09-17T03:35:46.953195+0000","last_peered":"2024-09-17T03:35:46.953195+0000","last_clean":"2024-09-17T03:35:46.953195+0000","last_became_active":"2024-09-17T03:35:45.962169+0000","last_became_peered":"2024-09-17T03:35:45.962169+0000","last_unstale":"2024-09-17T03:35:46.953195+0000","last_undegraded":"2024-09-17T03:35:46.953195+0000","last_fullsized":"2024-09-17T03:35:46.953195+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:20.774078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942572+0000","last_change":"2024-09-17T03:35:44.956080+0000","last_active":"2024-09-17T03:35:46.942572+0000","last_peered":"2024-09-17T03:35:46.942572+0000","last_clean":"2024-09-17T03:35:46.942572+0000","last_became_active":"2024-09-17T03:35:44.955748+0000","last_became_peered":"2024-09-17T03:35:44.955748+0000","last_unstale":"2024-09-17T03:35:46.942572+0000","last_undegraded":"2024-09-17T03:35:46.942572+0000","last_fullsized":"2024-09-17T03:35:46.942572+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:04.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963292+0000","last_change":"2024-09-17T03:35:45.963696+0000","last_active":"2024-09-17T03:35:46.963292+0000","last_peered":"2024-09-17T03:35:46.963292+0000","last_clean":"2024-09-17T03:35:46.963292+0000","last_became_active":"2024-09-17T03:35:45.963323+0000","last_became_peered":"2024-09-17T03:35:45.963323+0000","last_unstale":"2024-09-17T03:35:46.963292+0000","last_undegraded":"2024-09-17T03:35:46.963292+0000","last_fullsized":"2024-09-17T03:35:46.963292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:59.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115974+0000","last_change":"2024-09-17T03:35:44.953364+0000","last_active":"2024-09-17T03:35:47.115974+0000","last_peered":"2024-09-17T03:35:47.115974+0000","last_clean":"2024-09-17T03:35:47.115974+0000","last_became_active":"2024-09-17T03:35:44.953034+0000","last_became_peered":"2024-09-17T03:35:44.953034+0000","last_unstale":"2024-09-17T03:35:47.115974+0000","last_undegraded":"2024-09-17T03:35:47.115974+0000","last_fullsized":"2024-09-17T03:35:47.115974+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:23.777269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.966928+0000","last_change":"2024-09-17T03:35:45.963406+0000","last_active":"2024-09-17T03:35:45.966928+0000","last_peered":"2024-09-17T03:35:45.966928+0000","last_clean":"2024-09-17T03:35:45.966928+0000","last_became_active":"2024-09-17T03:35:45.963028+0000","last_became_peered":"2024-09-17T03:35:45.963028+0000","last_unstale":"2024-09-17T03:35:45.966928+0000","last_undegraded":"2024-09-17T03:35:45.966928+0000","last_fullsized":"2024-09-17T03:35:45.966928+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:26.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963480+0000","last_change":"2024-09-17T03:35:44.961829+0000","last_active":"2024-09-17T03:35:46.963480+0000","last_peered":"2024-09-17T03:35:46.963480+0000","last_clean":"2024-09-17T03:35:46.963480+0000","last_became_active":"2024-09-17T03:35:44.961479+0000","last_became_peered":"2024-09-17T03:35:44.961479+0000","last_unstale":"2024-09-17T03:35:46.963480+0000","last_undegraded":"2024-09-17T03:35:46.963480+0000","last_fullsized":"2024-09-17T03:35:46.963480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945631+0000","last_change":"2024-09-17T03:35:45.954094+0000","last_active":"2024-09-17T03:35:46.945631+0000","last_peered":"2024-09-17T03:35:46.945631+0000","last_clean":"2024-09-17T03:35:46.945631+0000","last_became_active":"2024-09-17T03:35:45.952010+0000","last_became_peered":"2024-09-17T03:35:45.952010+0000","last_unstale":"2024-09-17T03:35:46.945631+0000","last_undegraded":"2024-09-17T03:35:46.945631+0000","last_fullsized":"2024-09-17T03:35:46.945631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.979607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945521+0000","last_change":"2024-09-17T03:35:44.944467+0000","last_active":"2024-09-17T03:35:46.945521+0000","last_peered":"2024-09-17T03:35:46.945521+0000","last_clean":"2024-09-17T03:35:46.945521+0000","last_became_active":"2024-09-17T03:35:44.944155+0000","last_became_peered":"2024-09-17T03:35:44.944155+0000","last_unstale":"2024-09-17T03:35:46.945521+0000","last_undegraded":"2024-09-17T03:35:46.945521+0000","last_fullsized":"2024-09-17T03:35:46.945521+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:00.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115398+0000","last_change":"2024-09-17T03:35:45.954997+0000","last_active":"2024-09-17T03:35:47.115398+0000","last_peered":"2024-09-17T03:35:47.115398+0000","last_clean":"2024-09-17T03:35:47.115398+0000","last_became_active":"2024-09-17T03:35:45.954184+0000","last_became_peered":"2024-09-17T03:35:45.954184+0000","last_unstale":"2024-09-17T03:35:47.115398+0000","last_undegraded":"2024-09-17T03:35:47.115398+0000","last_fullsized":"2024-09-17T03:35:47.115398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.149975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942555+0000","last_change":"2024-09-17T03:35:44.948114+0000","last_active":"2024-09-17T03:35:46.942555+0000","last_peered":"2024-09-17T03:35:46.942555+0000","last_clean":"2024-09-17T03:35:46.942555+0000","last_became_active":"2024-09-17T03:35:44.947433+0000","last_became_peered":"2024-09-17T03:35:44.947433+0000","last_unstale":"2024-09-17T03:35:46.942555+0000","last_undegraded":"2024-09-17T03:35:46.942555+0000","last_fullsized":"2024-09-17T03:35:46.942555+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:22.126761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963403+0000","last_change":"2024-09-17T03:35:45.968401+0000","last_active":"2024-09-17T03:35:46.963403+0000","last_peered":"2024-09-17T03:35:46.963403+0000","last_clean":"2024-09-17T03:35:46.963403+0000","last_became_active":"2024-09-17T03:35:45.968050+0000","last_became_peered":"2024-09-17T03:35:45.968050+0000","last_unstale":"2024-09-17T03:35:46.963403+0000","last_undegraded":"2024-09-17T03:35:46.963403+0000","last_fullsized":"2024-09-17T03:35:46.963403+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963509+0000","last_change":"2024-09-17T03:35:44.963161+0000","last_active":"2024-09-17T03:35:46.963509+0000","last_peered":"2024-09-17T03:35:46.963509+0000","last_clean":"2024-09-17T03:35:46.963509+0000","last_became_active":"2024-09-17T03:35:44.962822+0000","last_became_peered":"2024-09-17T03:35:44.962822+0000","last_unstale":"2024-09-17T03:35:46.963509+0000","last_undegraded":"2024-09-17T03:35:46.963509+0000","last_fullsized":"2024-09-17T03:35:46.963509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:12.300948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":309,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.962781+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:35:46.962781+0000","last_peered":"2024-09-17T03:35:46.962781+0000","last_clean":"2024-09-17T03:35:46.962781+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:35:46.962781+0000","last_undegraded":"2024-09-17T03:35:46.962781+0000","last_fullsized":"2024-09-17T03:35:46.962781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509621,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136000,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235840,"statfs":{"total":96636764160,"available":96497500160,"internally_reserved":0,"allocated":2207744,"data_stored":1286611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145140,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226700,"statfs":{"total":96636764160,"available":96488140800,"internally_reserved":0,"allocated":1601536,"data_stored":695605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140968,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230872,"statfs":{"total":96636764160,"available":96492412928,"internally_reserved":0,"allocated":1593344,"data_stored":694063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127084,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244756,"statfs":{"total":96636764160,"available":96506630144,"internally_reserved":0,"allocated":1597440,"data_stored":694153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163180,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208660,"statfs":{"total":96636764160,"available":96469667840,"internally_reserved":0,"allocated":1597440,"data_stored":694609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145704,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226136,"statfs":{"total":96636764160,"available":96487563264,"internally_reserved":0,"allocated":2183168,"data_stored":1284361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542325,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127068,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244772,"statfs":{"total":96636764160,"available":96506646528,"internally_reserved":0,"allocated":1576960,"data_stored":687190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167340,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204500,"statfs":{"total":96636764160,"available":96465408000,"internally_reserved":0,"allocated":1589248,"data_stored":693993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:35:50.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 -- 172.21.15.46:0/3047302375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea01527b0 msgr2=0x7f9ea0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 --2- 172.21.15.46:0/3047302375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea01527b0 0x7f9ea0074e30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0072e30 tx=0x7f9e8c01c800 comp rx=0 tx=0).stop 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 -- 172.21.15.46:0/3047302375 shutdown_connections 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 --2- 172.21.15.46:0/3047302375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea01527b0 0x7f9ea0074e30 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 --2- 172.21.15.46:0/3047302375 >> v2:172.21.15.99:3300/0 conn(0x7f9ea00b4de0 0x7f9ea00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 --2- 172.21.15.46:0/3047302375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ea00b5c70 0x7f9ea015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 -- 172.21.15.46:0/3047302375 >> 172.21.15.46:0/3047302375 conn(0x7f9ea00b2040 msgr2=0x7f9ea00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:50.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 -- 172.21.15.46:0/3047302375 shutdown_connections 2024-09-17T03:35:50.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.765+0000 7f9ea6800640 1 -- 172.21.15.46:0/3047302375 wait complete. 2024-09-17T03:35:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 Processor -- start 2024-09-17T03:35:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 -- start start 2024-09-17T03:35:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 0x7f9ea014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ea00b5c70 0x7f9ea01f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ea01527b0 0x7f9ea01f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea015f700 con 0x7f9ea00b4de0 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea015f580 con 0x7f9ea00b5c70 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea015f880 con 0x7f9ea01527b0 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 0x7f9ea014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 0x7f9ea014fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33586/0 (socket says 172.21.15.46:33586) 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 -- 172.21.15.46:0/2811250349 learned_addr learned my addr 172.21.15.46:0/2811250349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9fe00640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ea01527b0 0x7f9ea01f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9ea00640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ea00b5c70 0x7f9ea01f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 -- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ea01527b0 msgr2=0x7f9ea01f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ea01527b0 0x7f9ea01f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 -- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ea00b5c70 msgr2=0x7f9ea01f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ea00b5c70 0x7f9ea01f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 -- 172.21.15.46:0/2811250349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea0152f50 con 0x7f9ea00b4de0 2024-09-17T03:35:50.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e9f400640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 0x7f9ea014fbd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000d750 tx=0x7f9e9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:50.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90017070 con 0x7f9ea00b4de0 2024-09-17T03:35:50.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e90009d20 con 0x7f9ea00b4de0 2024-09-17T03:35:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90005280 con 0x7f9ea00b4de0 2024-09-17T03:35:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea0153230 con 0x7f9ea00b4de0 2024-09-17T03:35:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.769+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea01537f0 con 0x7f9ea00b4de0 2024-09-17T03:35:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.773+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea015fda0 con 0x7f9ea00b4de0 2024-09-17T03:35:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.773+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f9e90005420 con 0x7f9ea00b4de0 2024-09-17T03:35:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.773+0000 7f9e97400640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e80075240 0x7f9e80077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.773+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f9e900933d0 con 0x7f9ea00b4de0 2024-09-17T03:35:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.773+0000 7f9e9ea00640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e80075240 0x7f9e80077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.777+0000 7f9e9ea00640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e80075240 0x7f9e80077750 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0af000 tx=0x7f9e8c0a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.777+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9e9000de60 con 0x7f9ea00b4de0 2024-09-17T03:35:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.953+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ea006b540 con 0x7f9e80075240 2024-09-17T03:35:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.957+0000 7f9e97400640 1 -- 172.21.15.46:0/2811250349 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252337 (secure 0 0 0) 0x7f9e8c0b3040 con 0x7f9e80075240 2024-09-17T03:35:50.961 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e80075240 msgr2=0x7f9e80077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e80075240 0x7f9e80077750 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0af000 tx=0x7f9e8c0a7040 comp rx=0 tx=0).stop 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 msgr2=0x7f9ea014fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 0x7f9ea014fbd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000d750 tx=0x7f9e9000dc10 comp rx=0 tx=0).stop 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 shutdown_connections 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e80075240 0x7f9e80077750 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ea01527b0 0x7f9ea01f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ea00b5c70 0x7f9ea01f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 --2- 172.21.15.46:0/2811250349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b4de0 0x7f9ea014fbd0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:50.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 >> 172.21.15.46:0/2811250349 conn(0x7f9ea00b2040 msgr2=0x7f9ea006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:50.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 shutdown_connections 2024-09-17T03:35:50.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:50.961+0000 7f9ea6800640 1 -- 172.21.15.46:0/2811250349 wait complete. 2024-09-17T03:35:50.984 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":569,"stamp":"2024-09-17T03:35:49.561285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":140,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1152484,"kb_used_data":13620,"kb_used_omap":12,"kb_used_meta":1138675,"kb_avail":753822236,"statfs":{"total":773094113280,"available":771913969664,"internally_reserved":0,"allocated":13946880,"data_stored":6730585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1166003760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.642018"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116128+0000","last_change":"2024-09-17T03:35:45.955819+0000","last_active":"2024-09-17T03:35:47.116128+0000","last_peered":"2024-09-17T03:35:47.116128+0000","last_clean":"2024-09-17T03:35:47.116128+0000","last_became_active":"2024-09-17T03:35:45.954935+0000","last_became_peered":"2024-09-17T03:35:45.954935+0000","last_unstale":"2024-09-17T03:35:47.116128+0000","last_undegraded":"2024-09-17T03:35:47.116128+0000","last_fullsized":"2024-09-17T03:35:47.116128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.363688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116488+0000","last_change":"2024-09-17T03:35:45.945240+0000","last_active":"2024-09-17T03:35:47.116488+0000","last_peered":"2024-09-17T03:35:47.116488+0000","last_clean":"2024-09-17T03:35:47.116488+0000","last_became_active":"2024-09-17T03:35:45.944788+0000","last_became_peered":"2024-09-17T03:35:45.944788+0000","last_unstale":"2024-09-17T03:35:47.116488+0000","last_undegraded":"2024-09-17T03:35:47.116488+0000","last_fullsized":"2024-09-17T03:35:47.116488+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:02.551861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.973644+0000","last_change":"2024-09-17T03:35:45.965545+0000","last_active":"2024-09-17T03:35:45.973644+0000","last_peered":"2024-09-17T03:35:45.973644+0000","last_clean":"2024-09-17T03:35:45.973644+0000","last_became_active":"2024-09-17T03:35:45.961480+0000","last_became_peered":"2024-09-17T03:35:45.961480+0000","last_unstale":"2024-09-17T03:35:45.973644+0000","last_undegraded":"2024-09-17T03:35:45.973644+0000","last_fullsized":"2024-09-17T03:35:45.973644+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:35.059698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115130+0000","last_change":"2024-09-17T03:35:45.960750+0000","last_active":"2024-09-17T03:35:47.115130+0000","last_peered":"2024-09-17T03:35:47.115130+0000","last_clean":"2024-09-17T03:35:47.115130+0000","last_became_active":"2024-09-17T03:35:45.960431+0000","last_became_peered":"2024-09-17T03:35:45.960431+0000","last_unstale":"2024-09-17T03:35:47.115130+0000","last_undegraded":"2024-09-17T03:35:47.115130+0000","last_fullsized":"2024-09-17T03:35:47.115130+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:59.838096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963365+0000","last_change":"2024-09-17T03:35:45.971487+0000","last_active":"2024-09-17T03:35:46.963365+0000","last_peered":"2024-09-17T03:35:46.963365+0000","last_clean":"2024-09-17T03:35:46.963365+0000","last_became_active":"2024-09-17T03:35:45.970309+0000","last_became_peered":"2024-09-17T03:35:45.970309+0000","last_unstale":"2024-09-17T03:35:46.963365+0000","last_undegraded":"2024-09-17T03:35:46.963365+0000","last_fullsized":"2024-09-17T03:35:46.963365+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953008+0000","last_change":"2024-09-17T03:35:45.961513+0000","last_active":"2024-09-17T03:35:46.953008+0000","last_peered":"2024-09-17T03:35:46.953008+0000","last_clean":"2024-09-17T03:35:46.953008+0000","last_became_active":"2024-09-17T03:35:45.961192+0000","last_became_peered":"2024-09-17T03:35:45.961192+0000","last_unstale":"2024-09-17T03:35:46.953008+0000","last_undegraded":"2024-09-17T03:35:46.953008+0000","last_fullsized":"2024-09-17T03:35:46.953008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:45.692040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952824+0000","last_change":"2024-09-17T03:35:45.955196+0000","last_active":"2024-09-17T03:35:46.952824+0000","last_peered":"2024-09-17T03:35:46.952824+0000","last_clean":"2024-09-17T03:35:46.952824+0000","last_became_active":"2024-09-17T03:35:45.954869+0000","last_became_peered":"2024-09-17T03:35:45.954869+0000","last_unstale":"2024-09-17T03:35:46.952824+0000","last_undegraded":"2024-09-17T03:35:46.952824+0000","last_fullsized":"2024-09-17T03:35:46.952824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:27.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978568+0000","last_change":"2024-09-17T03:35:45.968695+0000","last_active":"2024-09-17T03:35:45.978568+0000","last_peered":"2024-09-17T03:35:45.978568+0000","last_clean":"2024-09-17T03:35:45.978568+0000","last_became_active":"2024-09-17T03:35:45.966016+0000","last_became_peered":"2024-09-17T03:35:45.966016+0000","last_unstale":"2024-09-17T03:35:45.978568+0000","last_undegraded":"2024-09-17T03:35:45.978568+0000","last_fullsized":"2024-09-17T03:35:45.978568+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:29.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975084+0000","last_change":"2024-09-17T03:35:45.969067+0000","last_active":"2024-09-17T03:35:45.975084+0000","last_peered":"2024-09-17T03:35:45.975084+0000","last_clean":"2024-09-17T03:35:45.975084+0000","last_became_active":"2024-09-17T03:35:45.968667+0000","last_became_peered":"2024-09-17T03:35:45.968667+0000","last_unstale":"2024-09-17T03:35:45.975084+0000","last_undegraded":"2024-09-17T03:35:45.975084+0000","last_fullsized":"2024-09-17T03:35:45.975084+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:05:08.235660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116451+0000","last_change":"2024-09-17T03:35:45.945553+0000","last_active":"2024-09-17T03:35:47.116451+0000","last_peered":"2024-09-17T03:35:47.116451+0000","last_clean":"2024-09-17T03:35:47.116451+0000","last_became_active":"2024-09-17T03:35:45.945233+0000","last_became_peered":"2024-09-17T03:35:45.945233+0000","last_unstale":"2024-09-17T03:35:47.116451+0000","last_undegraded":"2024-09-17T03:35:47.116451+0000","last_fullsized":"2024-09-17T03:35:47.116451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:52.862294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942689+0000","last_change":"2024-09-17T03:35:45.972155+0000","last_active":"2024-09-17T03:35:46.942689+0000","last_peered":"2024-09-17T03:35:46.942689+0000","last_clean":"2024-09-17T03:35:46.942689+0000","last_became_active":"2024-09-17T03:35:45.971845+0000","last_became_peered":"2024-09-17T03:35:45.971845+0000","last_unstale":"2024-09-17T03:35:46.942689+0000","last_undegraded":"2024-09-17T03:35:46.942689+0000","last_fullsized":"2024-09-17T03:35:46.942689+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:13.103673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977386+0000","last_change":"2024-09-17T03:35:45.965493+0000","last_active":"2024-09-17T03:35:45.977386+0000","last_peered":"2024-09-17T03:35:45.977386+0000","last_clean":"2024-09-17T03:35:45.977386+0000","last_became_active":"2024-09-17T03:35:45.962965+0000","last_became_peered":"2024-09-17T03:35:45.962965+0000","last_unstale":"2024-09-17T03:35:45.977386+0000","last_undegraded":"2024-09-17T03:35:45.977386+0000","last_fullsized":"2024-09-17T03:35:45.977386+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:59.517858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115407+0000","last_change":"2024-09-17T03:35:45.957732+0000","last_active":"2024-09-17T03:35:47.115407+0000","last_peered":"2024-09-17T03:35:47.115407+0000","last_clean":"2024-09-17T03:35:47.115407+0000","last_became_active":"2024-09-17T03:35:45.957405+0000","last_became_peered":"2024-09-17T03:35:45.957405+0000","last_unstale":"2024-09-17T03:35:47.115407+0000","last_undegraded":"2024-09-17T03:35:47.115407+0000","last_fullsized":"2024-09-17T03:35:47.115407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:23.451219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115934+0000","last_change":"2024-09-17T03:35:45.952592+0000","last_active":"2024-09-17T03:35:47.115934+0000","last_peered":"2024-09-17T03:35:47.115934+0000","last_clean":"2024-09-17T03:35:47.115934+0000","last_became_active":"2024-09-17T03:35:45.949816+0000","last_became_peered":"2024-09-17T03:35:45.949816+0000","last_unstale":"2024-09-17T03:35:47.115934+0000","last_undegraded":"2024-09-17T03:35:47.115934+0000","last_fullsized":"2024-09-17T03:35:47.115934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:16.316918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114925+0000","last_change":"2024-09-17T03:35:45.958139+0000","last_active":"2024-09-17T03:35:47.114925+0000","last_peered":"2024-09-17T03:35:47.114925+0000","last_clean":"2024-09-17T03:35:47.114925+0000","last_became_active":"2024-09-17T03:35:45.957813+0000","last_became_peered":"2024-09-17T03:35:45.957813+0000","last_unstale":"2024-09-17T03:35:47.114925+0000","last_undegraded":"2024-09-17T03:35:47.114925+0000","last_fullsized":"2024-09-17T03:35:47.114925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:28.891920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945605+0000","last_change":"2024-09-17T03:35:45.950827+0000","last_active":"2024-09-17T03:35:46.945605+0000","last_peered":"2024-09-17T03:35:46.945605+0000","last_clean":"2024-09-17T03:35:46.945605+0000","last_became_active":"2024-09-17T03:35:45.950514+0000","last_became_peered":"2024-09-17T03:35:45.950514+0000","last_unstale":"2024-09-17T03:35:46.945605+0000","last_undegraded":"2024-09-17T03:35:46.945605+0000","last_fullsized":"2024-09-17T03:35:46.945605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:52.381399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953466+0000","last_change":"2024-09-17T03:35:45.964355+0000","last_active":"2024-09-17T03:35:46.953466+0000","last_peered":"2024-09-17T03:35:46.953466+0000","last_clean":"2024-09-17T03:35:46.953466+0000","last_became_active":"2024-09-17T03:35:45.962165+0000","last_became_peered":"2024-09-17T03:35:45.962165+0000","last_unstale":"2024-09-17T03:35:46.953466+0000","last_undegraded":"2024-09-17T03:35:46.953466+0000","last_fullsized":"2024-09-17T03:35:46.953466+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:26.092966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946215+0000","last_change":"2024-09-17T03:35:45.955274+0000","last_active":"2024-09-17T03:35:46.946215+0000","last_peered":"2024-09-17T03:35:46.946215+0000","last_clean":"2024-09-17T03:35:46.946215+0000","last_became_active":"2024-09-17T03:35:45.953802+0000","last_became_peered":"2024-09-17T03:35:45.953802+0000","last_unstale":"2024-09-17T03:35:46.946215+0000","last_undegraded":"2024-09-17T03:35:46.946215+0000","last_fullsized":"2024-09-17T03:35:46.946215+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942937+0000","last_change":"2024-09-17T03:35:45.963307+0000","last_active":"2024-09-17T03:35:46.942937+0000","last_peered":"2024-09-17T03:35:46.942937+0000","last_clean":"2024-09-17T03:35:46.942937+0000","last_became_active":"2024-09-17T03:35:45.963007+0000","last_became_peered":"2024-09-17T03:35:45.963007+0000","last_unstale":"2024-09-17T03:35:46.942937+0000","last_undegraded":"2024-09-17T03:35:46.942937+0000","last_fullsized":"2024-09-17T03:35:46.942937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:09.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945919+0000","last_change":"2024-09-17T03:35:45.950245+0000","last_active":"2024-09-17T03:35:46.945919+0000","last_peered":"2024-09-17T03:35:46.945919+0000","last_clean":"2024-09-17T03:35:46.945919+0000","last_became_active":"2024-09-17T03:35:45.949852+0000","last_became_peered":"2024-09-17T03:35:45.949852+0000","last_unstale":"2024-09-17T03:35:46.945919+0000","last_undegraded":"2024-09-17T03:35:46.945919+0000","last_fullsized":"2024-09-17T03:35:46.945919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:09.687961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963523+0000","last_change":"2024-09-17T03:35:45.967983+0000","last_active":"2024-09-17T03:35:46.963523+0000","last_peered":"2024-09-17T03:35:46.963523+0000","last_clean":"2024-09-17T03:35:46.963523+0000","last_became_active":"2024-09-17T03:35:45.964201+0000","last_became_peered":"2024-09-17T03:35:45.964201+0000","last_unstale":"2024-09-17T03:35:46.963523+0000","last_undegraded":"2024-09-17T03:35:46.963523+0000","last_fullsized":"2024-09-17T03:35:46.963523+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:26.825390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942483+0000","last_change":"2024-09-17T03:35:45.969671+0000","last_active":"2024-09-17T03:35:46.942483+0000","last_peered":"2024-09-17T03:35:46.942483+0000","last_clean":"2024-09-17T03:35:46.942483+0000","last_became_active":"2024-09-17T03:35:45.969304+0000","last_became_peered":"2024-09-17T03:35:45.969304+0000","last_unstale":"2024-09-17T03:35:46.942483+0000","last_undegraded":"2024-09-17T03:35:46.942483+0000","last_fullsized":"2024-09-17T03:35:46.942483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:30.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.972932+0000","last_change":"2024-09-17T03:35:45.962436+0000","last_active":"2024-09-17T03:35:45.972932+0000","last_peered":"2024-09-17T03:35:45.972932+0000","last_clean":"2024-09-17T03:35:45.972932+0000","last_became_active":"2024-09-17T03:35:45.962092+0000","last_became_peered":"2024-09-17T03:35:45.962092+0000","last_unstale":"2024-09-17T03:35:45.972932+0000","last_undegraded":"2024-09-17T03:35:45.972932+0000","last_fullsized":"2024-09-17T03:35:45.972932+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:33.151389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942550+0000","last_change":"2024-09-17T03:35:45.968660+0000","last_active":"2024-09-17T03:35:46.942550+0000","last_peered":"2024-09-17T03:35:46.942550+0000","last_clean":"2024-09-17T03:35:46.942550+0000","last_became_active":"2024-09-17T03:35:45.968290+0000","last_became_peered":"2024-09-17T03:35:45.968290+0000","last_unstale":"2024-09-17T03:35:46.942550+0000","last_undegraded":"2024-09-17T03:35:46.942550+0000","last_fullsized":"2024-09-17T03:35:46.942550+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:23.605758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116395+0000","last_change":"2024-09-17T03:35:45.950767+0000","last_active":"2024-09-17T03:35:47.116395+0000","last_peered":"2024-09-17T03:35:47.116395+0000","last_clean":"2024-09-17T03:35:47.116395+0000","last_became_active":"2024-09-17T03:35:45.950459+0000","last_became_peered":"2024-09-17T03:35:45.950459+0000","last_unstale":"2024-09-17T03:35:47.116395+0000","last_undegraded":"2024-09-17T03:35:47.116395+0000","last_fullsized":"2024-09-17T03:35:47.116395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:00.727158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946018+0000","last_change":"2024-09-17T03:35:45.951322+0000","last_active":"2024-09-17T03:35:46.946018+0000","last_peered":"2024-09-17T03:35:46.946018+0000","last_clean":"2024-09-17T03:35:46.946018+0000","last_became_active":"2024-09-17T03:35:45.950888+0000","last_became_peered":"2024-09-17T03:35:45.950888+0000","last_unstale":"2024-09-17T03:35:46.946018+0000","last_undegraded":"2024-09-17T03:35:46.946018+0000","last_fullsized":"2024-09-17T03:35:46.946018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115304+0000","last_change":"2024-09-17T03:35:45.958772+0000","last_active":"2024-09-17T03:35:47.115304+0000","last_peered":"2024-09-17T03:35:47.115304+0000","last_clean":"2024-09-17T03:35:47.115304+0000","last_became_active":"2024-09-17T03:35:45.958227+0000","last_became_peered":"2024-09-17T03:35:45.958227+0000","last_unstale":"2024-09-17T03:35:47.115304+0000","last_undegraded":"2024-09-17T03:35:47.115304+0000","last_fullsized":"2024-09-17T03:35:47.115304+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:36.638860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":3,"reported_epoch":128,"state":"creating+activating","last_fresh":"2024-09-17T03:35:45.948224+0000","last_change":"2024-09-17T03:35:45.948224+0000","last_active":"2024-09-17T03:35:44.919052+0000","last_peered":"2024-09-17T03:35:44.919052+0000","last_clean":"2024-09-17T03:35:44.919052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:35:45.948224+0000","last_undegraded":"2024-09-17T03:35:45.948224+0000","last_fullsized":"2024-09-17T03:35:45.948224+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115694+0000","last_change":"2024-09-17T03:35:45.957427+0000","last_active":"2024-09-17T03:35:47.115694+0000","last_peered":"2024-09-17T03:35:47.115694+0000","last_clean":"2024-09-17T03:35:47.115694+0000","last_became_active":"2024-09-17T03:35:45.956963+0000","last_became_peered":"2024-09-17T03:35:45.956963+0000","last_unstale":"2024-09-17T03:35:47.115694+0000","last_undegraded":"2024-09-17T03:35:47.115694+0000","last_fullsized":"2024-09-17T03:35:47.115694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.736866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115434+0000","last_change":"2024-09-17T03:35:45.959532+0000","last_active":"2024-09-17T03:35:47.115434+0000","last_peered":"2024-09-17T03:35:47.115434+0000","last_clean":"2024-09-17T03:35:47.115434+0000","last_became_active":"2024-09-17T03:35:45.952528+0000","last_became_peered":"2024-09-17T03:35:45.952528+0000","last_unstale":"2024-09-17T03:35:47.115434+0000","last_undegraded":"2024-09-17T03:35:47.115434+0000","last_fullsized":"2024-09-17T03:35:47.115434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:49.407151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963303+0000","last_change":"2024-09-17T03:35:45.975662+0000","last_active":"2024-09-17T03:35:46.963303+0000","last_peered":"2024-09-17T03:35:46.963303+0000","last_clean":"2024-09-17T03:35:46.963303+0000","last_became_active":"2024-09-17T03:35:45.975295+0000","last_became_peered":"2024-09-17T03:35:45.975295+0000","last_unstale":"2024-09-17T03:35:46.963303+0000","last_undegraded":"2024-09-17T03:35:46.963303+0000","last_fullsized":"2024-09-17T03:35:46.963303+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952833+0000","last_change":"2024-09-17T03:35:45.947162+0000","last_active":"2024-09-17T03:35:46.952833+0000","last_peered":"2024-09-17T03:35:46.952833+0000","last_clean":"2024-09-17T03:35:46.952833+0000","last_became_active":"2024-09-17T03:35:45.946769+0000","last_became_peered":"2024-09-17T03:35:45.946769+0000","last_unstale":"2024-09-17T03:35:46.952833+0000","last_undegraded":"2024-09-17T03:35:46.952833+0000","last_fullsized":"2024-09-17T03:35:46.952833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:12.153774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946273+0000","last_change":"2024-09-17T03:35:45.951693+0000","last_active":"2024-09-17T03:35:46.946273+0000","last_peered":"2024-09-17T03:35:46.946273+0000","last_clean":"2024-09-17T03:35:46.946273+0000","last_became_active":"2024-09-17T03:35:45.951350+0000","last_became_peered":"2024-09-17T03:35:45.951350+0000","last_unstale":"2024-09-17T03:35:46.946273+0000","last_undegraded":"2024-09-17T03:35:46.946273+0000","last_fullsized":"2024-09-17T03:35:46.946273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963772+0000","last_change":"2024-09-17T03:35:44.961465+0000","last_active":"2024-09-17T03:35:46.963772+0000","last_peered":"2024-09-17T03:35:46.963772+0000","last_clean":"2024-09-17T03:35:46.963772+0000","last_became_active":"2024-09-17T03:35:44.961034+0000","last_became_peered":"2024-09-17T03:35:44.961034+0000","last_unstale":"2024-09-17T03:35:46.963772+0000","last_undegraded":"2024-09-17T03:35:46.963772+0000","last_fullsized":"2024-09-17T03:35:46.963772+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.962934+0000","last_change":"2024-09-17T03:35:45.959757+0000","last_active":"2024-09-17T03:35:45.962934+0000","last_peered":"2024-09-17T03:35:45.962934+0000","last_clean":"2024-09-17T03:35:45.962934+0000","last_became_active":"2024-09-17T03:35:45.959088+0000","last_became_peered":"2024-09-17T03:35:45.959088+0000","last_unstale":"2024-09-17T03:35:45.962934+0000","last_undegraded":"2024-09-17T03:35:45.962934+0000","last_fullsized":"2024-09-17T03:35:45.962934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:09.348110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950367+0000","last_change":"2024-09-17T03:35:44.962725+0000","last_active":"2024-09-17T03:35:45.950367+0000","last_peered":"2024-09-17T03:35:45.950367+0000","last_clean":"2024-09-17T03:35:45.950367+0000","last_became_active":"2024-09-17T03:35:44.962372+0000","last_became_peered":"2024-09-17T03:35:44.962372+0000","last_unstale":"2024-09-17T03:35:45.950367+0000","last_undegraded":"2024-09-17T03:35:45.950367+0000","last_fullsized":"2024-09-17T03:35:45.950367+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:27.452690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963667+0000","last_change":"2024-09-17T03:35:45.971272+0000","last_active":"2024-09-17T03:35:46.963667+0000","last_peered":"2024-09-17T03:35:46.963667+0000","last_clean":"2024-09-17T03:35:46.963667+0000","last_became_active":"2024-09-17T03:35:45.967588+0000","last_became_peered":"2024-09-17T03:35:45.967588+0000","last_unstale":"2024-09-17T03:35:46.963667+0000","last_undegraded":"2024-09-17T03:35:46.963667+0000","last_fullsized":"2024-09-17T03:35:46.963667+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:57.132363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942929+0000","last_change":"2024-09-17T03:35:44.955317+0000","last_active":"2024-09-17T03:35:46.942929+0000","last_peered":"2024-09-17T03:35:46.942929+0000","last_clean":"2024-09-17T03:35:46.942929+0000","last_became_active":"2024-09-17T03:35:44.952983+0000","last_became_peered":"2024-09-17T03:35:44.952983+0000","last_unstale":"2024-09-17T03:35:46.942929+0000","last_undegraded":"2024-09-17T03:35:46.942929+0000","last_fullsized":"2024-09-17T03:35:46.942929+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:52.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963619+0000","last_change":"2024-09-17T03:35:45.976105+0000","last_active":"2024-09-17T03:35:46.963619+0000","last_peered":"2024-09-17T03:35:46.963619+0000","last_clean":"2024-09-17T03:35:46.963619+0000","last_became_active":"2024-09-17T03:35:45.975807+0000","last_became_peered":"2024-09-17T03:35:45.975807+0000","last_unstale":"2024-09-17T03:35:46.963619+0000","last_undegraded":"2024-09-17T03:35:46.963619+0000","last_fullsized":"2024-09-17T03:35:46.963619+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:26.601742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116356+0000","last_change":"2024-09-17T03:35:44.959733+0000","last_active":"2024-09-17T03:35:47.116356+0000","last_peered":"2024-09-17T03:35:47.116356+0000","last_clean":"2024-09-17T03:35:47.116356+0000","last_became_active":"2024-09-17T03:35:44.959330+0000","last_became_peered":"2024-09-17T03:35:44.959330+0000","last_unstale":"2024-09-17T03:35:47.116356+0000","last_undegraded":"2024-09-17T03:35:47.116356+0000","last_fullsized":"2024-09-17T03:35:47.116356+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:34.223217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945544+0000","last_change":"2024-09-17T03:35:45.945250+0000","last_active":"2024-09-17T03:35:46.945544+0000","last_peered":"2024-09-17T03:35:46.945544+0000","last_clean":"2024-09-17T03:35:46.945544+0000","last_became_active":"2024-09-17T03:35:45.944882+0000","last_became_peered":"2024-09-17T03:35:45.944882+0000","last_unstale":"2024-09-17T03:35:46.945544+0000","last_undegraded":"2024-09-17T03:35:46.945544+0000","last_fullsized":"2024-09-17T03:35:46.945544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:03.455251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115803+0000","last_change":"2024-09-17T03:35:44.957196+0000","last_active":"2024-09-17T03:35:47.115803+0000","last_peered":"2024-09-17T03:35:47.115803+0000","last_clean":"2024-09-17T03:35:47.115803+0000","last_became_active":"2024-09-17T03:35:44.956846+0000","last_became_peered":"2024-09-17T03:35:44.956846+0000","last_unstale":"2024-09-17T03:35:47.115803+0000","last_undegraded":"2024-09-17T03:35:47.115803+0000","last_fullsized":"2024-09-17T03:35:47.115803+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:28.310407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953166+0000","last_change":"2024-09-17T03:35:45.975695+0000","last_active":"2024-09-17T03:35:46.953166+0000","last_peered":"2024-09-17T03:35:46.953166+0000","last_clean":"2024-09-17T03:35:46.953166+0000","last_became_active":"2024-09-17T03:35:45.975349+0000","last_became_peered":"2024-09-17T03:35:45.975349+0000","last_unstale":"2024-09-17T03:35:46.953166+0000","last_undegraded":"2024-09-17T03:35:46.953166+0000","last_fullsized":"2024-09-17T03:35:46.953166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:24.952949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945624+0000","last_change":"2024-09-17T03:35:44.952496+0000","last_active":"2024-09-17T03:35:46.945624+0000","last_peered":"2024-09-17T03:35:46.945624+0000","last_clean":"2024-09-17T03:35:46.945624+0000","last_became_active":"2024-09-17T03:35:44.952172+0000","last_became_peered":"2024-09-17T03:35:44.952172+0000","last_unstale":"2024-09-17T03:35:46.945624+0000","last_undegraded":"2024-09-17T03:35:46.945624+0000","last_fullsized":"2024-09-17T03:35:46.945624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.251927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978605+0000","last_change":"2024-09-17T03:35:45.970502+0000","last_active":"2024-09-17T03:35:45.978605+0000","last_peered":"2024-09-17T03:35:45.978605+0000","last_clean":"2024-09-17T03:35:45.978605+0000","last_became_active":"2024-09-17T03:35:45.967366+0000","last_became_peered":"2024-09-17T03:35:45.967366+0000","last_unstale":"2024-09-17T03:35:45.978605+0000","last_undegraded":"2024-09-17T03:35:45.978605+0000","last_fullsized":"2024-09-17T03:35:45.978605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:38.209115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952670+0000","last_change":"2024-09-17T03:35:44.958325+0000","last_active":"2024-09-17T03:35:46.952670+0000","last_peered":"2024-09-17T03:35:46.952670+0000","last_clean":"2024-09-17T03:35:46.952670+0000","last_became_active":"2024-09-17T03:35:44.958015+0000","last_became_peered":"2024-09-17T03:35:44.958015+0000","last_unstale":"2024-09-17T03:35:46.952670+0000","last_undegraded":"2024-09-17T03:35:46.952670+0000","last_fullsized":"2024-09-17T03:35:46.952670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:32.736499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114786+0000","last_change":"2024-09-17T03:35:45.955593+0000","last_active":"2024-09-17T03:35:47.114786+0000","last_peered":"2024-09-17T03:35:47.114786+0000","last_clean":"2024-09-17T03:35:47.114786+0000","last_became_active":"2024-09-17T03:35:45.954658+0000","last_became_peered":"2024-09-17T03:35:45.954658+0000","last_unstale":"2024-09-17T03:35:47.114786+0000","last_undegraded":"2024-09-17T03:35:47.114786+0000","last_fullsized":"2024-09-17T03:35:47.114786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114901+0000","last_change":"2024-09-17T03:35:44.951328+0000","last_active":"2024-09-17T03:35:47.114901+0000","last_peered":"2024-09-17T03:35:47.114901+0000","last_clean":"2024-09-17T03:35:47.114901+0000","last_became_active":"2024-09-17T03:35:44.951004+0000","last_became_peered":"2024-09-17T03:35:44.951004+0000","last_unstale":"2024-09-17T03:35:47.114901+0000","last_undegraded":"2024-09-17T03:35:47.114901+0000","last_fullsized":"2024-09-17T03:35:47.114901+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:57.623042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115774+0000","last_change":"2024-09-17T03:35:45.963042+0000","last_active":"2024-09-17T03:35:47.115774+0000","last_peered":"2024-09-17T03:35:47.115774+0000","last_clean":"2024-09-17T03:35:47.115774+0000","last_became_active":"2024-09-17T03:35:45.962752+0000","last_became_peered":"2024-09-17T03:35:45.962752+0000","last_unstale":"2024-09-17T03:35:47.115774+0000","last_undegraded":"2024-09-17T03:35:47.115774+0000","last_fullsized":"2024-09-17T03:35:47.115774+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:51.335684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950811+0000","last_change":"2024-09-17T03:35:44.962808+0000","last_active":"2024-09-17T03:35:45.950811+0000","last_peered":"2024-09-17T03:35:45.950811+0000","last_clean":"2024-09-17T03:35:45.950811+0000","last_became_active":"2024-09-17T03:35:44.962473+0000","last_became_peered":"2024-09-17T03:35:44.962473+0000","last_unstale":"2024-09-17T03:35:45.950811+0000","last_undegraded":"2024-09-17T03:35:45.950811+0000","last_fullsized":"2024-09-17T03:35:45.950811+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:59.547888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.967805+0000","last_change":"2024-09-17T03:35:45.963711+0000","last_active":"2024-09-17T03:35:45.967805+0000","last_peered":"2024-09-17T03:35:45.967805+0000","last_clean":"2024-09-17T03:35:45.967805+0000","last_became_active":"2024-09-17T03:35:45.963372+0000","last_became_peered":"2024-09-17T03:35:45.963372+0000","last_unstale":"2024-09-17T03:35:45.967805+0000","last_undegraded":"2024-09-17T03:35:45.967805+0000","last_fullsized":"2024-09-17T03:35:45.967805+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:56.111252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946333+0000","last_change":"2024-09-17T03:35:44.946163+0000","last_active":"2024-09-17T03:35:46.946333+0000","last_peered":"2024-09-17T03:35:46.946333+0000","last_clean":"2024-09-17T03:35:46.946333+0000","last_became_active":"2024-09-17T03:35:44.945021+0000","last_became_peered":"2024-09-17T03:35:44.945021+0000","last_unstale":"2024-09-17T03:35:46.946333+0000","last_undegraded":"2024-09-17T03:35:46.946333+0000","last_fullsized":"2024-09-17T03:35:46.946333+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963634+0000","last_change":"2024-09-17T03:35:45.972309+0000","last_active":"2024-09-17T03:35:46.963634+0000","last_peered":"2024-09-17T03:35:46.963634+0000","last_clean":"2024-09-17T03:35:46.963634+0000","last_became_active":"2024-09-17T03:35:45.971924+0000","last_became_peered":"2024-09-17T03:35:45.971924+0000","last_unstale":"2024-09-17T03:35:46.963634+0000","last_undegraded":"2024-09-17T03:35:46.963634+0000","last_fullsized":"2024-09-17T03:35:46.963634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:53.929773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.949681+0000","last_change":"2024-09-17T03:35:44.961523+0000","last_active":"2024-09-17T03:35:45.949681+0000","last_peered":"2024-09-17T03:35:45.949681+0000","last_clean":"2024-09-17T03:35:45.949681+0000","last_became_active":"2024-09-17T03:35:44.960356+0000","last_became_peered":"2024-09-17T03:35:44.960356+0000","last_unstale":"2024-09-17T03:35:45.949681+0000","last_undegraded":"2024-09-17T03:35:45.949681+0000","last_fullsized":"2024-09-17T03:35:45.949681+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.980673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963629+0000","last_change":"2024-09-17T03:35:45.964975+0000","last_active":"2024-09-17T03:35:46.963629+0000","last_peered":"2024-09-17T03:35:46.963629+0000","last_clean":"2024-09-17T03:35:46.963629+0000","last_became_active":"2024-09-17T03:35:45.964625+0000","last_became_peered":"2024-09-17T03:35:45.964625+0000","last_unstale":"2024-09-17T03:35:46.963629+0000","last_undegraded":"2024-09-17T03:35:46.963629+0000","last_fullsized":"2024-09-17T03:35:46.963629+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:14.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"128'6","reported_seq":19,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.988197+0000","last_change":"2024-09-17T03:35:44.957186+0000","last_active":"2024-09-17T03:35:45.988197+0000","last_peered":"2024-09-17T03:35:45.988197+0000","last_clean":"2024-09-17T03:35:45.988197+0000","last_became_active":"2024-09-17T03:35:44.955711+0000","last_became_peered":"2024-09-17T03:35:44.955711+0000","last_unstale":"2024-09-17T03:35:45.988197+0000","last_undegraded":"2024-09-17T03:35:45.988197+0000","last_fullsized":"2024-09-17T03:35:45.988197+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:14.988765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115670+0000","last_change":"2024-09-17T03:35:45.966154+0000","last_active":"2024-09-17T03:35:47.115670+0000","last_peered":"2024-09-17T03:35:47.115670+0000","last_clean":"2024-09-17T03:35:47.115670+0000","last_became_active":"2024-09-17T03:35:45.965306+0000","last_became_peered":"2024-09-17T03:35:45.965306+0000","last_unstale":"2024-09-17T03:35:47.115670+0000","last_undegraded":"2024-09-17T03:35:47.115670+0000","last_fullsized":"2024-09-17T03:35:47.115670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:33.592156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115553+0000","last_change":"2024-09-17T03:35:44.943415+0000","last_active":"2024-09-17T03:35:47.115553+0000","last_peered":"2024-09-17T03:35:47.115553+0000","last_clean":"2024-09-17T03:35:47.115553+0000","last_became_active":"2024-09-17T03:35:44.943067+0000","last_became_peered":"2024-09-17T03:35:44.943067+0000","last_unstale":"2024-09-17T03:35:47.115553+0000","last_undegraded":"2024-09-17T03:35:47.115553+0000","last_fullsized":"2024-09-17T03:35:47.115553+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:05.563987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114777+0000","last_change":"2024-09-17T03:35:45.952030+0000","last_active":"2024-09-17T03:35:47.114777+0000","last_peered":"2024-09-17T03:35:47.114777+0000","last_clean":"2024-09-17T03:35:47.114777+0000","last_became_active":"2024-09-17T03:35:45.951717+0000","last_became_peered":"2024-09-17T03:35:45.951717+0000","last_unstale":"2024-09-17T03:35:47.114777+0000","last_undegraded":"2024-09-17T03:35:47.114777+0000","last_fullsized":"2024-09-17T03:35:47.114777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:58.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115584+0000","last_change":"2024-09-17T03:35:44.952448+0000","last_active":"2024-09-17T03:35:47.115584+0000","last_peered":"2024-09-17T03:35:47.115584+0000","last_clean":"2024-09-17T03:35:47.115584+0000","last_became_active":"2024-09-17T03:35:44.952088+0000","last_became_peered":"2024-09-17T03:35:44.952088+0000","last_unstale":"2024-09-17T03:35:47.115584+0000","last_undegraded":"2024-09-17T03:35:47.115584+0000","last_fullsized":"2024-09-17T03:35:47.115584+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.522764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975135+0000","last_change":"2024-09-17T03:35:45.967590+0000","last_active":"2024-09-17T03:35:45.975135+0000","last_peered":"2024-09-17T03:35:45.975135+0000","last_clean":"2024-09-17T03:35:45.975135+0000","last_became_active":"2024-09-17T03:35:45.966946+0000","last_became_peered":"2024-09-17T03:35:45.966946+0000","last_unstale":"2024-09-17T03:35:45.975135+0000","last_undegraded":"2024-09-17T03:35:45.975135+0000","last_fullsized":"2024-09-17T03:35:45.975135+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:22.418085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"128'8","reported_seq":21,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.982528+0000","last_change":"2024-09-17T03:35:44.957549+0000","last_active":"2024-09-17T03:35:45.982528+0000","last_peered":"2024-09-17T03:35:45.982528+0000","last_clean":"2024-09-17T03:35:45.982528+0000","last_became_active":"2024-09-17T03:35:44.957170+0000","last_became_peered":"2024-09-17T03:35:44.957170+0000","last_unstale":"2024-09-17T03:35:45.982528+0000","last_undegraded":"2024-09-17T03:35:45.982528+0000","last_fullsized":"2024-09-17T03:35:45.982528+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:17.448836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977529+0000","last_change":"2024-09-17T03:35:45.972995+0000","last_active":"2024-09-17T03:35:45.977529+0000","last_peered":"2024-09-17T03:35:45.977529+0000","last_clean":"2024-09-17T03:35:45.977529+0000","last_became_active":"2024-09-17T03:35:45.972604+0000","last_became_peered":"2024-09-17T03:35:45.972604+0000","last_unstale":"2024-09-17T03:35:45.977529+0000","last_undegraded":"2024-09-17T03:35:45.977529+0000","last_fullsized":"2024-09-17T03:35:45.977529+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:25.307232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950936+0000","last_change":"2024-09-17T03:35:44.958004+0000","last_active":"2024-09-17T03:35:45.950936+0000","last_peered":"2024-09-17T03:35:45.950936+0000","last_clean":"2024-09-17T03:35:45.950936+0000","last_became_active":"2024-09-17T03:35:44.957642+0000","last_became_peered":"2024-09-17T03:35:44.957642+0000","last_unstale":"2024-09-17T03:35:45.950936+0000","last_undegraded":"2024-09-17T03:35:45.950936+0000","last_fullsized":"2024-09-17T03:35:45.950936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:31.183310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":3,"reported_epoch":128,"state":"creating+activating","last_fresh":"2024-09-17T03:35:45.947281+0000","last_change":"2024-09-17T03:35:45.947281+0000","last_active":"2024-09-17T03:35:44.919052+0000","last_peered":"2024-09-17T03:35:44.919052+0000","last_clean":"2024-09-17T03:35:44.919052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:35:45.947281+0000","last_undegraded":"2024-09-17T03:35:45.947281+0000","last_fullsized":"2024-09-17T03:35:45.947281+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942821+0000","last_change":"2024-09-17T03:35:44.954770+0000","last_active":"2024-09-17T03:35:46.942821+0000","last_peered":"2024-09-17T03:35:46.942821+0000","last_clean":"2024-09-17T03:35:46.942821+0000","last_became_active":"2024-09-17T03:35:44.953523+0000","last_became_peered":"2024-09-17T03:35:44.953523+0000","last_unstale":"2024-09-17T03:35:46.942821+0000","last_undegraded":"2024-09-17T03:35:46.942821+0000","last_fullsized":"2024-09-17T03:35:46.942821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:40.501189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945946+0000","last_change":"2024-09-17T03:35:45.950548+0000","last_active":"2024-09-17T03:35:46.945946+0000","last_peered":"2024-09-17T03:35:46.945946+0000","last_clean":"2024-09-17T03:35:46.945946+0000","last_became_active":"2024-09-17T03:35:45.949460+0000","last_became_peered":"2024-09-17T03:35:45.949460+0000","last_unstale":"2024-09-17T03:35:46.945946+0000","last_undegraded":"2024-09-17T03:35:46.945946+0000","last_fullsized":"2024-09-17T03:35:46.945946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953125+0000","last_change":"2024-09-17T03:35:44.961601+0000","last_active":"2024-09-17T03:35:46.953125+0000","last_peered":"2024-09-17T03:35:46.953125+0000","last_clean":"2024-09-17T03:35:46.953125+0000","last_became_active":"2024-09-17T03:35:44.961301+0000","last_became_peered":"2024-09-17T03:35:44.961301+0000","last_unstale":"2024-09-17T03:35:46.953125+0000","last_undegraded":"2024-09-17T03:35:46.953125+0000","last_fullsized":"2024-09-17T03:35:46.953125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:55.987785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116185+0000","last_change":"2024-09-17T03:35:45.958025+0000","last_active":"2024-09-17T03:35:47.116185+0000","last_peered":"2024-09-17T03:35:47.116185+0000","last_clean":"2024-09-17T03:35:47.116185+0000","last_became_active":"2024-09-17T03:35:45.957707+0000","last_became_peered":"2024-09-17T03:35:45.957707+0000","last_unstale":"2024-09-17T03:35:47.116185+0000","last_undegraded":"2024-09-17T03:35:47.116185+0000","last_fullsized":"2024-09-17T03:35:47.116185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.185058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.950067+0000","last_change":"2024-09-17T03:35:44.961854+0000","last_active":"2024-09-17T03:35:45.950067+0000","last_peered":"2024-09-17T03:35:45.950067+0000","last_clean":"2024-09-17T03:35:45.950067+0000","last_became_active":"2024-09-17T03:35:44.961082+0000","last_became_peered":"2024-09-17T03:35:44.961082+0000","last_unstale":"2024-09-17T03:35:45.950067+0000","last_undegraded":"2024-09-17T03:35:45.950067+0000","last_fullsized":"2024-09-17T03:35:45.950067+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:42.433406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115537+0000","last_change":"2024-09-17T03:35:45.960079+0000","last_active":"2024-09-17T03:35:47.115537+0000","last_peered":"2024-09-17T03:35:47.115537+0000","last_clean":"2024-09-17T03:35:47.115537+0000","last_became_active":"2024-09-17T03:35:45.959680+0000","last_became_peered":"2024-09-17T03:35:45.959680+0000","last_unstale":"2024-09-17T03:35:47.115537+0000","last_undegraded":"2024-09-17T03:35:47.115537+0000","last_fullsized":"2024-09-17T03:35:47.115537+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:24.060327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.949362+0000","last_change":"2024-09-17T03:35:44.958405+0000","last_active":"2024-09-17T03:35:45.949362+0000","last_peered":"2024-09-17T03:35:45.949362+0000","last_clean":"2024-09-17T03:35:45.949362+0000","last_became_active":"2024-09-17T03:35:44.958102+0000","last_became_peered":"2024-09-17T03:35:44.958102+0000","last_unstale":"2024-09-17T03:35:45.949362+0000","last_undegraded":"2024-09-17T03:35:45.949362+0000","last_fullsized":"2024-09-17T03:35:45.949362+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.755778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953353+0000","last_change":"2024-09-17T03:35:45.955137+0000","last_active":"2024-09-17T03:35:46.953353+0000","last_peered":"2024-09-17T03:35:46.953353+0000","last_clean":"2024-09-17T03:35:46.953353+0000","last_became_active":"2024-09-17T03:35:45.954787+0000","last_became_peered":"2024-09-17T03:35:45.954787+0000","last_unstale":"2024-09-17T03:35:46.953353+0000","last_undegraded":"2024-09-17T03:35:46.953353+0000","last_fullsized":"2024-09-17T03:35:46.953353+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:10.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945674+0000","last_change":"2024-09-17T03:35:44.950660+0000","last_active":"2024-09-17T03:35:46.945674+0000","last_peered":"2024-09-17T03:35:46.945674+0000","last_clean":"2024-09-17T03:35:46.945674+0000","last_became_active":"2024-09-17T03:35:44.949123+0000","last_became_peered":"2024-09-17T03:35:44.949123+0000","last_unstale":"2024-09-17T03:35:46.945674+0000","last_undegraded":"2024-09-17T03:35:46.945674+0000","last_fullsized":"2024-09-17T03:35:46.945674+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114873+0000","last_change":"2024-09-17T03:35:45.956952+0000","last_active":"2024-09-17T03:35:47.114873+0000","last_peered":"2024-09-17T03:35:47.114873+0000","last_clean":"2024-09-17T03:35:47.114873+0000","last_became_active":"2024-09-17T03:35:45.956620+0000","last_became_peered":"2024-09-17T03:35:45.956620+0000","last_unstale":"2024-09-17T03:35:47.114873+0000","last_undegraded":"2024-09-17T03:35:47.114873+0000","last_fullsized":"2024-09-17T03:35:47.114873+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:11.263633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945421+0000","last_change":"2024-09-17T03:35:44.947967+0000","last_active":"2024-09-17T03:35:46.945421+0000","last_peered":"2024-09-17T03:35:46.945421+0000","last_clean":"2024-09-17T03:35:46.945421+0000","last_became_active":"2024-09-17T03:35:44.947694+0000","last_became_peered":"2024-09-17T03:35:44.947694+0000","last_unstale":"2024-09-17T03:35:46.945421+0000","last_undegraded":"2024-09-17T03:35:46.945421+0000","last_fullsized":"2024-09-17T03:35:46.945421+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:00.392947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963284+0000","last_change":"2024-09-17T03:35:45.973217+0000","last_active":"2024-09-17T03:35:46.963284+0000","last_peered":"2024-09-17T03:35:46.963284+0000","last_clean":"2024-09-17T03:35:46.963284+0000","last_became_active":"2024-09-17T03:35:45.972910+0000","last_became_peered":"2024-09-17T03:35:45.972910+0000","last_unstale":"2024-09-17T03:35:46.963284+0000","last_undegraded":"2024-09-17T03:35:46.963284+0000","last_fullsized":"2024-09-17T03:35:46.963284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942268+0000","last_change":"2024-09-17T03:35:44.947744+0000","last_active":"2024-09-17T03:35:46.942268+0000","last_peered":"2024-09-17T03:35:46.942268+0000","last_clean":"2024-09-17T03:35:46.942268+0000","last_became_active":"2024-09-17T03:35:44.947045+0000","last_became_peered":"2024-09-17T03:35:44.947045+0000","last_unstale":"2024-09-17T03:35:46.942268+0000","last_undegraded":"2024-09-17T03:35:46.942268+0000","last_fullsized":"2024-09-17T03:35:46.942268+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:56.123881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953525+0000","last_change":"2024-09-17T03:35:45.963241+0000","last_active":"2024-09-17T03:35:46.953525+0000","last_peered":"2024-09-17T03:35:46.953525+0000","last_clean":"2024-09-17T03:35:46.953525+0000","last_became_active":"2024-09-17T03:35:45.962942+0000","last_became_peered":"2024-09-17T03:35:45.962942+0000","last_unstale":"2024-09-17T03:35:46.953525+0000","last_undegraded":"2024-09-17T03:35:46.953525+0000","last_fullsized":"2024-09-17T03:35:46.953525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:47.099040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115936+0000","last_change":"2024-09-17T03:35:44.950227+0000","last_active":"2024-09-17T03:35:47.115936+0000","last_peered":"2024-09-17T03:35:47.115936+0000","last_clean":"2024-09-17T03:35:47.115936+0000","last_became_active":"2024-09-17T03:35:44.949838+0000","last_became_peered":"2024-09-17T03:35:44.949838+0000","last_unstale":"2024-09-17T03:35:47.115936+0000","last_undegraded":"2024-09-17T03:35:47.115936+0000","last_fullsized":"2024-09-17T03:35:47.115936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:38.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115830+0000","last_change":"2024-09-17T03:35:45.952300+0000","last_active":"2024-09-17T03:35:47.115830+0000","last_peered":"2024-09-17T03:35:47.115830+0000","last_clean":"2024-09-17T03:35:47.115830+0000","last_became_active":"2024-09-17T03:35:45.951940+0000","last_became_peered":"2024-09-17T03:35:45.951940+0000","last_unstale":"2024-09-17T03:35:47.115830+0000","last_undegraded":"2024-09-17T03:35:47.115830+0000","last_fullsized":"2024-09-17T03:35:47.115830+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.804587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.965242+0000","last_change":"2024-09-17T03:35:44.960044+0000","last_active":"2024-09-17T03:35:46.965242+0000","last_peered":"2024-09-17T03:35:46.965242+0000","last_clean":"2024-09-17T03:35:46.965242+0000","last_became_active":"2024-09-17T03:35:44.959679+0000","last_became_peered":"2024-09-17T03:35:44.959679+0000","last_unstale":"2024-09-17T03:35:46.965242+0000","last_undegraded":"2024-09-17T03:35:46.965242+0000","last_fullsized":"2024-09-17T03:35:46.965242+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:26.474116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116521+0000","last_change":"2024-09-17T03:35:45.957578+0000","last_active":"2024-09-17T03:35:47.116521+0000","last_peered":"2024-09-17T03:35:47.116521+0000","last_clean":"2024-09-17T03:35:47.116521+0000","last_became_active":"2024-09-17T03:35:45.956972+0000","last_became_peered":"2024-09-17T03:35:45.956972+0000","last_unstale":"2024-09-17T03:35:47.116521+0000","last_undegraded":"2024-09-17T03:35:47.116521+0000","last_fullsized":"2024-09-17T03:35:47.116521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:37.224976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116267+0000","last_change":"2024-09-17T03:35:44.953819+0000","last_active":"2024-09-17T03:35:47.116267+0000","last_peered":"2024-09-17T03:35:47.116267+0000","last_clean":"2024-09-17T03:35:47.116267+0000","last_became_active":"2024-09-17T03:35:44.951031+0000","last_became_peered":"2024-09-17T03:35:44.951031+0000","last_unstale":"2024-09-17T03:35:47.116267+0000","last_undegraded":"2024-09-17T03:35:47.116267+0000","last_fullsized":"2024-09-17T03:35:47.116267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:41.554774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953195+0000","last_change":"2024-09-17T03:35:45.962505+0000","last_active":"2024-09-17T03:35:46.953195+0000","last_peered":"2024-09-17T03:35:46.953195+0000","last_clean":"2024-09-17T03:35:46.953195+0000","last_became_active":"2024-09-17T03:35:45.962169+0000","last_became_peered":"2024-09-17T03:35:45.962169+0000","last_unstale":"2024-09-17T03:35:46.953195+0000","last_undegraded":"2024-09-17T03:35:46.953195+0000","last_fullsized":"2024-09-17T03:35:46.953195+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:20.774078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942572+0000","last_change":"2024-09-17T03:35:44.956080+0000","last_active":"2024-09-17T03:35:46.942572+0000","last_peered":"2024-09-17T03:35:46.942572+0000","last_clean":"2024-09-17T03:35:46.942572+0000","last_became_active":"2024-09-17T03:35:44.955748+0000","last_became_peered":"2024-09-17T03:35:44.955748+0000","last_unstale":"2024-09-17T03:35:46.942572+0000","last_undegraded":"2024-09-17T03:35:46.942572+0000","last_fullsized":"2024-09-17T03:35:46.942572+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:04.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963292+0000","last_change":"2024-09-17T03:35:45.963696+0000","last_active":"2024-09-17T03:35:46.963292+0000","last_peered":"2024-09-17T03:35:46.963292+0000","last_clean":"2024-09-17T03:35:46.963292+0000","last_became_active":"2024-09-17T03:35:45.963323+0000","last_became_peered":"2024-09-17T03:35:45.963323+0000","last_unstale":"2024-09-17T03:35:46.963292+0000","last_undegraded":"2024-09-17T03:35:46.963292+0000","last_fullsized":"2024-09-17T03:35:46.963292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:59.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115974+0000","last_change":"2024-09-17T03:35:44.953364+0000","last_active":"2024-09-17T03:35:47.115974+0000","last_peered":"2024-09-17T03:35:47.115974+0000","last_clean":"2024-09-17T03:35:47.115974+0000","last_became_active":"2024-09-17T03:35:44.953034+0000","last_became_peered":"2024-09-17T03:35:44.953034+0000","last_unstale":"2024-09-17T03:35:47.115974+0000","last_undegraded":"2024-09-17T03:35:47.115974+0000","last_fullsized":"2024-09-17T03:35:47.115974+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:23.777269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.966928+0000","last_change":"2024-09-17T03:35:45.963406+0000","last_active":"2024-09-17T03:35:45.966928+0000","last_peered":"2024-09-17T03:35:45.966928+0000","last_clean":"2024-09-17T03:35:45.966928+0000","last_became_active":"2024-09-17T03:35:45.963028+0000","last_became_peered":"2024-09-17T03:35:45.963028+0000","last_unstale":"2024-09-17T03:35:45.966928+0000","last_undegraded":"2024-09-17T03:35:45.966928+0000","last_fullsized":"2024-09-17T03:35:45.966928+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:26.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963480+0000","last_change":"2024-09-17T03:35:44.961829+0000","last_active":"2024-09-17T03:35:46.963480+0000","last_peered":"2024-09-17T03:35:46.963480+0000","last_clean":"2024-09-17T03:35:46.963480+0000","last_became_active":"2024-09-17T03:35:44.961479+0000","last_became_peered":"2024-09-17T03:35:44.961479+0000","last_unstale":"2024-09-17T03:35:46.963480+0000","last_undegraded":"2024-09-17T03:35:46.963480+0000","last_fullsized":"2024-09-17T03:35:46.963480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945631+0000","last_change":"2024-09-17T03:35:45.954094+0000","last_active":"2024-09-17T03:35:46.945631+0000","last_peered":"2024-09-17T03:35:46.945631+0000","last_clean":"2024-09-17T03:35:46.945631+0000","last_became_active":"2024-09-17T03:35:45.952010+0000","last_became_peered":"2024-09-17T03:35:45.952010+0000","last_unstale":"2024-09-17T03:35:46.945631+0000","last_undegraded":"2024-09-17T03:35:46.945631+0000","last_fullsized":"2024-09-17T03:35:46.945631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.979607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945521+0000","last_change":"2024-09-17T03:35:44.944467+0000","last_active":"2024-09-17T03:35:46.945521+0000","last_peered":"2024-09-17T03:35:46.945521+0000","last_clean":"2024-09-17T03:35:46.945521+0000","last_became_active":"2024-09-17T03:35:44.944155+0000","last_became_peered":"2024-09-17T03:35:44.944155+0000","last_unstale":"2024-09-17T03:35:46.945521+0000","last_undegraded":"2024-09-17T03:35:46.945521+0000","last_fullsized":"2024-09-17T03:35:46.945521+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:00.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115398+0000","last_change":"2024-09-17T03:35:45.954997+0000","last_active":"2024-09-17T03:35:47.115398+0000","last_peered":"2024-09-17T03:35:47.115398+0000","last_clean":"2024-09-17T03:35:47.115398+0000","last_became_active":"2024-09-17T03:35:45.954184+0000","last_became_peered":"2024-09-17T03:35:45.954184+0000","last_unstale":"2024-09-17T03:35:47.115398+0000","last_undegraded":"2024-09-17T03:35:47.115398+0000","last_fullsized":"2024-09-17T03:35:47.115398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.149975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942555+0000","last_change":"2024-09-17T03:35:44.948114+0000","last_active":"2024-09-17T03:35:46.942555+0000","last_peered":"2024-09-17T03:35:46.942555+0000","last_clean":"2024-09-17T03:35:46.942555+0000","last_became_active":"2024-09-17T03:35:44.947433+0000","last_became_peered":"2024-09-17T03:35:44.947433+0000","last_unstale":"2024-09-17T03:35:46.942555+0000","last_undegraded":"2024-09-17T03:35:46.942555+0000","last_fullsized":"2024-09-17T03:35:46.942555+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:22.126761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963403+0000","last_change":"2024-09-17T03:35:45.968401+0000","last_active":"2024-09-17T03:35:46.963403+0000","last_peered":"2024-09-17T03:35:46.963403+0000","last_clean":"2024-09-17T03:35:46.963403+0000","last_became_active":"2024-09-17T03:35:45.968050+0000","last_became_peered":"2024-09-17T03:35:45.968050+0000","last_unstale":"2024-09-17T03:35:46.963403+0000","last_undegraded":"2024-09-17T03:35:46.963403+0000","last_fullsized":"2024-09-17T03:35:46.963403+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963509+0000","last_change":"2024-09-17T03:35:44.963161+0000","last_active":"2024-09-17T03:35:46.963509+0000","last_peered":"2024-09-17T03:35:46.963509+0000","last_clean":"2024-09-17T03:35:46.963509+0000","last_became_active":"2024-09-17T03:35:44.962822+0000","last_became_peered":"2024-09-17T03:35:44.962822+0000","last_unstale":"2024-09-17T03:35:46.963509+0000","last_undegraded":"2024-09-17T03:35:46.963509+0000","last_fullsized":"2024-09-17T03:35:46.963509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:12.300948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":309,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.962781+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:35:46.962781+0000","last_peered":"2024-09-17T03:35:46.962781+0000","last_clean":"2024-09-17T03:35:46.962781+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:35:46.962781+0000","last_undegraded":"2024-09-17T03:35:46.962781+0000","last_fullsized":"2024-09-17T03:35:46.962781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509621,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136000,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235840,"statfs":{"total":96636764160,"available":96497500160,"internally_reserved":0,"allocated":2207744,"data_stored":1286611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145140,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226700,"statfs":{"total":96636764160,"available":96488140800,"internally_reserved":0,"allocated":1601536,"data_stored":695605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140968,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230872,"statfs":{"total":96636764160,"available":96492412928,"internally_reserved":0,"allocated":1593344,"data_stored":694063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127084,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244756,"statfs":{"total":96636764160,"available":96506630144,"internally_reserved":0,"allocated":1597440,"data_stored":694153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163180,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208660,"statfs":{"total":96636764160,"available":96469667840,"internally_reserved":0,"allocated":1597440,"data_stored":694609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145704,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226136,"statfs":{"total":96636764160,"available":96487563264,"internally_reserved":0,"allocated":2183168,"data_stored":1284361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542325,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127068,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244772,"statfs":{"total":96636764160,"available":96506646528,"internally_reserved":0,"allocated":1576960,"data_stored":687190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167340,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204500,"statfs":{"total":96636764160,"available":96465408000,"internally_reserved":0,"allocated":1589248,"data_stored":693993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:35:50.988 INFO:tasks.ceph.ceph_manager.ceph:PG 19.24 is not active+clean 2024-09-17T03:35:50.988 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 128, 'state': 'creating+activating', 'last_fresh': '2024-09-17T03:35:45.948224+0000', 'last_change': '2024-09-17T03:35:45.948224+0000', 'last_active': '2024-09-17T03:35:44.919052+0000', 'last_peered': '2024-09-17T03:35:44.919052+0000', 'last_clean': '2024-09-17T03:35:44.919052+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:35:45.948224+0000', 'last_undegraded': '2024-09-17T03:35:45.948224+0000', 'last_fullsized': '2024-09-17T03:35:45.948224+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:35:44.919052+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:35:44.919052+0000', 'last_clean_scrub_stamp': '2024-09-17T03:35:44.919052+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:35:50.988 INFO:tasks.ceph.ceph_manager.ceph:PG 19.f is not active+clean 2024-09-17T03:35:50.988 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 128, 'state': 'creating+activating', 'last_fresh': '2024-09-17T03:35:45.947281+0000', 'last_change': '2024-09-17T03:35:45.947281+0000', 'last_active': '2024-09-17T03:35:44.919052+0000', 'last_peered': '2024-09-17T03:35:44.919052+0000', 'last_clean': '2024-09-17T03:35:44.919052+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:35:45.947281+0000', 'last_undegraded': '2024-09-17T03:35:45.947281+0000', 'last_fullsized': '2024-09-17T03:35:45.947281+0000', 'mapping_epoch': 127, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 127, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:35:44.919052+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:35:44.919052+0000', 'last_clean_scrub_stamp': '2024-09-17T03:35:44.919052+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:35:50.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:35:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 -- 172.21.15.46:0/2455444664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06fc90 msgr2=0x7f7fbc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2455444664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06fc90 0x7f7fbc0700a0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0d14d0 tx=0x7f7fb401caf0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 -- 172.21.15.46:0/2455444664 shutdown_connections 2024-09-17T03:35:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2455444664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06fc90 0x7f7fbc0700a0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2455444664 >> v2:172.21.15.99:3300/0 conn(0x7f7fbc06cdf0 0x7f7fbc06d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 -- 172.21.15.46:0/2455444664 >> 172.21.15.46:0/2455444664 conn(0x7f7fbc0b35f0 msgr2=0x7f7fbc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 -- 172.21.15.46:0/2455444664 shutdown_connections 2024-09-17T03:35:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.185+0000 7f7fc2400640 1 -- 172.21.15.46:0/2455444664 wait complete. 2024-09-17T03:35:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 Processor -- start 2024-09-17T03:35:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 -- start start 2024-09-17T03:35:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 0x7f7fbc0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fbc06fc90 0x7f7fbc153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fbc159130 0x7f7fbc1646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc168da0 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc168c20 con 0x7f7fbc159130 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc168f20 con 0x7f7fbc06fc90 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 0x7f7fbc0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 0x7f7fbc0cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33604/0 (socket says 172.21.15.46:33604) 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 -- 172.21.15.46:0/2018985044 learned_addr learned my addr 172.21.15.46:0/2018985044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbbe00640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fbc159130 0x7f7fbc1646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbaa00640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fbc06fc90 0x7f7fbc153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 -- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fbc06fc90 msgr2=0x7f7fbc153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fbc06fc90 0x7f7fbc153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 -- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fbc159130 msgr2=0x7f7fbc1646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fbc159130 0x7f7fbc1646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 -- 172.21.15.46:0/2018985044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc1f8e80 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbbe00640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fbc159130 0x7f7fbc1646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fbb400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 0x7f7fbc0cf820 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001ec0 tx=0x7f7fac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac017070 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fac003600 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc1f9160 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.189+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac003a50 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.193+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1f9720 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.193+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc06d200 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.193+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f7fac01a070 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.193+0000 7f7fb3400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f9c075240 0x7f7f9c077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.193+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f7fac0928c0 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.193+0000 7f7fbaa00640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f9c075240 0x7f7f9c077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.197+0000 7f7fbaa00640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f9c075240 0x7f7f9c077750 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc073a30 tx=0x7f7fa800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.197+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7fac021020 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.433+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7fbc050f70 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.433+0000 7f7fb3400640 1 -- 172.21.15.46:0/2018985044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1149 (secure 0 0 0) 0x7f7fac05ee70 con 0x7f7fbc06cdf0 2024-09-17T03:35:51.435 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:51.435 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":909,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1726543255,"num_in_osds":8,"osd_in_since":1726543245,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"creating+activating","count":2}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592666,"bytes_used":1180143616,"bytes_avail":771913969664,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":2177,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":84,"btime":"2024-09-17T03:35:48:955180+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":248335}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-09-17T03:35:27.556456+0000","services":{}},"progress_events":{}} 2024-09-17T03:35:51.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f9c075240 msgr2=0x7f7f9c077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f9c075240 0x7f7f9c077750 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc073a30 tx=0x7f7fa800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 msgr2=0x7f7fbc0cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 0x7f7fbc0cf820 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001ec0 tx=0x7f7fac00c890 comp rx=0 tx=0).stop 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 shutdown_connections 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7f9c075240 0x7f7f9c077750 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fbc159130 0x7f7fbc1646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7fbc06fc90 0x7f7fbc153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 --2- 172.21.15.46:0/2018985044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc06cdf0 0x7f7fbc0cf820 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 >> 172.21.15.46:0/2018985044 conn(0x7f7fbc0b35f0 msgr2=0x7f7fbc073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 shutdown_connections 2024-09-17T03:35:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.437+0000 7f7fc2400640 1 -- 172.21.15.46:0/2018985044 wait complete. 2024-09-17T03:35:51.458 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:35:51.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 -- 172.21.15.46:0/2677362178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c0b54f0 msgr2=0x7f322c155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 --2- 172.21.15.46:0/2677362178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c0b54f0 0x7f322c155b50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f3218009840 tx=0x7f321801c790 comp rx=0 tx=0).stop 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 -- 172.21.15.46:0/2677362178 shutdown_connections 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 --2- 172.21.15.46:0/2677362178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c0b54f0 0x7f322c155b50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 -- 172.21.15.46:0/2677362178 >> 172.21.15.46:0/2677362178 conn(0x7f322c0b2040 msgr2=0x7f322c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 -- 172.21.15.46:0/2677362178 shutdown_connections 2024-09-17T03:35:51.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.617+0000 7f3234000640 1 -- 172.21.15.46:0/2677362178 wait complete. 2024-09-17T03:35:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 Processor -- start 2024-09-17T03:35:51.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 -- start start 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 0x7f322c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c073a90 0x7f322c1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f322c152580 0x7f322c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f322c169ed0 con 0x7f322c072100 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f322c169d50 con 0x7f322c073a90 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f322c16a050 con 0x7f322c152580 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 0x7f322c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 0x7f322c144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33618/0 (socket says 172.21.15.46:33618) 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 -- 172.21.15.46:0/3402543286 learned_addr learned my addr 172.21.15.46:0/3402543286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:51.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3230c00640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c073a90 0x7f322c1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3232000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f322c152580 0x7f322c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 -- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f322c152580 msgr2=0x7f322c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f322c152580 0x7f322c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 -- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c073a90 msgr2=0x7f322c1482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c073a90 0x7f322c1482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 -- 172.21.15.46:0/3402543286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c0b16b0 con 0x7f322c072100 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3230c00640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c073a90 0x7f322c1482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:51.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3232000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f322c152580 0x7f322c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:51.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3231600640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 0x7f322c144db0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f322000c430 tx=0x7f322000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:51.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220011790 con 0x7f322c072100 2024-09-17T03:35:51.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3220017070 con 0x7f322c072100 2024-09-17T03:35:51.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c0afe70 con 0x7f322c072100 2024-09-17T03:35:51.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220011ca0 con 0x7f322c072100 2024-09-17T03:35:51.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.621+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c145db0 con 0x7f322c072100 2024-09-17T03:35:51.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.625+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c155b50 con 0x7f322c072100 2024-09-17T03:35:51.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.625+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f322001c460 con 0x7f322c072100 2024-09-17T03:35:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.625+0000 7f3229600640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f320c075240 0x7f320c077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.625+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f3220093b40 con 0x7f322c072100 2024-09-17T03:35:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.625+0000 7f3230c00640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f320c075240 0x7f320c077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.625+0000 7f3230c00640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f320c075240 0x7f320c077750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f32180af000 tx=0x7f321801afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:51.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.629+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32200606d0 con 0x7f322c072100 2024-09-17T03:35:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.801+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f322c06b540 con 0x7f320c075240 2024-09-17T03:35:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.805+0000 7f3229600640 1 -- 172.21.15.46:0/3402543286 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252496 (secure 0 0 0) 0x7f32180050c0 con 0x7f320c075240 2024-09-17T03:35:51.809 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:35:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f320c075240 msgr2=0x7f320c077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f320c075240 0x7f320c077750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f32180af000 tx=0x7f321801afd0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 msgr2=0x7f322c144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 0x7f322c144db0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f322000c430 tx=0x7f322000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 shutdown_connections 2024-09-17T03:35:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f320c075240 0x7f320c077750 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f322c152580 0x7f322c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f322c073a90 0x7f322c1482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 --2- 172.21.15.46:0/3402543286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f322c072100 0x7f322c144db0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 >> 172.21.15.46:0/3402543286 conn(0x7f322c0b2040 msgr2=0x7f322c14e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 shutdown_connections 2024-09-17T03:35:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:51.809+0000 7f3234000640 1 -- 172.21.15.46:0/3402543286 wait complete. 2024-09-17T03:35:51.830 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":570,"stamp":"2024-09-17T03:35:51.561693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1152512,"kb_used_data":13652,"kb_used_omap":12,"kb_used_meta":1138675,"kb_avail":753822208,"statfs":{"total":773094113280,"available":771913940992,"internally_reserved":0,"allocated":13979648,"data_stored":6743828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1166003760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.632097"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116128+0000","last_change":"2024-09-17T03:35:45.955819+0000","last_active":"2024-09-17T03:35:47.116128+0000","last_peered":"2024-09-17T03:35:47.116128+0000","last_clean":"2024-09-17T03:35:47.116128+0000","last_became_active":"2024-09-17T03:35:45.954935+0000","last_became_peered":"2024-09-17T03:35:45.954935+0000","last_unstale":"2024-09-17T03:35:47.116128+0000","last_undegraded":"2024-09-17T03:35:47.116128+0000","last_fullsized":"2024-09-17T03:35:47.116128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.363688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116488+0000","last_change":"2024-09-17T03:35:45.945240+0000","last_active":"2024-09-17T03:35:47.116488+0000","last_peered":"2024-09-17T03:35:47.116488+0000","last_clean":"2024-09-17T03:35:47.116488+0000","last_became_active":"2024-09-17T03:35:45.944788+0000","last_became_peered":"2024-09-17T03:35:45.944788+0000","last_unstale":"2024-09-17T03:35:47.116488+0000","last_undegraded":"2024-09-17T03:35:47.116488+0000","last_fullsized":"2024-09-17T03:35:47.116488+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:02.551861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.973644+0000","last_change":"2024-09-17T03:35:45.965545+0000","last_active":"2024-09-17T03:35:45.973644+0000","last_peered":"2024-09-17T03:35:45.973644+0000","last_clean":"2024-09-17T03:35:45.973644+0000","last_became_active":"2024-09-17T03:35:45.961480+0000","last_became_peered":"2024-09-17T03:35:45.961480+0000","last_unstale":"2024-09-17T03:35:45.973644+0000","last_undegraded":"2024-09-17T03:35:45.973644+0000","last_fullsized":"2024-09-17T03:35:45.973644+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:35.059698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115130+0000","last_change":"2024-09-17T03:35:45.960750+0000","last_active":"2024-09-17T03:35:47.115130+0000","last_peered":"2024-09-17T03:35:47.115130+0000","last_clean":"2024-09-17T03:35:47.115130+0000","last_became_active":"2024-09-17T03:35:45.960431+0000","last_became_peered":"2024-09-17T03:35:45.960431+0000","last_unstale":"2024-09-17T03:35:47.115130+0000","last_undegraded":"2024-09-17T03:35:47.115130+0000","last_fullsized":"2024-09-17T03:35:47.115130+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:59.838096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963365+0000","last_change":"2024-09-17T03:35:45.971487+0000","last_active":"2024-09-17T03:35:46.963365+0000","last_peered":"2024-09-17T03:35:46.963365+0000","last_clean":"2024-09-17T03:35:46.963365+0000","last_became_active":"2024-09-17T03:35:45.970309+0000","last_became_peered":"2024-09-17T03:35:45.970309+0000","last_unstale":"2024-09-17T03:35:46.963365+0000","last_undegraded":"2024-09-17T03:35:46.963365+0000","last_fullsized":"2024-09-17T03:35:46.963365+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953008+0000","last_change":"2024-09-17T03:35:45.961513+0000","last_active":"2024-09-17T03:35:46.953008+0000","last_peered":"2024-09-17T03:35:46.953008+0000","last_clean":"2024-09-17T03:35:46.953008+0000","last_became_active":"2024-09-17T03:35:45.961192+0000","last_became_peered":"2024-09-17T03:35:45.961192+0000","last_unstale":"2024-09-17T03:35:46.953008+0000","last_undegraded":"2024-09-17T03:35:46.953008+0000","last_fullsized":"2024-09-17T03:35:46.953008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:45.692040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952824+0000","last_change":"2024-09-17T03:35:45.955196+0000","last_active":"2024-09-17T03:35:46.952824+0000","last_peered":"2024-09-17T03:35:46.952824+0000","last_clean":"2024-09-17T03:35:46.952824+0000","last_became_active":"2024-09-17T03:35:45.954869+0000","last_became_peered":"2024-09-17T03:35:45.954869+0000","last_unstale":"2024-09-17T03:35:46.952824+0000","last_undegraded":"2024-09-17T03:35:46.952824+0000","last_fullsized":"2024-09-17T03:35:46.952824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:27.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978568+0000","last_change":"2024-09-17T03:35:45.968695+0000","last_active":"2024-09-17T03:35:45.978568+0000","last_peered":"2024-09-17T03:35:45.978568+0000","last_clean":"2024-09-17T03:35:45.978568+0000","last_became_active":"2024-09-17T03:35:45.966016+0000","last_became_peered":"2024-09-17T03:35:45.966016+0000","last_unstale":"2024-09-17T03:35:45.978568+0000","last_undegraded":"2024-09-17T03:35:45.978568+0000","last_fullsized":"2024-09-17T03:35:45.978568+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:29.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975084+0000","last_change":"2024-09-17T03:35:45.969067+0000","last_active":"2024-09-17T03:35:45.975084+0000","last_peered":"2024-09-17T03:35:45.975084+0000","last_clean":"2024-09-17T03:35:45.975084+0000","last_became_active":"2024-09-17T03:35:45.968667+0000","last_became_peered":"2024-09-17T03:35:45.968667+0000","last_unstale":"2024-09-17T03:35:45.975084+0000","last_undegraded":"2024-09-17T03:35:45.975084+0000","last_fullsized":"2024-09-17T03:35:45.975084+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:05:08.235660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116451+0000","last_change":"2024-09-17T03:35:45.945553+0000","last_active":"2024-09-17T03:35:47.116451+0000","last_peered":"2024-09-17T03:35:47.116451+0000","last_clean":"2024-09-17T03:35:47.116451+0000","last_became_active":"2024-09-17T03:35:45.945233+0000","last_became_peered":"2024-09-17T03:35:45.945233+0000","last_unstale":"2024-09-17T03:35:47.116451+0000","last_undegraded":"2024-09-17T03:35:47.116451+0000","last_fullsized":"2024-09-17T03:35:47.116451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:52.862294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942689+0000","last_change":"2024-09-17T03:35:45.972155+0000","last_active":"2024-09-17T03:35:46.942689+0000","last_peered":"2024-09-17T03:35:46.942689+0000","last_clean":"2024-09-17T03:35:46.942689+0000","last_became_active":"2024-09-17T03:35:45.971845+0000","last_became_peered":"2024-09-17T03:35:45.971845+0000","last_unstale":"2024-09-17T03:35:46.942689+0000","last_undegraded":"2024-09-17T03:35:46.942689+0000","last_fullsized":"2024-09-17T03:35:46.942689+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:13.103673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977386+0000","last_change":"2024-09-17T03:35:45.965493+0000","last_active":"2024-09-17T03:35:45.977386+0000","last_peered":"2024-09-17T03:35:45.977386+0000","last_clean":"2024-09-17T03:35:45.977386+0000","last_became_active":"2024-09-17T03:35:45.962965+0000","last_became_peered":"2024-09-17T03:35:45.962965+0000","last_unstale":"2024-09-17T03:35:45.977386+0000","last_undegraded":"2024-09-17T03:35:45.977386+0000","last_fullsized":"2024-09-17T03:35:45.977386+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:59.517858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115407+0000","last_change":"2024-09-17T03:35:45.957732+0000","last_active":"2024-09-17T03:35:47.115407+0000","last_peered":"2024-09-17T03:35:47.115407+0000","last_clean":"2024-09-17T03:35:47.115407+0000","last_became_active":"2024-09-17T03:35:45.957405+0000","last_became_peered":"2024-09-17T03:35:45.957405+0000","last_unstale":"2024-09-17T03:35:47.115407+0000","last_undegraded":"2024-09-17T03:35:47.115407+0000","last_fullsized":"2024-09-17T03:35:47.115407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:23.451219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115934+0000","last_change":"2024-09-17T03:35:45.952592+0000","last_active":"2024-09-17T03:35:47.115934+0000","last_peered":"2024-09-17T03:35:47.115934+0000","last_clean":"2024-09-17T03:35:47.115934+0000","last_became_active":"2024-09-17T03:35:45.949816+0000","last_became_peered":"2024-09-17T03:35:45.949816+0000","last_unstale":"2024-09-17T03:35:47.115934+0000","last_undegraded":"2024-09-17T03:35:47.115934+0000","last_fullsized":"2024-09-17T03:35:47.115934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:16.316918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114925+0000","last_change":"2024-09-17T03:35:45.958139+0000","last_active":"2024-09-17T03:35:47.114925+0000","last_peered":"2024-09-17T03:35:47.114925+0000","last_clean":"2024-09-17T03:35:47.114925+0000","last_became_active":"2024-09-17T03:35:45.957813+0000","last_became_peered":"2024-09-17T03:35:45.957813+0000","last_unstale":"2024-09-17T03:35:47.114925+0000","last_undegraded":"2024-09-17T03:35:47.114925+0000","last_fullsized":"2024-09-17T03:35:47.114925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:28.891920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945605+0000","last_change":"2024-09-17T03:35:45.950827+0000","last_active":"2024-09-17T03:35:46.945605+0000","last_peered":"2024-09-17T03:35:46.945605+0000","last_clean":"2024-09-17T03:35:46.945605+0000","last_became_active":"2024-09-17T03:35:45.950514+0000","last_became_peered":"2024-09-17T03:35:45.950514+0000","last_unstale":"2024-09-17T03:35:46.945605+0000","last_undegraded":"2024-09-17T03:35:46.945605+0000","last_fullsized":"2024-09-17T03:35:46.945605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:52.381399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953466+0000","last_change":"2024-09-17T03:35:45.964355+0000","last_active":"2024-09-17T03:35:46.953466+0000","last_peered":"2024-09-17T03:35:46.953466+0000","last_clean":"2024-09-17T03:35:46.953466+0000","last_became_active":"2024-09-17T03:35:45.962165+0000","last_became_peered":"2024-09-17T03:35:45.962165+0000","last_unstale":"2024-09-17T03:35:46.953466+0000","last_undegraded":"2024-09-17T03:35:46.953466+0000","last_fullsized":"2024-09-17T03:35:46.953466+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:26.092966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946215+0000","last_change":"2024-09-17T03:35:45.955274+0000","last_active":"2024-09-17T03:35:46.946215+0000","last_peered":"2024-09-17T03:35:46.946215+0000","last_clean":"2024-09-17T03:35:46.946215+0000","last_became_active":"2024-09-17T03:35:45.953802+0000","last_became_peered":"2024-09-17T03:35:45.953802+0000","last_unstale":"2024-09-17T03:35:46.946215+0000","last_undegraded":"2024-09-17T03:35:46.946215+0000","last_fullsized":"2024-09-17T03:35:46.946215+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942937+0000","last_change":"2024-09-17T03:35:45.963307+0000","last_active":"2024-09-17T03:35:46.942937+0000","last_peered":"2024-09-17T03:35:46.942937+0000","last_clean":"2024-09-17T03:35:46.942937+0000","last_became_active":"2024-09-17T03:35:45.963007+0000","last_became_peered":"2024-09-17T03:35:45.963007+0000","last_unstale":"2024-09-17T03:35:46.942937+0000","last_undegraded":"2024-09-17T03:35:46.942937+0000","last_fullsized":"2024-09-17T03:35:46.942937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:09.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945919+0000","last_change":"2024-09-17T03:35:45.950245+0000","last_active":"2024-09-17T03:35:46.945919+0000","last_peered":"2024-09-17T03:35:46.945919+0000","last_clean":"2024-09-17T03:35:46.945919+0000","last_became_active":"2024-09-17T03:35:45.949852+0000","last_became_peered":"2024-09-17T03:35:45.949852+0000","last_unstale":"2024-09-17T03:35:46.945919+0000","last_undegraded":"2024-09-17T03:35:46.945919+0000","last_fullsized":"2024-09-17T03:35:46.945919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:09.687961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963523+0000","last_change":"2024-09-17T03:35:45.967983+0000","last_active":"2024-09-17T03:35:46.963523+0000","last_peered":"2024-09-17T03:35:46.963523+0000","last_clean":"2024-09-17T03:35:46.963523+0000","last_became_active":"2024-09-17T03:35:45.964201+0000","last_became_peered":"2024-09-17T03:35:45.964201+0000","last_unstale":"2024-09-17T03:35:46.963523+0000","last_undegraded":"2024-09-17T03:35:46.963523+0000","last_fullsized":"2024-09-17T03:35:46.963523+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:26.825390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942483+0000","last_change":"2024-09-17T03:35:45.969671+0000","last_active":"2024-09-17T03:35:46.942483+0000","last_peered":"2024-09-17T03:35:46.942483+0000","last_clean":"2024-09-17T03:35:46.942483+0000","last_became_active":"2024-09-17T03:35:45.969304+0000","last_became_peered":"2024-09-17T03:35:45.969304+0000","last_unstale":"2024-09-17T03:35:46.942483+0000","last_undegraded":"2024-09-17T03:35:46.942483+0000","last_fullsized":"2024-09-17T03:35:46.942483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:30.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.972932+0000","last_change":"2024-09-17T03:35:45.962436+0000","last_active":"2024-09-17T03:35:45.972932+0000","last_peered":"2024-09-17T03:35:45.972932+0000","last_clean":"2024-09-17T03:35:45.972932+0000","last_became_active":"2024-09-17T03:35:45.962092+0000","last_became_peered":"2024-09-17T03:35:45.962092+0000","last_unstale":"2024-09-17T03:35:45.972932+0000","last_undegraded":"2024-09-17T03:35:45.972932+0000","last_fullsized":"2024-09-17T03:35:45.972932+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:33.151389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942550+0000","last_change":"2024-09-17T03:35:45.968660+0000","last_active":"2024-09-17T03:35:46.942550+0000","last_peered":"2024-09-17T03:35:46.942550+0000","last_clean":"2024-09-17T03:35:46.942550+0000","last_became_active":"2024-09-17T03:35:45.968290+0000","last_became_peered":"2024-09-17T03:35:45.968290+0000","last_unstale":"2024-09-17T03:35:46.942550+0000","last_undegraded":"2024-09-17T03:35:46.942550+0000","last_fullsized":"2024-09-17T03:35:46.942550+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:23.605758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116395+0000","last_change":"2024-09-17T03:35:45.950767+0000","last_active":"2024-09-17T03:35:47.116395+0000","last_peered":"2024-09-17T03:35:47.116395+0000","last_clean":"2024-09-17T03:35:47.116395+0000","last_became_active":"2024-09-17T03:35:45.950459+0000","last_became_peered":"2024-09-17T03:35:45.950459+0000","last_unstale":"2024-09-17T03:35:47.116395+0000","last_undegraded":"2024-09-17T03:35:47.116395+0000","last_fullsized":"2024-09-17T03:35:47.116395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:00.727158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946018+0000","last_change":"2024-09-17T03:35:45.951322+0000","last_active":"2024-09-17T03:35:46.946018+0000","last_peered":"2024-09-17T03:35:46.946018+0000","last_clean":"2024-09-17T03:35:46.946018+0000","last_became_active":"2024-09-17T03:35:45.950888+0000","last_became_peered":"2024-09-17T03:35:45.950888+0000","last_unstale":"2024-09-17T03:35:46.946018+0000","last_undegraded":"2024-09-17T03:35:46.946018+0000","last_fullsized":"2024-09-17T03:35:46.946018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115304+0000","last_change":"2024-09-17T03:35:45.958772+0000","last_active":"2024-09-17T03:35:47.115304+0000","last_peered":"2024-09-17T03:35:47.115304+0000","last_clean":"2024-09-17T03:35:47.115304+0000","last_became_active":"2024-09-17T03:35:45.958227+0000","last_became_peered":"2024-09-17T03:35:45.958227+0000","last_unstale":"2024-09-17T03:35:47.115304+0000","last_undegraded":"2024-09-17T03:35:47.115304+0000","last_fullsized":"2024-09-17T03:35:47.115304+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:36.638860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122499+0000","last_change":"2024-09-17T03:35:45.971522+0000","last_active":"2024-09-17T03:35:47.122499+0000","last_peered":"2024-09-17T03:35:47.122499+0000","last_clean":"2024-09-17T03:35:47.122499+0000","last_became_active":"2024-09-17T03:35:45.970943+0000","last_became_peered":"2024-09-17T03:35:45.970943+0000","last_unstale":"2024-09-17T03:35:47.122499+0000","last_undegraded":"2024-09-17T03:35:47.122499+0000","last_fullsized":"2024-09-17T03:35:47.122499+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:39.283255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115694+0000","last_change":"2024-09-17T03:35:45.957427+0000","last_active":"2024-09-17T03:35:47.115694+0000","last_peered":"2024-09-17T03:35:47.115694+0000","last_clean":"2024-09-17T03:35:47.115694+0000","last_became_active":"2024-09-17T03:35:45.956963+0000","last_became_peered":"2024-09-17T03:35:45.956963+0000","last_unstale":"2024-09-17T03:35:47.115694+0000","last_undegraded":"2024-09-17T03:35:47.115694+0000","last_fullsized":"2024-09-17T03:35:47.115694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.736866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115434+0000","last_change":"2024-09-17T03:35:45.959532+0000","last_active":"2024-09-17T03:35:47.115434+0000","last_peered":"2024-09-17T03:35:47.115434+0000","last_clean":"2024-09-17T03:35:47.115434+0000","last_became_active":"2024-09-17T03:35:45.952528+0000","last_became_peered":"2024-09-17T03:35:45.952528+0000","last_unstale":"2024-09-17T03:35:47.115434+0000","last_undegraded":"2024-09-17T03:35:47.115434+0000","last_fullsized":"2024-09-17T03:35:47.115434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:49.407151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963303+0000","last_change":"2024-09-17T03:35:45.975662+0000","last_active":"2024-09-17T03:35:46.963303+0000","last_peered":"2024-09-17T03:35:46.963303+0000","last_clean":"2024-09-17T03:35:46.963303+0000","last_became_active":"2024-09-17T03:35:45.975295+0000","last_became_peered":"2024-09-17T03:35:45.975295+0000","last_unstale":"2024-09-17T03:35:46.963303+0000","last_undegraded":"2024-09-17T03:35:46.963303+0000","last_fullsized":"2024-09-17T03:35:46.963303+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952833+0000","last_change":"2024-09-17T03:35:45.947162+0000","last_active":"2024-09-17T03:35:46.952833+0000","last_peered":"2024-09-17T03:35:46.952833+0000","last_clean":"2024-09-17T03:35:46.952833+0000","last_became_active":"2024-09-17T03:35:45.946769+0000","last_became_peered":"2024-09-17T03:35:45.946769+0000","last_unstale":"2024-09-17T03:35:46.952833+0000","last_undegraded":"2024-09-17T03:35:46.952833+0000","last_fullsized":"2024-09-17T03:35:46.952833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:12.153774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946273+0000","last_change":"2024-09-17T03:35:45.951693+0000","last_active":"2024-09-17T03:35:46.946273+0000","last_peered":"2024-09-17T03:35:46.946273+0000","last_clean":"2024-09-17T03:35:46.946273+0000","last_became_active":"2024-09-17T03:35:45.951350+0000","last_became_peered":"2024-09-17T03:35:45.951350+0000","last_unstale":"2024-09-17T03:35:46.946273+0000","last_undegraded":"2024-09-17T03:35:46.946273+0000","last_fullsized":"2024-09-17T03:35:46.946273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963772+0000","last_change":"2024-09-17T03:35:44.961465+0000","last_active":"2024-09-17T03:35:46.963772+0000","last_peered":"2024-09-17T03:35:46.963772+0000","last_clean":"2024-09-17T03:35:46.963772+0000","last_became_active":"2024-09-17T03:35:44.961034+0000","last_became_peered":"2024-09-17T03:35:44.961034+0000","last_unstale":"2024-09-17T03:35:46.963772+0000","last_undegraded":"2024-09-17T03:35:46.963772+0000","last_fullsized":"2024-09-17T03:35:46.963772+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.962934+0000","last_change":"2024-09-17T03:35:45.959757+0000","last_active":"2024-09-17T03:35:45.962934+0000","last_peered":"2024-09-17T03:35:45.962934+0000","last_clean":"2024-09-17T03:35:45.962934+0000","last_became_active":"2024-09-17T03:35:45.959088+0000","last_became_peered":"2024-09-17T03:35:45.959088+0000","last_unstale":"2024-09-17T03:35:45.962934+0000","last_undegraded":"2024-09-17T03:35:45.962934+0000","last_fullsized":"2024-09-17T03:35:45.962934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:09.348110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122174+0000","last_change":"2024-09-17T03:35:44.962725+0000","last_active":"2024-09-17T03:35:47.122174+0000","last_peered":"2024-09-17T03:35:47.122174+0000","last_clean":"2024-09-17T03:35:47.122174+0000","last_became_active":"2024-09-17T03:35:44.962372+0000","last_became_peered":"2024-09-17T03:35:44.962372+0000","last_unstale":"2024-09-17T03:35:47.122174+0000","last_undegraded":"2024-09-17T03:35:47.122174+0000","last_fullsized":"2024-09-17T03:35:47.122174+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:27.452690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963667+0000","last_change":"2024-09-17T03:35:45.971272+0000","last_active":"2024-09-17T03:35:46.963667+0000","last_peered":"2024-09-17T03:35:46.963667+0000","last_clean":"2024-09-17T03:35:46.963667+0000","last_became_active":"2024-09-17T03:35:45.967588+0000","last_became_peered":"2024-09-17T03:35:45.967588+0000","last_unstale":"2024-09-17T03:35:46.963667+0000","last_undegraded":"2024-09-17T03:35:46.963667+0000","last_fullsized":"2024-09-17T03:35:46.963667+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:57.132363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942929+0000","last_change":"2024-09-17T03:35:44.955317+0000","last_active":"2024-09-17T03:35:46.942929+0000","last_peered":"2024-09-17T03:35:46.942929+0000","last_clean":"2024-09-17T03:35:46.942929+0000","last_became_active":"2024-09-17T03:35:44.952983+0000","last_became_peered":"2024-09-17T03:35:44.952983+0000","last_unstale":"2024-09-17T03:35:46.942929+0000","last_undegraded":"2024-09-17T03:35:46.942929+0000","last_fullsized":"2024-09-17T03:35:46.942929+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:52.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963619+0000","last_change":"2024-09-17T03:35:45.976105+0000","last_active":"2024-09-17T03:35:46.963619+0000","last_peered":"2024-09-17T03:35:46.963619+0000","last_clean":"2024-09-17T03:35:46.963619+0000","last_became_active":"2024-09-17T03:35:45.975807+0000","last_became_peered":"2024-09-17T03:35:45.975807+0000","last_unstale":"2024-09-17T03:35:46.963619+0000","last_undegraded":"2024-09-17T03:35:46.963619+0000","last_fullsized":"2024-09-17T03:35:46.963619+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:26.601742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116356+0000","last_change":"2024-09-17T03:35:44.959733+0000","last_active":"2024-09-17T03:35:47.116356+0000","last_peered":"2024-09-17T03:35:47.116356+0000","last_clean":"2024-09-17T03:35:47.116356+0000","last_became_active":"2024-09-17T03:35:44.959330+0000","last_became_peered":"2024-09-17T03:35:44.959330+0000","last_unstale":"2024-09-17T03:35:47.116356+0000","last_undegraded":"2024-09-17T03:35:47.116356+0000","last_fullsized":"2024-09-17T03:35:47.116356+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:34.223217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945544+0000","last_change":"2024-09-17T03:35:45.945250+0000","last_active":"2024-09-17T03:35:46.945544+0000","last_peered":"2024-09-17T03:35:46.945544+0000","last_clean":"2024-09-17T03:35:46.945544+0000","last_became_active":"2024-09-17T03:35:45.944882+0000","last_became_peered":"2024-09-17T03:35:45.944882+0000","last_unstale":"2024-09-17T03:35:46.945544+0000","last_undegraded":"2024-09-17T03:35:46.945544+0000","last_fullsized":"2024-09-17T03:35:46.945544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:03.455251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115803+0000","last_change":"2024-09-17T03:35:44.957196+0000","last_active":"2024-09-17T03:35:47.115803+0000","last_peered":"2024-09-17T03:35:47.115803+0000","last_clean":"2024-09-17T03:35:47.115803+0000","last_became_active":"2024-09-17T03:35:44.956846+0000","last_became_peered":"2024-09-17T03:35:44.956846+0000","last_unstale":"2024-09-17T03:35:47.115803+0000","last_undegraded":"2024-09-17T03:35:47.115803+0000","last_fullsized":"2024-09-17T03:35:47.115803+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:28.310407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953166+0000","last_change":"2024-09-17T03:35:45.975695+0000","last_active":"2024-09-17T03:35:46.953166+0000","last_peered":"2024-09-17T03:35:46.953166+0000","last_clean":"2024-09-17T03:35:46.953166+0000","last_became_active":"2024-09-17T03:35:45.975349+0000","last_became_peered":"2024-09-17T03:35:45.975349+0000","last_unstale":"2024-09-17T03:35:46.953166+0000","last_undegraded":"2024-09-17T03:35:46.953166+0000","last_fullsized":"2024-09-17T03:35:46.953166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:24.952949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945624+0000","last_change":"2024-09-17T03:35:44.952496+0000","last_active":"2024-09-17T03:35:46.945624+0000","last_peered":"2024-09-17T03:35:46.945624+0000","last_clean":"2024-09-17T03:35:46.945624+0000","last_became_active":"2024-09-17T03:35:44.952172+0000","last_became_peered":"2024-09-17T03:35:44.952172+0000","last_unstale":"2024-09-17T03:35:46.945624+0000","last_undegraded":"2024-09-17T03:35:46.945624+0000","last_fullsized":"2024-09-17T03:35:46.945624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.251927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.978605+0000","last_change":"2024-09-17T03:35:45.970502+0000","last_active":"2024-09-17T03:35:45.978605+0000","last_peered":"2024-09-17T03:35:45.978605+0000","last_clean":"2024-09-17T03:35:45.978605+0000","last_became_active":"2024-09-17T03:35:45.967366+0000","last_became_peered":"2024-09-17T03:35:45.967366+0000","last_unstale":"2024-09-17T03:35:45.978605+0000","last_undegraded":"2024-09-17T03:35:45.978605+0000","last_fullsized":"2024-09-17T03:35:45.978605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:38.209115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952670+0000","last_change":"2024-09-17T03:35:44.958325+0000","last_active":"2024-09-17T03:35:46.952670+0000","last_peered":"2024-09-17T03:35:46.952670+0000","last_clean":"2024-09-17T03:35:46.952670+0000","last_became_active":"2024-09-17T03:35:44.958015+0000","last_became_peered":"2024-09-17T03:35:44.958015+0000","last_unstale":"2024-09-17T03:35:46.952670+0000","last_undegraded":"2024-09-17T03:35:46.952670+0000","last_fullsized":"2024-09-17T03:35:46.952670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:32.736499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114786+0000","last_change":"2024-09-17T03:35:45.955593+0000","last_active":"2024-09-17T03:35:47.114786+0000","last_peered":"2024-09-17T03:35:47.114786+0000","last_clean":"2024-09-17T03:35:47.114786+0000","last_became_active":"2024-09-17T03:35:45.954658+0000","last_became_peered":"2024-09-17T03:35:45.954658+0000","last_unstale":"2024-09-17T03:35:47.114786+0000","last_undegraded":"2024-09-17T03:35:47.114786+0000","last_fullsized":"2024-09-17T03:35:47.114786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114901+0000","last_change":"2024-09-17T03:35:44.951328+0000","last_active":"2024-09-17T03:35:47.114901+0000","last_peered":"2024-09-17T03:35:47.114901+0000","last_clean":"2024-09-17T03:35:47.114901+0000","last_became_active":"2024-09-17T03:35:44.951004+0000","last_became_peered":"2024-09-17T03:35:44.951004+0000","last_unstale":"2024-09-17T03:35:47.114901+0000","last_undegraded":"2024-09-17T03:35:47.114901+0000","last_fullsized":"2024-09-17T03:35:47.114901+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:57.623042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115774+0000","last_change":"2024-09-17T03:35:45.963042+0000","last_active":"2024-09-17T03:35:47.115774+0000","last_peered":"2024-09-17T03:35:47.115774+0000","last_clean":"2024-09-17T03:35:47.115774+0000","last_became_active":"2024-09-17T03:35:45.962752+0000","last_became_peered":"2024-09-17T03:35:45.962752+0000","last_unstale":"2024-09-17T03:35:47.115774+0000","last_undegraded":"2024-09-17T03:35:47.115774+0000","last_fullsized":"2024-09-17T03:35:47.115774+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:51.335684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122505+0000","last_change":"2024-09-17T03:35:44.962808+0000","last_active":"2024-09-17T03:35:47.122505+0000","last_peered":"2024-09-17T03:35:47.122505+0000","last_clean":"2024-09-17T03:35:47.122505+0000","last_became_active":"2024-09-17T03:35:44.962473+0000","last_became_peered":"2024-09-17T03:35:44.962473+0000","last_unstale":"2024-09-17T03:35:47.122505+0000","last_undegraded":"2024-09-17T03:35:47.122505+0000","last_fullsized":"2024-09-17T03:35:47.122505+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:59.547888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122180+0000","last_change":"2024-09-17T03:35:45.963711+0000","last_active":"2024-09-17T03:35:47.122180+0000","last_peered":"2024-09-17T03:35:47.122180+0000","last_clean":"2024-09-17T03:35:47.122180+0000","last_became_active":"2024-09-17T03:35:45.963372+0000","last_became_peered":"2024-09-17T03:35:45.963372+0000","last_unstale":"2024-09-17T03:35:47.122180+0000","last_undegraded":"2024-09-17T03:35:47.122180+0000","last_fullsized":"2024-09-17T03:35:47.122180+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:20.404664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946333+0000","last_change":"2024-09-17T03:35:44.946163+0000","last_active":"2024-09-17T03:35:46.946333+0000","last_peered":"2024-09-17T03:35:46.946333+0000","last_clean":"2024-09-17T03:35:46.946333+0000","last_became_active":"2024-09-17T03:35:44.945021+0000","last_became_peered":"2024-09-17T03:35:44.945021+0000","last_unstale":"2024-09-17T03:35:46.946333+0000","last_undegraded":"2024-09-17T03:35:46.946333+0000","last_fullsized":"2024-09-17T03:35:46.946333+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963634+0000","last_change":"2024-09-17T03:35:45.972309+0000","last_active":"2024-09-17T03:35:46.963634+0000","last_peered":"2024-09-17T03:35:46.963634+0000","last_clean":"2024-09-17T03:35:46.963634+0000","last_became_active":"2024-09-17T03:35:45.971924+0000","last_became_peered":"2024-09-17T03:35:45.971924+0000","last_unstale":"2024-09-17T03:35:46.963634+0000","last_undegraded":"2024-09-17T03:35:46.963634+0000","last_fullsized":"2024-09-17T03:35:46.963634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:53.929773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"128'3","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.121869+0000","last_change":"2024-09-17T03:35:44.961523+0000","last_active":"2024-09-17T03:35:47.121869+0000","last_peered":"2024-09-17T03:35:47.121869+0000","last_clean":"2024-09-17T03:35:47.121869+0000","last_became_active":"2024-09-17T03:35:44.960356+0000","last_became_peered":"2024-09-17T03:35:44.960356+0000","last_unstale":"2024-09-17T03:35:47.121869+0000","last_undegraded":"2024-09-17T03:35:47.121869+0000","last_fullsized":"2024-09-17T03:35:47.121869+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.980673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963629+0000","last_change":"2024-09-17T03:35:45.964975+0000","last_active":"2024-09-17T03:35:46.963629+0000","last_peered":"2024-09-17T03:35:46.963629+0000","last_clean":"2024-09-17T03:35:46.963629+0000","last_became_active":"2024-09-17T03:35:45.964625+0000","last_became_peered":"2024-09-17T03:35:45.964625+0000","last_unstale":"2024-09-17T03:35:46.963629+0000","last_undegraded":"2024-09-17T03:35:46.963629+0000","last_fullsized":"2024-09-17T03:35:46.963629+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:14.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"128'6","reported_seq":19,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.988197+0000","last_change":"2024-09-17T03:35:44.957186+0000","last_active":"2024-09-17T03:35:45.988197+0000","last_peered":"2024-09-17T03:35:45.988197+0000","last_clean":"2024-09-17T03:35:45.988197+0000","last_became_active":"2024-09-17T03:35:44.955711+0000","last_became_peered":"2024-09-17T03:35:44.955711+0000","last_unstale":"2024-09-17T03:35:45.988197+0000","last_undegraded":"2024-09-17T03:35:45.988197+0000","last_fullsized":"2024-09-17T03:35:45.988197+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:14.988765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115670+0000","last_change":"2024-09-17T03:35:45.966154+0000","last_active":"2024-09-17T03:35:47.115670+0000","last_peered":"2024-09-17T03:35:47.115670+0000","last_clean":"2024-09-17T03:35:47.115670+0000","last_became_active":"2024-09-17T03:35:45.965306+0000","last_became_peered":"2024-09-17T03:35:45.965306+0000","last_unstale":"2024-09-17T03:35:47.115670+0000","last_undegraded":"2024-09-17T03:35:47.115670+0000","last_fullsized":"2024-09-17T03:35:47.115670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:33.592156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115553+0000","last_change":"2024-09-17T03:35:44.943415+0000","last_active":"2024-09-17T03:35:47.115553+0000","last_peered":"2024-09-17T03:35:47.115553+0000","last_clean":"2024-09-17T03:35:47.115553+0000","last_became_active":"2024-09-17T03:35:44.943067+0000","last_became_peered":"2024-09-17T03:35:44.943067+0000","last_unstale":"2024-09-17T03:35:47.115553+0000","last_undegraded":"2024-09-17T03:35:47.115553+0000","last_fullsized":"2024-09-17T03:35:47.115553+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:05.563987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114777+0000","last_change":"2024-09-17T03:35:45.952030+0000","last_active":"2024-09-17T03:35:47.114777+0000","last_peered":"2024-09-17T03:35:47.114777+0000","last_clean":"2024-09-17T03:35:47.114777+0000","last_became_active":"2024-09-17T03:35:45.951717+0000","last_became_peered":"2024-09-17T03:35:45.951717+0000","last_unstale":"2024-09-17T03:35:47.114777+0000","last_undegraded":"2024-09-17T03:35:47.114777+0000","last_fullsized":"2024-09-17T03:35:47.114777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:58.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115584+0000","last_change":"2024-09-17T03:35:44.952448+0000","last_active":"2024-09-17T03:35:47.115584+0000","last_peered":"2024-09-17T03:35:47.115584+0000","last_clean":"2024-09-17T03:35:47.115584+0000","last_became_active":"2024-09-17T03:35:44.952088+0000","last_became_peered":"2024-09-17T03:35:44.952088+0000","last_unstale":"2024-09-17T03:35:47.115584+0000","last_undegraded":"2024-09-17T03:35:47.115584+0000","last_fullsized":"2024-09-17T03:35:47.115584+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.522764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.975135+0000","last_change":"2024-09-17T03:35:45.967590+0000","last_active":"2024-09-17T03:35:45.975135+0000","last_peered":"2024-09-17T03:35:45.975135+0000","last_clean":"2024-09-17T03:35:45.975135+0000","last_became_active":"2024-09-17T03:35:45.966946+0000","last_became_peered":"2024-09-17T03:35:45.966946+0000","last_unstale":"2024-09-17T03:35:45.975135+0000","last_undegraded":"2024-09-17T03:35:45.975135+0000","last_fullsized":"2024-09-17T03:35:45.975135+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:22.418085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"128'8","reported_seq":21,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.982528+0000","last_change":"2024-09-17T03:35:44.957549+0000","last_active":"2024-09-17T03:35:45.982528+0000","last_peered":"2024-09-17T03:35:45.982528+0000","last_clean":"2024-09-17T03:35:45.982528+0000","last_became_active":"2024-09-17T03:35:44.957170+0000","last_became_peered":"2024-09-17T03:35:44.957170+0000","last_unstale":"2024-09-17T03:35:45.982528+0000","last_undegraded":"2024-09-17T03:35:45.982528+0000","last_fullsized":"2024-09-17T03:35:45.982528+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:17.448836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T03:35:45.977529+0000","last_change":"2024-09-17T03:35:45.972995+0000","last_active":"2024-09-17T03:35:45.977529+0000","last_peered":"2024-09-17T03:35:45.977529+0000","last_clean":"2024-09-17T03:35:45.977529+0000","last_became_active":"2024-09-17T03:35:45.972604+0000","last_became_peered":"2024-09-17T03:35:45.972604+0000","last_unstale":"2024-09-17T03:35:45.977529+0000","last_undegraded":"2024-09-17T03:35:45.977529+0000","last_fullsized":"2024-09-17T03:35:45.977529+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:25.307232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122635+0000","last_change":"2024-09-17T03:35:44.958004+0000","last_active":"2024-09-17T03:35:47.122635+0000","last_peered":"2024-09-17T03:35:47.122635+0000","last_clean":"2024-09-17T03:35:47.122635+0000","last_became_active":"2024-09-17T03:35:44.957642+0000","last_became_peered":"2024-09-17T03:35:44.957642+0000","last_unstale":"2024-09-17T03:35:47.122635+0000","last_undegraded":"2024-09-17T03:35:47.122635+0000","last_fullsized":"2024-09-17T03:35:47.122635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:31.183310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122299+0000","last_change":"2024-09-17T03:35:45.976953+0000","last_active":"2024-09-17T03:35:47.122299+0000","last_peered":"2024-09-17T03:35:47.122299+0000","last_clean":"2024-09-17T03:35:47.122299+0000","last_became_active":"2024-09-17T03:35:45.976267+0000","last_became_peered":"2024-09-17T03:35:45.976267+0000","last_unstale":"2024-09-17T03:35:47.122299+0000","last_undegraded":"2024-09-17T03:35:47.122299+0000","last_fullsized":"2024-09-17T03:35:47.122299+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942821+0000","last_change":"2024-09-17T03:35:44.954770+0000","last_active":"2024-09-17T03:35:46.942821+0000","last_peered":"2024-09-17T03:35:46.942821+0000","last_clean":"2024-09-17T03:35:46.942821+0000","last_became_active":"2024-09-17T03:35:44.953523+0000","last_became_peered":"2024-09-17T03:35:44.953523+0000","last_unstale":"2024-09-17T03:35:46.942821+0000","last_undegraded":"2024-09-17T03:35:46.942821+0000","last_fullsized":"2024-09-17T03:35:46.942821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:40.501189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945946+0000","last_change":"2024-09-17T03:35:45.950548+0000","last_active":"2024-09-17T03:35:46.945946+0000","last_peered":"2024-09-17T03:35:46.945946+0000","last_clean":"2024-09-17T03:35:46.945946+0000","last_became_active":"2024-09-17T03:35:45.949460+0000","last_became_peered":"2024-09-17T03:35:45.949460+0000","last_unstale":"2024-09-17T03:35:46.945946+0000","last_undegraded":"2024-09-17T03:35:46.945946+0000","last_fullsized":"2024-09-17T03:35:46.945946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953125+0000","last_change":"2024-09-17T03:35:44.961601+0000","last_active":"2024-09-17T03:35:46.953125+0000","last_peered":"2024-09-17T03:35:46.953125+0000","last_clean":"2024-09-17T03:35:46.953125+0000","last_became_active":"2024-09-17T03:35:44.961301+0000","last_became_peered":"2024-09-17T03:35:44.961301+0000","last_unstale":"2024-09-17T03:35:46.953125+0000","last_undegraded":"2024-09-17T03:35:46.953125+0000","last_fullsized":"2024-09-17T03:35:46.953125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:55.987785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116185+0000","last_change":"2024-09-17T03:35:45.958025+0000","last_active":"2024-09-17T03:35:47.116185+0000","last_peered":"2024-09-17T03:35:47.116185+0000","last_clean":"2024-09-17T03:35:47.116185+0000","last_became_active":"2024-09-17T03:35:45.957707+0000","last_became_peered":"2024-09-17T03:35:45.957707+0000","last_unstale":"2024-09-17T03:35:47.116185+0000","last_undegraded":"2024-09-17T03:35:47.116185+0000","last_fullsized":"2024-09-17T03:35:47.116185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.185058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122700+0000","last_change":"2024-09-17T03:35:44.961854+0000","last_active":"2024-09-17T03:35:47.122700+0000","last_peered":"2024-09-17T03:35:47.122700+0000","last_clean":"2024-09-17T03:35:47.122700+0000","last_became_active":"2024-09-17T03:35:44.961082+0000","last_became_peered":"2024-09-17T03:35:44.961082+0000","last_unstale":"2024-09-17T03:35:47.122700+0000","last_undegraded":"2024-09-17T03:35:47.122700+0000","last_fullsized":"2024-09-17T03:35:47.122700+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:42.433406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115537+0000","last_change":"2024-09-17T03:35:45.960079+0000","last_active":"2024-09-17T03:35:47.115537+0000","last_peered":"2024-09-17T03:35:47.115537+0000","last_clean":"2024-09-17T03:35:47.115537+0000","last_became_active":"2024-09-17T03:35:45.959680+0000","last_became_peered":"2024-09-17T03:35:45.959680+0000","last_unstale":"2024-09-17T03:35:47.115537+0000","last_undegraded":"2024-09-17T03:35:47.115537+0000","last_fullsized":"2024-09-17T03:35:47.115537+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:24.060327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122129+0000","last_change":"2024-09-17T03:35:44.958405+0000","last_active":"2024-09-17T03:35:47.122129+0000","last_peered":"2024-09-17T03:35:47.122129+0000","last_clean":"2024-09-17T03:35:47.122129+0000","last_became_active":"2024-09-17T03:35:44.958102+0000","last_became_peered":"2024-09-17T03:35:44.958102+0000","last_unstale":"2024-09-17T03:35:47.122129+0000","last_undegraded":"2024-09-17T03:35:47.122129+0000","last_fullsized":"2024-09-17T03:35:47.122129+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.755778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953353+0000","last_change":"2024-09-17T03:35:45.955137+0000","last_active":"2024-09-17T03:35:46.953353+0000","last_peered":"2024-09-17T03:35:46.953353+0000","last_clean":"2024-09-17T03:35:46.953353+0000","last_became_active":"2024-09-17T03:35:45.954787+0000","last_became_peered":"2024-09-17T03:35:45.954787+0000","last_unstale":"2024-09-17T03:35:46.953353+0000","last_undegraded":"2024-09-17T03:35:46.953353+0000","last_fullsized":"2024-09-17T03:35:46.953353+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:10.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945674+0000","last_change":"2024-09-17T03:35:44.950660+0000","last_active":"2024-09-17T03:35:46.945674+0000","last_peered":"2024-09-17T03:35:46.945674+0000","last_clean":"2024-09-17T03:35:46.945674+0000","last_became_active":"2024-09-17T03:35:44.949123+0000","last_became_peered":"2024-09-17T03:35:44.949123+0000","last_unstale":"2024-09-17T03:35:46.945674+0000","last_undegraded":"2024-09-17T03:35:46.945674+0000","last_fullsized":"2024-09-17T03:35:46.945674+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114873+0000","last_change":"2024-09-17T03:35:45.956952+0000","last_active":"2024-09-17T03:35:47.114873+0000","last_peered":"2024-09-17T03:35:47.114873+0000","last_clean":"2024-09-17T03:35:47.114873+0000","last_became_active":"2024-09-17T03:35:45.956620+0000","last_became_peered":"2024-09-17T03:35:45.956620+0000","last_unstale":"2024-09-17T03:35:47.114873+0000","last_undegraded":"2024-09-17T03:35:47.114873+0000","last_fullsized":"2024-09-17T03:35:47.114873+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:11.263633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945421+0000","last_change":"2024-09-17T03:35:44.947967+0000","last_active":"2024-09-17T03:35:46.945421+0000","last_peered":"2024-09-17T03:35:46.945421+0000","last_clean":"2024-09-17T03:35:46.945421+0000","last_became_active":"2024-09-17T03:35:44.947694+0000","last_became_peered":"2024-09-17T03:35:44.947694+0000","last_unstale":"2024-09-17T03:35:46.945421+0000","last_undegraded":"2024-09-17T03:35:46.945421+0000","last_fullsized":"2024-09-17T03:35:46.945421+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:00.392947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963284+0000","last_change":"2024-09-17T03:35:45.973217+0000","last_active":"2024-09-17T03:35:46.963284+0000","last_peered":"2024-09-17T03:35:46.963284+0000","last_clean":"2024-09-17T03:35:46.963284+0000","last_became_active":"2024-09-17T03:35:45.972910+0000","last_became_peered":"2024-09-17T03:35:45.972910+0000","last_unstale":"2024-09-17T03:35:46.963284+0000","last_undegraded":"2024-09-17T03:35:46.963284+0000","last_fullsized":"2024-09-17T03:35:46.963284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942268+0000","last_change":"2024-09-17T03:35:44.947744+0000","last_active":"2024-09-17T03:35:46.942268+0000","last_peered":"2024-09-17T03:35:46.942268+0000","last_clean":"2024-09-17T03:35:46.942268+0000","last_became_active":"2024-09-17T03:35:44.947045+0000","last_became_peered":"2024-09-17T03:35:44.947045+0000","last_unstale":"2024-09-17T03:35:46.942268+0000","last_undegraded":"2024-09-17T03:35:46.942268+0000","last_fullsized":"2024-09-17T03:35:46.942268+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:56.123881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953525+0000","last_change":"2024-09-17T03:35:45.963241+0000","last_active":"2024-09-17T03:35:46.953525+0000","last_peered":"2024-09-17T03:35:46.953525+0000","last_clean":"2024-09-17T03:35:46.953525+0000","last_became_active":"2024-09-17T03:35:45.962942+0000","last_became_peered":"2024-09-17T03:35:45.962942+0000","last_unstale":"2024-09-17T03:35:46.953525+0000","last_undegraded":"2024-09-17T03:35:46.953525+0000","last_fullsized":"2024-09-17T03:35:46.953525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:47.099040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115936+0000","last_change":"2024-09-17T03:35:44.950227+0000","last_active":"2024-09-17T03:35:47.115936+0000","last_peered":"2024-09-17T03:35:47.115936+0000","last_clean":"2024-09-17T03:35:47.115936+0000","last_became_active":"2024-09-17T03:35:44.949838+0000","last_became_peered":"2024-09-17T03:35:44.949838+0000","last_unstale":"2024-09-17T03:35:47.115936+0000","last_undegraded":"2024-09-17T03:35:47.115936+0000","last_fullsized":"2024-09-17T03:35:47.115936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:38.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115830+0000","last_change":"2024-09-17T03:35:45.952300+0000","last_active":"2024-09-17T03:35:47.115830+0000","last_peered":"2024-09-17T03:35:47.115830+0000","last_clean":"2024-09-17T03:35:47.115830+0000","last_became_active":"2024-09-17T03:35:45.951940+0000","last_became_peered":"2024-09-17T03:35:45.951940+0000","last_unstale":"2024-09-17T03:35:47.115830+0000","last_undegraded":"2024-09-17T03:35:47.115830+0000","last_fullsized":"2024-09-17T03:35:47.115830+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.804587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.965242+0000","last_change":"2024-09-17T03:35:44.960044+0000","last_active":"2024-09-17T03:35:46.965242+0000","last_peered":"2024-09-17T03:35:46.965242+0000","last_clean":"2024-09-17T03:35:46.965242+0000","last_became_active":"2024-09-17T03:35:44.959679+0000","last_became_peered":"2024-09-17T03:35:44.959679+0000","last_unstale":"2024-09-17T03:35:46.965242+0000","last_undegraded":"2024-09-17T03:35:46.965242+0000","last_fullsized":"2024-09-17T03:35:46.965242+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:26.474116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116521+0000","last_change":"2024-09-17T03:35:45.957578+0000","last_active":"2024-09-17T03:35:47.116521+0000","last_peered":"2024-09-17T03:35:47.116521+0000","last_clean":"2024-09-17T03:35:47.116521+0000","last_became_active":"2024-09-17T03:35:45.956972+0000","last_became_peered":"2024-09-17T03:35:45.956972+0000","last_unstale":"2024-09-17T03:35:47.116521+0000","last_undegraded":"2024-09-17T03:35:47.116521+0000","last_fullsized":"2024-09-17T03:35:47.116521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:37.224976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116267+0000","last_change":"2024-09-17T03:35:44.953819+0000","last_active":"2024-09-17T03:35:47.116267+0000","last_peered":"2024-09-17T03:35:47.116267+0000","last_clean":"2024-09-17T03:35:47.116267+0000","last_became_active":"2024-09-17T03:35:44.951031+0000","last_became_peered":"2024-09-17T03:35:44.951031+0000","last_unstale":"2024-09-17T03:35:47.116267+0000","last_undegraded":"2024-09-17T03:35:47.116267+0000","last_fullsized":"2024-09-17T03:35:47.116267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:41.554774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953195+0000","last_change":"2024-09-17T03:35:45.962505+0000","last_active":"2024-09-17T03:35:46.953195+0000","last_peered":"2024-09-17T03:35:46.953195+0000","last_clean":"2024-09-17T03:35:46.953195+0000","last_became_active":"2024-09-17T03:35:45.962169+0000","last_became_peered":"2024-09-17T03:35:45.962169+0000","last_unstale":"2024-09-17T03:35:46.953195+0000","last_undegraded":"2024-09-17T03:35:46.953195+0000","last_fullsized":"2024-09-17T03:35:46.953195+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:20.774078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942572+0000","last_change":"2024-09-17T03:35:44.956080+0000","last_active":"2024-09-17T03:35:46.942572+0000","last_peered":"2024-09-17T03:35:46.942572+0000","last_clean":"2024-09-17T03:35:46.942572+0000","last_became_active":"2024-09-17T03:35:44.955748+0000","last_became_peered":"2024-09-17T03:35:44.955748+0000","last_unstale":"2024-09-17T03:35:46.942572+0000","last_undegraded":"2024-09-17T03:35:46.942572+0000","last_fullsized":"2024-09-17T03:35:46.942572+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:04.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963292+0000","last_change":"2024-09-17T03:35:45.963696+0000","last_active":"2024-09-17T03:35:46.963292+0000","last_peered":"2024-09-17T03:35:46.963292+0000","last_clean":"2024-09-17T03:35:46.963292+0000","last_became_active":"2024-09-17T03:35:45.963323+0000","last_became_peered":"2024-09-17T03:35:45.963323+0000","last_unstale":"2024-09-17T03:35:46.963292+0000","last_undegraded":"2024-09-17T03:35:46.963292+0000","last_fullsized":"2024-09-17T03:35:46.963292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:59.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115974+0000","last_change":"2024-09-17T03:35:44.953364+0000","last_active":"2024-09-17T03:35:47.115974+0000","last_peered":"2024-09-17T03:35:47.115974+0000","last_clean":"2024-09-17T03:35:47.115974+0000","last_became_active":"2024-09-17T03:35:44.953034+0000","last_became_peered":"2024-09-17T03:35:44.953034+0000","last_unstale":"2024-09-17T03:35:47.115974+0000","last_undegraded":"2024-09-17T03:35:47.115974+0000","last_fullsized":"2024-09-17T03:35:47.115974+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:23.777269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122087+0000","last_change":"2024-09-17T03:35:45.963406+0000","last_active":"2024-09-17T03:35:47.122087+0000","last_peered":"2024-09-17T03:35:47.122087+0000","last_clean":"2024-09-17T03:35:47.122087+0000","last_became_active":"2024-09-17T03:35:45.963028+0000","last_became_peered":"2024-09-17T03:35:45.963028+0000","last_unstale":"2024-09-17T03:35:47.122087+0000","last_undegraded":"2024-09-17T03:35:47.122087+0000","last_fullsized":"2024-09-17T03:35:47.122087+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:44.922295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963480+0000","last_change":"2024-09-17T03:35:44.961829+0000","last_active":"2024-09-17T03:35:46.963480+0000","last_peered":"2024-09-17T03:35:46.963480+0000","last_clean":"2024-09-17T03:35:46.963480+0000","last_became_active":"2024-09-17T03:35:44.961479+0000","last_became_peered":"2024-09-17T03:35:44.961479+0000","last_unstale":"2024-09-17T03:35:46.963480+0000","last_undegraded":"2024-09-17T03:35:46.963480+0000","last_fullsized":"2024-09-17T03:35:46.963480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945631+0000","last_change":"2024-09-17T03:35:45.954094+0000","last_active":"2024-09-17T03:35:46.945631+0000","last_peered":"2024-09-17T03:35:46.945631+0000","last_clean":"2024-09-17T03:35:46.945631+0000","last_became_active":"2024-09-17T03:35:45.952010+0000","last_became_peered":"2024-09-17T03:35:45.952010+0000","last_unstale":"2024-09-17T03:35:46.945631+0000","last_undegraded":"2024-09-17T03:35:46.945631+0000","last_fullsized":"2024-09-17T03:35:46.945631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.979607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945521+0000","last_change":"2024-09-17T03:35:44.944467+0000","last_active":"2024-09-17T03:35:46.945521+0000","last_peered":"2024-09-17T03:35:46.945521+0000","last_clean":"2024-09-17T03:35:46.945521+0000","last_became_active":"2024-09-17T03:35:44.944155+0000","last_became_peered":"2024-09-17T03:35:44.944155+0000","last_unstale":"2024-09-17T03:35:46.945521+0000","last_undegraded":"2024-09-17T03:35:46.945521+0000","last_fullsized":"2024-09-17T03:35:46.945521+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:00.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115398+0000","last_change":"2024-09-17T03:35:45.954997+0000","last_active":"2024-09-17T03:35:47.115398+0000","last_peered":"2024-09-17T03:35:47.115398+0000","last_clean":"2024-09-17T03:35:47.115398+0000","last_became_active":"2024-09-17T03:35:45.954184+0000","last_became_peered":"2024-09-17T03:35:45.954184+0000","last_unstale":"2024-09-17T03:35:47.115398+0000","last_undegraded":"2024-09-17T03:35:47.115398+0000","last_fullsized":"2024-09-17T03:35:47.115398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.149975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942555+0000","last_change":"2024-09-17T03:35:44.948114+0000","last_active":"2024-09-17T03:35:46.942555+0000","last_peered":"2024-09-17T03:35:46.942555+0000","last_clean":"2024-09-17T03:35:46.942555+0000","last_became_active":"2024-09-17T03:35:44.947433+0000","last_became_peered":"2024-09-17T03:35:44.947433+0000","last_unstale":"2024-09-17T03:35:46.942555+0000","last_undegraded":"2024-09-17T03:35:46.942555+0000","last_fullsized":"2024-09-17T03:35:46.942555+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:22.126761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963403+0000","last_change":"2024-09-17T03:35:45.968401+0000","last_active":"2024-09-17T03:35:46.963403+0000","last_peered":"2024-09-17T03:35:46.963403+0000","last_clean":"2024-09-17T03:35:46.963403+0000","last_became_active":"2024-09-17T03:35:45.968050+0000","last_became_peered":"2024-09-17T03:35:45.968050+0000","last_unstale":"2024-09-17T03:35:46.963403+0000","last_undegraded":"2024-09-17T03:35:46.963403+0000","last_fullsized":"2024-09-17T03:35:46.963403+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963509+0000","last_change":"2024-09-17T03:35:44.963161+0000","last_active":"2024-09-17T03:35:46.963509+0000","last_peered":"2024-09-17T03:35:46.963509+0000","last_clean":"2024-09-17T03:35:46.963509+0000","last_became_active":"2024-09-17T03:35:44.962822+0000","last_became_peered":"2024-09-17T03:35:44.962822+0000","last_unstale":"2024-09-17T03:35:46.963509+0000","last_undegraded":"2024-09-17T03:35:46.963509+0000","last_fullsized":"2024-09-17T03:35:46.963509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:12.300948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":309,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.962781+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:35:46.962781+0000","last_peered":"2024-09-17T03:35:46.962781+0000","last_clean":"2024-09-17T03:35:46.962781+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:35:46.962781+0000","last_undegraded":"2024-09-17T03:35:46.962781+0000","last_fullsized":"2024-09-17T03:35:46.962781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509621,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136000,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235840,"statfs":{"total":96636764160,"available":96497500160,"internally_reserved":0,"allocated":2207744,"data_stored":1286611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145140,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226700,"statfs":{"total":96636764160,"available":96488140800,"internally_reserved":0,"allocated":1601536,"data_stored":695605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140968,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230872,"statfs":{"total":96636764160,"available":96492412928,"internally_reserved":0,"allocated":1593344,"data_stored":694063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509622,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127084,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244756,"statfs":{"total":96636764160,"available":96506630144,"internally_reserved":0,"allocated":1597440,"data_stored":694153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509621,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163180,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208660,"statfs":{"total":96636764160,"available":96469667840,"internally_reserved":0,"allocated":1597440,"data_stored":694609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509621,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145704,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226136,"statfs":{"total":96636764160,"available":96487563264,"internally_reserved":0,"allocated":2183168,"data_stored":1284361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127096,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244744,"statfs":{"total":96636764160,"available":96506617856,"internally_reserved":0,"allocated":1609728,"data_stored":700433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542325,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167340,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204500,"statfs":{"total":96636764160,"available":96465408000,"internally_reserved":0,"allocated":1589248,"data_stored":693993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:35:54.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 -- 172.21.15.46:0/1125731578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80cefa0 msgr2=0x7f48a80cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 --2- 172.21.15.46:0/1125731578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80cefa0 0x7f48a80cf3b0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f48a80cfc90 tx=0x7f489001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 -- 172.21.15.46:0/1125731578 shutdown_connections 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 --2- 172.21.15.46:0/1125731578 >> v2:172.21.15.99:3300/0 conn(0x7f48a80b3340 0x7f48a80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 --2- 172.21.15.46:0/1125731578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80cefa0 0x7f48a80cf3b0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 -- 172.21.15.46:0/1125731578 >> 172.21.15.46:0/1125731578 conn(0x7f48a80b2040 msgr2=0x7f48a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 -- 172.21.15.46:0/1125731578 shutdown_connections 2024-09-17T03:35:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 -- 172.21.15.46:0/1125731578 wait complete. 2024-09-17T03:35:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 Processor -- start 2024-09-17T03:35:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.013+0000 7f48ace00640 1 -- start start 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 0x7f48a81f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48a80b3340 0x7f48a8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a80cefa0 0x7f48a81fc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8169d20 con 0x7f48a80749a0 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8169ba0 con 0x7f48a80cefa0 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f48a8169ea0 con 0x7f48a80b3340 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 0x7f48a81f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 0x7f48a81f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40550/0 (socket says 172.21.15.46:40550) 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 -- 172.21.15.46:0/2027670942 learned_addr learned my addr 172.21.15.46:0/2027670942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f489fe00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48a80b3340 0x7f48a8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a6800640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a80cefa0 0x7f48a81fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 -- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48a80b3340 msgr2=0x7f48a8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48a80b3340 0x7f48a8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 -- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a80cefa0 msgr2=0x7f48a81fc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a80cefa0 0x7f48a81fc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 -- 172.21.15.46:0/2027670942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a8145da0 con 0x7f48a80749a0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a6800640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a80cefa0 0x7f48a81fc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48a5e00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 0x7f48a81f8ad0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f489800c3f0 tx=0x7f489800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898017070 con 0x7f48a80749a0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48980053c0 con 0x7f48a80749a0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a8146080 con 0x7f48a80749a0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898003860 con 0x7f48a80749a0 2024-09-17T03:35:55.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a813ef10 con 0x7f48a80749a0 2024-09-17T03:35:55.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a80b3750 con 0x7f48a80749a0 2024-09-17T03:35:55.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.017+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f489801a070 con 0x7f48a80749a0 2024-09-17T03:35:55.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.021+0000 7f489e000640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4884075210 0x7f4884077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.021+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f4898092550 con 0x7f48a80749a0 2024-09-17T03:35:55.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.021+0000 7f489fe00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4884075210 0x7f4884077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.021+0000 7f489fe00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4884075210 0x7f4884077720 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f48900af000 tx=0x7f48900a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:55.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.025+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f489805ea10 con 0x7f48a80749a0 2024-09-17T03:35:55.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.197+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f48a8145b50 con 0x7f4884075210 2024-09-17T03:35:55.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.201+0000 7f489e000640 1 -- 172.21.15.46:0/2027670942 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252411 (secure 0 0 0) 0x7f48900faa10 con 0x7f4884075210 2024-09-17T03:35:55.203 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4884075210 msgr2=0x7f4884077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4884075210 0x7f4884077720 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f48900af000 tx=0x7f48900a7040 comp rx=0 tx=0).stop 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 msgr2=0x7f48a81f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 0x7f48a81f8ad0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f489800c3f0 tx=0x7f489800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 shutdown_connections 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4884075210 0x7f4884077720 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a80cefa0 0x7f48a81fc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48a80b3340 0x7f48a8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 --2- 172.21.15.46:0/2027670942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48a80749a0 0x7f48a81f8ad0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 >> 172.21.15.46:0/2027670942 conn(0x7f48a80b2040 msgr2=0x7f48a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 shutdown_connections 2024-09-17T03:35:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.205+0000 7f48ace00640 1 -- 172.21.15.46:0/2027670942 wait complete. 2024-09-17T03:35:55.226 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":571,"stamp":"2024-09-17T03:35:53.562183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1152612,"kb_used_data":13736,"kb_used_omap":12,"kb_used_meta":1138675,"kb_avail":753822108,"statfs":{"total":773094113280,"available":771913838592,"internally_reserved":0,"allocated":14065664,"data_stored":6788124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1166003760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001742"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116128+0000","last_change":"2024-09-17T03:35:45.955819+0000","last_active":"2024-09-17T03:35:47.116128+0000","last_peered":"2024-09-17T03:35:47.116128+0000","last_clean":"2024-09-17T03:35:47.116128+0000","last_became_active":"2024-09-17T03:35:45.954935+0000","last_became_peered":"2024-09-17T03:35:45.954935+0000","last_unstale":"2024-09-17T03:35:47.116128+0000","last_undegraded":"2024-09-17T03:35:47.116128+0000","last_fullsized":"2024-09-17T03:35:47.116128+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:34.363688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116488+0000","last_change":"2024-09-17T03:35:45.945240+0000","last_active":"2024-09-17T03:35:47.116488+0000","last_peered":"2024-09-17T03:35:47.116488+0000","last_clean":"2024-09-17T03:35:47.116488+0000","last_became_active":"2024-09-17T03:35:45.944788+0000","last_became_peered":"2024-09-17T03:35:45.944788+0000","last_unstale":"2024-09-17T03:35:47.116488+0000","last_undegraded":"2024-09-17T03:35:47.116488+0000","last_fullsized":"2024-09-17T03:35:47.116488+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:02.551861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122681+0000","last_change":"2024-09-17T03:35:45.965545+0000","last_active":"2024-09-17T03:35:47.122681+0000","last_peered":"2024-09-17T03:35:47.122681+0000","last_clean":"2024-09-17T03:35:47.122681+0000","last_became_active":"2024-09-17T03:35:45.961480+0000","last_became_peered":"2024-09-17T03:35:45.961480+0000","last_unstale":"2024-09-17T03:35:47.122681+0000","last_undegraded":"2024-09-17T03:35:47.122681+0000","last_fullsized":"2024-09-17T03:35:47.122681+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:00.523912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115130+0000","last_change":"2024-09-17T03:35:45.960750+0000","last_active":"2024-09-17T03:35:47.115130+0000","last_peered":"2024-09-17T03:35:47.115130+0000","last_clean":"2024-09-17T03:35:47.115130+0000","last_became_active":"2024-09-17T03:35:45.960431+0000","last_became_peered":"2024-09-17T03:35:45.960431+0000","last_unstale":"2024-09-17T03:35:47.115130+0000","last_undegraded":"2024-09-17T03:35:47.115130+0000","last_fullsized":"2024-09-17T03:35:47.115130+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:59.838096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963365+0000","last_change":"2024-09-17T03:35:45.971487+0000","last_active":"2024-09-17T03:35:46.963365+0000","last_peered":"2024-09-17T03:35:46.963365+0000","last_clean":"2024-09-17T03:35:46.963365+0000","last_became_active":"2024-09-17T03:35:45.970309+0000","last_became_peered":"2024-09-17T03:35:45.970309+0000","last_unstale":"2024-09-17T03:35:46.963365+0000","last_undegraded":"2024-09-17T03:35:46.963365+0000","last_fullsized":"2024-09-17T03:35:46.963365+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953008+0000","last_change":"2024-09-17T03:35:45.961513+0000","last_active":"2024-09-17T03:35:46.953008+0000","last_peered":"2024-09-17T03:35:46.953008+0000","last_clean":"2024-09-17T03:35:46.953008+0000","last_became_active":"2024-09-17T03:35:45.961192+0000","last_became_peered":"2024-09-17T03:35:45.961192+0000","last_unstale":"2024-09-17T03:35:46.953008+0000","last_undegraded":"2024-09-17T03:35:46.953008+0000","last_fullsized":"2024-09-17T03:35:46.953008+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:45.692040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952824+0000","last_change":"2024-09-17T03:35:45.955196+0000","last_active":"2024-09-17T03:35:46.952824+0000","last_peered":"2024-09-17T03:35:46.952824+0000","last_clean":"2024-09-17T03:35:46.952824+0000","last_became_active":"2024-09-17T03:35:45.954869+0000","last_became_peered":"2024-09-17T03:35:45.954869+0000","last_unstale":"2024-09-17T03:35:46.952824+0000","last_undegraded":"2024-09-17T03:35:46.952824+0000","last_fullsized":"2024-09-17T03:35:46.952824+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:27.814662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.121791+0000","last_change":"2024-09-17T03:35:45.968695+0000","last_active":"2024-09-17T03:35:47.121791+0000","last_peered":"2024-09-17T03:35:47.121791+0000","last_clean":"2024-09-17T03:35:47.121791+0000","last_became_active":"2024-09-17T03:35:45.966016+0000","last_became_peered":"2024-09-17T03:35:45.966016+0000","last_unstale":"2024-09-17T03:35:47.121791+0000","last_undegraded":"2024-09-17T03:35:47.121791+0000","last_fullsized":"2024-09-17T03:35:47.121791+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:51.352494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122889+0000","last_change":"2024-09-17T03:35:45.969067+0000","last_active":"2024-09-17T03:35:47.122889+0000","last_peered":"2024-09-17T03:35:47.122889+0000","last_clean":"2024-09-17T03:35:47.122889+0000","last_became_active":"2024-09-17T03:35:45.968667+0000","last_became_peered":"2024-09-17T03:35:45.968667+0000","last_unstale":"2024-09-17T03:35:47.122889+0000","last_undegraded":"2024-09-17T03:35:47.122889+0000","last_fullsized":"2024-09-17T03:35:47.122889+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:55.802747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116451+0000","last_change":"2024-09-17T03:35:45.945553+0000","last_active":"2024-09-17T03:35:47.116451+0000","last_peered":"2024-09-17T03:35:47.116451+0000","last_clean":"2024-09-17T03:35:47.116451+0000","last_became_active":"2024-09-17T03:35:45.945233+0000","last_became_peered":"2024-09-17T03:35:45.945233+0000","last_unstale":"2024-09-17T03:35:47.116451+0000","last_undegraded":"2024-09-17T03:35:47.116451+0000","last_fullsized":"2024-09-17T03:35:47.116451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:52.862294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942689+0000","last_change":"2024-09-17T03:35:45.972155+0000","last_active":"2024-09-17T03:35:46.942689+0000","last_peered":"2024-09-17T03:35:46.942689+0000","last_clean":"2024-09-17T03:35:46.942689+0000","last_became_active":"2024-09-17T03:35:45.971845+0000","last_became_peered":"2024-09-17T03:35:45.971845+0000","last_unstale":"2024-09-17T03:35:46.942689+0000","last_undegraded":"2024-09-17T03:35:46.942689+0000","last_fullsized":"2024-09-17T03:35:46.942689+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:13.103673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122569+0000","last_change":"2024-09-17T03:35:45.965493+0000","last_active":"2024-09-17T03:35:47.122569+0000","last_peered":"2024-09-17T03:35:47.122569+0000","last_clean":"2024-09-17T03:35:47.122569+0000","last_became_active":"2024-09-17T03:35:45.962965+0000","last_became_peered":"2024-09-17T03:35:45.962965+0000","last_unstale":"2024-09-17T03:35:47.122569+0000","last_undegraded":"2024-09-17T03:35:47.122569+0000","last_fullsized":"2024-09-17T03:35:47.122569+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:12.153774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115407+0000","last_change":"2024-09-17T03:35:45.957732+0000","last_active":"2024-09-17T03:35:47.115407+0000","last_peered":"2024-09-17T03:35:47.115407+0000","last_clean":"2024-09-17T03:35:47.115407+0000","last_became_active":"2024-09-17T03:35:45.957405+0000","last_became_peered":"2024-09-17T03:35:45.957405+0000","last_unstale":"2024-09-17T03:35:47.115407+0000","last_undegraded":"2024-09-17T03:35:47.115407+0000","last_fullsized":"2024-09-17T03:35:47.115407+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:23.451219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115934+0000","last_change":"2024-09-17T03:35:45.952592+0000","last_active":"2024-09-17T03:35:47.115934+0000","last_peered":"2024-09-17T03:35:47.115934+0000","last_clean":"2024-09-17T03:35:47.115934+0000","last_became_active":"2024-09-17T03:35:45.949816+0000","last_became_peered":"2024-09-17T03:35:45.949816+0000","last_unstale":"2024-09-17T03:35:47.115934+0000","last_undegraded":"2024-09-17T03:35:47.115934+0000","last_fullsized":"2024-09-17T03:35:47.115934+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:16.316918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114925+0000","last_change":"2024-09-17T03:35:45.958139+0000","last_active":"2024-09-17T03:35:47.114925+0000","last_peered":"2024-09-17T03:35:47.114925+0000","last_clean":"2024-09-17T03:35:47.114925+0000","last_became_active":"2024-09-17T03:35:45.957813+0000","last_became_peered":"2024-09-17T03:35:45.957813+0000","last_unstale":"2024-09-17T03:35:47.114925+0000","last_undegraded":"2024-09-17T03:35:47.114925+0000","last_fullsized":"2024-09-17T03:35:47.114925+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:28.891920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945605+0000","last_change":"2024-09-17T03:35:45.950827+0000","last_active":"2024-09-17T03:35:46.945605+0000","last_peered":"2024-09-17T03:35:46.945605+0000","last_clean":"2024-09-17T03:35:46.945605+0000","last_became_active":"2024-09-17T03:35:45.950514+0000","last_became_peered":"2024-09-17T03:35:45.950514+0000","last_unstale":"2024-09-17T03:35:46.945605+0000","last_undegraded":"2024-09-17T03:35:46.945605+0000","last_fullsized":"2024-09-17T03:35:46.945605+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:52.381399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953466+0000","last_change":"2024-09-17T03:35:45.964355+0000","last_active":"2024-09-17T03:35:46.953466+0000","last_peered":"2024-09-17T03:35:46.953466+0000","last_clean":"2024-09-17T03:35:46.953466+0000","last_became_active":"2024-09-17T03:35:45.962165+0000","last_became_peered":"2024-09-17T03:35:45.962165+0000","last_unstale":"2024-09-17T03:35:46.953466+0000","last_undegraded":"2024-09-17T03:35:46.953466+0000","last_fullsized":"2024-09-17T03:35:46.953466+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:26.092966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946215+0000","last_change":"2024-09-17T03:35:45.955274+0000","last_active":"2024-09-17T03:35:46.946215+0000","last_peered":"2024-09-17T03:35:46.946215+0000","last_clean":"2024-09-17T03:35:46.946215+0000","last_became_active":"2024-09-17T03:35:45.953802+0000","last_became_peered":"2024-09-17T03:35:45.953802+0000","last_unstale":"2024-09-17T03:35:46.946215+0000","last_undegraded":"2024-09-17T03:35:46.946215+0000","last_fullsized":"2024-09-17T03:35:46.946215+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942937+0000","last_change":"2024-09-17T03:35:45.963307+0000","last_active":"2024-09-17T03:35:46.942937+0000","last_peered":"2024-09-17T03:35:46.942937+0000","last_clean":"2024-09-17T03:35:46.942937+0000","last_became_active":"2024-09-17T03:35:45.963007+0000","last_became_peered":"2024-09-17T03:35:45.963007+0000","last_unstale":"2024-09-17T03:35:46.942937+0000","last_undegraded":"2024-09-17T03:35:46.942937+0000","last_fullsized":"2024-09-17T03:35:46.942937+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:09.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945919+0000","last_change":"2024-09-17T03:35:45.950245+0000","last_active":"2024-09-17T03:35:46.945919+0000","last_peered":"2024-09-17T03:35:46.945919+0000","last_clean":"2024-09-17T03:35:46.945919+0000","last_became_active":"2024-09-17T03:35:45.949852+0000","last_became_peered":"2024-09-17T03:35:45.949852+0000","last_unstale":"2024-09-17T03:35:46.945919+0000","last_undegraded":"2024-09-17T03:35:46.945919+0000","last_fullsized":"2024-09-17T03:35:46.945919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:09.687961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963523+0000","last_change":"2024-09-17T03:35:45.967983+0000","last_active":"2024-09-17T03:35:46.963523+0000","last_peered":"2024-09-17T03:35:46.963523+0000","last_clean":"2024-09-17T03:35:46.963523+0000","last_became_active":"2024-09-17T03:35:45.964201+0000","last_became_peered":"2024-09-17T03:35:45.964201+0000","last_unstale":"2024-09-17T03:35:46.963523+0000","last_undegraded":"2024-09-17T03:35:46.963523+0000","last_fullsized":"2024-09-17T03:35:46.963523+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:08:26.825390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942483+0000","last_change":"2024-09-17T03:35:45.969671+0000","last_active":"2024-09-17T03:35:46.942483+0000","last_peered":"2024-09-17T03:35:46.942483+0000","last_clean":"2024-09-17T03:35:46.942483+0000","last_became_active":"2024-09-17T03:35:45.969304+0000","last_became_peered":"2024-09-17T03:35:45.969304+0000","last_unstale":"2024-09-17T03:35:46.942483+0000","last_undegraded":"2024-09-17T03:35:46.942483+0000","last_fullsized":"2024-09-17T03:35:46.942483+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:30.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.121999+0000","last_change":"2024-09-17T03:35:45.962436+0000","last_active":"2024-09-17T03:35:47.121999+0000","last_peered":"2024-09-17T03:35:47.121999+0000","last_clean":"2024-09-17T03:35:47.121999+0000","last_became_active":"2024-09-17T03:35:45.962092+0000","last_became_peered":"2024-09-17T03:35:45.962092+0000","last_unstale":"2024-09-17T03:35:47.121999+0000","last_undegraded":"2024-09-17T03:35:47.121999+0000","last_fullsized":"2024-09-17T03:35:47.121999+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:51.763376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942550+0000","last_change":"2024-09-17T03:35:45.968660+0000","last_active":"2024-09-17T03:35:46.942550+0000","last_peered":"2024-09-17T03:35:46.942550+0000","last_clean":"2024-09-17T03:35:46.942550+0000","last_became_active":"2024-09-17T03:35:45.968290+0000","last_became_peered":"2024-09-17T03:35:45.968290+0000","last_unstale":"2024-09-17T03:35:46.942550+0000","last_undegraded":"2024-09-17T03:35:46.942550+0000","last_fullsized":"2024-09-17T03:35:46.942550+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:23.605758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116395+0000","last_change":"2024-09-17T03:35:45.950767+0000","last_active":"2024-09-17T03:35:47.116395+0000","last_peered":"2024-09-17T03:35:47.116395+0000","last_clean":"2024-09-17T03:35:47.116395+0000","last_became_active":"2024-09-17T03:35:45.950459+0000","last_became_peered":"2024-09-17T03:35:45.950459+0000","last_unstale":"2024-09-17T03:35:47.116395+0000","last_undegraded":"2024-09-17T03:35:47.116395+0000","last_fullsized":"2024-09-17T03:35:47.116395+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:00.727158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946018+0000","last_change":"2024-09-17T03:35:45.951322+0000","last_active":"2024-09-17T03:35:46.946018+0000","last_peered":"2024-09-17T03:35:46.946018+0000","last_clean":"2024-09-17T03:35:46.946018+0000","last_became_active":"2024-09-17T03:35:45.950888+0000","last_became_peered":"2024-09-17T03:35:45.950888+0000","last_unstale":"2024-09-17T03:35:46.946018+0000","last_undegraded":"2024-09-17T03:35:46.946018+0000","last_fullsized":"2024-09-17T03:35:46.946018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115304+0000","last_change":"2024-09-17T03:35:45.958772+0000","last_active":"2024-09-17T03:35:47.115304+0000","last_peered":"2024-09-17T03:35:47.115304+0000","last_clean":"2024-09-17T03:35:47.115304+0000","last_became_active":"2024-09-17T03:35:45.958227+0000","last_became_peered":"2024-09-17T03:35:45.958227+0000","last_unstale":"2024-09-17T03:35:47.115304+0000","last_undegraded":"2024-09-17T03:35:47.115304+0000","last_fullsized":"2024-09-17T03:35:47.115304+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:36.638860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122499+0000","last_change":"2024-09-17T03:35:45.971522+0000","last_active":"2024-09-17T03:35:47.122499+0000","last_peered":"2024-09-17T03:35:47.122499+0000","last_clean":"2024-09-17T03:35:47.122499+0000","last_became_active":"2024-09-17T03:35:45.970943+0000","last_became_peered":"2024-09-17T03:35:45.970943+0000","last_unstale":"2024-09-17T03:35:47.122499+0000","last_undegraded":"2024-09-17T03:35:47.122499+0000","last_fullsized":"2024-09-17T03:35:47.122499+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:39.283255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115694+0000","last_change":"2024-09-17T03:35:45.957427+0000","last_active":"2024-09-17T03:35:47.115694+0000","last_peered":"2024-09-17T03:35:47.115694+0000","last_clean":"2024-09-17T03:35:47.115694+0000","last_became_active":"2024-09-17T03:35:45.956963+0000","last_became_peered":"2024-09-17T03:35:45.956963+0000","last_unstale":"2024-09-17T03:35:47.115694+0000","last_undegraded":"2024-09-17T03:35:47.115694+0000","last_fullsized":"2024-09-17T03:35:47.115694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:17.736866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115434+0000","last_change":"2024-09-17T03:35:45.959532+0000","last_active":"2024-09-17T03:35:47.115434+0000","last_peered":"2024-09-17T03:35:47.115434+0000","last_clean":"2024-09-17T03:35:47.115434+0000","last_became_active":"2024-09-17T03:35:45.952528+0000","last_became_peered":"2024-09-17T03:35:45.952528+0000","last_unstale":"2024-09-17T03:35:47.115434+0000","last_undegraded":"2024-09-17T03:35:47.115434+0000","last_fullsized":"2024-09-17T03:35:47.115434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:49.407151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963303+0000","last_change":"2024-09-17T03:35:45.975662+0000","last_active":"2024-09-17T03:35:46.963303+0000","last_peered":"2024-09-17T03:35:46.963303+0000","last_clean":"2024-09-17T03:35:46.963303+0000","last_became_active":"2024-09-17T03:35:45.975295+0000","last_became_peered":"2024-09-17T03:35:45.975295+0000","last_unstale":"2024-09-17T03:35:46.963303+0000","last_undegraded":"2024-09-17T03:35:46.963303+0000","last_fullsized":"2024-09-17T03:35:46.963303+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:55.585022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952833+0000","last_change":"2024-09-17T03:35:45.947162+0000","last_active":"2024-09-17T03:35:46.952833+0000","last_peered":"2024-09-17T03:35:46.952833+0000","last_clean":"2024-09-17T03:35:46.952833+0000","last_became_active":"2024-09-17T03:35:45.946769+0000","last_became_peered":"2024-09-17T03:35:45.946769+0000","last_unstale":"2024-09-17T03:35:46.952833+0000","last_undegraded":"2024-09-17T03:35:46.952833+0000","last_fullsized":"2024-09-17T03:35:46.952833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:12.153774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946273+0000","last_change":"2024-09-17T03:35:45.951693+0000","last_active":"2024-09-17T03:35:46.946273+0000","last_peered":"2024-09-17T03:35:46.946273+0000","last_clean":"2024-09-17T03:35:46.946273+0000","last_became_active":"2024-09-17T03:35:45.951350+0000","last_became_peered":"2024-09-17T03:35:45.951350+0000","last_unstale":"2024-09-17T03:35:46.946273+0000","last_undegraded":"2024-09-17T03:35:46.946273+0000","last_fullsized":"2024-09-17T03:35:46.946273+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:57.242973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963772+0000","last_change":"2024-09-17T03:35:44.961465+0000","last_active":"2024-09-17T03:35:46.963772+0000","last_peered":"2024-09-17T03:35:46.963772+0000","last_clean":"2024-09-17T03:35:46.963772+0000","last_became_active":"2024-09-17T03:35:44.961034+0000","last_became_peered":"2024-09-17T03:35:44.961034+0000","last_unstale":"2024-09-17T03:35:46.963772+0000","last_undegraded":"2024-09-17T03:35:46.963772+0000","last_fullsized":"2024-09-17T03:35:46.963772+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.123053+0000","last_change":"2024-09-17T03:35:45.959757+0000","last_active":"2024-09-17T03:35:47.123053+0000","last_peered":"2024-09-17T03:35:47.123053+0000","last_clean":"2024-09-17T03:35:47.123053+0000","last_became_active":"2024-09-17T03:35:45.959088+0000","last_became_peered":"2024-09-17T03:35:45.959088+0000","last_unstale":"2024-09-17T03:35:47.123053+0000","last_undegraded":"2024-09-17T03:35:47.123053+0000","last_fullsized":"2024-09-17T03:35:47.123053+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:09.600007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122174+0000","last_change":"2024-09-17T03:35:44.962725+0000","last_active":"2024-09-17T03:35:47.122174+0000","last_peered":"2024-09-17T03:35:47.122174+0000","last_clean":"2024-09-17T03:35:47.122174+0000","last_became_active":"2024-09-17T03:35:44.962372+0000","last_became_peered":"2024-09-17T03:35:44.962372+0000","last_unstale":"2024-09-17T03:35:47.122174+0000","last_undegraded":"2024-09-17T03:35:47.122174+0000","last_fullsized":"2024-09-17T03:35:47.122174+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:27.452690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963667+0000","last_change":"2024-09-17T03:35:45.971272+0000","last_active":"2024-09-17T03:35:46.963667+0000","last_peered":"2024-09-17T03:35:46.963667+0000","last_clean":"2024-09-17T03:35:46.963667+0000","last_became_active":"2024-09-17T03:35:45.967588+0000","last_became_peered":"2024-09-17T03:35:45.967588+0000","last_unstale":"2024-09-17T03:35:46.963667+0000","last_undegraded":"2024-09-17T03:35:46.963667+0000","last_fullsized":"2024-09-17T03:35:46.963667+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:57.132363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942929+0000","last_change":"2024-09-17T03:35:44.955317+0000","last_active":"2024-09-17T03:35:46.942929+0000","last_peered":"2024-09-17T03:35:46.942929+0000","last_clean":"2024-09-17T03:35:46.942929+0000","last_became_active":"2024-09-17T03:35:44.952983+0000","last_became_peered":"2024-09-17T03:35:44.952983+0000","last_unstale":"2024-09-17T03:35:46.942929+0000","last_undegraded":"2024-09-17T03:35:46.942929+0000","last_fullsized":"2024-09-17T03:35:46.942929+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:52.312743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963619+0000","last_change":"2024-09-17T03:35:45.976105+0000","last_active":"2024-09-17T03:35:46.963619+0000","last_peered":"2024-09-17T03:35:46.963619+0000","last_clean":"2024-09-17T03:35:46.963619+0000","last_became_active":"2024-09-17T03:35:45.975807+0000","last_became_peered":"2024-09-17T03:35:45.975807+0000","last_unstale":"2024-09-17T03:35:46.963619+0000","last_undegraded":"2024-09-17T03:35:46.963619+0000","last_fullsized":"2024-09-17T03:35:46.963619+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:26.601742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116356+0000","last_change":"2024-09-17T03:35:44.959733+0000","last_active":"2024-09-17T03:35:47.116356+0000","last_peered":"2024-09-17T03:35:47.116356+0000","last_clean":"2024-09-17T03:35:47.116356+0000","last_became_active":"2024-09-17T03:35:44.959330+0000","last_became_peered":"2024-09-17T03:35:44.959330+0000","last_unstale":"2024-09-17T03:35:47.116356+0000","last_undegraded":"2024-09-17T03:35:47.116356+0000","last_fullsized":"2024-09-17T03:35:47.116356+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:34.223217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945544+0000","last_change":"2024-09-17T03:35:45.945250+0000","last_active":"2024-09-17T03:35:46.945544+0000","last_peered":"2024-09-17T03:35:46.945544+0000","last_clean":"2024-09-17T03:35:46.945544+0000","last_became_active":"2024-09-17T03:35:45.944882+0000","last_became_peered":"2024-09-17T03:35:45.944882+0000","last_unstale":"2024-09-17T03:35:46.945544+0000","last_undegraded":"2024-09-17T03:35:46.945544+0000","last_fullsized":"2024-09-17T03:35:46.945544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:03.455251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115803+0000","last_change":"2024-09-17T03:35:44.957196+0000","last_active":"2024-09-17T03:35:47.115803+0000","last_peered":"2024-09-17T03:35:47.115803+0000","last_clean":"2024-09-17T03:35:47.115803+0000","last_became_active":"2024-09-17T03:35:44.956846+0000","last_became_peered":"2024-09-17T03:35:44.956846+0000","last_unstale":"2024-09-17T03:35:47.115803+0000","last_undegraded":"2024-09-17T03:35:47.115803+0000","last_fullsized":"2024-09-17T03:35:47.115803+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:28.310407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953166+0000","last_change":"2024-09-17T03:35:45.975695+0000","last_active":"2024-09-17T03:35:46.953166+0000","last_peered":"2024-09-17T03:35:46.953166+0000","last_clean":"2024-09-17T03:35:46.953166+0000","last_became_active":"2024-09-17T03:35:45.975349+0000","last_became_peered":"2024-09-17T03:35:45.975349+0000","last_unstale":"2024-09-17T03:35:46.953166+0000","last_undegraded":"2024-09-17T03:35:46.953166+0000","last_fullsized":"2024-09-17T03:35:46.953166+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:11:24.952949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945624+0000","last_change":"2024-09-17T03:35:44.952496+0000","last_active":"2024-09-17T03:35:46.945624+0000","last_peered":"2024-09-17T03:35:46.945624+0000","last_clean":"2024-09-17T03:35:46.945624+0000","last_became_active":"2024-09-17T03:35:44.952172+0000","last_became_peered":"2024-09-17T03:35:44.952172+0000","last_unstale":"2024-09-17T03:35:46.945624+0000","last_undegraded":"2024-09-17T03:35:46.945624+0000","last_fullsized":"2024-09-17T03:35:46.945624+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:01.251927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122861+0000","last_change":"2024-09-17T03:35:45.970502+0000","last_active":"2024-09-17T03:35:47.122861+0000","last_peered":"2024-09-17T03:35:47.122861+0000","last_clean":"2024-09-17T03:35:47.122861+0000","last_became_active":"2024-09-17T03:35:45.967366+0000","last_became_peered":"2024-09-17T03:35:45.967366+0000","last_unstale":"2024-09-17T03:35:47.122861+0000","last_undegraded":"2024-09-17T03:35:47.122861+0000","last_fullsized":"2024-09-17T03:35:47.122861+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:35.952340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.952670+0000","last_change":"2024-09-17T03:35:44.958325+0000","last_active":"2024-09-17T03:35:46.952670+0000","last_peered":"2024-09-17T03:35:46.952670+0000","last_clean":"2024-09-17T03:35:46.952670+0000","last_became_active":"2024-09-17T03:35:44.958015+0000","last_became_peered":"2024-09-17T03:35:44.958015+0000","last_unstale":"2024-09-17T03:35:46.952670+0000","last_undegraded":"2024-09-17T03:35:46.952670+0000","last_fullsized":"2024-09-17T03:35:46.952670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:32.736499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114786+0000","last_change":"2024-09-17T03:35:45.955593+0000","last_active":"2024-09-17T03:35:47.114786+0000","last_peered":"2024-09-17T03:35:47.114786+0000","last_clean":"2024-09-17T03:35:47.114786+0000","last_became_active":"2024-09-17T03:35:45.954658+0000","last_became_peered":"2024-09-17T03:35:45.954658+0000","last_unstale":"2024-09-17T03:35:47.114786+0000","last_undegraded":"2024-09-17T03:35:47.114786+0000","last_fullsized":"2024-09-17T03:35:47.114786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114901+0000","last_change":"2024-09-17T03:35:44.951328+0000","last_active":"2024-09-17T03:35:47.114901+0000","last_peered":"2024-09-17T03:35:47.114901+0000","last_clean":"2024-09-17T03:35:47.114901+0000","last_became_active":"2024-09-17T03:35:44.951004+0000","last_became_peered":"2024-09-17T03:35:44.951004+0000","last_unstale":"2024-09-17T03:35:47.114901+0000","last_undegraded":"2024-09-17T03:35:47.114901+0000","last_fullsized":"2024-09-17T03:35:47.114901+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:57.623042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115774+0000","last_change":"2024-09-17T03:35:45.963042+0000","last_active":"2024-09-17T03:35:47.115774+0000","last_peered":"2024-09-17T03:35:47.115774+0000","last_clean":"2024-09-17T03:35:47.115774+0000","last_became_active":"2024-09-17T03:35:45.962752+0000","last_became_peered":"2024-09-17T03:35:45.962752+0000","last_unstale":"2024-09-17T03:35:47.115774+0000","last_undegraded":"2024-09-17T03:35:47.115774+0000","last_fullsized":"2024-09-17T03:35:47.115774+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:51.335684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122505+0000","last_change":"2024-09-17T03:35:44.962808+0000","last_active":"2024-09-17T03:35:47.122505+0000","last_peered":"2024-09-17T03:35:47.122505+0000","last_clean":"2024-09-17T03:35:47.122505+0000","last_became_active":"2024-09-17T03:35:44.962473+0000","last_became_peered":"2024-09-17T03:35:44.962473+0000","last_unstale":"2024-09-17T03:35:47.122505+0000","last_undegraded":"2024-09-17T03:35:47.122505+0000","last_fullsized":"2024-09-17T03:35:47.122505+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:59.547888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122180+0000","last_change":"2024-09-17T03:35:45.963711+0000","last_active":"2024-09-17T03:35:47.122180+0000","last_peered":"2024-09-17T03:35:47.122180+0000","last_clean":"2024-09-17T03:35:47.122180+0000","last_became_active":"2024-09-17T03:35:45.963372+0000","last_became_peered":"2024-09-17T03:35:45.963372+0000","last_unstale":"2024-09-17T03:35:47.122180+0000","last_undegraded":"2024-09-17T03:35:47.122180+0000","last_fullsized":"2024-09-17T03:35:47.122180+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:20.404664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.946333+0000","last_change":"2024-09-17T03:35:44.946163+0000","last_active":"2024-09-17T03:35:46.946333+0000","last_peered":"2024-09-17T03:35:46.946333+0000","last_clean":"2024-09-17T03:35:46.946333+0000","last_became_active":"2024-09-17T03:35:44.945021+0000","last_became_peered":"2024-09-17T03:35:44.945021+0000","last_unstale":"2024-09-17T03:35:46.946333+0000","last_undegraded":"2024-09-17T03:35:46.946333+0000","last_fullsized":"2024-09-17T03:35:46.946333+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:32.319093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963634+0000","last_change":"2024-09-17T03:35:45.972309+0000","last_active":"2024-09-17T03:35:46.963634+0000","last_peered":"2024-09-17T03:35:46.963634+0000","last_clean":"2024-09-17T03:35:46.963634+0000","last_became_active":"2024-09-17T03:35:45.971924+0000","last_became_peered":"2024-09-17T03:35:45.971924+0000","last_unstale":"2024-09-17T03:35:46.963634+0000","last_undegraded":"2024-09-17T03:35:46.963634+0000","last_fullsized":"2024-09-17T03:35:46.963634+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:53.929773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"128'3","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.121869+0000","last_change":"2024-09-17T03:35:44.961523+0000","last_active":"2024-09-17T03:35:47.121869+0000","last_peered":"2024-09-17T03:35:47.121869+0000","last_clean":"2024-09-17T03:35:47.121869+0000","last_became_active":"2024-09-17T03:35:44.960356+0000","last_became_peered":"2024-09-17T03:35:44.960356+0000","last_unstale":"2024-09-17T03:35:47.121869+0000","last_undegraded":"2024-09-17T03:35:47.121869+0000","last_fullsized":"2024-09-17T03:35:47.121869+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:50.980673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963629+0000","last_change":"2024-09-17T03:35:45.964975+0000","last_active":"2024-09-17T03:35:46.963629+0000","last_peered":"2024-09-17T03:35:46.963629+0000","last_clean":"2024-09-17T03:35:46.963629+0000","last_became_active":"2024-09-17T03:35:45.964625+0000","last_became_peered":"2024-09-17T03:35:45.964625+0000","last_unstale":"2024-09-17T03:35:46.963629+0000","last_undegraded":"2024-09-17T03:35:46.963629+0000","last_fullsized":"2024-09-17T03:35:46.963629+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:14.777696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"128'6","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122770+0000","last_change":"2024-09-17T03:35:44.957186+0000","last_active":"2024-09-17T03:35:47.122770+0000","last_peered":"2024-09-17T03:35:47.122770+0000","last_clean":"2024-09-17T03:35:47.122770+0000","last_became_active":"2024-09-17T03:35:44.955711+0000","last_became_peered":"2024-09-17T03:35:44.955711+0000","last_unstale":"2024-09-17T03:35:47.122770+0000","last_undegraded":"2024-09-17T03:35:47.122770+0000","last_fullsized":"2024-09-17T03:35:47.122770+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:14.988765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115670+0000","last_change":"2024-09-17T03:35:45.966154+0000","last_active":"2024-09-17T03:35:47.115670+0000","last_peered":"2024-09-17T03:35:47.115670+0000","last_clean":"2024-09-17T03:35:47.115670+0000","last_became_active":"2024-09-17T03:35:45.965306+0000","last_became_peered":"2024-09-17T03:35:45.965306+0000","last_unstale":"2024-09-17T03:35:47.115670+0000","last_undegraded":"2024-09-17T03:35:47.115670+0000","last_fullsized":"2024-09-17T03:35:47.115670+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:33.592156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115553+0000","last_change":"2024-09-17T03:35:44.943415+0000","last_active":"2024-09-17T03:35:47.115553+0000","last_peered":"2024-09-17T03:35:47.115553+0000","last_clean":"2024-09-17T03:35:47.115553+0000","last_became_active":"2024-09-17T03:35:44.943067+0000","last_became_peered":"2024-09-17T03:35:44.943067+0000","last_unstale":"2024-09-17T03:35:47.115553+0000","last_undegraded":"2024-09-17T03:35:47.115553+0000","last_fullsized":"2024-09-17T03:35:47.115553+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:05.563987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114777+0000","last_change":"2024-09-17T03:35:45.952030+0000","last_active":"2024-09-17T03:35:47.114777+0000","last_peered":"2024-09-17T03:35:47.114777+0000","last_clean":"2024-09-17T03:35:47.114777+0000","last_became_active":"2024-09-17T03:35:45.951717+0000","last_became_peered":"2024-09-17T03:35:45.951717+0000","last_unstale":"2024-09-17T03:35:47.114777+0000","last_undegraded":"2024-09-17T03:35:47.114777+0000","last_fullsized":"2024-09-17T03:35:47.114777+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:58.634767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115584+0000","last_change":"2024-09-17T03:35:44.952448+0000","last_active":"2024-09-17T03:35:47.115584+0000","last_peered":"2024-09-17T03:35:47.115584+0000","last_clean":"2024-09-17T03:35:47.115584+0000","last_became_active":"2024-09-17T03:35:44.952088+0000","last_became_peered":"2024-09-17T03:35:44.952088+0000","last_unstale":"2024-09-17T03:35:47.115584+0000","last_undegraded":"2024-09-17T03:35:47.115584+0000","last_fullsized":"2024-09-17T03:35:47.115584+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.522764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.121829+0000","last_change":"2024-09-17T03:35:45.967590+0000","last_active":"2024-09-17T03:35:47.121829+0000","last_peered":"2024-09-17T03:35:47.121829+0000","last_clean":"2024-09-17T03:35:47.121829+0000","last_became_active":"2024-09-17T03:35:45.966946+0000","last_became_peered":"2024-09-17T03:35:45.966946+0000","last_unstale":"2024-09-17T03:35:47.121829+0000","last_undegraded":"2024-09-17T03:35:47.121829+0000","last_fullsized":"2024-09-17T03:35:47.121829+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:16.210499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"128'8","reported_seq":23,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122006+0000","last_change":"2024-09-17T03:35:44.957549+0000","last_active":"2024-09-17T03:35:47.122006+0000","last_peered":"2024-09-17T03:35:47.122006+0000","last_clean":"2024-09-17T03:35:47.122006+0000","last_became_active":"2024-09-17T03:35:44.957170+0000","last_became_peered":"2024-09-17T03:35:44.957170+0000","last_unstale":"2024-09-17T03:35:47.122006+0000","last_undegraded":"2024-09-17T03:35:47.122006+0000","last_fullsized":"2024-09-17T03:35:47.122006+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:17.448836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122798+0000","last_change":"2024-09-17T03:35:45.972995+0000","last_active":"2024-09-17T03:35:47.122798+0000","last_peered":"2024-09-17T03:35:47.122798+0000","last_clean":"2024-09-17T03:35:47.122798+0000","last_became_active":"2024-09-17T03:35:45.972604+0000","last_became_peered":"2024-09-17T03:35:45.972604+0000","last_unstale":"2024-09-17T03:35:47.122798+0000","last_undegraded":"2024-09-17T03:35:47.122798+0000","last_fullsized":"2024-09-17T03:35:47.122798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:32.541741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122635+0000","last_change":"2024-09-17T03:35:44.958004+0000","last_active":"2024-09-17T03:35:47.122635+0000","last_peered":"2024-09-17T03:35:47.122635+0000","last_clean":"2024-09-17T03:35:47.122635+0000","last_became_active":"2024-09-17T03:35:44.957642+0000","last_became_peered":"2024-09-17T03:35:44.957642+0000","last_unstale":"2024-09-17T03:35:47.122635+0000","last_undegraded":"2024-09-17T03:35:47.122635+0000","last_fullsized":"2024-09-17T03:35:47.122635+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:31.183310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122299+0000","last_change":"2024-09-17T03:35:45.976953+0000","last_active":"2024-09-17T03:35:47.122299+0000","last_peered":"2024-09-17T03:35:47.122299+0000","last_clean":"2024-09-17T03:35:47.122299+0000","last_became_active":"2024-09-17T03:35:45.976267+0000","last_became_peered":"2024-09-17T03:35:45.976267+0000","last_unstale":"2024-09-17T03:35:47.122299+0000","last_undegraded":"2024-09-17T03:35:47.122299+0000","last_fullsized":"2024-09-17T03:35:47.122299+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:13.619428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942821+0000","last_change":"2024-09-17T03:35:44.954770+0000","last_active":"2024-09-17T03:35:46.942821+0000","last_peered":"2024-09-17T03:35:46.942821+0000","last_clean":"2024-09-17T03:35:46.942821+0000","last_became_active":"2024-09-17T03:35:44.953523+0000","last_became_peered":"2024-09-17T03:35:44.953523+0000","last_unstale":"2024-09-17T03:35:46.942821+0000","last_undegraded":"2024-09-17T03:35:46.942821+0000","last_fullsized":"2024-09-17T03:35:46.942821+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:40.501189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945946+0000","last_change":"2024-09-17T03:35:45.950548+0000","last_active":"2024-09-17T03:35:46.945946+0000","last_peered":"2024-09-17T03:35:46.945946+0000","last_clean":"2024-09-17T03:35:46.945946+0000","last_became_active":"2024-09-17T03:35:45.949460+0000","last_became_peered":"2024-09-17T03:35:45.949460+0000","last_unstale":"2024-09-17T03:35:46.945946+0000","last_undegraded":"2024-09-17T03:35:46.945946+0000","last_fullsized":"2024-09-17T03:35:46.945946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953125+0000","last_change":"2024-09-17T03:35:44.961601+0000","last_active":"2024-09-17T03:35:46.953125+0000","last_peered":"2024-09-17T03:35:46.953125+0000","last_clean":"2024-09-17T03:35:46.953125+0000","last_became_active":"2024-09-17T03:35:44.961301+0000","last_became_peered":"2024-09-17T03:35:44.961301+0000","last_unstale":"2024-09-17T03:35:46.953125+0000","last_undegraded":"2024-09-17T03:35:46.953125+0000","last_fullsized":"2024-09-17T03:35:46.953125+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:55.987785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116185+0000","last_change":"2024-09-17T03:35:45.958025+0000","last_active":"2024-09-17T03:35:47.116185+0000","last_peered":"2024-09-17T03:35:47.116185+0000","last_clean":"2024-09-17T03:35:47.116185+0000","last_became_active":"2024-09-17T03:35:45.957707+0000","last_became_peered":"2024-09-17T03:35:45.957707+0000","last_unstale":"2024-09-17T03:35:47.116185+0000","last_undegraded":"2024-09-17T03:35:47.116185+0000","last_fullsized":"2024-09-17T03:35:47.116185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.185058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122700+0000","last_change":"2024-09-17T03:35:44.961854+0000","last_active":"2024-09-17T03:35:47.122700+0000","last_peered":"2024-09-17T03:35:47.122700+0000","last_clean":"2024-09-17T03:35:47.122700+0000","last_became_active":"2024-09-17T03:35:44.961082+0000","last_became_peered":"2024-09-17T03:35:44.961082+0000","last_unstale":"2024-09-17T03:35:47.122700+0000","last_undegraded":"2024-09-17T03:35:47.122700+0000","last_fullsized":"2024-09-17T03:35:47.122700+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:42.433406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115537+0000","last_change":"2024-09-17T03:35:45.960079+0000","last_active":"2024-09-17T03:35:47.115537+0000","last_peered":"2024-09-17T03:35:47.115537+0000","last_clean":"2024-09-17T03:35:47.115537+0000","last_became_active":"2024-09-17T03:35:45.959680+0000","last_became_peered":"2024-09-17T03:35:45.959680+0000","last_unstale":"2024-09-17T03:35:47.115537+0000","last_undegraded":"2024-09-17T03:35:47.115537+0000","last_fullsized":"2024-09-17T03:35:47.115537+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:24.060327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122129+0000","last_change":"2024-09-17T03:35:44.958405+0000","last_active":"2024-09-17T03:35:47.122129+0000","last_peered":"2024-09-17T03:35:47.122129+0000","last_clean":"2024-09-17T03:35:47.122129+0000","last_became_active":"2024-09-17T03:35:44.958102+0000","last_became_peered":"2024-09-17T03:35:44.958102+0000","last_unstale":"2024-09-17T03:35:47.122129+0000","last_undegraded":"2024-09-17T03:35:47.122129+0000","last_fullsized":"2024-09-17T03:35:47.122129+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.755778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953353+0000","last_change":"2024-09-17T03:35:45.955137+0000","last_active":"2024-09-17T03:35:46.953353+0000","last_peered":"2024-09-17T03:35:46.953353+0000","last_clean":"2024-09-17T03:35:46.953353+0000","last_became_active":"2024-09-17T03:35:45.954787+0000","last_became_peered":"2024-09-17T03:35:45.954787+0000","last_unstale":"2024-09-17T03:35:46.953353+0000","last_undegraded":"2024-09-17T03:35:46.953353+0000","last_fullsized":"2024-09-17T03:35:46.953353+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:10.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945674+0000","last_change":"2024-09-17T03:35:44.950660+0000","last_active":"2024-09-17T03:35:46.945674+0000","last_peered":"2024-09-17T03:35:46.945674+0000","last_clean":"2024-09-17T03:35:46.945674+0000","last_became_active":"2024-09-17T03:35:44.949123+0000","last_became_peered":"2024-09-17T03:35:44.949123+0000","last_unstale":"2024-09-17T03:35:46.945674+0000","last_undegraded":"2024-09-17T03:35:46.945674+0000","last_fullsized":"2024-09-17T03:35:46.945674+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.114873+0000","last_change":"2024-09-17T03:35:45.956952+0000","last_active":"2024-09-17T03:35:47.114873+0000","last_peered":"2024-09-17T03:35:47.114873+0000","last_clean":"2024-09-17T03:35:47.114873+0000","last_became_active":"2024-09-17T03:35:45.956620+0000","last_became_peered":"2024-09-17T03:35:45.956620+0000","last_unstale":"2024-09-17T03:35:47.114873+0000","last_undegraded":"2024-09-17T03:35:47.114873+0000","last_fullsized":"2024-09-17T03:35:47.114873+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:11.263633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945421+0000","last_change":"2024-09-17T03:35:44.947967+0000","last_active":"2024-09-17T03:35:46.945421+0000","last_peered":"2024-09-17T03:35:46.945421+0000","last_clean":"2024-09-17T03:35:46.945421+0000","last_became_active":"2024-09-17T03:35:44.947694+0000","last_became_peered":"2024-09-17T03:35:44.947694+0000","last_unstale":"2024-09-17T03:35:46.945421+0000","last_undegraded":"2024-09-17T03:35:46.945421+0000","last_fullsized":"2024-09-17T03:35:46.945421+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:00.392947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963284+0000","last_change":"2024-09-17T03:35:45.973217+0000","last_active":"2024-09-17T03:35:46.963284+0000","last_peered":"2024-09-17T03:35:46.963284+0000","last_clean":"2024-09-17T03:35:46.963284+0000","last_became_active":"2024-09-17T03:35:45.972910+0000","last_became_peered":"2024-09-17T03:35:45.972910+0000","last_unstale":"2024-09-17T03:35:46.963284+0000","last_undegraded":"2024-09-17T03:35:46.963284+0000","last_fullsized":"2024-09-17T03:35:46.963284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:11.459235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942268+0000","last_change":"2024-09-17T03:35:44.947744+0000","last_active":"2024-09-17T03:35:46.942268+0000","last_peered":"2024-09-17T03:35:46.942268+0000","last_clean":"2024-09-17T03:35:46.942268+0000","last_became_active":"2024-09-17T03:35:44.947045+0000","last_became_peered":"2024-09-17T03:35:44.947045+0000","last_unstale":"2024-09-17T03:35:46.942268+0000","last_undegraded":"2024-09-17T03:35:46.942268+0000","last_fullsized":"2024-09-17T03:35:46.942268+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:56.123881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953525+0000","last_change":"2024-09-17T03:35:45.963241+0000","last_active":"2024-09-17T03:35:46.953525+0000","last_peered":"2024-09-17T03:35:46.953525+0000","last_clean":"2024-09-17T03:35:46.953525+0000","last_became_active":"2024-09-17T03:35:45.962942+0000","last_became_peered":"2024-09-17T03:35:45.962942+0000","last_unstale":"2024-09-17T03:35:46.953525+0000","last_undegraded":"2024-09-17T03:35:46.953525+0000","last_fullsized":"2024-09-17T03:35:46.953525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:47.099040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115936+0000","last_change":"2024-09-17T03:35:44.950227+0000","last_active":"2024-09-17T03:35:47.115936+0000","last_peered":"2024-09-17T03:35:47.115936+0000","last_clean":"2024-09-17T03:35:47.115936+0000","last_became_active":"2024-09-17T03:35:44.949838+0000","last_became_peered":"2024-09-17T03:35:44.949838+0000","last_unstale":"2024-09-17T03:35:47.115936+0000","last_undegraded":"2024-09-17T03:35:47.115936+0000","last_fullsized":"2024-09-17T03:35:47.115936+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:38.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115830+0000","last_change":"2024-09-17T03:35:45.952300+0000","last_active":"2024-09-17T03:35:47.115830+0000","last_peered":"2024-09-17T03:35:47.115830+0000","last_clean":"2024-09-17T03:35:47.115830+0000","last_became_active":"2024-09-17T03:35:45.951940+0000","last_became_peered":"2024-09-17T03:35:45.951940+0000","last_unstale":"2024-09-17T03:35:47.115830+0000","last_undegraded":"2024-09-17T03:35:47.115830+0000","last_fullsized":"2024-09-17T03:35:47.115830+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:57.804587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.965242+0000","last_change":"2024-09-17T03:35:44.960044+0000","last_active":"2024-09-17T03:35:46.965242+0000","last_peered":"2024-09-17T03:35:46.965242+0000","last_clean":"2024-09-17T03:35:46.965242+0000","last_became_active":"2024-09-17T03:35:44.959679+0000","last_became_peered":"2024-09-17T03:35:44.959679+0000","last_unstale":"2024-09-17T03:35:46.965242+0000","last_undegraded":"2024-09-17T03:35:46.965242+0000","last_fullsized":"2024-09-17T03:35:46.965242+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:26.474116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116521+0000","last_change":"2024-09-17T03:35:45.957578+0000","last_active":"2024-09-17T03:35:47.116521+0000","last_peered":"2024-09-17T03:35:47.116521+0000","last_clean":"2024-09-17T03:35:47.116521+0000","last_became_active":"2024-09-17T03:35:45.956972+0000","last_became_peered":"2024-09-17T03:35:45.956972+0000","last_unstale":"2024-09-17T03:35:47.116521+0000","last_undegraded":"2024-09-17T03:35:47.116521+0000","last_fullsized":"2024-09-17T03:35:47.116521+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:37.224976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.116267+0000","last_change":"2024-09-17T03:35:44.953819+0000","last_active":"2024-09-17T03:35:47.116267+0000","last_peered":"2024-09-17T03:35:47.116267+0000","last_clean":"2024-09-17T03:35:47.116267+0000","last_became_active":"2024-09-17T03:35:44.951031+0000","last_became_peered":"2024-09-17T03:35:44.951031+0000","last_unstale":"2024-09-17T03:35:47.116267+0000","last_undegraded":"2024-09-17T03:35:47.116267+0000","last_fullsized":"2024-09-17T03:35:47.116267+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:41.554774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.953195+0000","last_change":"2024-09-17T03:35:45.962505+0000","last_active":"2024-09-17T03:35:46.953195+0000","last_peered":"2024-09-17T03:35:46.953195+0000","last_clean":"2024-09-17T03:35:46.953195+0000","last_became_active":"2024-09-17T03:35:45.962169+0000","last_became_peered":"2024-09-17T03:35:45.962169+0000","last_unstale":"2024-09-17T03:35:46.953195+0000","last_undegraded":"2024-09-17T03:35:46.953195+0000","last_fullsized":"2024-09-17T03:35:46.953195+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:20.774078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942572+0000","last_change":"2024-09-17T03:35:44.956080+0000","last_active":"2024-09-17T03:35:46.942572+0000","last_peered":"2024-09-17T03:35:46.942572+0000","last_clean":"2024-09-17T03:35:46.942572+0000","last_became_active":"2024-09-17T03:35:44.955748+0000","last_became_peered":"2024-09-17T03:35:44.955748+0000","last_unstale":"2024-09-17T03:35:46.942572+0000","last_undegraded":"2024-09-17T03:35:46.942572+0000","last_fullsized":"2024-09-17T03:35:46.942572+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:04.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963292+0000","last_change":"2024-09-17T03:35:45.963696+0000","last_active":"2024-09-17T03:35:46.963292+0000","last_peered":"2024-09-17T03:35:46.963292+0000","last_clean":"2024-09-17T03:35:46.963292+0000","last_became_active":"2024-09-17T03:35:45.963323+0000","last_became_peered":"2024-09-17T03:35:45.963323+0000","last_unstale":"2024-09-17T03:35:46.963292+0000","last_undegraded":"2024-09-17T03:35:46.963292+0000","last_fullsized":"2024-09-17T03:35:46.963292+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:59.384483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115974+0000","last_change":"2024-09-17T03:35:44.953364+0000","last_active":"2024-09-17T03:35:47.115974+0000","last_peered":"2024-09-17T03:35:47.115974+0000","last_clean":"2024-09-17T03:35:47.115974+0000","last_became_active":"2024-09-17T03:35:44.953034+0000","last_became_peered":"2024-09-17T03:35:44.953034+0000","last_unstale":"2024-09-17T03:35:47.115974+0000","last_undegraded":"2024-09-17T03:35:47.115974+0000","last_fullsized":"2024-09-17T03:35:47.115974+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:23.777269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.122087+0000","last_change":"2024-09-17T03:35:45.963406+0000","last_active":"2024-09-17T03:35:47.122087+0000","last_peered":"2024-09-17T03:35:47.122087+0000","last_clean":"2024-09-17T03:35:47.122087+0000","last_became_active":"2024-09-17T03:35:45.963028+0000","last_became_peered":"2024-09-17T03:35:45.963028+0000","last_unstale":"2024-09-17T03:35:47.122087+0000","last_undegraded":"2024-09-17T03:35:47.122087+0000","last_fullsized":"2024-09-17T03:35:47.122087+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:44.922295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963480+0000","last_change":"2024-09-17T03:35:44.961829+0000","last_active":"2024-09-17T03:35:46.963480+0000","last_peered":"2024-09-17T03:35:46.963480+0000","last_clean":"2024-09-17T03:35:46.963480+0000","last_became_active":"2024-09-17T03:35:44.961479+0000","last_became_peered":"2024-09-17T03:35:44.961479+0000","last_unstale":"2024-09-17T03:35:46.963480+0000","last_undegraded":"2024-09-17T03:35:46.963480+0000","last_fullsized":"2024-09-17T03:35:46.963480+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:25.938155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945631+0000","last_change":"2024-09-17T03:35:45.954094+0000","last_active":"2024-09-17T03:35:46.945631+0000","last_peered":"2024-09-17T03:35:46.945631+0000","last_clean":"2024-09-17T03:35:46.945631+0000","last_became_active":"2024-09-17T03:35:45.952010+0000","last_became_peered":"2024-09-17T03:35:45.952010+0000","last_unstale":"2024-09-17T03:35:46.945631+0000","last_undegraded":"2024-09-17T03:35:46.945631+0000","last_fullsized":"2024-09-17T03:35:46.945631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.979607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.945521+0000","last_change":"2024-09-17T03:35:44.944467+0000","last_active":"2024-09-17T03:35:46.945521+0000","last_peered":"2024-09-17T03:35:46.945521+0000","last_clean":"2024-09-17T03:35:46.945521+0000","last_became_active":"2024-09-17T03:35:44.944155+0000","last_became_peered":"2024-09-17T03:35:44.944155+0000","last_unstale":"2024-09-17T03:35:46.945521+0000","last_undegraded":"2024-09-17T03:35:46.945521+0000","last_fullsized":"2024-09-17T03:35:46.945521+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:00.644286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:47.115398+0000","last_change":"2024-09-17T03:35:45.954997+0000","last_active":"2024-09-17T03:35:47.115398+0000","last_peered":"2024-09-17T03:35:47.115398+0000","last_clean":"2024-09-17T03:35:47.115398+0000","last_became_active":"2024-09-17T03:35:45.954184+0000","last_became_peered":"2024-09-17T03:35:45.954184+0000","last_unstale":"2024-09-17T03:35:47.115398+0000","last_undegraded":"2024-09-17T03:35:47.115398+0000","last_fullsized":"2024-09-17T03:35:47.115398+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.149975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.942555+0000","last_change":"2024-09-17T03:35:44.948114+0000","last_active":"2024-09-17T03:35:46.942555+0000","last_peered":"2024-09-17T03:35:46.942555+0000","last_clean":"2024-09-17T03:35:46.942555+0000","last_became_active":"2024-09-17T03:35:44.947433+0000","last_became_peered":"2024-09-17T03:35:44.947433+0000","last_unstale":"2024-09-17T03:35:46.942555+0000","last_undegraded":"2024-09-17T03:35:46.942555+0000","last_fullsized":"2024-09-17T03:35:46.942555+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:22.126761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963403+0000","last_change":"2024-09-17T03:35:45.968401+0000","last_active":"2024-09-17T03:35:46.963403+0000","last_peered":"2024-09-17T03:35:46.963403+0000","last_clean":"2024-09-17T03:35:46.963403+0000","last_became_active":"2024-09-17T03:35:45.968050+0000","last_became_peered":"2024-09-17T03:35:45.968050+0000","last_unstale":"2024-09-17T03:35:46.963403+0000","last_undegraded":"2024-09-17T03:35:46.963403+0000","last_fullsized":"2024-09-17T03:35:46.963403+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:44.919052+0000","last_clean_scrub_stamp":"2024-09-17T03:35:44.919052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.412069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.963509+0000","last_change":"2024-09-17T03:35:44.963161+0000","last_active":"2024-09-17T03:35:46.963509+0000","last_peered":"2024-09-17T03:35:46.963509+0000","last_clean":"2024-09-17T03:35:46.963509+0000","last_became_active":"2024-09-17T03:35:44.962822+0000","last_became_peered":"2024-09-17T03:35:44.962822+0000","last_unstale":"2024-09-17T03:35:46.963509+0000","last_undegraded":"2024-09-17T03:35:46.963509+0000","last_fullsized":"2024-09-17T03:35:46.963509+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:35:43.910570+0000","last_clean_scrub_stamp":"2024-09-17T03:35:43.910570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:12.300948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":309,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T03:35:46.962781+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:35:46.962781+0000","last_peered":"2024-09-17T03:35:46.962781+0000","last_clean":"2024-09-17T03:35:46.962781+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:35:46.962781+0000","last_undegraded":"2024-09-17T03:35:46.962781+0000","last_fullsized":"2024-09-17T03:35:46.962781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509622,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136016,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235824,"statfs":{"total":96636764160,"available":96497483776,"internally_reserved":0,"allocated":2220032,"data_stored":1292939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":137034186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509622,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145156,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226684,"statfs":{"total":96636764160,"available":96488124416,"internally_reserved":0,"allocated":1613824,"data_stored":701933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509622,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140984,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230856,"statfs":{"total":96636764160,"available":96492396544,"internally_reserved":0,"allocated":1605632,"data_stored":700391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509623,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127100,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244740,"statfs":{"total":96636764160,"available":96506613760,"internally_reserved":0,"allocated":1609728,"data_stored":700481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509622,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163196,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208644,"statfs":{"total":96636764160,"available":96469651456,"internally_reserved":0,"allocated":1609728,"data_stored":700937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509622,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145720,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226120,"statfs":{"total":96636764160,"available":96487546880,"internally_reserved":0,"allocated":2195456,"data_stored":1290689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127096,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244744,"statfs":{"total":96636764160,"available":96506617856,"internally_reserved":0,"allocated":1609728,"data_stored":700433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542326,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167344,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204496,"statfs":{"total":96636764160,"available":96465403904,"internally_reserved":0,"allocated":1601536,"data_stored":700321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:35:55.230 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:35:55.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:35:55.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 -- 172.21.15.46:0/593658467 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0074b20 msgr2=0x7fe1f015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 --2- 172.21.15.46:0/593658467 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0074b20 0x7fe1f015cf90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009bc0 tx=0x7fe1e801cba0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 -- 172.21.15.46:0/593658467 shutdown_connections 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 --2- 172.21.15.46:0/593658467 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0074b20 0x7fe1f015cf90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 --2- 172.21.15.46:0/593658467 >> v2:172.21.15.99:3300/0 conn(0x7fe1f0157030 0x7fe1f00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 -- 172.21.15.46:0/593658467 >> 172.21.15.46:0/593658467 conn(0x7fe1f00708c0 msgr2=0x7fe1f0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 -- 172.21.15.46:0/593658467 shutdown_connections 2024-09-17T03:35:55.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.433+0000 7fe1f5800640 1 -- 172.21.15.46:0/593658467 wait complete. 2024-09-17T03:35:55.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 Processor -- start 2024-09-17T03:35:55.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 -- start start 2024-09-17T03:35:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1f0157030 0x7fe1f0156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0160d50 0x7fe1f013fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 0x7fe1f00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f016a740 con 0x7fe1f01fd300 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f016a5c0 con 0x7fe1f0157030 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1f5800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f016a8c0 con 0x7fe1f0160d50 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 0x7fe1f00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 0x7fe1f00b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40564/0 (socket says 172.21.15.46:40564) 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 -- 172.21.15.46:0/4074133382 learned_addr learned my addr 172.21.15.46:0/4074133382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1eea00640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1f0157030 0x7fe1f0156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1e7e00640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0160d50 0x7fe1f013fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 -- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0160d50 msgr2=0x7fe1f013fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0160d50 0x7fe1f013fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 -- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1f0157030 msgr2=0x7fe1f0156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1f0157030 0x7fe1f0156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 -- 172.21.15.46:0/4074133382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f0140420 con 0x7fe1f01fd300 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1ef400640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 0x7fe1f00b3aa0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80b0040 tx=0x7fe1e8005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80b2040 con 0x7fe1f01fd300 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.437+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e8004040 con 0x7fe1f01fd300 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f01fc860 con 0x7fe1f01fd300 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8004470 con 0x7fe1f01fd300 2024-09-17T03:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01fcb00 con 0x7fe1f01fd300 2024-09-17T03:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f015cf90 con 0x7fe1f01fd300 2024-09-17T03:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fe1e80bd030 con 0x7fe1f01fd300 2024-09-17T03:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1e6a00640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1cc075240 0x7fe1cc077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fe1e812d430 con 0x7fe1f01fd300 2024-09-17T03:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.441+0000 7fe1eea00640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1cc075240 0x7fe1cc077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.445+0000 7fe1eea00640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1cc075240 0x7fe1cc077750 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009ad0 tx=0x7fe1e0009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.449+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe1e80c4060 con 0x7fe1f01fd300 2024-09-17T03:35:55.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.685+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe1f00d2260 con 0x7fe1f01fd300 2024-09-17T03:35:55.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1e6a00640 1 -- 172.21.15.46:0/4074133382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v43) ==== 190+0+147 (secure 0 0 0) 0x7fe1e8016790 con 0x7fe1f01fd300 2024-09-17T03:35:55.688 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:35:55.689 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:35:55.689 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:35:55.689 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:35:55.689 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:35:55.689 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1cc075240 msgr2=0x7fe1cc077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1cc075240 0x7fe1cc077750 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009ad0 tx=0x7fe1e0009260 comp rx=0 tx=0).stop 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 msgr2=0x7fe1f00b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 0x7fe1f00b3aa0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80b0040 tx=0x7fe1e8005a60 comp rx=0 tx=0).stop 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 shutdown_connections 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe1cc075240 0x7fe1cc077750 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1f01fd300 0x7fe1f00b3aa0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe1f0160d50 0x7fe1f013fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 --2- 172.21.15.46:0/4074133382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1f0157030 0x7fe1f0156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 >> 172.21.15.46:0/4074133382 conn(0x7fe1f00708c0 msgr2=0x7fe1f015ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 shutdown_connections 2024-09-17T03:35:55.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.689+0000 7fe1f5800640 1 -- 172.21.15.46:0/4074133382 wait complete. 2024-09-17T03:35:55.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 -- 172.21.15.46:0/764236417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8069500 msgr2=0x7f4bc80ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 --2- 172.21.15.46:0/764236417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8069500 0x7f4bc80ae8e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009c80 tx=0x7f4bc001cd70 comp rx=0 tx=0).stop 2024-09-17T03:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 -- 172.21.15.46:0/764236417 shutdown_connections 2024-09-17T03:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 --2- 172.21.15.46:0/764236417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8069500 0x7f4bc80ae8e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 --2- 172.21.15.46:0/764236417 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bc815efc0 0x7f4bc8070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 -- 172.21.15.46:0/764236417 >> 172.21.15.46:0/764236417 conn(0x7f4bc80b35f0 msgr2=0x7f4bc80b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 -- 172.21.15.46:0/764236417 shutdown_connections 2024-09-17T03:35:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.869+0000 7f4bce800640 1 -- 172.21.15.46:0/764236417 wait complete. 2024-09-17T03:35:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 Processor -- start 2024-09-17T03:35:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 -- start start 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bc80b0810 0x7f4bc80b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bc815efc0 0x7f4bc814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 0x7f4bc814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc816c960 con 0x7f4bc8154de0 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc8163500 con 0x7f4bc815efc0 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bce800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc816cae0 con 0x7f4bc80b0810 2024-09-17T03:35:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 0x7f4bc814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 0x7f4bc814ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40592/0 (socket says 172.21.15.46:40592) 2024-09-17T03:35:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bc7e00640 1 -- 172.21.15.46:0/2841805854 learned_addr learned my addr 172.21.15.46:0/2841805854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.873+0000 7f4bc7400640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bc80b0810 0x7f4bc80b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bbfe00640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bc815efc0 0x7f4bc814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bc7e00640 1 -- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bc80b0810 msgr2=0x7f4bc80b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bc7e00640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bc80b0810 0x7f4bc80b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bc7e00640 1 -- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bc815efc0 msgr2=0x7f4bc814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bc7e00640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bc815efc0 0x7f4bc814c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bc7e00640 1 -- 172.21.15.46:0/2841805854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc80b1a20 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bc7e00640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 0x7f4bc814ce70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4bc001c7e0 tx=0x7f4bc0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00bb070 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bc0004b40 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00b2040 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc80b1d00 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc80b22c0 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc80ae8e0 con 0x7f4bc8154de0 2024-09-17T03:35:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.877+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f4bc00be070 con 0x7f4bc8154de0 2024-09-17T03:35:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.881+0000 7f4bbf400640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4ba4075240 0x7f4ba4077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.881+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f4bc012d070 con 0x7f4bc8154de0 2024-09-17T03:35:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.881+0000 7f4bc7400640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4ba4075240 0x7f4ba4077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:55.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.881+0000 7f4bc7400640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4ba4075240 0x7f4ba4077750 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8004630 tx=0x7f4bb800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:55.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:55.885+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4bc00f2db0 con 0x7f4bc8154de0 2024-09-17T03:35:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.105+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bc80d0f80 con 0x7f4bc8154de0 2024-09-17T03:35:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.105+0000 7f4bbf400640 1 -- 172.21.15.46:0/2841805854 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5241 (secure 0 0 0) 0x7f4bc0016990 con 0x7f4bc8154de0 2024-09-17T03:35:56.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:35:56.106 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"btime":"2024-09-17T03:35:48:955180+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:35:45.367356+0000","modified":"2024-09-17T03:35:48.376364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248335":{"gid":248335,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2479724490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2479724490},{"type":"v1","addr":"172.21.15.46:6836","nonce":2479724490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248335,"qdb_cluster":[248335]},"id":9}]} 2024-09-17T03:35:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 84 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4ba4075240 msgr2=0x7f4ba4077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4ba4075240 0x7f4ba4077750 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8004630 tx=0x7f4bb800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 msgr2=0x7f4bc814ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 0x7f4bc814ce70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4bc001c7e0 tx=0x7f4bc0005b00 comp rx=0 tx=0).stop 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 shutdown_connections 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4ba4075240 0x7f4ba4077750 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bc8154de0 0x7f4bc814ce70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bc815efc0 0x7f4bc814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 --2- 172.21.15.46:0/2841805854 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bc80b0810 0x7f4bc80b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 >> 172.21.15.46:0/2841805854 conn(0x7f4bc80b35f0 msgr2=0x7f4bc80b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 shutdown_connections 2024-09-17T03:35:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.109+0000 7f4bce800640 1 -- 172.21.15.46:0/2841805854 wait complete. 2024-09-17T03:35:56.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:35:45.367356+0000', 'modified': '2024-09-17T03:35:48.376364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 248335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_248335': {'gid': 248335, 'name': 'a', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/2479724490', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2479724490}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 2479724490}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 248335, 'qdb_cluster': [248335]} 2024-09-17T03:35:56.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:35:56.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T03:35:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 -- 172.21.15.46:0/2602833609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 msgr2=0x7f9398074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 --2- 172.21.15.46:0/2602833609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f9398074e30 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9398071cb0 tx=0x7f938001c740 comp rx=0 tx=0).stop 2024-09-17T03:35:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 -- 172.21.15.46:0/2602833609 shutdown_connections 2024-09-17T03:35:56.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 --2- 172.21.15.46:0/2602833609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f9398074e30 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 --2- 172.21.15.46:0/2602833609 >> v2:172.21.15.99:3300/0 conn(0x7f93980b5c70 0x7f939815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 -- 172.21.15.46:0/2602833609 >> 172.21.15.46:0/2602833609 conn(0x7f93980b2040 msgr2=0x7f93980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 -- 172.21.15.46:0/2602833609 shutdown_connections 2024-09-17T03:35:56.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 -- 172.21.15.46:0/2602833609 wait complete. 2024-09-17T03:35:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 Processor -- start 2024-09-17T03:35:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.697+0000 7f939da00640 1 -- start start 2024-09-17T03:35:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93980b4de0 0x7f93981413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93980b5c70 0x7f93980cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f939815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9398168ab0 con 0x7f93981527b0 2024-09-17T03:35:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9398168930 con 0x7f93980b5c70 2024-09-17T03:35:56.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9398168c30 con 0x7f93980b4de0 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f939815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f939815a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40610/0 (socket says 172.21.15.46:40610) 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 -- 172.21.15.46:0/244540008 learned_addr learned my addr 172.21.15.46:0/244540008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9396a00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93980b4de0 0x7f93981413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f938fe00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93980b5c70 0x7f93980cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 -- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93980b4de0 msgr2=0x7f93981413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93980b4de0 0x7f93981413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 -- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93980b5c70 msgr2=0x7f93980cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93980b5c70 0x7f93980cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 -- 172.21.15.46:0/244540008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9398141c30 con 0x7f93981527b0 2024-09-17T03:35:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f9397400640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f939815a770 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f939000c730 tx=0x7f939000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f938ea00640 1 -- 172.21.15.46:0/244540008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390003b80 con 0x7f93981527b0 2024-09-17T03:35:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f938ea00640 1 -- 172.21.15.46:0/244540008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9390003d40 con 0x7f93981527b0 2024-09-17T03:35:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9398141f10 con 0x7f93981527b0 2024-09-17T03:35:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f938ea00640 1 -- 172.21.15.46:0/244540008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900052d0 con 0x7f93981527b0 2024-09-17T03:35:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.701+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981424d0 con 0x7f93981527b0 2024-09-17T03:35:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f938ea00640 1 -- 172.21.15.46:0/244540008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f9390005680 con 0x7f93981527b0 2024-09-17T03:35:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f938ea00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9374075210 0x7f9374077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f938ea00640 1 -- 172.21.15.46:0/244540008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f93900930d0 con 0x7f93981527b0 2024-09-17T03:35:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 Processor -- start 2024-09-17T03:35:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9374075210 0x7f9374077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 -- start start 2024-09-17T03:35:56.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9374075210 0x7f9374077720 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9388009b40 tx=0x7f93880091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e7331be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7332120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9e7206410 con 0x55d9e6df0460 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9e7143b40 con 0x55d9e71c9ca0 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55d9e7132f70 con 0x55d9e71e0950 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73316a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40620/0 (socket says 172.21.15.46:40620) 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 -- 172.21.15.46:0/3090494369 learned_addr learned my addr 172.21.15.46:0/3090494369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f938fe00640 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e7331be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9397400640 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7332120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 -- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 msgr2=0x55d9e7331be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e7331be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 -- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 msgr2=0x55d9e7332120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7332120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.705+0000 7f9396a00640 1 -- 172.21.15.46:0/3090494369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e70fa5b0 con 0x55d9e6df0460 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f9396a00640 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73316a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f93880058d0 tx=0x7f9388014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f4b6a1003276c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f9387400640 1 -- 172.21.15.46:0/3090494369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388003b80 con 0x55d9e6df0460 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f9387400640 1 -- 172.21.15.46:0/3090494369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9388003e10 con 0x55d9e6df0460 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f9387400640 1 -- 172.21.15.46:0/3090494369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388005280 con 0x55d9e6df0460 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 -- 172.21.15.46:0/3090494369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 msgr2=0x55d9e73316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73316a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f93880058d0 tx=0x7f9388014000 comp rx=0 tx=0).stop 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 -- 172.21.15.46:0/3090494369 shutdown_connections 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7332120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e7331be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 --2- 172.21.15.46:0/3090494369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73316a0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 -- 172.21.15.46:0/3090494369 >> 172.21.15.46:0/3090494369 conn(0x55d9e72b6d90 msgr2=0x55d9e72e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 -- 172.21.15.46:0/3090494369 shutdown_connections 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 -- 172.21.15.46:0/3090494369 wait complete. 2024-09-17T03:35:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 -- start start 2024-09-17T03:35:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.709+0000 7f93a5139000 1 Processor -- start 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e6df0460 0x55d9e7338920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e7338e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 0x55d9e73393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9e70cd5f0 con 0x55d9e71e0950 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9e7136d80 con 0x55d9e6df0460 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55d9e70d9520 con 0x55d9e71c9ca0 2024-09-17T03:35:56.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 10 client.0 _ll_register_callbacks cb 0x55d9e70dd830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 0x55d9e73393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 0x55d9e73393a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40622/0 (socket says 172.21.15.46:40622) 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 -- 172.21.15.46:0/28758423 learned_addr learned my addr 172.21.15.46:0/28758423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f938fe00640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e7338e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 -- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 msgr2=0x55d9e7338e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e7338e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9396a00640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e6df0460 0x55d9e7338920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 -- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e6df0460 msgr2=0x55d9e7338920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e6df0460 0x55d9e7338920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 -- 172.21.15.46:0/28758423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e70d1190 con 0x55d9e71e0950 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f938fe00640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e7338e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9396a00640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e6df0460 0x55d9e7338920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9397400640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 0x55d9e73393a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f939005c880 tx=0x7f9390068c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55da0c7365a0bee server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:35:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 1 -- 172.21.15.46:0/28758423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939005ecf0 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 1 -- 172.21.15.46:0/28758423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f939005e250 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9e70cb110 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 1 -- 172.21.15.46:0/28758423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f939005f090 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 10 client.248874 fetch_fsmap learned FSMap version 84 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55d9e71b7f10 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 1 -- 172.21.15.46:0/28758423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d9e70cb110 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 1 -- 172.21.15.46:0/28758423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f939005e450 con 0x55d9e71e0950 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 10 client.248874 fetch_fsmap finished waiting for FSMap version 84 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 10 client.248874 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 20 client.248874 populate_metadata read hostname 'smithi046' 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dd57d0 0x55d9e7342d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 4 client.248874 mds_command: new command op to 248335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 --> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9e7145a90 con 0x55d9e6dd57d0 2024-09-17T03:35:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9396a00640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dd57d0 0x55d9e7342d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f9396a00640 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dd57d0 0x55d9e7342d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea17f97defb0131 server_cookie=74eae0607cb7fd91 in_seq=0 out_seq=0 2024-09-17T03:35:56.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 10 client.248874 ms_handle_connect on v2:172.21.15.46:6834/2479724490 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 1 -- 172.21.15.46:0/28758423 <== mds.0 v2:172.21.15.46:6834/2479724490 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9e7145a90 con 0x55d9e6dd57d0 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f937be00640 10 client.248874 handle_command_reply: tid=0 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 client.248874 shutdown 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 msgr2=0x55d9e73393a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.713+0000 7f93a5139000 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 0x55d9e73393a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f939005c880 tx=0x7f9390068c90 comp rx=0 tx=0).stop 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 shutdown_connections 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e71e0950 0x55d9e73393a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dd57d0 0x55d9e7342d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e7338e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 --2- 172.21.15.46:0/28758423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e6df0460 0x55d9e7338920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 >> 172.21.15.46:0/28758423 conn(0x55d9e72b6d90 msgr2=0x55d9e72ed440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f937be00640 0 client.248874 ms_handle_reset on v2:172.21.15.46:6834/2479724490 2024-09-17T03:35:56.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 shutdown_connections 2024-09-17T03:35:56.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 1 -- 172.21.15.46:0/28758423 wait complete. 2024-09-17T03:35:56.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.717+0000 7f93a5139000 20 client.248874 trim_cache size 0 max 16384 2024-09-17T03:35:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.733+0000 7f93a5139000 1 Processor -- start 2024-09-17T03:35:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.733+0000 7f93a5139000 1 -- start start 2024-09-17T03:35:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e73a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71e0950 0x55d9e73a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9398203040 con 0x55d9e6df0460 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9398202060 con 0x55d9e71e0950 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f93981436b0 con 0x55d9e71c9ca0 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73a7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40632/0 (socket says 172.21.15.46:40632) 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 -- 172.21.15.46:0/3696100117 learned_addr learned my addr 172.21.15.46:0/3696100117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f938fe00640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e73a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 -- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 msgr2=0x55d9e73a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e73a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 -- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71e0950 msgr2=0x55d9e73a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9397400640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71e0950 0x55d9e73a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71e0950 0x55d9e73a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 -- 172.21.15.46:0/3696100117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e718edb0 con 0x55d9e6df0460 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f938fe00640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e73a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9397400640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71e0950 0x55d9e73a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:35:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9396a00640 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73a7a30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9388022000 tx=0x7f9388011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba85e2ca6deab38 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9386a00640 1 -- 172.21.15.46:0/3696100117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388035040 con 0x55d9e6df0460 2024-09-17T03:35:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9386a00640 1 -- 172.21.15.46:0/3696100117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9388003e00 con 0x55d9e6df0460 2024-09-17T03:35:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f9386a00640 1 -- 172.21.15.46:0/3696100117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388005280 con 0x55d9e6df0460 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- 172.21.15.46:0/3696100117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 msgr2=0x55d9e73a7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73a7a30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9388022000 tx=0x7f9388011f90 comp rx=0 tx=0).stop 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- 172.21.15.46:0/3696100117 shutdown_connections 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71e0950 0x55d9e73a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71c9ca0 0x55d9e73a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 --2- 172.21.15.46:0/3696100117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e73a7a30 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- 172.21.15.46:0/3696100117 >> 172.21.15.46:0/3696100117 conn(0x55d9e72b6d90 msgr2=0x55d9e732f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- 172.21.15.46:0/3696100117 shutdown_connections 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.737+0000 7f93a5139000 1 -- 172.21.15.46:0/3696100117 wait complete. 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 -- start start 2024-09-17T03:35:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 Processor -- start 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e7367630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7367b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e73680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d9e7136090 con 0x55d9e6df0460 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9e713b3b0 con 0x55d9e71c9ca0 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x55d9e7142910 con 0x55d9e71e0950 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 10 client.0 _ll_register_callbacks cb 0x7f939005b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:35:56.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e7367630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e7367630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40648/0 (socket says 172.21.15.46:40648) 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 -- 172.21.15.46:0/3610349194 learned_addr learned my addr 172.21.15.46:0/3610349194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f938fe00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7367b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9397400640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e73680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 -- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 msgr2=0x55d9e73680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e73680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 -- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 msgr2=0x55d9e7367b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7367b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 -- 172.21.15.46:0/3610349194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e70f9ab0 con 0x55d9e6df0460 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f938fe00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7367b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9397400640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e73680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f9396a00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e7367630 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f93880058a0 tx=0x7f9388005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f70d6c734ad784 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f937be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:35:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f937be00640 1 -- 172.21.15.46:0/3610349194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388035040 con 0x55d9e6df0460 2024-09-17T03:35:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f937be00640 1 -- 172.21.15.46:0/3610349194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f9388033440 con 0x55d9e6df0460 2024-09-17T03:35:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9388003e00 con 0x55d9e6df0460 2024-09-17T03:35:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f937be00640 1 -- 172.21.15.46:0/3610349194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f938804a050 con 0x55d9e6df0460 2024-09-17T03:35:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 10 client.248880 fetch_fsmap learned FSMap version 84 2024-09-17T03:35:56.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55d9e70d8070 con 0x55d9e6df0460 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f937be00640 1 -- 172.21.15.46:0/3610349194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9388003e00 con 0x55d9e6df0460 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.741+0000 7f937be00640 1 -- 172.21.15.46:0/3610349194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f938804d020 con 0x55d9e6df0460 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 10 client.248880 fetch_fsmap finished waiting for FSMap version 84 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 10 client.248880 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 20 client.248880 populate_metadata read hostname 'smithi046' 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dc3830 0x55d9e73689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 4 client.248880 mds_command: new command op to 248335 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 --> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] -- command(tid 0: {"prefix": "status"}) -- 0x55d9e7141540 con 0x55d9e6dc3830 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f938fe00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dc3830 0x55d9e73689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f938fe00640 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dc3830 0x55d9e73689e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a45c3b6ab7bd5 server_cookie=88f7a5e05179eac9 in_seq=0 out_seq=0 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f937be00640 10 client.248880 ms_handle_connect on v2:172.21.15.46:6834/2479724490 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f937be00640 1 -- 172.21.15.46:0/3610349194 <== mds.0 v2:172.21.15.46:6834/2479724490 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55d9e7141540 con 0x55d9e6dc3830 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f937be00640 10 client.248880 handle_command_reply: tid=0 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 client.248880 shutdown 2024-09-17T03:35:56.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 msgr2=0x55d9e7367630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e7367630 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f93880058a0 tx=0x7f9388005e50 comp rx=0 tx=0).stop 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 shutdown_connections 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:6834/2479724490,v1:172.21.15.46:6836/2479724490] conn(0x55d9e6dc3830 0x55d9e73689e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x55d9e71e0950 0x55d9e73680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e71c9ca0 0x55d9e7367b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f937be00640 0 client.248880 ms_handle_reset on v2:172.21.15.46:6834/2479724490 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 --2- 172.21.15.46:0/3610349194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9e6df0460 0x55d9e7367630 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 >> 172.21.15.46:0/3610349194 conn(0x55d9e72b6d90 msgr2=0x55d9e73426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 shutdown_connections 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 1 -- 172.21.15.46:0/3610349194 wait complete. 2024-09-17T03:35:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f93a5139000 20 client.248880 trim_cache size 0 max 16384 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 248335, 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.801071105, 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 84, 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 129, 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 129, 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.198963243 2024-09-17T03:35:56.749 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:35:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9374075210 msgr2=0x7f9374077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9374075210 0x7f9374077720 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9388009b40 tx=0x7f93880091e0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 msgr2=0x7f939815a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f939815a770 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f939000c730 tx=0x7f939000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 shutdown_connections 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9374075210 0x7f9374077720 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93981527b0 0x7f939815a770 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93980b5c70 0x7f93980cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 --2- 172.21.15.46:0/244540008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f93980b4de0 0x7f93981413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 >> 172.21.15.46:0/244540008 conn(0x7f93980b2040 msgr2=0x7f9398154310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 shutdown_connections 2024-09-17T03:35:56.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.745+0000 7f939da00640 1 -- 172.21.15.46:0/244540008 wait complete. 2024-09-17T03:35:56.765 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:35:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90fb000640 1 -- 172.21.15.46:0/264914744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d8004d50 msgr2=0x7f90d8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90fb000640 1 --2- 172.21.15.46:0/264914744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d8004d50 0x7f90d8005160 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009ec0 tx=0x7f90ec01d5b0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90f9000640 1 -- 172.21.15.46:0/264914744 reap_dead start 2024-09-17T03:35:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90fb000640 1 -- 172.21.15.46:0/264914744 shutdown_connections 2024-09-17T03:35:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90fb000640 1 -- 172.21.15.46:0/264914744 >> 172.21.15.46:0/264914744 conn(0x7f90f40b2c40 msgr2=0x7f90f40b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90fb000640 1 -- 172.21.15.46:0/264914744 shutdown_connections 2024-09-17T03:35:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.925+0000 7f90fb000640 1 -- 172.21.15.46:0/264914744 wait complete. 2024-09-17T03:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 Processor -- start 2024-09-17T03:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 -- start start 2024-09-17T03:35:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d8004d50 0x7f90f415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 0x7f90f40afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90f40ae8e0 0x7f90f40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90f406f3b0 con 0x7f90f4158a70 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90f40b3d50 con 0x7f90f40ae8e0 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90fb000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f90f406f530 con 0x7f90d8004d50 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 0x7f90f40afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 0x7f90f40afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40664/0 (socket says 172.21.15.46:40664) 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 -- 172.21.15.46:0/484935662 learned_addr learned my addr 172.21.15.46:0/484935662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90f9000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90f40ae8e0 0x7f90f40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90f3e00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d8004d50 0x7f90f415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 -- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d8004d50 msgr2=0x7f90f415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d8004d50 0x7f90f415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 -- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90f40ae8e0 msgr2=0x7f90f40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90f40ae8e0 0x7f90f40aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 -- 172.21.15.46:0/484935662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f415e3c0 con 0x7f90f4158a70 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90ebe00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 0x7f90f40afc00 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f90dc011850 tx=0x7f90dc011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.929+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00fd50 con 0x7f90f4158a70 2024-09-17T03:35:56.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.933+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90dc004030 con 0x7f90f4158a70 2024-09-17T03:35:56.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.933+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc004460 con 0x7f90f4158a70 2024-09-17T03:35:56.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.933+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f41579d0 con 0x7f90f4158a70 2024-09-17T03:35:56.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.933+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f4157f90 con 0x7f90f4158a70 2024-09-17T03:35:56.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.933+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f406fac0 con 0x7f90f4158a70 2024-09-17T03:35:56.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.933+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f90dc003600 con 0x7f90f4158a70 2024-09-17T03:35:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.937+0000 7f90f0c00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90d0075210 0x7f90d0077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.937+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f90dc0946c0 con 0x7f90f4158a70 2024-09-17T03:35:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.937+0000 7f90f3e00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90d0075210 0x7f90d0077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.937+0000 7f90f3e00640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90d0075210 0x7f90d0077720 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f90e4000fc0 tx=0x7f90e4001750 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:56.937+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f90dc0612c0 con 0x7f90f4158a70 2024-09-17T03:35:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.153+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90f40cfb40 con 0x7f90f4158a70 2024-09-17T03:35:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.153+0000 7f90f0c00640 1 -- 172.21.15.46:0/484935662 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f90dc060bc0 con 0x7f90f4158a70 2024-09-17T03:35:57.153 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90d0075210 msgr2=0x7f90d0077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90d0075210 0x7f90d0077720 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f90e4000fc0 tx=0x7f90e4001750 comp rx=0 tx=0).stop 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 msgr2=0x7f90f40afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 0x7f90f40afc00 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f90dc011850 tx=0x7f90dc011d10 comp rx=0 tx=0).stop 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 shutdown_connections 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90d0075210 0x7f90d0077720 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90f40ae8e0 0x7f90f40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f4158a70 0x7f90f40afc00 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 --2- 172.21.15.46:0/484935662 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d8004d50 0x7f90f415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 >> 172.21.15.46:0/484935662 conn(0x7f90f40b2c40 msgr2=0x7f90f40cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 shutdown_connections 2024-09-17T03:35:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57.157+0000 7f90fb000640 1 -- 172.21.15.46:0/484935662 wait complete. 2024-09-17T03:35:57.176 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:35:57.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:35:57.177 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:35:57.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.177 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:35:57.188 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41531sec preferred_lft 41531sec 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:35:57.189 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:35:57.190 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.191 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:35:57.191 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:35:57.191 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:35:57.191 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:35:57.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.191 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:35:57.249 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:35:57.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.252 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:35:57.309 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:35:57.309 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:35:57.309 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:35:57.309 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:35:57.309 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:35:57.309 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:35:57.309 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:35:57.309 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:35:57.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:35:57.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:35:57.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:35:57.310 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:35:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:35:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:35:57.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.452 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:35:57.462 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:35:57.462 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:35:57.462 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:35:57.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:35:57.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:35:57.527 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:35:57.581 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:35:57.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:35:57.581 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:35:57.634 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:35:57.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:35:57.737 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:35:57.737 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:35:57.752 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:35:57.752 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:35:57.753 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:35:57.753 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:35:57.753 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:35:57.757 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:35:57.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:35:57.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:35:57.777 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:35:57.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:35:57.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:35:57.884 INFO:teuthology.orchestra.run.smithi046.stdout:client247535 2024-09-17T03:35:57.884 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:35:57.884 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:35:57.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:35:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 -- 172.21.15.46:0/1225059343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 msgr2=0x7ff68c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 --2- 172.21.15.46:0/1225059343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c15bb60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009900 tx=0x7ff67c01c6d0 comp rx=0 tx=0).stop 2024-09-17T03:35:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 -- 172.21.15.46:0/1225059343 shutdown_connections 2024-09-17T03:35:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 --2- 172.21.15.46:0/1225059343 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff68c0b4de0 0x7ff68c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 --2- 172.21.15.46:0/1225059343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c15bb60 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 -- 172.21.15.46:0/1225059343 >> 172.21.15.46:0/1225059343 conn(0x7ff68c0b2040 msgr2=0x7ff68c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 -- 172.21.15.46:0/1225059343 shutdown_connections 2024-09-17T03:35:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.097+0000 7ff691a00640 1 -- 172.21.15.46:0/1225059343 wait complete. 2024-09-17T03:35:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 Processor -- start 2024-09-17T03:35:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 -- start start 2024-09-17T03:35:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff68c0b4de0 0x7ff68c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68c1527b0 0x7ff68c1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c168a50 con 0x7ff68c0b5c70 2024-09-17T03:35:58.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1688d0 con 0x7ff68c1527b0 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff68c168bd0 con 0x7ff68c0b4de0 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c0ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40680/0 (socket says 172.21.15.46:40680) 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 -- 172.21.15.46:0/1712669992 learned_addr learned my addr 172.21.15.46:0/1712669992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68aa00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff68c0b4de0 0x7ff68c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68b400640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68c1527b0 0x7ff68c1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 -- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff68c0b4de0 msgr2=0x7ff68c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff68c0b4de0 0x7ff68c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 -- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68c1527b0 msgr2=0x7ff68c1f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68c1527b0 0x7ff68c1f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 -- 172.21.15.46:0/1712669992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c0cf5b0 con 0x7ff68c0b5c70 2024-09-17T03:35:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff68a000640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c0ad420 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ff67800b410 tx=0x7ff67800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678015030 con 0x7ff68c0b5c70 2024-09-17T03:35:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff678004020 con 0x7ff68c0b5c70 2024-09-17T03:35:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678004470 con 0x7ff68c0b5c70 2024-09-17T03:35:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.101+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c0cdfd0 con 0x7ff68c0b5c70 2024-09-17T03:35:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.105+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c0ce590 con 0x7ff68c0b5c70 2024-09-17T03:35:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.105+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c0cecb0 con 0x7ff68c0b5c70 2024-09-17T03:35:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.105+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7ff678018070 con 0x7ff68c0b5c70 2024-09-17T03:35:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.105+0000 7ff682a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff66c075210 0x7ff66c077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.105+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7ff678092d00 con 0x7ff68c0b5c70 2024-09-17T03:35:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.105+0000 7ff68aa00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff66c075210 0x7ff66c077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.109+0000 7ff68aa00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff66c075210 0x7ff66c077720 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0af000 tx=0x7ff67c01af10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.109+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff67805f810 con 0x7ff68c0b5c70 2024-09-17T03:35:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:58.317+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff68c0cdd60 con 0x7ff68c0b5c70 2024-09-17T03:35:59.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.001+0000 7ff682a00640 1 -- 172.21.15.46:0/1712669992 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v85) ==== 136+0+0 (secure 0 0 0) 0x7ff678066110 con 0x7ff68c0b5c70 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff66c075210 msgr2=0x7ff66c077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff66c075210 0x7ff66c077720 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0af000 tx=0x7ff67c01af10 comp rx=0 tx=0).stop 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 msgr2=0x7ff68c0ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c0ad420 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ff67800b410 tx=0x7ff67800b8d0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 shutdown_connections 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff66c075210 0x7ff66c077720 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68c1527b0 0x7ff68c1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff68c0b5c70 0x7ff68c0ad420 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 --2- 172.21.15.46:0/1712669992 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff68c0b4de0 0x7ff68c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 >> 172.21.15.46:0/1712669992 conn(0x7ff68c0b2040 msgr2=0x7ff68c1581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:59.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 shutdown_connections 2024-09-17T03:35:59.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.005+0000 7ff691a00640 1 -- 172.21.15.46:0/1712669992 wait complete. 2024-09-17T03:35:59.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:35:59.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 -- 172.21.15.46:0/4234580555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c1527b0 msgr2=0x7f2e4c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 --2- 172.21.15.46:0/4234580555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c1527b0 0x7f2e4c15bb60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0cca90 tx=0x7f2e3801c800 comp rx=0 tx=0).stop 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 -- 172.21.15.46:0/4234580555 shutdown_connections 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 --2- 172.21.15.46:0/4234580555 >> v2:172.21.15.99:3300/0 conn(0x7f2e4c0b4de0 0x7f2e4c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 --2- 172.21.15.46:0/4234580555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c1527b0 0x7f2e4c15bb60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 -- 172.21.15.46:0/4234580555 >> 172.21.15.46:0/4234580555 conn(0x7f2e4c0b2040 msgr2=0x7f2e4c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 -- 172.21.15.46:0/4234580555 shutdown_connections 2024-09-17T03:35:59.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.193+0000 7f2e50e00640 1 -- 172.21.15.46:0/4234580555 wait complete. 2024-09-17T03:35:59.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 Processor -- start 2024-09-17T03:35:59.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 -- start start 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e4c0b4de0 0x7f2e4c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 0x7f2e4c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e4c1527b0 0x7f2e4c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c168920 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1687a0 con 0x7f2e4c1527b0 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e50e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c168aa0 con 0x7f2e4c0b4de0 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 0x7f2e4c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 0x7f2e4c0ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40686/0 (socket says 172.21.15.46:40686) 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 -- 172.21.15.46:0/665589105 learned_addr learned my addr 172.21.15.46:0/665589105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e4c0b4de0 0x7f2e4c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e4a800640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e4c1527b0 0x7f2e4c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 -- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e4c0b4de0 msgr2=0x7f2e4c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e4c0b4de0 0x7f2e4c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 -- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e4c1527b0 msgr2=0x7f2e4c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e4c1527b0 0x7f2e4c0aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.197+0000 7f2e49400640 1 -- 172.21.15.46:0/665589105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c1561c0 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e49400640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 0x7f2e4c0ae8e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2e38019410 tx=0x7f2e380a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380bc070 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e380044b0 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380b2040 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c0cf5e0 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c0cfba0 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c0b51f0 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.201+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f2e38005290 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.205+0000 7f2e42000640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2e2c075240 0x7f2e2c077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.205+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f2e3812ca90 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.205+0000 7f2e49e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2e2c075240 0x7f2e2c077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.205+0000 7f2e49e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2e2c075240 0x7f2e2c077750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007fd0 tx=0x7f2e3c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:59.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.209+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2e380164b0 con 0x7f2e4c0b5c70 2024-09-17T03:35:59.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.401+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2e4c06b540 con 0x7f2e2c075240 2024-09-17T03:35:59.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e42000640 1 -- 172.21.15.46:0/665589105 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2e3c001d20 con 0x7f2e2c075240 2024-09-17T03:35:59.405 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:35:59.405 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:35:59.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:35:59.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:35:59.405 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:35:59.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2e2c075240 msgr2=0x7f2e2c077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2e2c075240 0x7f2e2c077750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007fd0 tx=0x7f2e3c009040 comp rx=0 tx=0).stop 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 msgr2=0x7f2e4c0ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 0x7f2e4c0ae8e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2e38019410 tx=0x7f2e380a5820 comp rx=0 tx=0).stop 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 shutdown_connections 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2e2c075240 0x7f2e2c077750 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e4c1527b0 0x7f2e4c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4c0b5c70 0x7f2e4c0ae8e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 --2- 172.21.15.46:0/665589105 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2e4c0b4de0 0x7f2e4c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 >> 172.21.15.46:0/665589105 conn(0x7f2e4c0b2040 msgr2=0x7f2e4c152cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:59.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 shutdown_connections 2024-09-17T03:35:59.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.405+0000 7f2e50e00640 1 -- 172.21.15.46:0/665589105 wait complete. 2024-09-17T03:35:59.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 -- 172.21.15.46:0/2032361144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b54f0 msgr2=0x7fb48415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 --2- 172.21.15.46:0/2032361144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b54f0 0x7fb48415bb60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009c80 tx=0x7fb47c01cce0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 -- 172.21.15.46:0/2032361144 shutdown_connections 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 --2- 172.21.15.46:0/2032361144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b54f0 0x7fb48415bb60 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 -- 172.21.15.46:0/2032361144 >> 172.21.15.46:0/2032361144 conn(0x7fb4840b2040 msgr2=0x7fb4840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 -- 172.21.15.46:0/2032361144 shutdown_connections 2024-09-17T03:35:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.589+0000 7fb48b200640 1 -- 172.21.15.46:0/2032361144 wait complete. 2024-09-17T03:35:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 Processor -- start 2024-09-17T03:35:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 -- start start 2024-09-17T03:35:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4840b5c70 0x7fb48414fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4841527b0 0x7fb484158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 0x7fb4841588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb484169bd0 con 0x7fb4840ad2f0 2024-09-17T03:35:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb484169a50 con 0x7fb4841527b0 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb48b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb484169d50 con 0x7fb4840b5c70 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb489200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 0x7fb4841588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb489200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 0x7fb4841588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40704/0 (socket says 172.21.15.46:40704) 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb489200640 1 -- 172.21.15.46:0/3639354483 learned_addr learned my addr 172.21.15.46:0/3639354483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb483e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4840b5c70 0x7fb48414fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.593+0000 7fb47b400640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4841527b0 0x7fb484158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb489200640 1 -- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4840b5c70 msgr2=0x7fb48414fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb489200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4840b5c70 0x7fb48414fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb489200640 1 -- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4841527b0 msgr2=0x7fb484158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb489200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4841527b0 0x7fb484158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb489200640 1 -- 172.21.15.46:0/3639354483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb484151af0 con 0x7fb4840ad2f0 2024-09-17T03:35:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb489200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 0x7fb4841588d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009bd0 tx=0x7fb47c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0bc070 con 0x7fb4840ad2f0 2024-09-17T03:35:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47c0ad8e0 con 0x7fb4840ad2f0 2024-09-17T03:35:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb484151db0 con 0x7fb4840ad2f0 2024-09-17T03:35:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0b2040 con 0x7fb4840ad2f0 2024-09-17T03:35:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4840af700 con 0x7fb4840ad2f0 2024-09-17T03:35:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48415bb60 con 0x7fb4840ad2f0 2024-09-17T03:35:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.597+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fb47c0bf070 con 0x7fb4840ad2f0 2024-09-17T03:35:59.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.601+0000 7fb480c00640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb460075210 0x7fb460077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.601+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fb47c12d190 con 0x7fb4840ad2f0 2024-09-17T03:35:59.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.601+0000 7fb483e00640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb460075210 0x7fb460077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.601+0000 7fb483e00640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb460075210 0x7fb460077720 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4740098e0 tx=0x7fb474009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:59.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.605+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb47c016990 con 0x7fb4840ad2f0 2024-09-17T03:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb4840b5560 con 0x7fb4840ad2f0 2024-09-17T03:35:59.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb480c00640 1 -- 172.21.15.46:0/3639354483 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v55)=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7fb47c0f96f0 con 0x7fb4840ad2f0 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb460075210 msgr2=0x7fb460077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb460075210 0x7fb460077720 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4740098e0 tx=0x7fb474009060 comp rx=0 tx=0).stop 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 msgr2=0x7fb4841588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 0x7fb4841588d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009bd0 tx=0x7fb47c0093d0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 shutdown_connections 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb460075210 0x7fb460077720 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840ad2f0 0x7fb4841588d0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4841527b0 0x7fb484158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 --2- 172.21.15.46:0/3639354483 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb4840b5c70 0x7fb48414fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 >> 172.21.15.46:0/3639354483 conn(0x7fb4840b2040 msgr2=0x7fb4840b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 shutdown_connections 2024-09-17T03:35:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.781+0000 7fb48b200640 1 -- 172.21.15.46:0/3639354483 wait complete. 2024-09-17T03:35:59.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:35:59.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 -- 172.21.15.46:0/1216705909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 msgr2=0x7f4f100d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1216705909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 0x7f4f100d14e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009840 tx=0x7f4ef801c820 comp rx=0 tx=0).stop 2024-09-17T03:35:59.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 -- 172.21.15.46:0/1216705909 shutdown_connections 2024-09-17T03:35:59.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1216705909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 0x7f4f100d14e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1216705909 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f100d25d0 0x7f4f100d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 -- 172.21.15.46:0/1216705909 >> 172.21.15.46:0/1216705909 conn(0x7f4f10073e00 msgr2=0x7f4f1006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:35:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.965+0000 7f4f14e00640 1 -- 172.21.15.46:0/1216705909 shutdown_connections 2024-09-17T03:35:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 -- 172.21.15.46:0/1216705909 wait complete. 2024-09-17T03:35:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 Processor -- start 2024-09-17T03:35:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 -- start start 2024-09-17T03:35:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 0x7f4f10158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f100d25d0 0x7f4f1015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 0x7f4f101f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1016c410 con 0x7f4f100b60d0 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1016c290 con 0x7f4f100d29e0 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f14e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1016c590 con 0x7f4f100d25d0 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.969+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 0x7f4f10158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 0x7f4f10158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40712/0 (socket says 172.21.15.46:40712) 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 -- 172.21.15.46:0/1258635623 learned_addr learned my addr 172.21.15.46:0/1258635623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f05400640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f100d25d0 0x7f4f1015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0e800640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 0x7f4f101f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 -- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f100d25d0 msgr2=0x7f4f1015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f100d25d0 0x7f4f1015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 -- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 msgr2=0x7f4f101f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 0x7f4f101f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:35:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 -- 172.21.15.46:0/1258635623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f100cd890 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f0de00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 0x7f4f10158e30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000e970 tx=0x7f4f0000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000cd80 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f00004020 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00004470 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f100cdb70 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f100ce130 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f100b64e0 con 0x7f4f100b60d0 2024-09-17T03:35:59.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.973+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f4f00003600 con 0x7f4f100b60d0 2024-09-17T03:35:59.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.977+0000 7f4f06a00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4eec075210 0x7f4eec077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:35:59.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.977+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f4f00093a90 con 0x7f4f100b60d0 2024-09-17T03:35:59.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.977+0000 7f4f05400640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4eec075210 0x7f4eec077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:35:59.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.977+0000 7f4f05400640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4eec075210 0x7f4eec077720 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80af000 tx=0x7f4ef801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:35:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:35:59.981+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4f0005fff0 con 0x7f4f100b60d0 2024-09-17T03:36:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.153+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f4f100ceea0 con 0x7f4f100b60d0 2024-09-17T03:36:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f06a00640 1 -- 172.21.15.46:0/1258635623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v55) ==== 116+0+2 (secure 0 0 0) 0x7f4f00064a20 con 0x7f4f100b60d0 2024-09-17T03:36:00.156 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:36:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4eec075210 msgr2=0x7f4eec077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4eec075210 0x7f4eec077720 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80af000 tx=0x7f4ef801b060 comp rx=0 tx=0).stop 2024-09-17T03:36:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 msgr2=0x7f4f10158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 0x7f4f10158e30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000e970 tx=0x7f4f0000ee30 comp rx=0 tx=0).stop 2024-09-17T03:36:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 shutdown_connections 2024-09-17T03:36:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4eec075210 0x7f4eec077720 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f100d29e0 0x7f4f101f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4f100d25d0 0x7f4f1015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 --2- 172.21.15.46:0/1258635623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f100b60d0 0x7f4f10158e30 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 >> 172.21.15.46:0/1258635623 conn(0x7f4f10073e00 msgr2=0x7f4f100b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 shutdown_connections 2024-09-17T03:36:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.157+0000 7f4f14e00640 1 -- 172.21.15.46:0/1258635623 wait complete. 2024-09-17T03:36:00.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:36:00.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 -- 172.21.15.46:0/197150406 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7244160060 msgr2=0x7f7244074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 --2- 172.21.15.46:0/197150406 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7244160060 0x7f7244074240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7230009870 tx=0x7f723001c5f0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 -- 172.21.15.46:0/197150406 shutdown_connections 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 --2- 172.21.15.46:0/197150406 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7244160060 0x7f7244074240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 -- 172.21.15.46:0/197150406 >> 172.21.15.46:0/197150406 conn(0x7f72440b4560 msgr2=0x7f72440d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 -- 172.21.15.46:0/197150406 shutdown_connections 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.341+0000 7f724a000640 1 -- 172.21.15.46:0/197150406 wait complete. 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 Processor -- start 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 -- start start 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72441536f0 0x7f7244146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 0x7f7244153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7244160470 0x7f724414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f724416b530 con 0x7f7244160060 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f724416b3b0 con 0x7f7244160470 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f724416b6b0 con 0x7f72441536f0 2024-09-17T03:36:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 0x7f7244153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 0x7f7244153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40714/0 (socket says 172.21.15.46:40714) 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 -- 172.21.15.46:0/528417807 learned_addr learned my addr 172.21.15.46:0/528417807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7248c00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72441536f0 0x7f7244146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7249600640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7244160470 0x7f724414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 -- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72441536f0 msgr2=0x7f7244146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72441536f0 0x7f7244146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 -- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7244160470 msgr2=0x7f724414ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7244160470 0x7f724414ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 -- 172.21.15.46:0/528417807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7244146850 con 0x7f7244160060 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7243e00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 0x7f7244153b90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f723001cad0 tx=0x7f72300aa710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72300aaaf0 con 0x7f7244160060 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72300aac90 con 0x7f7244160060 2024-09-17T03:36:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230003800 con 0x7f7244160060 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72441540d0 con 0x7f7244160060 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.345+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7244154830 con 0x7f7244160060 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.349+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7244074240 con 0x7f7244160060 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.349+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f72300039a0 con 0x7f7244160060 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.349+0000 7f7240c00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7224075240 0x7f7224077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.349+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f723012e5a0 con 0x7f7244160060 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.349+0000 7f7248c00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7224075240 0x7f7224077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.353+0000 7f7248c00640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7224075240 0x7f7224077750 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7238001640 tx=0x7f7238009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.353+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7244146850 con 0x7f7244160060 2024-09-17T03:36:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.529+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f72441542a0 con 0x7f7244160060 2024-09-17T03:36:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.533+0000 7f7240c00640 1 -- 172.21.15.46:0/528417807 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7f7244074b90 con 0x7f7244160060 2024-09-17T03:36:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7224075240 msgr2=0x7f7224077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7224075240 0x7f7224077750 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7238001640 tx=0x7f7238009040 comp rx=0 tx=0).stop 2024-09-17T03:36:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 msgr2=0x7f7244153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 0x7f7244153b90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f723001cad0 tx=0x7f72300aa710 comp rx=0 tx=0).stop 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 shutdown_connections 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7224075240 0x7f7224077750 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7244160470 0x7f724414ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7244160060 0x7f7244153b90 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 --2- 172.21.15.46:0/528417807 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72441536f0 0x7f7244146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 >> 172.21.15.46:0/528417807 conn(0x7f72440b4560 msgr2=0x7f72440b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 shutdown_connections 2024-09-17T03:36:00.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.537+0000 7f724a000640 1 -- 172.21.15.46:0/528417807 wait complete. 2024-09-17T03:36:00.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:36:00.665 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:00.557+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:00.665 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:00.557+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:00.665 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:00.557+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:00.665 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:00.557+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:00.666 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:00.557+0000 7fbe4fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:00.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:36:00.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:36:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 -- 172.21.15.46:0/3979856523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b201527b0 msgr2=0x7f2b20074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 --2- 172.21.15.46:0/3979856523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b201527b0 0x7f2b20074e30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009840 tx=0x7f2b0801c760 comp rx=0 tx=0).stop 2024-09-17T03:36:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 -- 172.21.15.46:0/3979856523 shutdown_connections 2024-09-17T03:36:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 --2- 172.21.15.46:0/3979856523 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b201527b0 0x7f2b20074e30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 --2- 172.21.15.46:0/3979856523 >> v2:172.21.15.99:3300/0 conn(0x7f2b200b5c70 0x7f2b2015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 -- 172.21.15.46:0/3979856523 >> 172.21.15.46:0/3979856523 conn(0x7f2b200b2040 msgr2=0x7f2b200b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 -- 172.21.15.46:0/3979856523 shutdown_connections 2024-09-17T03:36:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.829+0000 7f2b26800640 1 -- 172.21.15.46:0/3979856523 wait complete. 2024-09-17T03:36:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 Processor -- start 2024-09-17T03:36:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 -- start start 2024-09-17T03:36:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 0x7f2b200c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200b5c70 0x7f2b200ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b201527b0 0x7f2b200aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20168c50 con 0x7f2b200b5c70 2024-09-17T03:36:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20168ad0 con 0x7f2b201527b0 2024-09-17T03:36:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2b20168dd0 con 0x7f2b200b4de0 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200b5c70 0x7f2b200ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 0x7f2b200c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200b5c70 0x7f2b200ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40738/0 (socket says 172.21.15.46:40738) 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 0x7f2b200c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:52408/0 (socket says 172.21.15.46:52408) 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b16a00640 1 -- 172.21.15.46:0/1140819584 learned_addr learned my addr 172.21.15.46:0/1140819584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1fe00640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b201527b0 0x7f2b200aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 -- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b201527b0 msgr2=0x7f2b200aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b201527b0 0x7f2b200aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 -- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200b5c70 msgr2=0x7f2b200ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200b5c70 0x7f2b200ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:00.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 -- 172.21.15.46:0/1140819584 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b201f8d30 con 0x7f2b200b4de0 2024-09-17T03:36:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b1f400640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 0x7f2b200c5740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000ea20 tx=0x7f2b1000eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000ccc0 con 0x7f2b200b4de0 2024-09-17T03:36:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b1000ce60 con 0x7f2b200b4de0 2024-09-17T03:36:00.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004280 con 0x7f2b200b4de0 2024-09-17T03:36:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b201f9010 con 0x7f2b200b4de0 2024-09-17T03:36:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.833+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b200c5c80 con 0x7f2b200b4de0 2024-09-17T03:36:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.837+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b200cdba0 con 0x7f2b200b4de0 2024-09-17T03:36:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.837+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7f2b10003600 con 0x7f2b200b4de0 2024-09-17T03:36:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.837+0000 7f2b17e00640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2afc075210 0x7f2afc077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.837+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f2b10093a90 con 0x7f2b200b4de0 2024-09-17T03:36:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.837+0000 7f2b16a00640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2afc075210 0x7f2afc077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.841+0000 7f2b16a00640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2afc075210 0x7f2afc077720 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2b0801a390 tx=0x7f2b0801afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:00.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:00.841+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b1005ff50 con 0x7f2b200b4de0 2024-09-17T03:36:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.045+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f2b2006b540 con 0x7f2afc075210 2024-09-17T03:36:01.055 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:01.057+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:01.056 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:01.057+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:01.056 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:01.057+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:01.056 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:01.057+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:01.056 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:01.057+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.089+0000 7f2b17e00640 1 -- 172.21.15.46:0/1140819584 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b08004040 con 0x7f2afc075210 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2afc075210 msgr2=0x7f2afc077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2afc075210 0x7f2afc077720 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2b0801a390 tx=0x7f2b0801afa0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 msgr2=0x7f2b200c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 0x7f2b200c5740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000ea20 tx=0x7f2b1000eee0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 shutdown_connections 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2afc075210 0x7f2afc077720 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b201527b0 0x7f2b200aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200b5c70 0x7f2b200ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 --2- 172.21.15.46:0/1140819584 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2b200b4de0 0x7f2b200c5740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 >> 172.21.15.46:0/1140819584 conn(0x7f2b200b2040 msgr2=0x7f2b201583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 shutdown_connections 2024-09-17T03:36:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.093+0000 7f2b26800640 1 -- 172.21.15.46:0/1140819584 wait complete. 2024-09-17T03:36:01.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:36:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 -- 172.21.15.46:0/473811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0074b20 msgr2=0x7fb6b015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/473811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0074b20 0x7fb6b015cf90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00713e0 tx=0x7fb6a801caf0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 -- 172.21.15.46:0/473811979 shutdown_connections 2024-09-17T03:36:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/473811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0074b20 0x7fb6b015cf90 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/473811979 >> v2:172.21.15.99:3300/0 conn(0x7fb6b0157030 0x7fb6b00b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 -- 172.21.15.46:0/473811979 >> 172.21.15.46:0/473811979 conn(0x7fb6b00708c0 msgr2=0x7fb6b0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 -- 172.21.15.46:0/473811979 shutdown_connections 2024-09-17T03:36:01.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.273+0000 7fb6b5c00640 1 -- 172.21.15.46:0/473811979 wait complete. 2024-09-17T03:36:01.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 Processor -- start 2024-09-17T03:36:01.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 -- start start 2024-09-17T03:36:01.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6b0157030 0x7fb6b013bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 0x7fb6b01fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6b01545f0 0x7fb6b01fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b016a790 con 0x7fb6b0160d50 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b016a610 con 0x7fb6b01545f0 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.277+0000 7fb6b5c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b016a910 con 0x7fb6b0157030 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 0x7fb6b01fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 0x7fb6b01fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40768/0 (socket says 172.21.15.46:40768) 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 -- 172.21.15.46:0/540550450 learned_addr learned my addr 172.21.15.46:0/540550450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6af400640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6b01545f0 0x7fb6b01fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6aea00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6b0157030 0x7fb6b013bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 -- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6b0157030 msgr2=0x7fb6b013bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6b0157030 0x7fb6b013bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 -- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6b01545f0 msgr2=0x7fb6b01fcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6b01545f0 0x7fb6b01fcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 -- 172.21.15.46:0/540550450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b01fd050 con 0x7fb6b0160d50 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a7e00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 0x7fb6b01fc5d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fb69800c400 tx=0x7fb69800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698017070 con 0x7fb6b0160d50 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb698003600 con 0x7fb6b0160d50 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698003a50 con 0x7fb6b0160d50 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b013c330 con 0x7fb6b0160d50 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01fd4c0 con 0x7fb6b0160d50 2024-09-17T03:36:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.281+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b01fd220 con 0x7fb6b0160d50 2024-09-17T03:36:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.285+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7fb69801a070 con 0x7fb6b0160d50 2024-09-17T03:36:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.285+0000 7fb6a6a00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb68c075240 0x7fb68c077750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.285+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fb698092890 con 0x7fb6b0160d50 2024-09-17T03:36:01.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.285+0000 7fb6aea00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb68c075240 0x7fb68c077750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.285+0000 7fb6aea00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb68c075240 0x7fb68c077750 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009b40 tx=0x7fb6a00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:01.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.289+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb6b01fd220 con 0x7fb6b0160d50 2024-09-17T03:36:01.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.485+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb6b006b540 con 0x7fb68c075240 2024-09-17T03:36:01.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6a6a00640 1 -- 172.21.15.46:0/540550450 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb6a0011be0 con 0x7fb68c075240 2024-09-17T03:36:01.491 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb68c075240 msgr2=0x7fb68c077750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb68c075240 0x7fb68c077750 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009b40 tx=0x7fb6a00091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 msgr2=0x7fb6b01fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 0x7fb6b01fc5d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fb69800c400 tx=0x7fb69800c8c0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 shutdown_connections 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb68c075240 0x7fb68c077750 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6b01545f0 0x7fb6b01fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6b0160d50 0x7fb6b01fc5d0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 --2- 172.21.15.46:0/540550450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb6b0157030 0x7fb6b013bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 >> 172.21.15.46:0/540550450 conn(0x7fb6b00708c0 msgr2=0x7fb6b015f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 shutdown_connections 2024-09-17T03:36:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.493+0000 7fb6b5c00640 1 -- 172.21.15.46:0/540550450 wait complete. 2024-09-17T03:36:01.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 -- 172.21.15.46:0/1645878035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b5c70 msgr2=0x7effb815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 --2- 172.21.15.46:0/1645878035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b5c70 0x7effb815bb60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7effac009840 tx=0x7effac01c7f0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 -- 172.21.15.46:0/1645878035 shutdown_connections 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 --2- 172.21.15.46:0/1645878035 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effb80b4de0 0x7effb80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 --2- 172.21.15.46:0/1645878035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b5c70 0x7effb815bb60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 -- 172.21.15.46:0/1645878035 >> 172.21.15.46:0/1645878035 conn(0x7effb80b2040 msgr2=0x7effb80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 -- 172.21.15.46:0/1645878035 shutdown_connections 2024-09-17T03:36:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.677+0000 7effc0200640 1 -- 172.21.15.46:0/1645878035 wait complete. 2024-09-17T03:36:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 Processor -- start 2024-09-17T03:36:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 -- start start 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b4de0 0x7effb813a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 0x7effb814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effb81527b0 0x7effb815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7effb8168a20 con 0x7effb80b5c70 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effb81688a0 con 0x7effb80b4de0 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7effb8168ba0 con 0x7effb81527b0 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 0x7effb814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 0x7effb814db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40782/0 (socket says 172.21.15.46:40782) 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 -- 172.21.15.46:0/1279520888 learned_addr learned my addr 172.21.15.46:0/1279520888 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbd800640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b4de0 0x7effb813a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbe200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effb81527b0 0x7effb815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 -- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effb81527b0 msgr2=0x7effb815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effb81527b0 0x7effb815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 -- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b4de0 msgr2=0x7effb813a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b4de0 0x7effb813a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 -- 172.21.15.46:0/1279520888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb80ccc80 con 0x7effb80b5c70 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effbce00640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 0x7effb814db50 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7effa400c760 tx=0x7effa400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4016020 con 0x7effb80b5c70 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effa400a8f0 con 0x7effb80b5c70 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa400ad40 con 0x7effb80b5c70 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb80ccf60 con 0x7effb80b5c70 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.681+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb80cd520 con 0x7effb80b5c70 2024-09-17T03:36:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.685+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb80b51f0 con 0x7effb80b5c70 2024-09-17T03:36:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.685+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) ==== 100141+0+0 (secure 0 0 0) 0x7effa4019020 con 0x7effb80b5c70 2024-09-17T03:36:01.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.685+0000 7effb5600640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff98075210 0x7eff98077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:01.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.685+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7effa4093100 con 0x7effb80b5c70 2024-09-17T03:36:01.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.685+0000 7effbd800640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff98075210 0x7eff98077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:01.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.685+0000 7effbd800640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff98075210 0x7eff98077720 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7effac0af000 tx=0x7effac01b030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.689+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7effa4020020 con 0x7effb80b5c70 2024-09-17T03:36:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.889+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7effb81fadc0 con 0x7eff98075210 2024-09-17T03:36:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.893+0000 7effb5600640 1 -- 172.21.15.46:0/1279520888 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7effac0a56b0 con 0x7eff98075210 2024-09-17T03:36:01.895 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:01.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff98075210 msgr2=0x7eff98077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff98075210 0x7eff98077720 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7effac0af000 tx=0x7effac01b030 comp rx=0 tx=0).stop 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 msgr2=0x7effb814db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 0x7effb814db50 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7effa400c760 tx=0x7effa400cc20 comp rx=0 tx=0).stop 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 shutdown_connections 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7eff98075210 0x7eff98077720 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7effb81527b0 0x7effb815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effb80b5c70 0x7effb814db50 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 --2- 172.21.15.46:0/1279520888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb80b4de0 0x7effb813a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 >> 172.21.15.46:0/1279520888 conn(0x7effb80b2040 msgr2=0x7effb8154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 shutdown_connections 2024-09-17T03:36:01.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:01.897+0000 7effc0200640 1 -- 172.21.15.46:0/1279520888 wait complete. 2024-09-17T03:36:01.918 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:01.918 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:01.918 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:01.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:01.928 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:36:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:36:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:36:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.48251 s, 127 MB/s 2024-09-17T03:36:10.474 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:10.474 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:10.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:10.483 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:36:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:36:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:36:18.563 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.0256 s, 134 MB/s 2024-09-17T03:36:18.564 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:18.564 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:18.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:18.573 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:36:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:36:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:36:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.84846 s, 121 MB/s 2024-09-17T03:36:27.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:36:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 -- 172.21.15.46:0/4294154787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 msgr2=0x7f9cb00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 --2- 172.21.15.46:0/4294154787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 0x7f9cb00b51f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009c80 tx=0x7f9ca801cce0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 -- 172.21.15.46:0/4294154787 shutdown_connections 2024-09-17T03:36:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 --2- 172.21.15.46:0/4294154787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 0x7f9cb00b51f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 --2- 172.21.15.46:0/4294154787 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cb00b5c70 0x7f9cb015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 -- 172.21.15.46:0/4294154787 >> 172.21.15.46:0/4294154787 conn(0x7f9cb00b2040 msgr2=0x7f9cb00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 -- 172.21.15.46:0/4294154787 shutdown_connections 2024-09-17T03:36:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 -- 172.21.15.46:0/4294154787 wait complete. 2024-09-17T03:36:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 Processor -- start 2024-09-17T03:36:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.641+0000 7f9cb6400640 1 -- start start 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 0x7f9cb01516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cb00b5c70 0x7f9cb014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 0x7f9cb01f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0168ab0 con 0x7f9cb01527b0 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0168930 con 0x7f9cb00b4de0 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb0168c30 con 0x7f9cb00b5c70 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 0x7f9cb01f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 0x7f9cb01f8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48310/0 (socket says 172.21.15.46:48310) 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 -- 172.21.15.46:0/256975015 learned_addr learned my addr 172.21.15.46:0/256975015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9caf400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 0x7f9cb01516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cb00b5c70 0x7f9cb014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 -- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cb00b5c70 msgr2=0x7f9cb014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cb00b5c70 0x7f9cb014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 -- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 msgr2=0x7f9cb01516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 0x7f9cb01516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 -- 172.21.15.46:0/256975015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb01505e0 con 0x7f9cb01527b0 2024-09-17T03:36:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cafe00640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 0x7f9cb01f8240 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009bd0 tx=0x7f9ca8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80bb070 con 0x7f9cb01527b0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ca8004b40 con 0x7f9cb01527b0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb01508c0 con 0x7f9cb01527b0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80b2040 con 0x7f9cb01527b0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb00ceba0 con 0x7f9cb01527b0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.645+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb00b51f0 con 0x7f9cb01527b0 2024-09-17T03:36:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.649+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) ==== 100243+0+0 (secure 0 0 0) 0x7f9ca80be070 con 0x7f9cb01527b0 2024-09-17T03:36:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.649+0000 7f9ca7400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c8c075330 0x7f9c8c077840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.649+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f9ca812d400 con 0x7f9cb01527b0 2024-09-17T03:36:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.649+0000 7f9caf400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c8c075330 0x7f9c8c077840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.649+0000 7f9caf400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c8c075330 0x7f9c8c077840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009b40 tx=0x7f9ca00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.653+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ca8016990 con 0x7f9cb01527b0 2024-09-17T03:36:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.857+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f9cb006b540 con 0x7f9c8c075330 2024-09-17T03:36:27.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9ca7400640 1 -- 172.21.15.46:0/256975015 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ca0011be0 con 0x7f9c8c075330 2024-09-17T03:36:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c8c075330 msgr2=0x7f9c8c077840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c8c075330 0x7f9c8c077840 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009b40 tx=0x7f9ca00091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 msgr2=0x7f9cb01f8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 0x7f9cb01f8240 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009bd0 tx=0x7f9ca8005b00 comp rx=0 tx=0).stop 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 shutdown_connections 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9c8c075330 0x7f9c8c077840 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cb01527b0 0x7f9cb01f8240 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cb00b5c70 0x7f9cb014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 --2- 172.21.15.46:0/256975015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cb00b4de0 0x7f9cb01516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 >> 172.21.15.46:0/256975015 conn(0x7f9cb00b2040 msgr2=0x7f9cb0151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 shutdown_connections 2024-09-17T03:36:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:27.873+0000 7f9cb6400640 1 -- 172.21.15.46:0/256975015 wait complete. 2024-09-17T03:36:27.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:36:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 -- 172.21.15.46:0/1795714067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b5c70 msgr2=0x7f4bf015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/1795714067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b5c70 0x7f4bf015bb60 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00cca90 tx=0x7f4be001c800 comp rx=0 tx=0).stop 2024-09-17T03:36:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 -- 172.21.15.46:0/1795714067 shutdown_connections 2024-09-17T03:36:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/1795714067 >> v2:172.21.15.99:3300/0 conn(0x7f4bf00b4de0 0x7f4bf00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/1795714067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf00b5c70 0x7f4bf015bb60 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 -- 172.21.15.46:0/1795714067 >> 172.21.15.46:0/1795714067 conn(0x7f4bf00b2040 msgr2=0x7f4bf00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 -- 172.21.15.46:0/1795714067 shutdown_connections 2024-09-17T03:36:28.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 -- 172.21.15.46:0/1795714067 wait complete. 2024-09-17T03:36:28.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.057+0000 7f4bf7c00640 1 Processor -- start 2024-09-17T03:36:28.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 -- start start 2024-09-17T03:36:28.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf00b4de0 0x7f4bf00acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bf00b5c70 0x7f4bf00ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 0x7f4bf00abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0168910 con 0x7f4bf01527b0 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0168790 con 0x7f4bf00b4de0 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf0168a90 con 0x7f4bf00b5c70 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 0x7f4bf00abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 0x7f4bf00abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48320/0 (socket says 172.21.15.46:48320) 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 -- 172.21.15.46:0/4141186612 learned_addr learned my addr 172.21.15.46:0/4141186612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4be7e00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bf00b5c70 0x7f4bf00ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5200640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf00b4de0 0x7f4bf00acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 -- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bf00b5c70 msgr2=0x7f4bf00ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:28.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bf00b5c70 0x7f4bf00ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 -- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf00b4de0 msgr2=0x7f4bf00acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf00b4de0 0x7f4bf00acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 -- 172.21.15.46:0/4141186612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf00aa570 con 0x7f4bf01527b0 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf5c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 0x7f4bf00abb50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4be800eab0 tx=0x7f4be8009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be800ce40 con 0x7f4bf01527b0 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4be8004c20 con 0x7f4bf01527b0 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf00aa850 con 0x7f4bf01527b0 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8005020 con 0x7f4bf01527b0 2024-09-17T03:36:28.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.061+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf00aae10 con 0x7f4bf01527b0 2024-09-17T03:36:28.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.065+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf00b51f0 con 0x7f4bf01527b0 2024-09-17T03:36:28.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.065+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) ==== 100243+0+0 (secure 0 0 0) 0x7f4be80051e0 con 0x7f4bf01527b0 2024-09-17T03:36:28.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.065+0000 7f4bed600640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4bcc075360 0x7f4bcc077870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:28.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.065+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f4be8094010 con 0x7f4bf01527b0 2024-09-17T03:36:28.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.065+0000 7f4bf5200640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4bcc075360 0x7f4bcc077870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:28.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.065+0000 7f4bf5200640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4bcc075360 0x7f4bcc077870 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4be00af000 tx=0x7f4be001b040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:28.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.069+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4be80604d0 con 0x7f4bf01527b0 2024-09-17T03:36:28.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.273+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4bf006b540 con 0x7f4bcc075360 2024-09-17T03:36:28.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.277+0000 7f4bed600640 1 -- 172.21.15.46:0/4141186612 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f4be00a59b0 con 0x7f4bcc075360 2024-09-17T03:36:28.279 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4bcc075360 msgr2=0x7f4bcc077870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4bcc075360 0x7f4bcc077870 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4be00af000 tx=0x7f4be001b040 comp rx=0 tx=0).stop 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 msgr2=0x7f4bf00abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 0x7f4bf00abb50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4be800eab0 tx=0x7f4be8009f70 comp rx=0 tx=0).stop 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 shutdown_connections 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4bcc075360 0x7f4bcc077870 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf01527b0 0x7f4bf00abb50 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4bf00b5c70 0x7f4bf00ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 --2- 172.21.15.46:0/4141186612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf00b4de0 0x7f4bf00acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 >> 172.21.15.46:0/4141186612 conn(0x7f4bf00b2040 msgr2=0x7f4bf01581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 shutdown_connections 2024-09-17T03:36:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:28.281+0000 7f4bf7c00640 1 -- 172.21.15.46:0/4141186612 wait complete. 2024-09-17T03:36:28.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:28.301 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:28.301 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:36:28.311 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:28.311 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:36:28.311 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:28.312 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:36:30.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:30.313 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:30.313 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:36:30.325 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:30.325 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:36:30.325 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:30.326 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:36:30.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:36:30.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:36:32.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:32.327 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:32.327 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:36:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:32.338 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T03:36:32.339 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:32.339 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T03:36:32.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-17T03:36:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 -- 172.21.15.46:0/2675745768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c003ae0 msgr2=0x7fc91c003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 --2- 172.21.15.46:0/2675745768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c003ae0 0x7fc91c003f50 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc9280016f0 tx=0x7fc92801c760 comp rx=0 tx=0).stop 2024-09-17T03:36:32.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 -- 172.21.15.46:0/2675745768 shutdown_connections 2024-09-17T03:36:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 --2- 172.21.15.46:0/2675745768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c003ae0 0x7fc91c003f50 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 -- 172.21.15.46:0/2675745768 >> 172.21.15.46:0/2675745768 conn(0x7fc93406cdf0 msgr2=0x7fc93406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 -- 172.21.15.46:0/2675745768 shutdown_connections 2024-09-17T03:36:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.561+0000 7fc939e00640 1 -- 172.21.15.46:0/2675745768 wait complete. 2024-09-17T03:36:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 Processor -- start 2024-09-17T03:36:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 -- start start 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc91c003ae0 0x7fc9341f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 0x7fc93415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9340ab1d0 0x7fc93415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340b3b10 con 0x7fc9340aadc0 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340b3990 con 0x7fc9340ab1d0 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc9340b3c90 con 0x7fc91c003ae0 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 0x7fc93415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 0x7fc93415a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52206/0 (socket says 172.21.15.46:52206) 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 -- 172.21.15.46:0/3405014450 learned_addr learned my addr 172.21.15.46:0/3405014450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc938a00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc91c003ae0 0x7fc9341f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc939400640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9340ab1d0 0x7fc93415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 -- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc91c003ae0 msgr2=0x7fc9341f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc91c003ae0 0x7fc9341f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 -- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9340ab1d0 msgr2=0x7fc93415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9340ab1d0 0x7fc93415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 -- 172.21.15.46:0/3405014450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341f9d40 con 0x7fc9340aadc0 2024-09-17T03:36:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.565+0000 7fc933e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 0x7fc93415a4a0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc9200119e0 tx=0x7fc920011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92000fde0 con 0x7fc9340aadc0 2024-09-17T03:36:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc920004030 con 0x7fc9340aadc0 2024-09-17T03:36:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920004440 con 0x7fc9340aadc0 2024-09-17T03:36:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9340ab960 con 0x7fc9340aadc0 2024-09-17T03:36:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93415d190 con 0x7fc9340aadc0 2024-09-17T03:36:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9340ab6d0 con 0x7fc9340aadc0 2024-09-17T03:36:32.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) ==== 100243+0+0 (secure 0 0 0) 0x7fc920003600 con 0x7fc9340aadc0 2024-09-17T03:36:32.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc930c00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc914075330 0x7fc914077840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:32.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.569+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fc920093ff0 con 0x7fc9340aadc0 2024-09-17T03:36:32.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.573+0000 7fc938a00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc914075330 0x7fc914077840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:32.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.573+0000 7fc938a00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc914075330 0x7fc914077840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc9280016c0 tx=0x7fc9280a8040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.577+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc920060a70 con 0x7fc9340aadc0 2024-09-17T03:36:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.781+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fc93406b540 con 0x7fc914075330 2024-09-17T03:36:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.841+0000 7fc930c00640 1 -- 172.21.15.46:0/3405014450 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9280062b0 con 0x7fc914075330 2024-09-17T03:36:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.841+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc914075330 msgr2=0x7fc914077840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.841+0000 7fc939e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc914075330 0x7fc914077840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc9280016c0 tx=0x7fc9280a8040 comp rx=0 tx=0).stop 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.841+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 msgr2=0x7fc93415a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.841+0000 7fc939e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 0x7fc93415a4a0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc9200119e0 tx=0x7fc920011ea0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 shutdown_connections 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc914075330 0x7fc914077840 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9340ab1d0 0x7fc93415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9340aadc0 0x7fc93415a4a0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 --2- 172.21.15.46:0/3405014450 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc91c003ae0 0x7fc9341f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 >> 172.21.15.46:0/3405014450 conn(0x7fc93406cdf0 msgr2=0x7fc9340b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 shutdown_connections 2024-09-17T03:36:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fc939e00640 1 -- 172.21.15.46:0/3405014450 wait complete. 2024-09-17T03:36:32.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.845+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.864 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.865+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.865 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.865+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.865 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.865+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.865 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.865+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:32.865 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:32.865+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:33.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:33.853+0000 7fbdc4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:33.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:33.853+0000 7fbdc4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:33.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:33.853+0000 7fbdc4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:33.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:33.853+0000 7fbdc4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:33.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:33.853+0000 7fbdc4e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:33.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-17T03:36:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.161+0000 7f6508c00640 1 -- 172.21.15.46:0/3996039849 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64e8003cc0 msgr2=0x7f64e8004110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.161+0000 7f6508c00640 1 --2- 172.21.15.46:0/3996039849 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f64e8003cc0 0x7f64e8004110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001760 tx=0x7f64f401c800 comp rx=0 tx=0).stop 2024-09-17T03:36:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.161+0000 7f6502800640 1 -- 172.21.15.46:0/3996039849 reap_dead start 2024-09-17T03:36:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.161+0000 7f6508c00640 1 -- 172.21.15.46:0/3996039849 shutdown_connections 2024-09-17T03:36:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.161+0000 7f6508c00640 1 -- 172.21.15.46:0/3996039849 >> 172.21.15.46:0/3996039849 conn(0x7f65040b2040 msgr2=0x7f65040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.165+0000 7f6508c00640 1 -- 172.21.15.46:0/3996039849 shutdown_connections 2024-09-17T03:36:34.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.165+0000 7f6508c00640 1 -- 172.21.15.46:0/3996039849 wait complete. 2024-09-17T03:36:34.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 Processor -- start 2024-09-17T03:36:34.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 -- start start 2024-09-17T03:36:34.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65040ae560 0x7f6504150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65041f6cf0 0x7f65041f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 0x7f65041f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65040b3340 con 0x7f65041f7640 2024-09-17T03:36:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6504070580 con 0x7f65040ae560 2024-09-17T03:36:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6508c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f65040b34c0 con 0x7f65041f6cf0 2024-09-17T03:36:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6502800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 0x7f65041f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6502800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 0x7f65041f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52270/0 (socket says 172.21.15.46:52270) 2024-09-17T03:36:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.169+0000 7f6502800640 1 -- 172.21.15.46:0/453765832 learned_addr learned my addr 172.21.15.46:0/453765832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f64f9400640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65041f6cf0 0x7f65041f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6501e00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65040ae560 0x7f6504150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6502800640 1 -- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65041f6cf0 msgr2=0x7f65041f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6502800640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65041f6cf0 0x7f65041f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6502800640 1 -- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65040ae560 msgr2=0x7f6504150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6502800640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65040ae560 0x7f6504150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6502800640 1 -- 172.21.15.46:0/453765832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65040aeb20 con 0x7f65041f7640 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f64f9400640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65041f6cf0 0x7f65041f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f6502800640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 0x7f65041f3700 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f64fc003a00 tx=0x7f64fc00fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.173+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc01b070 con 0x7f65041f7640 2024-09-17T03:36:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.177+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64fc00a530 con 0x7f65041f7640 2024-09-17T03:36:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.177+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65040aee00 con 0x7f65041f7640 2024-09-17T03:36:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.177+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00a980 con 0x7f65041f7640 2024-09-17T03:36:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.177+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65040af3c0 con 0x7f65041f7640 2024-09-17T03:36:34.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.177+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65040b2710 con 0x7f65041f7640 2024-09-17T03:36:34.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.181+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 51) ==== 100345+0+0 (secure 0 0 0) 0x7f64fc01f3f0 con 0x7f65041f7640 2024-09-17T03:36:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.181+0000 7f64faa00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64e0075450 0x7f64e0077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.181+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f64fc095830 con 0x7f65041f7640 2024-09-17T03:36:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.181+0000 7f6501e00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64e0075450 0x7f64e0077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.185+0000 7f6501e00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64e0075450 0x7f64e0077960 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f64e80057f0 tx=0x7f64e8005840 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:34.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.193+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f64fc062350 con 0x7f65041f7640 2024-09-17T03:36:34.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.609+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f650415a240 con 0x7f64e0075450 2024-09-17T03:36:34.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.773+0000 7f64faa00640 1 -- 172.21.15.46:0/453765832 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f650415a240 con 0x7f64e0075450 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64e0075450 msgr2=0x7f64e0077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64e0075450 0x7f64e0077960 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f64e80057f0 tx=0x7f64e8005840 comp rx=0 tx=0).stop 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 msgr2=0x7f65041f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 0x7f65041f3700 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f64fc003a00 tx=0x7f64fc00fa60 comp rx=0 tx=0).stop 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 shutdown_connections 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f64e0075450 0x7f64e0077960 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65041f7640 0x7f65041f3700 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f65041f6cf0 0x7f65041f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 --2- 172.21.15.46:0/453765832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65040ae560 0x7f6504150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 >> 172.21.15.46:0/453765832 conn(0x7f65040b2040 msgr2=0x7f6504072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 shutdown_connections 2024-09-17T03:36:34.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.777+0000 7f6508c00640 1 -- 172.21.15.46:0/453765832 wait complete. 2024-09-17T03:36:34.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T03:36:34.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 -- 172.21.15.46:0/3422645342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd81527b0 msgr2=0x7f9cd8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 --2- 172.21.15.46:0/3422645342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd81527b0 0x7f9cd8074e30 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009900 tx=0x7f9cc001c820 comp rx=0 tx=0).stop 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 -- 172.21.15.46:0/3422645342 shutdown_connections 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 --2- 172.21.15.46:0/3422645342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd81527b0 0x7f9cd8074e30 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 --2- 172.21.15.46:0/3422645342 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cd80b5c70 0x7f9cd815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 -- 172.21.15.46:0/3422645342 >> 172.21.15.46:0/3422645342 conn(0x7f9cd80b2040 msgr2=0x7f9cd80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 -- 172.21.15.46:0/3422645342 shutdown_connections 2024-09-17T03:36:34.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.957+0000 7f9cde600640 1 -- 172.21.15.46:0/3422645342 wait complete. 2024-09-17T03:36:34.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 Processor -- start 2024-09-17T03:36:34.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 -- start start 2024-09-17T03:36:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd80b4de0 0x7f9cd8152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 0x7f9cd81499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cd81527b0 0x7f9cd81f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8168c00 con 0x7f9cd80b5c70 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8168a80 con 0x7f9cd80b4de0 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9cde600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd8168d80 con 0x7f9cd81527b0 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.961+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 0x7f9cd81499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 0x7f9cd81499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52280/0 (socket says 172.21.15.46:52280) 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 -- 172.21.15.46:0/192374061 learned_addr learned my addr 172.21.15.46:0/192374061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9cd7e00640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cd81527b0 0x7f9cd81f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9cd7400640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd80b4de0 0x7f9cd8152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 -- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cd81527b0 msgr2=0x7f9cd81f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cd81527b0 0x7f9cd81f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 -- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd80b4de0 msgr2=0x7f9cd8152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:34.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd80b4de0 0x7f9cd8152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 -- 172.21.15.46:0/192374061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd8158ca0 con 0x7f9cd80b5c70 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccea00640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 0x7f9cd81499b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00b0040 tx=0x7f9cc00a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b2040 con 0x7f9cd80b5c70 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cc00041b0 con 0x7f9cd80b5c70 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0003600 con 0x7f9cd80b5c70 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd8158f80 con 0x7f9cd80b5c70 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd81f8c80 con 0x7f9cd80b5c70 2024-09-17T03:36:34.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd814b490 con 0x7f9cd80b5c70 2024-09-17T03:36:34.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.965+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 51) ==== 100345+0+0 (secure 0 0 0) 0x7f9cc00bf020 con 0x7f9cd80b5c70 2024-09-17T03:36:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.969+0000 7f9ccfe00640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9cb4075450 0x7f9cb4077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.969+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f9cc012dbf0 con 0x7f9cd80b5c70 2024-09-17T03:36:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.969+0000 7f9cd7400640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9cb4075450 0x7f9cb4077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:34.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.969+0000 7f9cd7400640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9cb4075450 0x7f9cb4077960 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009b40 tx=0x7f9cc80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:34.969+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9cc00c6020 con 0x7f9cd80b5c70 2024-09-17T03:36:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:35.181+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f9cd806b540 con 0x7f9cb4075450 2024-09-17T03:36:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.133+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7f9cc00f6320 con 0x7f9cd80b5c70 2024-09-17T03:36:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9ccfe00640 1 -- 172.21.15.46:0/192374061 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f9cc8011be0 con 0x7f9cb4075450 2024-09-17T03:36:36.564 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:36:36.564 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:36.565 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:36:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9cb4075450 msgr2=0x7f9cb4077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9cb4075450 0x7f9cb4077960 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009b40 tx=0x7f9cc80091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 msgr2=0x7f9cd81499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 0x7f9cd81499b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00b0040 tx=0x7f9cc00a5780 comp rx=0 tx=0).stop 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 shutdown_connections 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9cb4075450 0x7f9cb4077960 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9cd81527b0 0x7f9cd81f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd80b5c70 0x7f9cd81499b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 --2- 172.21.15.46:0/192374061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd80b4de0 0x7f9cd8152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 >> 172.21.15.46:0/192374061 conn(0x7f9cd80b2040 msgr2=0x7f9cd80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.565+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 shutdown_connections 2024-09-17T03:36:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.569+0000 7f9cde600640 1 -- 172.21.15.46:0/192374061 wait complete. 2024-09-17T03:36:36.591 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:36:36.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:36:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 -- 172.21.15.46:0/975305167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0c20 msgr2=0x7f8c84069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 --2- 172.21.15.46:0/975305167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0c20 0x7f8c84069330 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f8c84072d40 tx=0x7f8c7c01cd80 comp rx=0 tx=0).stop 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 -- 172.21.15.46:0/975305167 shutdown_connections 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 --2- 172.21.15.46:0/975305167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0c20 0x7f8c84069330 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 --2- 172.21.15.46:0/975305167 >> v2:172.21.15.99:3300/0 conn(0x7f8c840b0810 0x7f8c84068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 -- 172.21.15.46:0/975305167 >> 172.21.15.46:0/975305167 conn(0x7f8c840b35f0 msgr2=0x7f8c840b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 -- 172.21.15.46:0/975305167 shutdown_connections 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.753+0000 7f8c88e00640 1 -- 172.21.15.46:0/975305167 wait complete. 2024-09-17T03:36:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 Processor -- start 2024-09-17T03:36:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 -- start start 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 0x7f8c840b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c840b0c20 0x7f8c8414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8c8415efc0 0x7f8c840cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84162e10 con 0x7f8c840b0810 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84162c90 con 0x7f8c840b0c20 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c88e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8c84162f90 con 0x7f8c8415efc0 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c81e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 0x7f8c840b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c81e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 0x7f8c840b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52304/0 (socket says 172.21.15.46:52304) 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.757+0000 7f8c81e00640 1 -- 172.21.15.46:0/1139090326 learned_addr learned my addr 172.21.15.46:0/1139090326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c79400640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c840b0c20 0x7f8c8414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:36.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c82800640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8c8415efc0 0x7f8c840cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c81e00640 1 -- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8c8415efc0 msgr2=0x7f8c840cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c81e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8c8415efc0 0x7f8c840cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c81e00640 1 -- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c840b0c20 msgr2=0x7f8c8414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c81e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c840b0c20 0x7f8c8414d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c81e00640 1 -- 172.21.15.46:0/1139090326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c840b1c30 con 0x7f8c840b0810 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c82800640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8c8415efc0 0x7f8c840cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:36.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c81e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 0x7f8c840b16f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400b410 tx=0x7f8c7400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:36.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74015030 con 0x7f8c840b0810 2024-09-17T03:36:36.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c74004020 con 0x7f8c840b0810 2024-09-17T03:36:36.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8414a520 con 0x7f8c840b0810 2024-09-17T03:36:36.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74004470 con 0x7f8c840b0810 2024-09-17T03:36:36.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c840d0730 con 0x7f8c840b0810 2024-09-17T03:36:36.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c84070910 con 0x7f8c840b0810 2024-09-17T03:36:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.761+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7f8c74018070 con 0x7f8c840b0810 2024-09-17T03:36:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.765+0000 7f8c7aa00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8c600754e0 0x7f8c600779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.765+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f8c74092f30 con 0x7f8c840b0810 2024-09-17T03:36:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.765+0000 7f8c79400640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8c600754e0 0x7f8c600779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:36.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.765+0000 7f8c79400640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8c600754e0 0x7f8c600779f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8c8414d4e0 tx=0x7f8c6c00a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:36.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:36.769+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c7405f930 con 0x7f8c840b0810 2024-09-17T03:36:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.009+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8c8406e5e0 con 0x7f8c840b0810 2024-09-17T03:36:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.009+0000 7f8c7aa00640 1 -- 172.21.15.46:0/1139090326 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1882 (secure 0 0 0) 0x7f8c7405f230 con 0x7f8c840b0810 2024-09-17T03:36:37.009 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:36:37.010 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 954, 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:36:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 129, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:36:37.012 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 793, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 3222008369, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 7624466432, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 765469646848, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 89457019, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 0, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 21 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.013 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 85, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:35:58:996234+0000", 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 9, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "a", 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:36:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 248335 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:36:37.015 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 22, 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:35:27.556456+0000", 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:36:37.016 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8c600754e0 msgr2=0x7f8c600779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8c600754e0 0x7f8c600779f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8c8414d4e0 tx=0x7f8c6c00a250 comp rx=0 tx=0).stop 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 msgr2=0x7f8c840b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 0x7f8c840b16f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400b410 tx=0x7f8c7400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 shutdown_connections 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8c600754e0 0x7f8c600779f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8c8415efc0 0x7f8c840cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c840b0c20 0x7f8c8414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 --2- 172.21.15.46:0/1139090326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c840b0810 0x7f8c840b16f0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 >> 172.21.15.46:0/1139090326 conn(0x7f8c840b35f0 msgr2=0x7f8c84068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 shutdown_connections 2024-09-17T03:36:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.013+0000 7f8c88e00640 1 -- 172.21.15.46:0/1139090326 wait complete. 2024-09-17T03:36:37.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:37.032 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/bba461c5-b7bd-4b61-b187-7797ef01e7bb 2024-09-17T03:36:37.032 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:36:37.042 INFO:teuthology.orchestra.run.smithi046.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/bba461c5-b7bd-4b61-b187-7797ef01e7bb': No such file or directory 2024-09-17T03:36:37.043 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T03:36:37.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:36:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 -- 172.21.15.46:0/2919476759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 msgr2=0x7f31780b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 --2- 172.21.15.46:0/2919476759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f31780b3750 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3170009c80 tx=0x7f317001cda0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 -- 172.21.15.46:0/2919476759 shutdown_connections 2024-09-17T03:36:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 --2- 172.21.15.46:0/2919476759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f31780b3750 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 --2- 172.21.15.46:0/2919476759 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31780cefa0 0x7f31780cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 -- 172.21.15.46:0/2919476759 >> 172.21.15.46:0/2919476759 conn(0x7f31780b2040 msgr2=0x7f31780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:37.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 -- 172.21.15.46:0/2919476759 shutdown_connections 2024-09-17T03:36:37.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 -- 172.21.15.46:0/2919476759 wait complete. 2024-09-17T03:36:37.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 Processor -- start 2024-09-17T03:36:37.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.249+0000 7f3180000640 1 -- start start 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 0x7f31781fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f317813de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31780cefa0 0x7f31780aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3178169dc0 con 0x7f31780749a0 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3178169c40 con 0x7f31780b3340 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3178169f40 con 0x7f31780cefa0 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 0x7f31781fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 0x7f31781fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52314/0 (socket says 172.21.15.46:52314) 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 -- 172.21.15.46:0/1843164620 learned_addr learned my addr 172.21.15.46:0/1843164620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317cc00640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f317813de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317e000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31780cefa0 0x7f31780aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 -- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31780cefa0 msgr2=0x7f31780aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31780cefa0 0x7f31780aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 -- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 msgr2=0x7f317813de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f317813de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 -- 172.21.15.46:0/1843164620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3178160f00 con 0x7f31780749a0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317cc00640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f317813de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f317d600640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 0x7f31781fc110 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c3c0 tx=0x7f316c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c017070 con 0x7f31780749a0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31780b08d0 con 0x7f31780749a0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f316c0053c0 con 0x7f31780749a0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c003860 con 0x7f31780749a0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31780b0e90 con 0x7f31780749a0 2024-09-17T03:36:37.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.253+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31780b3750 con 0x7f31780749a0 2024-09-17T03:36:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.257+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7f316c01a070 con 0x7f31780749a0 2024-09-17T03:36:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.257+0000 7f3175600640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3158075510 0x7f3158077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.257+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f316c0927c0 con 0x7f31780749a0 2024-09-17T03:36:37.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.257+0000 7f317cc00640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3158075510 0x7f3158077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.257+0000 7f317cc00640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3158075510 0x7f3158077a20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f317813f010 tx=0x7f31640091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.261+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f316c05eb70 con 0x7f31780749a0 2024-09-17T03:36:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.457+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f317806b540 con 0x7f3158075510 2024-09-17T03:36:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3175600640 1 -- 172.21.15.46:0/1843164620 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3164004030 con 0x7f3158075510 2024-09-17T03:36:37.461 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:36:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:36:37.461 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:37.461 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3158075510 msgr2=0x7f3158077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3158075510 0x7f3158077a20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f317813f010 tx=0x7f31640091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 msgr2=0x7f31781fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 0x7f31781fc110 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c3c0 tx=0x7f316c00c880 comp rx=0 tx=0).stop 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 shutdown_connections 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3158075510 0x7f3158077a20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f31780cefa0 0x7f31780aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31780b3340 0x7f317813de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 --2- 172.21.15.46:0/1843164620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31780749a0 0x7f31781fc110 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 >> 172.21.15.46:0/1843164620 conn(0x7f31780b2040 msgr2=0x7f3178074db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 shutdown_connections 2024-09-17T03:36:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.461+0000 7f3180000640 1 -- 172.21.15.46:0/1843164620 wait complete. 2024-09-17T03:36:37.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:36:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 -- 172.21.15.46:0/2742047034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 msgr2=0x7ff8401606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 --2- 172.21.15.46:0/2742047034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8401606b0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff840160da0 tx=0x7ff82801c8a0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 -- 172.21.15.46:0/2742047034 shutdown_connections 2024-09-17T03:36:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 --2- 172.21.15.46:0/2742047034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8401606b0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 --2- 172.21.15.46:0/2742047034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 0x7ff84015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 --2- 172.21.15.46:0/2742047034 >> v2:172.21.15.99:3300/0 conn(0x7ff8400cd2e0 0x7ff840070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 -- 172.21.15.46:0/2742047034 >> 172.21.15.46:0/2742047034 conn(0x7ff8400b2c40 msgr2=0x7ff8400b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 -- 172.21.15.46:0/2742047034 shutdown_connections 2024-09-17T03:36:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.645+0000 7ff844a00640 1 -- 172.21.15.46:0/2742047034 wait complete. 2024-09-17T03:36:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 Processor -- start 2024-09-17T03:36:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 -- start start 2024-09-17T03:36:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8400cd2e0 0x7ff8401f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 0x7ff8400b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8400b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400cfef0 con 0x7ff8401602a0 2024-09-17T03:36:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400cfd70 con 0x7ff8400cd2e0 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400d0070 con 0x7ff84015cd20 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8400b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8400b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52336/0 (socket says 172.21.15.46:52336) 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 -- 172.21.15.46:0/2861928411 learned_addr learned my addr 172.21.15.46:0/2861928411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83de00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8400cd2e0 0x7ff8401f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff835400640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 0x7ff8400b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 -- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 msgr2=0x7ff8400b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 0x7ff8400b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 -- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8400cd2e0 msgr2=0x7ff8401f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8400cd2e0 0x7ff8401f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 -- 172.21.15.46:0/2861928411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8401f8e60 con 0x7ff8401602a0 2024-09-17T03:36:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff835400640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 0x7ff8400b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff83e800640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8400b5780 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff83800e710 tx=0x7ff83800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83800ca50 con 0x7ff8401602a0 2024-09-17T03:36:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff83800cbf0 con 0x7ff8401602a0 2024-09-17T03:36:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838005020 con 0x7ff8401602a0 2024-09-17T03:36:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff840144a90 con 0x7ff8401602a0 2024-09-17T03:36:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.649+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401f9120 con 0x7ff8401602a0 2024-09-17T03:36:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.653+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84015d130 con 0x7ff8401602a0 2024-09-17T03:36:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.653+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7ff8380053d0 con 0x7ff8401602a0 2024-09-17T03:36:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.653+0000 7ff836a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff81c0754e0 0x7ff81c0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.653+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7ff838093e50 con 0x7ff8401602a0 2024-09-17T03:36:37.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.653+0000 7ff83de00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff81c0754e0 0x7ff81c0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:37.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.653+0000 7ff83de00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff81c0754e0 0x7ff81c0779f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff830009eb0 tx=0x7ff830009500 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:37.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.657+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff838060960 con 0x7ff8401602a0 2024-09-17T03:36:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.861+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff84006b540 con 0x7ff81c0754e0 2024-09-17T03:36:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff836a00640 1 -- 172.21.15.46:0/2861928411 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7ff830009b00 con 0x7ff81c0754e0 2024-09-17T03:36:37.865 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:37.865 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:36:37.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:37.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:36:37.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:37.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:37.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1clone1" 2024-09-17T03:36:37.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:37.866 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff81c0754e0 msgr2=0x7ff81c0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff81c0754e0 0x7ff81c0779f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff830009eb0 tx=0x7ff830009500 comp rx=0 tx=0).stop 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 msgr2=0x7ff8400b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8400b5780 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff83800e710 tx=0x7ff83800ebd0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 shutdown_connections 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff81c0754e0 0x7ff81c0779f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8401602a0 0x7ff8400b5780 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff84015cd20 0x7ff8400b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.865+0000 7ff844a00640 1 --2- 172.21.15.46:0/2861928411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8400cd2e0 0x7ff8401f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.869+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 >> 172.21.15.46:0/2861928411 conn(0x7ff8400b2c40 msgr2=0x7ff84013f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.869+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 shutdown_connections 2024-09-17T03:36:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:37.869+0000 7ff844a00640 1 -- 172.21.15.46:0/2861928411 wait complete. 2024-09-17T03:36:37.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:36:38.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.049+0000 7fd737600640 1 -- 172.21.15.46:0/2079090748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7300b4ce0 msgr2=0x7fd7300b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.049+0000 7fd737600640 1 --2- 172.21.15.46:0/2079090748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7300b4ce0 0x7fd7300b50f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd72800acd0 tx=0x7fd72801de60 comp rx=0 tx=0).stop 2024-09-17T03:36:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.049+0000 7fd737600640 1 -- 172.21.15.46:0/2079090748 shutdown_connections 2024-09-17T03:36:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.049+0000 7fd737600640 1 --2- 172.21.15.46:0/2079090748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7300b4ce0 0x7fd7300b50f0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.049+0000 7fd737600640 1 -- 172.21.15.46:0/2079090748 >> 172.21.15.46:0/2079090748 conn(0x7fd7300b2040 msgr2=0x7fd7300b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.049+0000 7fd737600640 1 -- 172.21.15.46:0/2079090748 shutdown_connections 2024-09-17T03:36:38.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 -- 172.21.15.46:0/2079090748 wait complete. 2024-09-17T03:36:38.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 Processor -- start 2024-09-17T03:36:38.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 -- start start 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7300b4ce0 0x7fd73012f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7300cf900 0x7fd7301f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 0x7fd7301fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd730169010 con 0x7fd73013de10 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd730168e90 con 0x7fd7300cf900 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.053+0000 7fd737600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd730169190 con 0x7fd7300b4ce0 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 0x7fd7301fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 0x7fd7301fa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52360/0 (socket says 172.21.15.46:52360) 2024-09-17T03:36:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 -- 172.21.15.46:0/2584698638 learned_addr learned my addr 172.21.15.46:0/2584698638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd734c00640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7300b4ce0 0x7fd73012f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd727e00640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7300cf900 0x7fd7301f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 -- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7300b4ce0 msgr2=0x7fd73012f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7300b4ce0 0x7fd73012f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 -- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7300cf900 msgr2=0x7fd7301f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7300cf900 0x7fd7301f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 -- 172.21.15.46:0/2584698638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd730075250 con 0x7fd73013de10 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd727e00640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7300cf900 0x7fd7301f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd735600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 0x7fd7301fa1c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd7280a8fd0 tx=0x7fd7280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280b3040 con 0x7fd73013de10 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7280045d0 con 0x7fd73013de10 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728004a50 con 0x7fd73013de10 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd730075530 con 0x7fd73013de10 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd730073e90 con 0x7fd73013de10 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7300b50f0 con 0x7fd73013de10 2024-09-17T03:36:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.057+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7fd728004bf0 con 0x7fd73013de10 2024-09-17T03:36:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.061+0000 7fd72d600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd70c0754e0 0x7fd70c0779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.061+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fd72812d8a0 con 0x7fd73013de10 2024-09-17T03:36:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.061+0000 7fd734c00640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd70c0754e0 0x7fd70c0779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.061+0000 7fd734c00640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd70c0754e0 0x7fd70c0779f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd720007fd0 tx=0x7fd720009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.065+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd7280bd060 con 0x7fd73013de10 2024-09-17T03:36:38.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.269+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd730075000 con 0x7fd70c0754e0 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.281+0000 7fd72d600640 1 -- 172.21.15.46:0/2584698638 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd730075000 con 0x7fd70c0754e0 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:38.282 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:36:38.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd70c0754e0 msgr2=0x7fd70c0779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd70c0754e0 0x7fd70c0779f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd720007fd0 tx=0x7fd720009040 comp rx=0 tx=0).stop 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 msgr2=0x7fd7301fa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 0x7fd7301fa1c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd7280a8fd0 tx=0x7fd7280042f0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 shutdown_connections 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd70c0754e0 0x7fd70c0779f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd73013de10 0x7fd7301fa1c0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7300cf900 0x7fd7301f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 --2- 172.21.15.46:0/2584698638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd7300b4ce0 0x7fd73012f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 >> 172.21.15.46:0/2584698638 conn(0x7fd7300b2040 msgr2=0x7fd73015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 shutdown_connections 2024-09-17T03:36:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.285+0000 7fd737600640 1 -- 172.21.15.46:0/2584698638 wait complete. 2024-09-17T03:36:38.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:36:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 -- 172.21.15.46:0/1310076606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340cefa0 msgr2=0x7f5c340cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 --2- 172.21.15.46:0/1310076606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340cefa0 0x7f5c340cf3b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009840 tx=0x7f5c2401c6b0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 -- 172.21.15.46:0/1310076606 shutdown_connections 2024-09-17T03:36:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 --2- 172.21.15.46:0/1310076606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340cefa0 0x7f5c340cf3b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 -- 172.21.15.46:0/1310076606 >> 172.21.15.46:0/1310076606 conn(0x7f5c340b2040 msgr2=0x7f5c340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 -- 172.21.15.46:0/1310076606 shutdown_connections 2024-09-17T03:36:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 -- 172.21.15.46:0/1310076606 wait complete. 2024-09-17T03:36:38.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 Processor -- start 2024-09-17T03:36:38.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 -- start start 2024-09-17T03:36:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.473+0000 7f5c3c800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5c340749a0 0x7f5c3413e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 0x7f5c340ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c340cefa0 0x7f5c340aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c340b41f0 con 0x7f5c340b3340 2024-09-17T03:36:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c340b4070 con 0x7f5c340cefa0 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5c340b4370 con 0x7f5c340749a0 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 0x7f5c340ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 0x7f5c340ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52374/0 (socket says 172.21.15.46:52374) 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 -- 172.21.15.46:0/3624025431 learned_addr learned my addr 172.21.15.46:0/3624025431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39e00640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5c340749a0 0x7f5c3413e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3a800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c340cefa0 0x7f5c340aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 -- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5c340749a0 msgr2=0x7f5c3413e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5c340749a0 0x7f5c3413e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 -- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c340cefa0 msgr2=0x7f5c340aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c340cefa0 0x7f5c340aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 -- 172.21.15.46:0/3624025431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c34143510 con 0x7f5c340b3340 2024-09-17T03:36:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c39400640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 0x7f5c340ae8e0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f5c2401cb90 tx=0x7f5c24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240b2040 con 0x7f5c340b3340 2024-09-17T03:36:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c24004460 con 0x7f5c340b3340 2024-09-17T03:36:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3406ecb0 con 0x7f5c340b3340 2024-09-17T03:36:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003800 con 0x7f5c340b3340 2024-09-17T03:36:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c340af670 con 0x7f5c340b3340 2024-09-17T03:36:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c340cf3b0 con 0x7f5c340b3340 2024-09-17T03:36:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.477+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7f5c240bd030 con 0x7f5c340b3340 2024-09-17T03:36:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.481+0000 7f5c2e000640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5c180754e0 0x7f5c180779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.481+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f5c2412d210 con 0x7f5c340b3340 2024-09-17T03:36:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.481+0000 7f5c39e00640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5c180754e0 0x7f5c180779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.481+0000 7f5c39e00640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5c180754e0 0x7f5c180779f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000f70 tx=0x7f5c28009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:38.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.485+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5c240c4060 con 0x7f5c340b3340 2024-09-17T03:36:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.689+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c3406b540 con 0x7f5c180754e0 2024-09-17T03:36:38.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c2e000640 1 -- 172.21.15.46:0/3624025431 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c280012b0 con 0x7f5c180754e0 2024-09-17T03:36:38.720 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5c180754e0 msgr2=0x7f5c180779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5c180754e0 0x7f5c180779f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000f70 tx=0x7f5c28009040 comp rx=0 tx=0).stop 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 msgr2=0x7f5c340ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 0x7f5c340ae8e0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f5c2401cb90 tx=0x7f5c24004270 comp rx=0 tx=0).stop 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 shutdown_connections 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5c180754e0 0x7f5c180779f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c340cefa0 0x7f5c340aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c340b3340 0x7f5c340ae8e0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 --2- 172.21.15.46:0/3624025431 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5c340749a0 0x7f5c3413e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 >> 172.21.15.46:0/3624025431 conn(0x7f5c340b2040 msgr2=0x7f5c340b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 shutdown_connections 2024-09-17T03:36:38.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.721+0000 7f5c3c800640 1 -- 172.21.15.46:0/3624025431 wait complete. 2024-09-17T03:36:38.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:36:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 -- 172.21.15.46:0/2520471697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b4de0 msgr2=0x7f03c80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 --2- 172.21.15.46:0/2520471697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b4de0 0x7f03c80b51f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f03c8071cb0 tx=0x7f03c001cce0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 -- 172.21.15.46:0/2520471697 shutdown_connections 2024-09-17T03:36:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 --2- 172.21.15.46:0/2520471697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b4de0 0x7f03c80b51f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 --2- 172.21.15.46:0/2520471697 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03c81527b0 0x7f03c8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 --2- 172.21.15.46:0/2520471697 >> v2:172.21.15.99:3300/0 conn(0x7f03c80b5c70 0x7f03c815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 -- 172.21.15.46:0/2520471697 >> 172.21.15.46:0/2520471697 conn(0x7f03c80b2040 msgr2=0x7f03c80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 -- 172.21.15.46:0/2520471697 shutdown_connections 2024-09-17T03:36:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 -- 172.21.15.46:0/2520471697 wait complete. 2024-09-17T03:36:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 Processor -- start 2024-09-17T03:36:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.905+0000 7f03cea00640 1 -- start start 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80b4de0 0x7f03c8144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 0x7f03c80b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03c81527b0 0x7f03c80b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03c815f730 con 0x7f03c80b5c70 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03c815f5b0 con 0x7f03c80b4de0 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f03c815f8b0 con 0x7f03c81527b0 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 0x7f03c80b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 0x7f03c80b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52388/0 (socket says 172.21.15.46:52388) 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 -- 172.21.15.46:0/2398036267 learned_addr learned my addr 172.21.15.46:0/2398036267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03c7e00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80b4de0 0x7f03c8144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cca00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03c81527b0 0x7f03c80b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 -- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03c81527b0 msgr2=0x7f03c80b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03c81527b0 0x7f03c80b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 -- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80b4de0 msgr2=0x7f03c8144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80b4de0 0x7f03c8144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 -- 172.21.15.46:0/2398036267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c81fbcf0 con 0x7f03c80b5c70 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03bfe00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 0x7f03c80b0b80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b3c0 tx=0x7f03b000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0011830 con 0x7f03c80b5c70 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03b0016070 con 0x7f03c80b5c70 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0011d60 con 0x7f03c80b5c70 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c81fbfd0 con 0x7f03c80b5c70 2024-09-17T03:36:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c81fc590 con 0x7f03c80b5c70 2024-09-17T03:36:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.909+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c815fdd0 con 0x7f03c80b5c70 2024-09-17T03:36:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.913+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7f03b001b460 con 0x7f03c80b5c70 2024-09-17T03:36:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.913+0000 7f03c4c00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f03a40754e0 0x7f03a40779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.913+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f03b0092f80 con 0x7f03c80b5c70 2024-09-17T03:36:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.913+0000 7f03c7e00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f03a40754e0 0x7f03a40779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:38.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.913+0000 7f03c7e00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f03a40754e0 0x7f03a40779f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f03b800ab40 tx=0x7f03b800a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:38.917+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f03b005f3b0 con 0x7f03c80b5c70 2024-09-17T03:36:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.125+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f03c806b540 con 0x7f03a40754e0 2024-09-17T03:36:39.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03c4c00640 1 -- 172.21.15.46:0/2398036267 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f03b800fb60 con 0x7f03a40754e0 2024-09-17T03:36:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f03a40754e0 msgr2=0x7f03a40779f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f03a40754e0 0x7f03a40779f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f03b800ab40 tx=0x7f03b800a250 comp rx=0 tx=0).stop 2024-09-17T03:36:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 msgr2=0x7f03c80b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 0x7f03c80b0b80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b3c0 tx=0x7f03b000b880 comp rx=0 tx=0).stop 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 shutdown_connections 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f03a40754e0 0x7f03a40779f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03c81527b0 0x7f03c80b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c80b5c70 0x7f03c80b0b80 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 --2- 172.21.15.46:0/2398036267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80b4de0 0x7f03c8144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 >> 172.21.15.46:0/2398036267 conn(0x7f03c80b2040 msgr2=0x7f03c8159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 shutdown_connections 2024-09-17T03:36:39.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.141+0000 7f03cea00640 1 -- 172.21.15.46:0/2398036267 wait complete. 2024-09-17T03:36:39.155 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:39.153+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:39.155 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:39.153+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:39.155 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:39.153+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:39.155 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:39.153+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:39.155 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:36:39.153+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:36:39.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T03:36:39.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 -- 172.21.15.46:0/2187928773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160060 msgr2=0x7f53d8074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 --2- 172.21.15.46:0/2187928773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160060 0x7f53d8074240 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f53d8073cf0 tx=0x7f53c001c650 comp rx=0 tx=0).stop 2024-09-17T03:36:39.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 -- 172.21.15.46:0/2187928773 shutdown_connections 2024-09-17T03:36:39.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 --2- 172.21.15.46:0/2187928773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160060 0x7f53d8074240 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 --2- 172.21.15.46:0/2187928773 >> v2:172.21.15.99:3300/0 conn(0x7f53d81536f0 0x7f53d815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 -- 172.21.15.46:0/2187928773 >> 172.21.15.46:0/2187928773 conn(0x7f53d80b4560 msgr2=0x7f53d80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:39.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 -- 172.21.15.46:0/2187928773 shutdown_connections 2024-09-17T03:36:39.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.333+0000 7f53d7200640 1 -- 172.21.15.46:0/2187928773 wait complete. 2024-09-17T03:36:39.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 Processor -- start 2024-09-17T03:36:39.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 -- start start 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53d81536f0 0x7f53d81562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8160060 0x7f53d8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 0x7f53d8159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f53d816a3f0 con 0x7f53d8160470 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53d816a270 con 0x7f53d8160060 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d7200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f53d816a570 con 0x7f53d81536f0 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 0x7f53d8159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 0x7f53d8159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52428/0 (socket says 172.21.15.46:52428) 2024-09-17T03:36:39.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d6800640 1 -- 172.21.15.46:0/2567601783 learned_addr learned my addr 172.21.15.46:0/2567601783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53cd400640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8160060 0x7f53d8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.337+0000 7f53d5e00640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53d81536f0 0x7f53d81562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d6800640 1 -- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53d81536f0 msgr2=0x7f53d81562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d6800640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53d81536f0 0x7f53d81562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d6800640 1 -- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8160060 msgr2=0x7f53d8155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d6800640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8160060 0x7f53d8155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d6800640 1 -- 172.21.15.46:0/2567601783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d81f7420 con 0x7f53d8160470 2024-09-17T03:36:39.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d6800640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 0x7f53d8159d90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b970 tx=0x7f53d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d5e00640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53d81536f0 0x7f53d81562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53cd400640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8160060 0x7f53d8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0019070 con 0x7f53d8160470 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d000cc30 con 0x7f53d8160470 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d81f7700 con 0x7f53d8160470 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00146a0 con 0x7f53d8160470 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d81f7cc0 con 0x7f53d8160470 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d815d480 con 0x7f53d8160470 2024-09-17T03:36:39.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.341+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) ==== 100396+0+0 (secure 0 0 0) 0x7f53d001c070 con 0x7f53d8160470 2024-09-17T03:36:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.345+0000 7f53cea00640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53b4075510 0x7f53b4077a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.345+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f53d00933a0 con 0x7f53d8160470 2024-09-17T03:36:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.345+0000 7f53d5e00640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53b4075510 0x7f53b4077a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:39.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.345+0000 7f53d5e00640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53b4075510 0x7f53b4077a20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c8005dc0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:39.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.349+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53d000ce20 con 0x7f53d8160470 2024-09-17T03:36:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.569+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f53d81f71d0 con 0x7f53b4075510 2024-09-17T03:36:39.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.873+0000 7f53cea00640 1 -- 172.21.15.46:0/2567601783 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53d81f71d0 con 0x7f53b4075510 2024-09-17T03:36:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53b4075510 msgr2=0x7f53b4077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53b4075510 0x7f53b4077a20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c8005dc0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 msgr2=0x7f53d8159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 0x7f53d8159d90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b970 tx=0x7f53d000be30 comp rx=0 tx=0).stop 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 shutdown_connections 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f53b4075510 0x7f53b4077a20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d8160470 0x7f53d8159d90 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d8160060 0x7f53d8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 --2- 172.21.15.46:0/2567601783 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f53d81536f0 0x7f53d81562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 >> 172.21.15.46:0/2567601783 conn(0x7f53d80b4560 msgr2=0x7f53d80d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 shutdown_connections 2024-09-17T03:36:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39.877+0000 7f53d7200640 1 -- 172.21.15.46:0/2567601783 wait complete. 2024-09-17T03:36:39.900 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:36:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:36:39.956 INFO:teuthology.orchestra.run.smithi046.stdout:3 2024-09-17T03:36:39.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:36:44.965 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:36:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:36:45.024 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:36:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:36:45.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:36:45.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 -- 172.21.15.46:0/277345031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25740b4de0 msgr2=0x7f25740b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 --2- 172.21.15.46:0/277345031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25740b4de0 0x7f25740b51f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2560009900 tx=0x7f256001c8e0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 -- 172.21.15.46:0/277345031 shutdown_connections 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 --2- 172.21.15.46:0/277345031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25740b4de0 0x7f25740b51f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 --2- 172.21.15.46:0/277345031 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 0x7f2574155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 -- 172.21.15.46:0/277345031 >> 172.21.15.46:0/277345031 conn(0x7f25740b2040 msgr2=0x7f25740b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 -- 172.21.15.46:0/277345031 shutdown_connections 2024-09-17T03:36:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.197+0000 7f257c600640 1 -- 172.21.15.46:0/277345031 wait complete. 2024-09-17T03:36:45.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 Processor -- start 2024-09-17T03:36:45.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 -- start start 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 0x7f25741fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25740b4de0 0x7f25740b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257415f0e0 0x7f257415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2574169420 con 0x7f257415f0e0 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25741692a0 con 0x7f25740b4de0 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257c600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f25741695a0 con 0x7f2574072100 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257415f0e0 0x7f257415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f2579c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 0x7f25741fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f2579200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25740b4de0 0x7f25740b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f2579c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 0x7f25741fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:38526/0 (socket says 172.21.15.46:38526) 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f2579c00640 1 -- 172.21.15.46:0/2665307494 learned_addr learned my addr 172.21.15.46:0/2665307494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 -- 172.21.15.46:0/2665307494 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 msgr2=0x7f25741fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 0x7f25741fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 -- 172.21.15.46:0/2665307494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25740b4de0 msgr2=0x7f25740b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25740b4de0 0x7f25740b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 -- 172.21.15.46:0/2665307494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25740b1260 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.201+0000 7f257a600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257415f0e0 0x7f257415a070 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f25740731f0 tx=0x7f256c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c003b80 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f256c003d20 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c005330 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25741592b0 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2574159870 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257415b3a0 con 0x7f257415f0e0 2024-09-17T03:36:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f256c0054d0 con 0x7f257415f0e0 2024-09-17T03:36:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2571600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2554075570 0x7f2554077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f256c092f10 con 0x7f257415f0e0 2024-09-17T03:36:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.205+0000 7f2579c00640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2554075570 0x7f2554077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.209+0000 7f2579c00640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2554075570 0x7f2554077a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2568005fd0 tx=0x7f256800b410 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.209+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f25740b5730 con 0x7f257415f0e0 2024-09-17T03:36:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.385+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f2574155b50 con 0x7f257415f0e0 2024-09-17T03:36:45.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.385+0000 7f2571600640 1 -- 172.21.15.46:0/2665307494 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7f256c05f470 con 0x7f257415f0e0 2024-09-17T03:36:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2554075570 msgr2=0x7f2554077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2554075570 0x7f2554077a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2568005fd0 tx=0x7f256800b410 comp rx=0 tx=0).stop 2024-09-17T03:36:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257415f0e0 msgr2=0x7f257415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257415f0e0 0x7f257415a070 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f25740731f0 tx=0x7f256c00cae0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 shutdown_connections 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2554075570 0x7f2554077a80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257415f0e0 0x7f257415a070 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25740b4de0 0x7f25740b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 --2- 172.21.15.46:0/2665307494 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2574072100 0x7f25741fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 >> 172.21.15.46:0/2665307494 conn(0x7f25740b2040 msgr2=0x7f25741545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 shutdown_connections 2024-09-17T03:36:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.389+0000 7f257c600640 1 -- 172.21.15.46:0/2665307494 wait complete. 2024-09-17T03:36:45.410 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:36:45.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:36:45.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:36:45.598 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:36:45.598 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:36:45.599 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:45.599 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:45.607 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:45.608 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:45.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 -- 172.21.15.46:0/1308703995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 msgr2=0x7ff29415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 --2- 172.21.15.46:0/1308703995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff29415bb60 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff294074b20 tx=0x7ff28801c760 comp rx=0 tx=0).stop 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 -- 172.21.15.46:0/1308703995 shutdown_connections 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 --2- 172.21.15.46:0/1308703995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff29415bb60 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 -- 172.21.15.46:0/1308703995 >> 172.21.15.46:0/1308703995 conn(0x7ff2940b2040 msgr2=0x7ff2940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 -- 172.21.15.46:0/1308703995 shutdown_connections 2024-09-17T03:36:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 -- 172.21.15.46:0/1308703995 wait complete. 2024-09-17T03:36:45.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 Processor -- start 2024-09-17T03:36:45.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.869+0000 7ff29c400640 1 -- start start 2024-09-17T03:36:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2940b4de0 0x7ff294154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff294140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2941527b0 0x7ff29413d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff294169bd0 con 0x7ff2940b5c70 2024-09-17T03:36:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff294169a50 con 0x7ff2941527b0 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff294169d50 con 0x7ff2940b4de0 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff294140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff294140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42940/0 (socket says 172.21.15.46:42940) 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 -- 172.21.15.46:0/523661598 learned_addr learned my addr 172.21.15.46:0/523661598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299a00640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2940b4de0 0x7ff294154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29a400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2941527b0 0x7ff29413d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 -- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2940b4de0 msgr2=0x7ff294154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2940b4de0 0x7ff294154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 -- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2941527b0 msgr2=0x7ff29413d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2941527b0 0x7ff29413d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 -- 172.21.15.46:0/523661598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff294154ed0 con 0x7ff2940b5c70 2024-09-17T03:36:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff299000640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff294140b30 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c450 tx=0x7ff28000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800052d0 con 0x7ff2940b5c70 2024-09-17T03:36:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff280005470 con 0x7ff2940b5c70 2024-09-17T03:36:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2941551b0 con 0x7ff2940b5c70 2024-09-17T03:36:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280012870 con 0x7ff2940b5c70 2024-09-17T03:36:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29413e0f0 con 0x7ff2940b5c70 2024-09-17T03:36:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.873+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29413dee0 con 0x7ff2940b5c70 2024-09-17T03:36:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.877+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7ff280003600 con 0x7ff2940b5c70 2024-09-17T03:36:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.877+0000 7ff291600640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff274075570 0x7ff274077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.877+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7ff280092d80 con 0x7ff2940b5c70 2024-09-17T03:36:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.877+0000 7ff299a00640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff274075570 0x7ff274077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.877+0000 7ff299a00640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff274075570 0x7ff274077a80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff2880af000 tx=0x7ff28801afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:45.881+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff28005f240 con 0x7ff2940b5c70 2024-09-17T03:36:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.065+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff29415bb60 con 0x7ff2940b5c70 2024-09-17T03:36:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff291600640 1 -- 172.21.15.46:0/523661598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7ff280063c70 con 0x7ff2940b5c70 2024-09-17T03:36:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff274075570 msgr2=0x7ff274077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff274075570 0x7ff274077a80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff2880af000 tx=0x7ff28801afa0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 msgr2=0x7ff294140b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff294140b30 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c450 tx=0x7ff28000c910 comp rx=0 tx=0).stop 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 shutdown_connections 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff274075570 0x7ff274077a80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2941527b0 0x7ff29413d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2940b5c70 0x7ff294140b30 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 --2- 172.21.15.46:0/523661598 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff2940b4de0 0x7ff294154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 >> 172.21.15.46:0/523661598 conn(0x7ff2940b2040 msgr2=0x7ff2940ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 shutdown_connections 2024-09-17T03:36:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.069+0000 7ff29c400640 1 -- 172.21.15.46:0/523661598 wait complete. 2024-09-17T03:36:46.090 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:36:46.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:46.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:36:46.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:36:46.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:36:46.152 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:36:46.170 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:36:46.170 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:36:46.170 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:46.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:36:46.181 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:36:46.181 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:46.181 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:46.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:36:46.220 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:36:46.220 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:36:46.220 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:46.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:36:46.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:36:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 -- 172.21.15.46:0/1159019711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00749a0 msgr2=0x7f90d0074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 --2- 172.21.15.46:0/1159019711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00749a0 0x7f90d0074db0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009840 tx=0x7f90c401c820 comp rx=0 tx=0).stop 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 -- 172.21.15.46:0/1159019711 shutdown_connections 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 --2- 172.21.15.46:0/1159019711 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00b3340 0x7f90d00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 --2- 172.21.15.46:0/1159019711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00749a0 0x7f90d0074db0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 -- 172.21.15.46:0/1159019711 >> 172.21.15.46:0/1159019711 conn(0x7f90d00b2040 msgr2=0x7f90d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 -- 172.21.15.46:0/1159019711 shutdown_connections 2024-09-17T03:36:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.421+0000 7f90d7800640 1 -- 172.21.15.46:0/1159019711 wait complete. 2024-09-17T03:36:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 Processor -- start 2024-09-17T03:36:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 -- start start 2024-09-17T03:36:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 0x7f90d01fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00b3340 0x7f90d013fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00cefa0 0x7f90d00b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90d0169e10 con 0x7f90d00749a0 2024-09-17T03:36:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90d0169c90 con 0x7f90d00b3340 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d7800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f90d0169f90 con 0x7f90d00cefa0 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 0x7f90d01fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 0x7f90d01fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42946/0 (socket says 172.21.15.46:42946) 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.425+0000 7f90d4e00640 1 -- 172.21.15.46:0/6342638 learned_addr learned my addr 172.21.15.46:0/6342638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90cfe00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00b3340 0x7f90d013fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d5800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00cefa0 0x7f90d00b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d4e00640 1 -- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00cefa0 msgr2=0x7f90d00b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d4e00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00cefa0 0x7f90d00b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d4e00640 1 -- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00b3340 msgr2=0x7f90d013fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d4e00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00b3340 0x7f90d013fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d4e00640 1 -- 172.21.15.46:0/6342638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d015e1c0 con 0x7f90d00749a0 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90cfe00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00b3340 0x7f90d013fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d5800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00cefa0 0x7f90d00b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d4e00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 0x7f90d01fcb20 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009790 tx=0x7f90c4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40b2040 con 0x7f90d00749a0 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90c40043e0 con 0x7f90d00749a0 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d015e4a0 con 0x7f90d00749a0 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40052d0 con 0x7f90d00749a0 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d015ea60 con 0x7f90d00749a0 2024-09-17T03:36:46.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d0074db0 con 0x7f90d00749a0 2024-09-17T03:36:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.429+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f90c4005470 con 0x7f90d00749a0 2024-09-17T03:36:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.433+0000 7f90ccc00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90b00755a0 0x7f90b0077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.433+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f90c412d0e0 con 0x7f90d00749a0 2024-09-17T03:36:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.433+0000 7f90cfe00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90b00755a0 0x7f90b0077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.433+0000 7f90cfe00640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90b00755a0 0x7f90b0077ab0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f90d01614a0 tx=0x7f90bc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.437+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f90c40bd060 con 0x7f90d00749a0 2024-09-17T03:36:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90d015a6e0 con 0x7f90d00749a0 2024-09-17T03:36:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90ccc00640 1 -- 172.21.15.46:0/6342638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5241 (secure 0 0 0) 0x7f90c4016410 con 0x7f90d00749a0 2024-09-17T03:36:46.645 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:46.645 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"btime":"2024-09-17T03:35:58:996234+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:35:45.367356+0000","modified":"2024-09-17T03:35:48.376364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248335":{"gid":248335,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2479724490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2479724490},{"type":"v1","addr":"172.21.15.46:6836","nonce":2479724490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248335,"qdb_cluster":[248335]},"id":9}]} 2024-09-17T03:36:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-09-17T03:36:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90b00755a0 msgr2=0x7f90b0077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90b00755a0 0x7f90b0077ab0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f90d01614a0 tx=0x7f90bc009040 comp rx=0 tx=0).stop 2024-09-17T03:36:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 msgr2=0x7f90d01fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 0x7f90d01fcb20 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009790 tx=0x7f90c4005d90 comp rx=0 tx=0).stop 2024-09-17T03:36:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 shutdown_connections 2024-09-17T03:36:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f90b00755a0 0x7f90b0077ab0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f90d00cefa0 0x7f90d00b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90d00b3340 0x7f90d013fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 --2- 172.21.15.46:0/6342638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90d00749a0 0x7f90d01fcb20 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 >> 172.21.15.46:0/6342638 conn(0x7f90d00b2040 msgr2=0x7f90d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:46.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.645+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 shutdown_connections 2024-09-17T03:36:46.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.649+0000 7f90d7800640 1 -- 172.21.15.46:0/6342638 wait complete. 2024-09-17T03:36:46.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 -- 172.21.15.46:0/689033108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 msgr2=0x7f72340b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 --2- 172.21.15.46:0/689033108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340b0730 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7234072250 tx=0x7f722401cb10 comp rx=0 tx=0).stop 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 -- 172.21.15.46:0/689033108 shutdown_connections 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 --2- 172.21.15.46:0/689033108 >> v2:172.21.15.99:3300/0 conn(0x7f723406dbc0 0x7f723406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 --2- 172.21.15.46:0/689033108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340b0730 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 -- 172.21.15.46:0/689033108 >> 172.21.15.46:0/689033108 conn(0x7f72340b41f0 msgr2=0x7f72340b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 -- 172.21.15.46:0/689033108 shutdown_connections 2024-09-17T03:36:46.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.825+0000 7f7233e00640 1 -- 172.21.15.46:0/689033108 wait complete. 2024-09-17T03:36:46.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 Processor -- start 2024-09-17T03:36:46.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 -- start start 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f723406dbc0 0x7f723415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f723415f5c0 0x7f72340cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7234168c60 con 0x7f72340d1140 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7234160230 con 0x7f723415f5c0 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7234168de0 con 0x7f723406dbc0 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42964/0 (socket says 172.21.15.46:42964) 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7232a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f723406dbc0 0x7f723415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 -- 172.21.15.46:0/4260179616 learned_addr learned my addr 172.21.15.46:0/4260179616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:46.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f7233400640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f723415f5c0 0x7f72340cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 -- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f723406dbc0 msgr2=0x7f723415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f723406dbc0 0x7f723415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 -- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f723415f5c0 msgr2=0x7f72340cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f723415f5c0 0x7f72340cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.829+0000 7f722a000640 1 -- 172.21.15.46:0/4260179616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340ce3c0 con 0x7f72340d1140 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f722a000640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340cf950 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f721c00d720 tx=0x7f721c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c017070 con 0x7f72340d1140 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f721c009d20 con 0x7f72340d1140 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341f9ab0 con 0x7f72340d1140 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c005280 con 0x7f72340d1140 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72341fa070 con 0x7f72340d1140 2024-09-17T03:36:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72340ce130 con 0x7f72340d1140 2024-09-17T03:36:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f721c005420 con 0x7f72340d1140 2024-09-17T03:36:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.833+0000 7f722b400640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7210075570 0x7f7210077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.837+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f721c093480 con 0x7f72340d1140 2024-09-17T03:36:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.837+0000 7f7232a00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7210075570 0x7f7210077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:46.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.837+0000 7f7232a00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7210075570 0x7f7210077a80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f72240af000 tx=0x7f722401b350 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:46.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:46.841+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f721c05f940 con 0x7f72340d1140 2024-09-17T03:36:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.049+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7234069a50 con 0x7f72340d1140 2024-09-17T03:36:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.049+0000 7f722b400640 1 -- 172.21.15.46:0/4260179616 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5241 (secure 0 0 0) 0x7f721c064370 con 0x7f72340d1140 2024-09-17T03:36:47.050 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:47.050 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"btime":"2024-09-17T03:35:58:996234+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:35:45.367356+0000","modified":"2024-09-17T03:35:48.376364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248335":{"gid":248335,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2479724490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2479724490},{"type":"v1","addr":"172.21.15.46:6836","nonce":2479724490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248335,"qdb_cluster":[248335]},"id":9}]} 2024-09-17T03:36:47.050 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-09-17T03:36:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7210075570 msgr2=0x7f7210077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7210075570 0x7f7210077a80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f72240af000 tx=0x7f722401b350 comp rx=0 tx=0).stop 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 msgr2=0x7f72340cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340cf950 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f721c00d720 tx=0x7f721c00dbe0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 shutdown_connections 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7210075570 0x7f7210077a80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f723415f5c0 0x7f72340cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72340d1140 0x7f72340cf950 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 --2- 172.21.15.46:0/4260179616 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f723406dbc0 0x7f723415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 >> 172.21.15.46:0/4260179616 conn(0x7f72340b41f0 msgr2=0x7f723406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 shutdown_connections 2024-09-17T03:36:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.053+0000 7f7233e00640 1 -- 172.21.15.46:0/4260179616 wait complete. 2024-09-17T03:36:47.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 -- 172.21.15.46:0/3263313738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2328004da0 msgr2=0x7f2328005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 --2- 172.21.15.46:0/3263313738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2328004da0 0x7f2328005230 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f233c002120 tx=0x7f233c01f390 comp rx=0 tx=0).stop 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2343400640 1 -- 172.21.15.46:0/3263313738 reap_dead start 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 -- 172.21.15.46:0/3263313738 shutdown_connections 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 -- 172.21.15.46:0/3263313738 >> 172.21.15.46:0/3263313738 conn(0x7f23440b2c40 msgr2=0x7f23440b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 -- 172.21.15.46:0/3263313738 shutdown_connections 2024-09-17T03:36:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 -- 172.21.15.46:0/3263313738 wait complete. 2024-09-17T03:36:47.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 Processor -- start 2024-09-17T03:36:47.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.233+0000 7f2349c00640 1 -- start start 2024-09-17T03:36:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2328004da0 0x7f23440afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 0x7f23441f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23441f9030 0x7f2344072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2344074270 con 0x7f2344149070 2024-09-17T03:36:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23440740f0 con 0x7f23441f9030 2024-09-17T03:36:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f234406eed0 con 0x7f2328004da0 2024-09-17T03:36:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 0x7f23441f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 0x7f23441f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42984/0 (socket says 172.21.15.46:42984) 2024-09-17T03:36:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 -- 172.21.15.46:0/2867498608 learned_addr learned my addr 172.21.15.46:0/2867498608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2342a00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2328004da0 0x7f23440afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2343400640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23441f9030 0x7f2344072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 -- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2328004da0 msgr2=0x7f23440afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2328004da0 0x7f23440afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 -- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23441f9030 msgr2=0x7f2344072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23441f9030 0x7f2344072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 -- 172.21.15.46:0/2867498608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23441f97e0 con 0x7f2344149070 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2342a00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2328004da0 0x7f23440afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2343400640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23441f9030 0x7f2344072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233be00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 0x7f23441f8af0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f232c006f40 tx=0x7f232c003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c015070 con 0x7f2344149070 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f232c00f9e0 con 0x7f2344149070 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c00fe30 con 0x7f2344149070 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2344148590 con 0x7f2344149070 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2344148b50 con 0x7f2344149070 2024-09-17T03:36:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.237+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2344069880 con 0x7f2344149070 2024-09-17T03:36:47.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.241+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f232c011430 con 0x7f2344149070 2024-09-17T03:36:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.241+0000 7f233aa00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2320075570 0x7f2320077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.241+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f232c0942d0 con 0x7f2344149070 2024-09-17T03:36:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.241+0000 7f2342a00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2320075570 0x7f2320077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.241+0000 7f2342a00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2320075570 0x7f2320077a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f23440b5720 tx=0x7f23340091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.245+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23440b4b90 con 0x7f2344149070 2024-09-17T03:36:47.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.421+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23441f9570 con 0x7f2344149070 2024-09-17T03:36:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f233aa00640 1 -- 172.21.15.46:0/2867498608 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16859 (secure 0 0 0) 0x7f232c060d60 con 0x7f2344149070 2024-09-17T03:36:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:47.424 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":130,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:36:42.382562+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:35:43.106524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T03:35:44.323553+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2320075570 msgr2=0x7f2320077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2320075570 0x7f2320077a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f23440b5720 tx=0x7f23340091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 msgr2=0x7f23441f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 0x7f23441f8af0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f232c006f40 tx=0x7f232c003a90 comp rx=0 tx=0).stop 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 shutdown_connections 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2320075570 0x7f2320077a80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23441f9030 0x7f2344072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2344149070 0x7f23441f8af0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 --2- 172.21.15.46:0/2867498608 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2328004da0 0x7f23440afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 >> 172.21.15.46:0/2867498608 conn(0x7f23440b2c40 msgr2=0x7f23440b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 shutdown_connections 2024-09-17T03:36:47.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.425+0000 7f2349c00640 1 -- 172.21.15.46:0/2867498608 wait complete. 2024-09-17T03:36:47.446 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:36:47.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 -- 172.21.15.46:0/1998032034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de8160330 msgr2=0x7f8de8160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 --2- 172.21.15.46:0/1998032034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de8160330 0x7f8de8160740 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8de80703e0 tx=0x7f8de001cd80 comp rx=0 tx=0).stop 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 -- 172.21.15.46:0/1998032034 shutdown_connections 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 --2- 172.21.15.46:0/1998032034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de8160330 0x7f8de8160740 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 --2- 172.21.15.46:0/1998032034 >> v2:172.21.15.99:3300/0 conn(0x7f8de80cefa0 0x7f8de80cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 --2- 172.21.15.46:0/1998032034 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de80749a0 0x7f8de8074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 -- 172.21.15.46:0/1998032034 >> 172.21.15.46:0/1998032034 conn(0x7f8de80b2040 msgr2=0x7f8de80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 -- 172.21.15.46:0/1998032034 shutdown_connections 2024-09-17T03:36:47.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.605+0000 7f8dee000640 1 -- 172.21.15.46:0/1998032034 wait complete. 2024-09-17T03:36:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 Processor -- start 2024-09-17T03:36:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 -- start start 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 0x7f8de812db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80cefa0 0x7f8de81fd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de8160330 0x7f8de815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80b3ee0 con 0x7f8de80749a0 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80b3d60 con 0x7f8de80cefa0 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8de80b4060 con 0x7f8de8160330 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 0x7f8de812db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 0x7f8de812db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42990/0 (socket says 172.21.15.46:42990) 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 -- 172.21.15.46:0/3905400749 learned_addr learned my addr 172.21.15.46:0/3905400749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8ddfe00640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80cefa0 0x7f8de81fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7e00640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de8160330 0x7f8de815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 -- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de8160330 msgr2=0x7f8de815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de8160330 0x7f8de815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 -- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80cefa0 msgr2=0x7f8de81fd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80cefa0 0x7f8de81fd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 -- 172.21.15.46:0/3905400749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de8146140 con 0x7f8de80749a0 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8ddfe00640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80cefa0 0x7f8de81fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8de7400640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 0x7f8de812db00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800d700 tx=0x7f8dd800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8017070 con 0x7f8de80749a0 2024-09-17T03:36:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dd8009b10 con 0x7f8de80749a0 2024-09-17T03:36:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de8145b80 con 0x7f8de80749a0 2024-09-17T03:36:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8010460 con 0x7f8de80749a0 2024-09-17T03:36:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.609+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de81fc810 con 0x7f8de80749a0 2024-09-17T03:36:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.613+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de80cf3b0 con 0x7f8de80749a0 2024-09-17T03:36:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.613+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f8dd801a070 con 0x7f8de80749a0 2024-09-17T03:36:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.613+0000 7f8ddf400640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc40755a0 0x7f8dc4077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.613+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f8dd80934f0 con 0x7f8de80749a0 2024-09-17T03:36:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.613+0000 7f8ddfe00640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc40755a0 0x7f8dc4077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.613+0000 7f8ddfe00640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc40755a0 0x7f8dc4077ab0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005fd0 tx=0x7f8dd0005f20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.617+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8dd8060000 con 0x7f8de80749a0 2024-09-17T03:36:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.829+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8de80736e0 con 0x7f8de80749a0 2024-09-17T03:36:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.829+0000 7f8ddf400640 1 -- 172.21.15.46:0/3905400749 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5241 (secure 0 0 0) 0x7f8dd805f900 con 0x7f8de80749a0 2024-09-17T03:36:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:47.830 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"btime":"2024-09-17T03:35:58:996234+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:35:45.367356+0000","modified":"2024-09-17T03:35:48.376364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248335":{"gid":248335,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2479724490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2479724490},{"type":"v1","addr":"172.21.15.46:6836","nonce":2479724490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248335,"qdb_cluster":[248335]},"id":9}]} 2024-09-17T03:36:47.830 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc40755a0 msgr2=0x7f8dc4077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc40755a0 0x7f8dc4077ab0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005fd0 tx=0x7f8dd0005f20 comp rx=0 tx=0).stop 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 msgr2=0x7f8de812db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 0x7f8de812db00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800d700 tx=0x7f8dd800dbc0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 shutdown_connections 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8dc40755a0 0x7f8dc4077ab0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8de8160330 0x7f8de815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8de80cefa0 0x7f8de81fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 --2- 172.21.15.46:0/3905400749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de80749a0 0x7f8de812db00 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:47.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 >> 172.21.15.46:0/3905400749 conn(0x7f8de80b2040 msgr2=0x7f8de81415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:47.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 shutdown_connections 2024-09-17T03:36:47.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:47.833+0000 7f8dee000640 1 -- 172.21.15.46:0/3905400749 wait complete. 2024-09-17T03:36:47.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 -- 172.21.15.46:0/1628148470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c1527b0 msgr2=0x7f1e5c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 --2- 172.21.15.46:0/1628148470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c074e30 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c071cb0 tx=0x7f1e4801c740 comp rx=0 tx=0).stop 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 -- 172.21.15.46:0/1628148470 shutdown_connections 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 --2- 172.21.15.46:0/1628148470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c074e30 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 --2- 172.21.15.46:0/1628148470 >> v2:172.21.15.99:3300/0 conn(0x7f1e5c0b5c70 0x7f1e5c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 -- 172.21.15.46:0/1628148470 >> 172.21.15.46:0/1628148470 conn(0x7f1e5c0b2040 msgr2=0x7f1e5c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 -- 172.21.15.46:0/1628148470 shutdown_connections 2024-09-17T03:36:48.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 -- 172.21.15.46:0/1628148470 wait complete. 2024-09-17T03:36:48.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 Processor -- start 2024-09-17T03:36:48.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.013+0000 7f1e63c00640 1 -- start start 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e5c0b4de0 0x7f1e5c13caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 0x7f1e5c1fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c1fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c168a80 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c168900 con 0x7f1e5c1527b0 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c168c00 con 0x7f1e5c0b4de0 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 0x7f1e5c1fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 0x7f1e5c1fbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43004/0 (socket says 172.21.15.46:43004) 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 -- 172.21.15.46:0/929497934 learned_addr learned my addr 172.21.15.46:0/929497934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e61c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c1fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 -- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e5c0b4de0 msgr2=0x7f1e5c13caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e61200640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e5c0b4de0 0x7f1e5c13caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e5c0b4de0 0x7f1e5c13caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 -- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c1527b0 msgr2=0x7f1e5c1fc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c1fc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 -- 172.21.15.46:0/929497934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1f9b20 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e61c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c1fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e61200640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e5c0b4de0 0x7f1e5c13caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e5be00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 0x7f1e5c1fbe20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000c10 tx=0x7f1e48004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480b2040 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e48005020 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1f9dc0 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480053d0 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c1fa380 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.017+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c15bb60 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.021+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f1e480be060 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.021+0000 7f1e58c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e3c075570 0x7f1e3c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.021+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f1e4812d150 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.021+0000 7f1e61200640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e3c075570 0x7f1e3c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.021+0000 7f1e61200640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e3c075570 0x7f1e3c077a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005fd0 tx=0x7f1e50005d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:48.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.025+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e5c075580 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.233+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e5c1546c0 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.233+0000 7f1e58c00640 1 -- 172.21.15.46:0/929497934 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5241 (secure 0 0 0) 0x7f1e480163f0 con 0x7f1e5c0b5c70 2024-09-17T03:36:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:48.237 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"btime":"2024-09-17T03:35:58:996234+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:35:45.367356+0000","modified":"2024-09-17T03:35:48.376364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248335":{"gid":248335,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2479724490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2479724490},{"type":"v1","addr":"172.21.15.46:6836","nonce":2479724490}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248335,"qdb_cluster":[248335]},"id":9}]} 2024-09-17T03:36:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-09-17T03:36:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e3c075570 msgr2=0x7f1e3c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e3c075570 0x7f1e3c077a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005fd0 tx=0x7f1e50005d00 comp rx=0 tx=0).stop 2024-09-17T03:36:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 msgr2=0x7f1e5c1fbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 0x7f1e5c1fbe20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000c10 tx=0x7f1e48004040 comp rx=0 tx=0).stop 2024-09-17T03:36:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 shutdown_connections 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e3c075570 0x7f1e3c077a80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e5c1527b0 0x7f1e5c1fc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e5c0b5c70 0x7f1e5c1fbe20 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 --2- 172.21.15.46:0/929497934 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e5c0b4de0 0x7f1e5c13caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 >> 172.21.15.46:0/929497934 conn(0x7f1e5c0b2040 msgr2=0x7f1e5c151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 shutdown_connections 2024-09-17T03:36:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.237+0000 7f1e63c00640 1 -- 172.21.15.46:0/929497934 wait complete. 2024-09-17T03:36:48.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:36:48.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 -- 172.21.15.46:0/3840310989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 msgr2=0x7f2be006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 --2- 172.21.15.46:0/3840310989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be006dfd0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009870 tx=0x7f2bcc01c790 comp rx=0 tx=0).stop 2024-09-17T03:36:48.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 -- 172.21.15.46:0/3840310989 shutdown_connections 2024-09-17T03:36:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 --2- 172.21.15.46:0/3840310989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be006dfd0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 --2- 172.21.15.46:0/3840310989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2be00d1140 0x7f2be00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 -- 172.21.15.46:0/3840310989 >> 172.21.15.46:0/3840310989 conn(0x7f2be00b41f0 msgr2=0x7f2be00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 -- 172.21.15.46:0/3840310989 shutdown_connections 2024-09-17T03:36:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 -- 172.21.15.46:0/3840310989 wait complete. 2024-09-17T03:36:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 Processor -- start 2024-09-17T03:36:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.417+0000 7f2bdf200640 1 -- start start 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be00d1140 0x7f2be00cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2be015f5c0 0x7f2be00cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0168de0 con 0x7f2be006dbc0 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0168c60 con 0x7f2be00d1140 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2be0168f60 con 0x7f2be015f5c0 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be0157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43014/0 (socket says 172.21.15.46:43014) 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 -- 172.21.15.46:0/1944880899 learned_addr learned my addr 172.21.15.46:0/1944880899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdd400640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be00d1140 0x7f2be00cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bde800640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2be015f5c0 0x7f2be00cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 -- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2be015f5c0 msgr2=0x7f2be00cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2be015f5c0 0x7f2be00cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 -- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be00d1140 msgr2=0x7f2be00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be00d1140 0x7f2be00cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 -- 172.21.15.46:0/1944880899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be00cc9a0 con 0x7f2be006dbc0 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdd400640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be00d1140 0x7f2be00cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdde00640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be0157410 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e7f0 tx=0x7f2bd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000ccc0 con 0x7f2be006dbc0 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd000ce80 con 0x7f2be006dbc0 2024-09-17T03:36:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0005280 con 0x7f2be006dbc0 2024-09-17T03:36:48.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be00cdef0 con 0x7f2be006dbc0 2024-09-17T03:36:48.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be015ecd0 con 0x7f2be006dbc0 2024-09-17T03:36:48.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.421+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be015eac0 con 0x7f2be006dbc0 2024-09-17T03:36:48.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.425+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f2bd0005420 con 0x7f2be006dbc0 2024-09-17T03:36:48.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.425+0000 7f2bd6000640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bc0075570 0x7f2bc0077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.425+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7f2bd0093e50 con 0x7f2be006dbc0 2024-09-17T03:36:48.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.425+0000 7f2bdd400640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bc0075570 0x7f2bc0077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.425+0000 7f2bdd400640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bc0075570 0x7f2bc0077a80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0af000 tx=0x7f2bcc01afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:48.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.429+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2bd00608d0 con 0x7f2be006dbc0 2024-09-17T03:36:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.605+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2be006dfd0 con 0x7f2be006dbc0 2024-09-17T03:36:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.605+0000 7f2bd6000640 1 -- 172.21.15.46:0/1944880899 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16859 (secure 0 0 0) 0x7f2bd00601d0 con 0x7f2be006dbc0 2024-09-17T03:36:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:48.606 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":130,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:36:42.382562+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:35:43.106524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T03:35:44.323553+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":130,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bc0075570 msgr2=0x7f2bc0077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bc0075570 0x7f2bc0077a80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0af000 tx=0x7f2bcc01afd0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 msgr2=0x7f2be0157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be0157410 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e7f0 tx=0x7f2bd000ecb0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 shutdown_connections 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2bc0075570 0x7f2bc0077a80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2be015f5c0 0x7f2be00cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be00d1140 0x7f2be00cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 --2- 172.21.15.46:0/1944880899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2be006dbc0 0x7f2be0157410 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 >> 172.21.15.46:0/1944880899 conn(0x7f2be00b41f0 msgr2=0x7f2be0069320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 shutdown_connections 2024-09-17T03:36:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.609+0000 7f2bdf200640 1 -- 172.21.15.46:0/1944880899 wait complete. 2024-09-17T03:36:48.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:36:48.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 -- 172.21.15.46:0/167646833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8072100 msgr2=0x7febc8155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 --2- 172.21.15.46:0/167646833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8072100 0x7febc8155b50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7febc8071130 tx=0x7febb801c8c0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 -- 172.21.15.46:0/167646833 shutdown_connections 2024-09-17T03:36:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 --2- 172.21.15.46:0/167646833 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febc815f0e0 0x7febc815f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 --2- 172.21.15.46:0/167646833 >> v2:172.21.15.99:3300/0 conn(0x7febc80b4de0 0x7febc80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 --2- 172.21.15.46:0/167646833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8072100 0x7febc8155b50 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 -- 172.21.15.46:0/167646833 >> 172.21.15.46:0/167646833 conn(0x7febc80b2040 msgr2=0x7febc80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 -- 172.21.15.46:0/167646833 shutdown_connections 2024-09-17T03:36:48.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.789+0000 7febce000640 1 -- 172.21.15.46:0/167646833 wait complete. 2024-09-17T03:36:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 Processor -- start 2024-09-17T03:36:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 -- start start 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febc8072100 0x7febc81464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 0x7febc814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febc815f0e0 0x7febc815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febc8073a90 con 0x7febc80b4de0 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7febc80b4aa0 con 0x7febc815f0e0 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7febc8073c10 con 0x7febc8072100 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 0x7febc814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 0x7febc814c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43028/0 (socket says 172.21.15.46:43028) 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 -- 172.21.15.46:0/1616478126 learned_addr learned my addr 172.21.15.46:0/1616478126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febc7e00640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febc815f0e0 0x7febc815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febc7400640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febc8072100 0x7febc81464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 -- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febc8072100 msgr2=0x7febc81464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febc8072100 0x7febc81464c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 -- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febc815f0e0 msgr2=0x7febc815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febc815f0e0 0x7febc815dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 -- 172.21.15.46:0/1616478126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc8146a00 con 0x7febc80b4de0 2024-09-17T03:36:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbfe00640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 0x7febc814c4c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7febb000c7b0 tx=0x7febb000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0007bb0 con 0x7febc80b4de0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb0007e40 con 0x7febc80b4de0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc814a140 con 0x7febc80b4de0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.793+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb000f6b0 con 0x7febc80b4de0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.797+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc814a700 con 0x7febc80b4de0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.797+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc80b51f0 con 0x7febc80b4de0 2024-09-17T03:36:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.797+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7febb0018400 con 0x7febc80b4de0 2024-09-17T03:36:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.797+0000 7febbf400640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feba4075570 0x7feba4077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.797+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5793+0+0 (secure 0 0 0) 0x7febb00940c0 con 0x7febc80b4de0 2024-09-17T03:36:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.797+0000 7febc7400640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feba4075570 0x7feba4077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:48.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.801+0000 7febc7400640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feba4075570 0x7feba4077a80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7febb80ae000 tx=0x7febb801b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:48.801+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7febb0026020 con 0x7febc80b4de0 2024-09-17T03:36:49.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.013+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7febc80b3740 con 0x7febc80b4de0 2024-09-17T03:36:49.013 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:36:49.013+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:36:49.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.017+0000 7febbf400640 1 -- 172.21.15.46:0/1616478126 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7febb0014030 con 0x7febc80b4de0 2024-09-17T03:36:49.016 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:36:49.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.017+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feba4075570 msgr2=0x7feba4077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:49.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.017+0000 7febce000640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feba4075570 0x7feba4077a80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7febb80ae000 tx=0x7febb801b100 comp rx=0 tx=0).stop 2024-09-17T03:36:49.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.017+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 msgr2=0x7febc814c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:49.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.017+0000 7febce000640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 0x7febc814c4c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7febb000c7b0 tx=0x7febb000cc70 comp rx=0 tx=0).stop 2024-09-17T03:36:49.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 shutdown_connections 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feba4075570 0x7feba4077a80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febc815f0e0 0x7febc815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc80b4de0 0x7febc814c4c0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 --2- 172.21.15.46:0/1616478126 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7febc8072100 0x7febc81464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 >> 172.21.15.46:0/1616478126 conn(0x7febc80b2040 msgr2=0x7febc814f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 shutdown_connections 2024-09-17T03:36:49.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.021+0000 7febce000640 1 -- 172.21.15.46:0/1616478126 wait complete. 2024-09-17T03:36:49.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:36:49.163 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:36:49.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 -- 172.21.15.46:0/131419203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c0b5c70 msgr2=0x7f5d8c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:49.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 --2- 172.21.15.46:0/131419203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c15bb60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009840 tx=0x7f5d8001c680 comp rx=0 tx=0).stop 2024-09-17T03:36:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 -- 172.21.15.46:0/131419203 shutdown_connections 2024-09-17T03:36:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 --2- 172.21.15.46:0/131419203 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d8c1527b0 0x7f5d8c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 --2- 172.21.15.46:0/131419203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c15bb60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 -- 172.21.15.46:0/131419203 >> 172.21.15.46:0/131419203 conn(0x7f5d8c0b2040 msgr2=0x7f5d8c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 -- 172.21.15.46:0/131419203 shutdown_connections 2024-09-17T03:36:49.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 -- 172.21.15.46:0/131419203 wait complete. 2024-09-17T03:36:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 Processor -- start 2024-09-17T03:36:49.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.209+0000 7f5d93e00640 1 -- start start 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d8c0b4de0 0x7f5d8c14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1527b0 0x7f5d8c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c168c50 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c168ad0 con 0x7f5d8c1527b0 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c168dd0 con 0x7f5d8c0b4de0 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c14b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43054/0 (socket says 172.21.15.46:43054) 2024-09-17T03:36:49.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 -- 172.21.15.46:0/2560770803 learned_addr learned my addr 172.21.15.46:0/2560770803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d91400640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d8c0b4de0 0x7f5d8c14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d91e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1527b0 0x7f5d8c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 -- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d8c0b4de0 msgr2=0x7f5d8c14d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d8c0b4de0 0x7f5d8c14d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 -- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1527b0 msgr2=0x7f5d8c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1527b0 0x7f5d8c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 -- 172.21.15.46:0/2560770803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c1f9c40 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d90a00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c14b380 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800e770 tx=0x7f5d7800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800ccc0 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d7800ee80 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c1f9ee0 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78010670 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c1fa4a0 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.213+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8c14d6c0 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.217+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f5d78010810 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.217+0000 7f5d89600640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d6c075570 0x7f5d6c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.217+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5851+0+0 (secure 0 0 0) 0x7f5d78093d80 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.217+0000 7f5d91400640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d6c075570 0x7f5d6c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.217+0000 7f5d91400640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d6c075570 0x7f5d6c077a80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5d800af000 tx=0x7f5d8001aec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:49.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.221+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5d780601b0 con 0x7f5d8c0b5c70 2024-09-17T03:36:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:49.429+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5d8c15bb60 con 0x7f5d8c0b5c70 2024-09-17T03:36:50.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.025+0000 7f5d89600640 1 -- 172.21.15.46:0/2560770803 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) ==== 104+0+0 (secure 0 0 0) 0x7f5d78064be0 con 0x7f5d8c0b5c70 2024-09-17T03:36:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d6c075570 msgr2=0x7f5d6c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d6c075570 0x7f5d6c077a80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5d800af000 tx=0x7f5d8001aec0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 msgr2=0x7f5d8c14b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c14b380 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800e770 tx=0x7f5d7800ec30 comp rx=0 tx=0).stop 2024-09-17T03:36:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 shutdown_connections 2024-09-17T03:36:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5d6c075570 0x7f5d6c077a80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d8c1527b0 0x7f5d8c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8c0b5c70 0x7f5d8c14b380 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 --2- 172.21.15.46:0/2560770803 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5d8c0b4de0 0x7f5d8c14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 >> 172.21.15.46:0/2560770803 conn(0x7f5d8c0b2040 msgr2=0x7f5d8c0b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 shutdown_connections 2024-09-17T03:36:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.029+0000 7f5d93e00640 1 -- 172.21.15.46:0/2560770803 wait complete. 2024-09-17T03:36:50.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.217+0000 7feb32e00640 1 -- 172.21.15.46:0/961888641 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 msgr2=0x7feb2c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.217+0000 7feb32e00640 1 --2- 172.21.15.46:0/961888641 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 0x7feb2c15cf90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feb24009bc0 tx=0x7feb2401cba0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- 172.21.15.46:0/961888641 shutdown_connections 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 --2- 172.21.15.46:0/961888641 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 0x7feb2c15cf90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 --2- 172.21.15.46:0/961888641 >> v2:172.21.15.99:3300/0 conn(0x7feb2c0b5a80 0x7feb2c074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- 172.21.15.46:0/961888641 >> 172.21.15.46:0/961888641 conn(0x7feb2c0708c0 msgr2=0x7feb2c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- 172.21.15.46:0/961888641 shutdown_connections 2024-09-17T03:36:50.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- 172.21.15.46:0/961888641 wait complete. 2024-09-17T03:36:50.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 Processor -- start 2024-09-17T03:36:50.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- start start 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 0x7feb2c0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb2c0b5a80 0x7feb2c153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 0x7feb2c148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c16a360 con 0x7feb2c157030 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c16a1e0 con 0x7feb2c0b5a80 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.221+0000 7feb32e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c16a4e0 con 0x7feb2c074950 2024-09-17T03:36:50.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 0x7feb2c148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb23400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb2c0b5a80 0x7feb2c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 0x7feb2c148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43064/0 (socket says 172.21.15.46:43064) 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 -- 172.21.15.46:0/1451008457 learned_addr learned my addr 172.21.15.46:0/1451008457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb2be00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 0x7feb2c0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 -- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 msgr2=0x7feb2c0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 0x7feb2c0b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 -- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb2c0b5a80 msgr2=0x7feb2c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb2c0b5a80 0x7feb2c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 -- 172.21.15.46:0/1451008457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c0b31c0 con 0x7feb2c157030 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb23400640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb2c0b5a80 0x7feb2c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:36:50.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb30e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 0x7feb2c148720 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7feb2401a7d0 tx=0x7feb24009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:50.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240bb070 con 0x7feb2c157030 2024-09-17T03:36:50.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb24004b40 con 0x7feb2c157030 2024-09-17T03:36:50.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c0b34a0 con 0x7feb2c157030 2024-09-17T03:36:50.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240b2040 con 0x7feb2c157030 2024-09-17T03:36:50.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c0b3a60 con 0x7feb2c157030 2024-09-17T03:36:50.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c0b2f30 con 0x7feb2c157030 2024-09-17T03:36:50.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.225+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7feb240be070 con 0x7feb2c157030 2024-09-17T03:36:50.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.229+0000 7feb28c00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb08075570 0x7feb08077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:50.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.229+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5909+0+0 (secure 0 0 0) 0x7feb2412ce40 con 0x7feb2c157030 2024-09-17T03:36:50.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.229+0000 7feb2be00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb08075570 0x7feb08077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:50.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.229+0000 7feb2be00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb08075570 0x7feb08077a80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7feb1c007fd0 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:50.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.233+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb24016790 con 0x7feb2c157030 2024-09-17T03:36:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:50.409+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feb2c15cf90 con 0x7feb2c157030 2024-09-17T03:36:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.021+0000 7feb28c00640 1 -- 172.21.15.46:0/1451008457 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7feb240f9270 con 0x7feb2c157030 2024-09-17T03:36:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:36:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb08075570 msgr2=0x7feb08077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb08075570 0x7feb08077a80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7feb1c007fd0 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2024-09-17T03:36:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 msgr2=0x7feb2c148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 0x7feb2c148720 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7feb2401a7d0 tx=0x7feb24009260 comp rx=0 tx=0).stop 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 shutdown_connections 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb08075570 0x7feb08077a80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb2c157030 0x7feb2c148720 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb2c0b5a80 0x7feb2c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 --2- 172.21.15.46:0/1451008457 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb2c074950 0x7feb2c0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 >> 172.21.15.46:0/1451008457 conn(0x7feb2c0708c0 msgr2=0x7feb2c14f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 shutdown_connections 2024-09-17T03:36:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.025+0000 7feb32e00640 1 -- 172.21.15.46:0/1451008457 wait complete. 2024-09-17T03:36:51.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:36:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 -- 172.21.15.46:0/4216772022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b5c70 msgr2=0x7f35e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 --2- 172.21.15.46:0/4216772022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b5c70 0x7f35e015bb60 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f35e0070710 tx=0x7f35d001cac0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 -- 172.21.15.46:0/4216772022 shutdown_connections 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 --2- 172.21.15.46:0/4216772022 >> v2:172.21.15.99:3300/0 conn(0x7f35e01527b0 0x7f35e0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 --2- 172.21.15.46:0/4216772022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b5c70 0x7f35e015bb60 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 -- 172.21.15.46:0/4216772022 >> 172.21.15.46:0/4216772022 conn(0x7f35e00b2040 msgr2=0x7f35e00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 -- 172.21.15.46:0/4216772022 shutdown_connections 2024-09-17T03:36:51.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 -- 172.21.15.46:0/4216772022 wait complete. 2024-09-17T03:36:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 Processor -- start 2024-09-17T03:36:51.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.237+0000 7f35e7800640 1 -- start start 2024-09-17T03:36:51.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 0x7f35e00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e00b5c70 0x7f35e0150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35e01527b0 0x7f35e00ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0168b20 con 0x7f35e00b4de0 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e01689a0 con 0x7f35e00b5c70 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f35e0168ca0 con 0x7f35e01527b0 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 0x7f35e00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 0x7f35e00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43078/0 (socket says 172.21.15.46:43078) 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 -- 172.21.15.46:0/465202049 learned_addr learned my addr 172.21.15.46:0/465202049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35dfe00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e00b5c70 0x7f35e0150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e5800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35e01527b0 0x7f35e00ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 -- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35e01527b0 msgr2=0x7f35e00ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35e01527b0 0x7f35e00ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 -- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e00b5c70 msgr2=0x7f35e0150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e00b5c70 0x7f35e0150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 -- 172.21.15.46:0/465202049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e015a2b0 con 0x7f35e00b4de0 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e5800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35e01527b0 0x7f35e00ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e4e00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 0x7f35e00b4680 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f90 tx=0x7f35d00a59b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00bb070 con 0x7f35e00b4de0 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35d00a5e50 con 0x7f35e00b4de0 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00b2040 con 0x7f35e00b4de0 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e015a590 con 0x7f35e00b4de0 2024-09-17T03:36:51.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e015ab50 con 0x7f35e00b4de0 2024-09-17T03:36:51.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.241+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e015bb60 con 0x7f35e00b4de0 2024-09-17T03:36:51.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.245+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f35d00040c0 con 0x7f35e00b4de0 2024-09-17T03:36:51.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.245+0000 7f35dcc00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35c0075570 0x7f35c0077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:51.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.245+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5471+0+0 (secure 0 0 0) 0x7f35d012d280 con 0x7f35e00b4de0 2024-09-17T03:36:51.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.245+0000 7f35dfe00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35c0075570 0x7f35c0077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:51.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.245+0000 7f35dfe00640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35c0075570 0x7f35c0077a80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f35e0155570 tx=0x7f35cc00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:51.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.249+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f35d0137020 con 0x7f35e00b4de0 2024-09-17T03:36:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:51.425+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35e0155b10 con 0x7f35e00b4de0 2024-09-17T03:36:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.425+0000 7f35dcc00640 1 -- 172.21.15.46:0/465202049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v135) ==== 169+0+0 (secure 0 0 0) 0x7f35e0075580 con 0x7f35e00b4de0 2024-09-17T03:36:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:36:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35c0075570 msgr2=0x7f35c0077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35c0075570 0x7f35c0077a80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f35e0155570 tx=0x7f35cc00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 msgr2=0x7f35e00b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 0x7f35e00b4680 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f90 tx=0x7f35d00a59b0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 shutdown_connections 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f35c0075570 0x7f35c0077a80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35e01527b0 0x7f35e00ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e00b5c70 0x7f35e0150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 --2- 172.21.15.46:0/465202049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35e00b4de0 0x7f35e00b4680 secure :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f90 tx=0x7f35d00a59b0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 >> 172.21.15.46:0/465202049 conn(0x7f35e00b2040 msgr2=0x7f35e00b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 shutdown_connections 2024-09-17T03:36:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.433+0000 7f35e7800640 1 -- 172.21.15.46:0/465202049 wait complete. 2024-09-17T03:36:52.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 -- 172.21.15.46:0/1732495858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a4005360 msgr2=0x7f27a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 --2- 172.21.15.46:0/1732495858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a4005360 0x7f27a400b9d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f27bc074950 tx=0x7f27b401f9e0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 -- 172.21.15.46:0/1732495858 shutdown_connections 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 --2- 172.21.15.46:0/1732495858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27a4005360 0x7f27a400b9d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 -- 172.21.15.46:0/1732495858 >> 172.21.15.46:0/1732495858 conn(0x7f27bc0b2040 msgr2=0x7f27bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 -- 172.21.15.46:0/1732495858 shutdown_connections 2024-09-17T03:36:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 -- 172.21.15.46:0/1732495858 wait complete. 2024-09-17T03:36:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 Processor -- start 2024-09-17T03:36:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.633+0000 7f27c3e00640 1 -- start start 2024-09-17T03:36:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27a4005360 0x7f27bc1e2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27bc144860 0x7f27bc0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 0x7f27bc0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc0751c0 con 0x7f27bc144c70 2024-09-17T03:36:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc075040 con 0x7f27a4005360 2024-09-17T03:36:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f27bc075340 con 0x7f27bc144860 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 0x7f27bc0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 0x7f27bc0aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46462/0 (socket says 172.21.15.46:46462) 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 -- 172.21.15.46:0/3021887502 learned_addr learned my addr 172.21.15.46:0/3021887502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1400640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27a4005360 0x7f27bc1e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c0a00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27bc144860 0x7f27bc0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 -- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27bc144860 msgr2=0x7f27bc0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27bc144860 0x7f27bc0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 -- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27a4005360 msgr2=0x7f27bc1e2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27a4005360 0x7f27bc1e2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 -- 172.21.15.46:0/3021887502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc1e1790 con 0x7f27bc144c70 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c1e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 0x7f27bc0aee20 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f27b400c6d0 tx=0x7f27b4001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4002080 con 0x7f27bc144c70 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27b40b7020 con 0x7f27bc144c70 2024-09-17T03:36:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc1e1a70 con 0x7f27bc144c70 2024-09-17T03:36:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40a95b0 con 0x7f27bc144c70 2024-09-17T03:36:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1e2030 con 0x7f27bc144c70 2024-09-17T03:36:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc0b53b0 con 0x7f27bc144c70 2024-09-17T03:36:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.637+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f27b40060d0 con 0x7f27bc144c70 2024-09-17T03:36:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.641+0000 7f27b9600640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f279c075570 0x7f279c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.641+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7f27b4130940 con 0x7f27bc144c70 2024-09-17T03:36:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.641+0000 7f27c1400640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f279c075570 0x7f279c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:52.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.641+0000 7f27c1400640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f279c075570 0x7f279c077a80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009be0 tx=0x7f27b00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:52.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.645+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27b40c1030 con 0x7f27bc144c70 2024-09-17T03:36:52.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.825+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f27bc0cc870 con 0x7f27bc144c70 2024-09-17T03:36:52.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.829+0000 7f27b9600640 1 -- 172.21.15.46:0/3021887502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7f27b4019790 con 0x7f27bc144c70 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.829+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f279c075570 msgr2=0x7f279c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.829+0000 7f27c3e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f279c075570 0x7f279c077a80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009be0 tx=0x7f27b00091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 msgr2=0x7f27bc0aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 0x7f27bc0aee20 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f27b400c6d0 tx=0x7f27b4001c60 comp rx=0 tx=0).stop 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 shutdown_connections 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f279c075570 0x7f279c077a80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27bc144c70 0x7f27bc0aee20 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f27bc144860 0x7f27bc0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 --2- 172.21.15.46:0/3021887502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27a4005360 0x7f27bc1e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 >> 172.21.15.46:0/3021887502 conn(0x7f27bc0b2040 msgr2=0x7f27bc0b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:52.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 shutdown_connections 2024-09-17T03:36:52.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:52.833+0000 7f27c3e00640 1 -- 172.21.15.46:0/3021887502 wait complete. 2024-09-17T03:36:52.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-17T03:36:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 -- 172.21.15.46:0/2776477055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4240b54f0 msgr2=0x7fe42415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 --2- 172.21.15.46:0/2776477055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4240b54f0 0x7fe42415bb60 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009840 tx=0x7fe41c01c760 comp rx=0 tx=0).stop 2024-09-17T03:36:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 -- 172.21.15.46:0/2776477055 shutdown_connections 2024-09-17T03:36:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 --2- 172.21.15.46:0/2776477055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4240b54f0 0x7fe42415bb60 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 -- 172.21.15.46:0/2776477055 >> 172.21.15.46:0/2776477055 conn(0x7fe4240b2040 msgr2=0x7fe4240b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 -- 172.21.15.46:0/2776477055 shutdown_connections 2024-09-17T03:36:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 -- 172.21.15.46:0/2776477055 wait complete. 2024-09-17T03:36:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 Processor -- start 2024-09-17T03:36:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.013+0000 7fe429000640 1 -- start start 2024-09-17T03:36:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4240b5c70 0x7fe42414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241527b0 0x7fe4240aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 0x7fe4240aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe424169bd0 con 0x7fe424150ce0 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe424169a50 con 0x7fe4241527b0 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe424169d50 con 0x7fe4240b5c70 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 0x7fe4240aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 0x7fe4240aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46472/0 (socket says 172.21.15.46:46472) 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 -- 172.21.15.46:0/2527072986 learned_addr learned my addr 172.21.15.46:0/2527072986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe421e00640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4240b5c70 0x7fe42414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe419400640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241527b0 0x7fe4240aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 -- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4240b5c70 msgr2=0x7fe42414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4240b5c70 0x7fe42414f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 -- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241527b0 msgr2=0x7fe4240aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241527b0 0x7fe4240aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 -- 172.21.15.46:0/2527072986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe424158dc0 con 0x7fe424150ce0 2024-09-17T03:36:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe422800640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 0x7fe4240aaa10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe41c01a390 tx=0x7fe41c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0bb070 con 0x7fe424150ce0 2024-09-17T03:36:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe41c005030 con 0x7fe424150ce0 2024-09-17T03:36:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0b2040 con 0x7fe424150ce0 2024-09-17T03:36:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241592c0 con 0x7fe424150ce0 2024-09-17T03:36:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4240ab460 con 0x7fe424150ce0 2024-09-17T03:36:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.017+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42415bb60 con 0x7fe424150ce0 2024-09-17T03:36:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.021+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fe41c005440 con 0x7fe424150ce0 2024-09-17T03:36:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.021+0000 7fe41aa00640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe400075570 0x7fe400077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.021+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7fe41c12cf70 con 0x7fe424150ce0 2024-09-17T03:36:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.021+0000 7fe421e00640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe400075570 0x7fe400077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.021+0000 7fe421e00640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe400075570 0x7fe400077a80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe414007fd0 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:53.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.025+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe41c016410 con 0x7fe424150ce0 2024-09-17T03:36:53.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.265+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7fe424159050 con 0x7fe424150ce0 2024-09-17T03:36:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe41aa00640 1 -- 172.21.15.46:0/2527072986 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v729) ==== 186+0+0 (secure 0 0 0) 0x7fe41c0f9320 con 0x7fe424150ce0 2024-09-17T03:36:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe400075570 msgr2=0x7fe400077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe400075570 0x7fe400077a80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe414007fd0 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-09-17T03:36:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 msgr2=0x7fe4240aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 0x7fe4240aaa10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe41c01a390 tx=0x7fe41c004040 comp rx=0 tx=0).stop 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 shutdown_connections 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe400075570 0x7fe400077a80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe424150ce0 0x7fe4240aaa10 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4241527b0 0x7fe4240aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 --2- 172.21.15.46:0/2527072986 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe4240b5c70 0x7fe42414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 >> 172.21.15.46:0/2527072986 conn(0x7fe4240b2040 msgr2=0x7fe4240b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 shutdown_connections 2024-09-17T03:36:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.429+0000 7fe429000640 1 -- 172.21.15.46:0/2527072986 wait complete. 2024-09-17T03:36:53.452 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-17T03:36:53.453 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-17T03:36:53.453 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T03:36:53.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:36:53.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 -- 172.21.15.46:0/2945768269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c004810 msgr2=0x7f407c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 --2- 172.21.15.46:0/2945768269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c004810 0x7f407c004c20 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4080009b40 tx=0x7f408001d250 comp rx=0 tx=0).stop 2024-09-17T03:36:53.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 -- 172.21.15.46:0/2945768269 shutdown_connections 2024-09-17T03:36:53.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 --2- 172.21.15.46:0/2945768269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c004810 0x7f407c004c20 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 -- 172.21.15.46:0/2945768269 >> 172.21.15.46:0/2945768269 conn(0x7f40980708c0 msgr2=0x7f4098070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:53.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 -- 172.21.15.46:0/2945768269 shutdown_connections 2024-09-17T03:36:53.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.613+0000 7f409cc00640 1 -- 172.21.15.46:0/2945768269 wait complete. 2024-09-17T03:36:53.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 Processor -- start 2024-09-17T03:36:53.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 -- start start 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f407c004810 0x7f409806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 0x7f409815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409815c1b0 0x7f4098153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f409815d920 con 0x7f40981fd9e0 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f409815d7a0 con 0x7f409815c1b0 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f409cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4098071c00 con 0x7f407c004810 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 0x7f409815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 0x7f409815bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46488/0 (socket says 172.21.15.46:46488) 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 -- 172.21.15.46:0/1498925917 learned_addr learned my addr 172.21.15.46:0/1498925917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f4095e00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f407c004810 0x7f409806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f4096800640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409815c1b0 0x7f4098153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 -- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f407c004810 msgr2=0x7f409806e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f407c004810 0x7f409806e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 -- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409815c1b0 msgr2=0x7f4098153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409815c1b0 0x7f4098153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.617+0000 7f408fe00640 1 -- 172.21.15.46:0/1498925917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409806ebb0 con 0x7f40981fd9e0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408fe00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 0x7f409815bc70 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f408001d080 tx=0x7f40800050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800bb030 con 0x7f40981fd9e0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4080004900 con 0x7f40981fd9e0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40981f9a80 con 0x7f40981fd9e0 2024-09-17T03:36:53.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080004d50 con 0x7f40981fd9e0 2024-09-17T03:36:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40981fa040 con 0x7f40981fd9e0 2024-09-17T03:36:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409806e920 con 0x7f40981fd9e0 2024-09-17T03:36:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f40800be030 con 0x7f40981fd9e0 2024-09-17T03:36:53.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408e000640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4074075570 0x7f4074077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.621+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7f408012ed10 con 0x7f40981fd9e0 2024-09-17T03:36:53.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.625+0000 7f4095e00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4074075570 0x7f4074077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.625+0000 7f4095e00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4074075570 0x7f4074077a80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4088000fc0 tx=0x7f4088001300 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.629+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4080016f60 con 0x7f40981fd9e0 2024-09-17T03:36:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.805+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4098072160 con 0x7f40981fd9e0 2024-09-17T03:36:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.805+0000 7f408e000640 1 -- 172.21.15.46:0/1498925917 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+13050 (secure 0 0 0) 0x7f40800fb1f0 con 0x7f40981fd9e0 2024-09-17T03:36:53.805 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:53.805 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":135,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:36:52.419624+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6836/2479724490":"2024-09-18T03:36:49.015808+0000","172.21.15.46:6834/2479724490":"2024-09-18T03:36:49.015808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.805+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4074075570 msgr2=0x7f4074077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.805+0000 7f409cc00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4074075570 0x7f4074077a80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4088000fc0 tx=0x7f4088001300 comp rx=0 tx=0).stop 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.805+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 msgr2=0x7f409815bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.805+0000 7f409cc00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 0x7f409815bc70 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f408001d080 tx=0x7f40800050a0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 shutdown_connections 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4074075570 0x7f4074077a80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409815c1b0 0x7f4098153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40981fd9e0 0x7f409815bc70 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 --2- 172.21.15.46:0/1498925917 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f407c004810 0x7f409806e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 >> 172.21.15.46:0/1498925917 conn(0x7f40980708c0 msgr2=0x7f409815f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 shutdown_connections 2024-09-17T03:36:53.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.809+0000 7f409cc00640 1 -- 172.21.15.46:0/1498925917 wait complete. 2024-09-17T03:36:53.828 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:20:55.230199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:36:53.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:36:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.989+0000 7fdd9a000640 1 -- 172.21.15.46:0/3991389105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd941527b0 msgr2=0x7fdd9415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.989+0000 7fdd9a000640 1 --2- 172.21.15.46:0/3991389105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd941527b0 0x7fdd9415bb60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fdd940cca90 tx=0x7fdd7c01c620 comp rx=0 tx=0).stop 2024-09-17T03:36:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 -- 172.21.15.46:0/3991389105 shutdown_connections 2024-09-17T03:36:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 --2- 172.21.15.46:0/3991389105 >> v2:172.21.15.99:3300/0 conn(0x7fdd940b4de0 0x7fdd940b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 --2- 172.21.15.46:0/3991389105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd941527b0 0x7fdd9415bb60 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 -- 172.21.15.46:0/3991389105 >> 172.21.15.46:0/3991389105 conn(0x7fdd940b2040 msgr2=0x7fdd940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 -- 172.21.15.46:0/3991389105 shutdown_connections 2024-09-17T03:36:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 -- 172.21.15.46:0/3991389105 wait complete. 2024-09-17T03:36:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 Processor -- start 2024-09-17T03:36:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.993+0000 7fdd9a000640 1 -- start start 2024-09-17T03:36:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 0x7fdd940adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd940b5c70 0x7fdd940af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd941527b0 0x7fdd941f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94168910 con 0x7fdd940b4de0 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94168790 con 0x7fdd940b5c70 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fdd94168a90 con 0x7fdd941527b0 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 0x7fdd940adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 0x7fdd940adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46520/0 (socket says 172.21.15.46:46520) 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 -- 172.21.15.46:0/1432047375 learned_addr learned my addr 172.21.15.46:0/1432047375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd940b5c70 0x7fdd940af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93e00640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd941527b0 0x7fdd941f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 -- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd941527b0 msgr2=0x7fdd941f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd941527b0 0x7fdd941f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 -- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd940b5c70 msgr2=0x7fdd940af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd940b5c70 0x7fdd940af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 -- 172.21.15.46:0/1432047375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd941fa0b0 con 0x7fdd940b4de0 2024-09-17T03:36:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93e00640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd941527b0 0x7fdd941f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:36:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd93400640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 0x7fdd940adc80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b410 tx=0x7fdd8400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84015020 con 0x7fdd940b4de0 2024-09-17T03:36:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd84011920 con 0x7fdd940b4de0 2024-09-17T03:36:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84011db0 con 0x7fdd940b4de0 2024-09-17T03:36:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd941fa390 con 0x7fdd940b4de0 2024-09-17T03:36:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd941fa950 con 0x7fdd940b4de0 2024-09-17T03:36:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd940b51f0 con 0x7fdd940b4de0 2024-09-17T03:36:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:53.997+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fdd84018020 con 0x7fdd940b4de0 2024-09-17T03:36:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.001+0000 7fdd8b400640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd70075570 0x7fdd70077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.001+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7fdd84093320 con 0x7fdd940b4de0 2024-09-17T03:36:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.001+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd70075570 0x7fdd70077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.001+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd70075570 0x7fdd70077a80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0af000 tx=0x7fdd7c01ae60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.005+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdd8401f020 con 0x7fdd940b4de0 2024-09-17T03:36:54.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.177+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd940cdad0 con 0x7fdd940b4de0 2024-09-17T03:36:54.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.177+0000 7fdd8b400640 1 -- 172.21.15.46:0/1432047375 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) ==== 93+0+10 (secure 0 0 0) 0x7fdd8405fda0 con 0x7fdd940b4de0 2024-09-17T03:36:54.179 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-17T03:36:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd70075570 msgr2=0x7fdd70077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd70075570 0x7fdd70077a80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0af000 tx=0x7fdd7c01ae60 comp rx=0 tx=0).stop 2024-09-17T03:36:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 msgr2=0x7fdd940adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 0x7fdd940adc80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b410 tx=0x7fdd8400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 shutdown_connections 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fdd70075570 0x7fdd70077a80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fdd941527b0 0x7fdd941f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd940b5c70 0x7fdd940af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 --2- 172.21.15.46:0/1432047375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd940b4de0 0x7fdd940adc80 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 >> 172.21.15.46:0/1432047375 conn(0x7fdd940b2040 msgr2=0x7fdd94159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 shutdown_connections 2024-09-17T03:36:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.181+0000 7fdd9a000640 1 -- 172.21.15.46:0/1432047375 wait complete. 2024-09-17T03:36:54.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-17T03:36:54.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 -- 172.21.15.46:0/2346954700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c072100 msgr2=0x7f297c155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:54.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 --2- 172.21.15.46:0/2346954700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c072100 0x7f297c155b50 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f297c071130 tx=0x7f296c01c830 comp rx=0 tx=0).stop 2024-09-17T03:36:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 -- 172.21.15.46:0/2346954700 shutdown_connections 2024-09-17T03:36:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 --2- 172.21.15.46:0/2346954700 >> v2:172.21.15.99:3300/0 conn(0x7f297c0b4de0 0x7f297c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 --2- 172.21.15.46:0/2346954700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c072100 0x7f297c155b50 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 -- 172.21.15.46:0/2346954700 >> 172.21.15.46:0/2346954700 conn(0x7f297c0b2040 msgr2=0x7f297c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 -- 172.21.15.46:0/2346954700 shutdown_connections 2024-09-17T03:36:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 -- 172.21.15.46:0/2346954700 wait complete. 2024-09-17T03:36:54.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.361+0000 7f2983400640 1 Processor -- start 2024-09-17T03:36:54.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- start start 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f297c072100 0x7f297c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0b4de0 0x7f297c1f9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 0x7f297c0af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1693b0 con 0x7f297c15f0e0 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f297c169230 con 0x7f297c0b4de0 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f297c169530 con 0x7f297c072100 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 0x7f297c0af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 0x7f297c0af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46534/0 (socket says 172.21.15.46:46534) 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 -- 172.21.15.46:0/2107562572 learned_addr learned my addr 172.21.15.46:0/2107562572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 -- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f297c072100 msgr2=0x7f297c1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2980a00640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f297c072100 0x7f297c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2973e00640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0b4de0 0x7f297c1f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f297c072100 0x7f297c1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 -- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0b4de0 msgr2=0x7f297c1f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0b4de0 0x7f297c1f9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 -- 172.21.15.46:0/2107562572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c0b2760 con 0x7f297c15f0e0 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2980a00640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f297c072100 0x7f297c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2973e00640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0b4de0 0x7f297c1f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2981400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 0x7f297c0af080 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f297400c760 tx=0x7f297400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974003db0 con 0x7f297c15f0e0 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f297400ce10 con 0x7f297c15f0e0 2024-09-17T03:36:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740052d0 con 0x7f297c15f0e0 2024-09-17T03:36:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c0adfe0 con 0x7f297c15f0e0 2024-09-17T03:36:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c0ae5a0 con 0x7f297c15f0e0 2024-09-17T03:36:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.365+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c155b50 con 0x7f297c15f0e0 2024-09-17T03:36:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.369+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f2974005470 con 0x7f297c15f0e0 2024-09-17T03:36:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.369+0000 7f2979600640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2958075570 0x7f2958077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.369+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7f2974093100 con 0x7f297c15f0e0 2024-09-17T03:36:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.369+0000 7f2980a00640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2958075570 0x7f2958077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.369+0000 7f2980a00640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2958075570 0x7f2958077a80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f296c0af000 tx=0x7f296c01b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.373+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f297c0b5730 con 0x7f297c15f0e0 2024-09-17T03:36:54.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:54.617+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f297c0add70 con 0x7f297c15f0e0 2024-09-17T03:36:55.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.437+0000 7f2979600640 1 -- 172.21.15.46:0/2107562572 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v731) ==== 189+0+0 (secure 0 0 0) 0x7f297405fcb0 con 0x7f297c15f0e0 2024-09-17T03:36:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2958075570 msgr2=0x7f2958077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2958075570 0x7f2958077a80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f296c0af000 tx=0x7f296c01b070 comp rx=0 tx=0).stop 2024-09-17T03:36:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 msgr2=0x7f297c0af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 0x7f297c0af080 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f297400c760 tx=0x7f297400cc20 comp rx=0 tx=0).stop 2024-09-17T03:36:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 shutdown_connections 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2958075570 0x7f2958077a80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f297c15f0e0 0x7f297c0af080 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f297c0b4de0 0x7f297c1f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 --2- 172.21.15.46:0/2107562572 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f297c072100 0x7f297c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 >> 172.21.15.46:0/2107562572 conn(0x7f297c0b2040 msgr2=0x7f297c073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 shutdown_connections 2024-09-17T03:36:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.441+0000 7f2983400640 1 -- 172.21.15.46:0/2107562572 wait complete. 2024-09-17T03:36:55.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:36:55.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 -- 172.21.15.46:0/1129717100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac0b5c70 msgr2=0x7fd3ac15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/1129717100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac0b5c70 0x7fd3ac15bb60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0cca90 tx=0x7fd39c01c620 comp rx=0 tx=0).stop 2024-09-17T03:36:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 -- 172.21.15.46:0/1129717100 shutdown_connections 2024-09-17T03:36:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/1129717100 >> v2:172.21.15.99:3300/0 conn(0x7fd3ac0b4de0 0x7fd3ac0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/1129717100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac0b5c70 0x7fd3ac15bb60 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 -- 172.21.15.46:0/1129717100 >> 172.21.15.46:0/1129717100 conn(0x7fd3ac0b2040 msgr2=0x7fd3ac0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 -- 172.21.15.46:0/1129717100 shutdown_connections 2024-09-17T03:36:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.629+0000 7fd3b1a00640 1 -- 172.21.15.46:0/1129717100 wait complete. 2024-09-17T03:36:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 Processor -- start 2024-09-17T03:36:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 -- start start 2024-09-17T03:36:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3ac0b4de0 0x7fd3ac1f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3ac0b5c70 0x7fd3ac14fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 0x7fd3ac1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac168920 con 0x7fd3ac1527b0 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1687a0 con 0x7fd3ac0b4de0 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ac168aa0 con 0x7fd3ac0b5c70 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 0x7fd3ac1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 0x7fd3ac1551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46554/0 (socket says 172.21.15.46:46554) 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 -- 172.21.15.46:0/207479011 learned_addr learned my addr 172.21.15.46:0/207479011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3aaa00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3ac0b4de0 0x7fd3ac1f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3a3e00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3ac0b5c70 0x7fd3ac14fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 -- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3ac0b5c70 msgr2=0x7fd3ac14fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3ac0b5c70 0x7fd3ac14fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 -- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3ac0b4de0 msgr2=0x7fd3ac1f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3ac0b4de0 0x7fd3ac1f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 -- 172.21.15.46:0/207479011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac1f9c60 con 0x7fd3ac1527b0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3ab400640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 0x7fd3ac1551c0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400e940 tx=0x7fd3a400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004d30 con 0x7fd3ac1527b0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3a400cb20 con 0x7fd3ac1527b0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4005020 con 0x7fd3ac1527b0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac15d990 con 0x7fd3ac1527b0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.633+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac15dc30 con 0x7fd3ac1527b0 2024-09-17T03:36:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.637+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac15f570 con 0x7fd3ac1527b0 2024-09-17T03:36:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.637+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fd3a40051c0 con 0x7fd3ac1527b0 2024-09-17T03:36:55.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.637+0000 7fd3a2a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd388075570 0x7fd388077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.637+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7fd3a4093f70 con 0x7fd3ac1527b0 2024-09-17T03:36:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.637+0000 7fd3aaa00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd388075570 0x7fd388077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:55.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.641+0000 7fd3aaa00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd388075570 0x7fd388077a80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0af000 tx=0x7fd39c01ae60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:55.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.641+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd3ac1f9c60 con 0x7fd3ac1527b0 2024-09-17T03:36:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.829+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fd3ac0b51f0 con 0x7fd3ac1527b0 2024-09-17T03:36:55.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.829+0000 7fd3a2a00640 1 -- 172.21.15.46:0/207479011 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v57) ==== 59+0+2294 (secure 0 0 0) 0x7fd3a4016030 con 0x7fd3ac1527b0 2024-09-17T03:36:55.829 INFO:teuthology.orchestra.run.smithi046.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:36:55.829 INFO:teuthology.orchestra.run.smithi046.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:36:55.829 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_client 20/20 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3753.438262 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5056.988604 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3768.847777 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.269903 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4247.059402 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5264.236629 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4517.967974 2024-09-17T03:36:55.830 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4303.040947 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds 20/20 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced debug_ms 2/2 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_frag true 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds dev mds_verify_scatter true 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced client_mount_timeout 600 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_client 20/20 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced debug_ms 1/1 2024-09-17T03:36:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:36:55.832 INFO:teuthology.orchestra.run.smithi046.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:36:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.829+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd388075570 msgr2=0x7fd388077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.829+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd388075570 0x7fd388077a80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0af000 tx=0x7fd39c01ae60 comp rx=0 tx=0).stop 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.829+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 msgr2=0x7fd3ac1551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.829+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 0x7fd3ac1551c0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400e940 tx=0x7fd3a400ee00 comp rx=0 tx=0).stop 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 shutdown_connections 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd388075570 0x7fd388077a80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac1527b0 0x7fd3ac1551c0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd3ac0b5c70 0x7fd3ac14fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 --2- 172.21.15.46:0/207479011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3ac0b4de0 0x7fd3ac1f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 >> 172.21.15.46:0/207479011 conn(0x7fd3ac0b2040 msgr2=0x7fd3ac151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 shutdown_connections 2024-09-17T03:36:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:55.833+0000 7fd3b1a00640 1 -- 172.21.15.46:0/207479011 wait complete. 2024-09-17T03:36:55.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 -- 172.21.15.46:0/3667776213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698004d50 msgr2=0x7f1698005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 --2- 172.21.15.46:0/3667776213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698004d50 0x7f1698005160 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009e90 tx=0x7f16ac01d590 comp rx=0 tx=0).stop 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16b9200640 1 -- 172.21.15.46:0/3667776213 reap_dead start 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 -- 172.21.15.46:0/3667776213 shutdown_connections 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 -- 172.21.15.46:0/3667776213 >> 172.21.15.46:0/3667776213 conn(0x7f16b40b2040 msgr2=0x7f16b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 -- 172.21.15.46:0/3667776213 shutdown_connections 2024-09-17T03:36:56.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 -- 172.21.15.46:0/3667776213 wait complete. 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 Processor -- start 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.017+0000 7f16bb200640 1 -- start start 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1698004d50 0x7f16b41f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16b4135480 0x7f16b40cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 0x7f16b40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40b3ae0 con 0x7f16b40cbde0 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40b3960 con 0x7f1698004d50 2024-09-17T03:36:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f16b40b3c60 con 0x7f16b4135480 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 0x7f16b40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 0x7f16b40ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46584/0 (socket says 172.21.15.46:46584) 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 -- 172.21.15.46:0/3686600822 learned_addr learned my addr 172.21.15.46:0/3686600822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1698004d50 0x7f16b41f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16ab400640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16b4135480 0x7f16b40cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 -- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16b4135480 msgr2=0x7f16b40cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16b4135480 0x7f16b40cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 -- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1698004d50 msgr2=0x7f16b41f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1698004d50 0x7f16b41f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 -- 172.21.15.46:0/3686600822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b4135c60 con 0x7f16b40cbde0 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b3e00640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1698004d50 0x7f16b41f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b9200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 0x7f16b40ae8e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009e90 tx=0x7f16ac009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac009a90 con 0x7f16b40cbde0 2024-09-17T03:36:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16ac0a94d0 con 0x7f16b40cbde0 2024-09-17T03:36:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0a9cd0 con 0x7f16b40cbde0 2024-09-17T03:36:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b4135f40 con 0x7f16b40cbde0 2024-09-17T03:36:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b40c6a60 con 0x7f16b40cbde0 2024-09-17T03:36:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.021+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b41359d0 con 0x7f16b40cbde0 2024-09-17T03:36:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.025+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f16ac004fe0 con 0x7f16b40cbde0 2024-09-17T03:36:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.025+0000 7f16b0c00640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1690075570 0x7f1690077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.025+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7f16ac12e380 con 0x7f16b40cbde0 2024-09-17T03:36:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.025+0000 7f16b3e00640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1690075570 0x7f1690077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.025+0000 7f16b3e00640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1690075570 0x7f1690077a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f16a40124a0 tx=0x7f16a40091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.029+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f16ac0c2030 con 0x7f16b40cbde0 2024-09-17T03:36:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.205+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f16b40c52b0 con 0x7f16b40cbde0 2024-09-17T03:36:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16b0c00640 1 -- 172.21.15.46:0/3686600822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) ==== 122+0+0 (secure 0 0 0) 0x7f16ac0ad070 con 0x7f16b40cbde0 2024-09-17T03:36:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1690075570 msgr2=0x7f1690077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1690075570 0x7f1690077a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f16a40124a0 tx=0x7f16a40091e0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 msgr2=0x7f16b40ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 0x7f16b40ae8e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009e90 tx=0x7f16ac009480 comp rx=0 tx=0).stop 2024-09-17T03:36:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 shutdown_connections 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1690075570 0x7f1690077a80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16b40cbde0 0x7f16b40ae8e0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f16b4135480 0x7f16b40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 --2- 172.21.15.46:0/3686600822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1698004d50 0x7f16b41f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 >> 172.21.15.46:0/3686600822 conn(0x7f16b40b2040 msgr2=0x7f16b40cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 shutdown_connections 2024-09-17T03:36:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.209+0000 7f16bb200640 1 -- 172.21.15.46:0/3686600822 wait complete. 2024-09-17T03:36:56.232 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:36:56.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:36:56.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:36:56.438 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:36:56.438 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:56.438 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:56.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:36:56.447 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:36:56.448 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:56.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:36:56.457 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:36:56.457 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:56.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:36:56.500 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:36:56.500 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:36:56.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:36:56.509 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:36:56.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:36:56.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 -- 172.21.15.46:0/4069162699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100004810 msgr2=0x7f1100004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 --2- 172.21.15.46:0/4069162699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100004810 0x7f1100004c20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f1104006f50 tx=0x7f110401ce60 comp rx=0 tx=0).stop 2024-09-17T03:36:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 -- 172.21.15.46:0/4069162699 shutdown_connections 2024-09-17T03:36:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 --2- 172.21.15.46:0/4069162699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100004810 0x7f1100004c20 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 -- 172.21.15.46:0/4069162699 >> 172.21.15.46:0/4069162699 conn(0x7f11180b2040 msgr2=0x7f11180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 -- 172.21.15.46:0/4069162699 shutdown_connections 2024-09-17T03:36:56.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.701+0000 7f111fe00640 1 -- 172.21.15.46:0/4069162699 wait complete. 2024-09-17T03:36:56.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 Processor -- start 2024-09-17T03:36:56.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 -- start start 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1100004810 0x7f111814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 0x7f111815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f111815e480 0x7f11181f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11180751f0 con 0x7f1118159ad0 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1118075070 con 0x7f1100004810 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1118075370 con 0x7f111815e480 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 0x7f111815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 0x7f111815df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46602/0 (socket says 172.21.15.46:46602) 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 -- 172.21.15.46:0/375041155 learned_addr learned my addr 172.21.15.46:0/375041155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111d400640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1100004810 0x7f111814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111de00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f111815e480 0x7f11181f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 -- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f111815e480 msgr2=0x7f11181f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f111815e480 0x7f11181f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 -- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1100004810 msgr2=0x7f111814e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1100004810 0x7f111814e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 -- 172.21.15.46:0/375041155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111815e890 con 0x7f1118159ad0 2024-09-17T03:36:56.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f111ca00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 0x7f111815df40 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1104001250 tx=0x7f1104006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.705+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11040015c0 con 0x7f1118159ad0 2024-09-17T03:36:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1104001760 con 0x7f1118159ad0 2024-09-17T03:36:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111813bd70 con 0x7f1118159ad0 2024-09-17T03:36:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11040064a0 con 0x7f1118159ad0 2024-09-17T03:36:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111813c550 con 0x7f1118159ad0 2024-09-17T03:36:56.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111813bfe0 con 0x7f1118159ad0 2024-09-17T03:36:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f1104003600 con 0x7f1118159ad0 2024-09-17T03:36:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.709+0000 7f1115600640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f10f8075570 0x7f10f8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:56.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.713+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7f110412d380 con 0x7f1118159ad0 2024-09-17T03:36:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.713+0000 7f111d400640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f10f8075570 0x7f10f8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:56.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.713+0000 7f111d400640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f10f8075570 0x7f10f8077a80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f110c000c10 tx=0x7f110c006b20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.717+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11040be060 con 0x7f1118159ad0 2024-09-17T03:36:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.925+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11180cf730 con 0x7f1118159ad0 2024-09-17T03:36:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.925+0000 7f1115600640 1 -- 172.21.15.46:0/375041155 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+3919 (secure 0 0 0) 0x7f111815e890 con 0x7f1118159ad0 2024-09-17T03:36:56.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:56.927 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"btime":"2024-09-17T03:36:50:018921+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3739416825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3739416825},{"type":"v1","addr":"172.21.15.99:6835","nonce":3739416825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":248326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4004835672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4004835672},{"type":"v1","addr":"172.21.15.99:6833","nonce":4004835672}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":248332,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2480347507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":2480347507},{"type":"v1","addr":"172.21.15.46:6837","nonce":2480347507}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":249117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/2510922453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2510922453},{"type":"v1","addr":"172.21.15.46:6836","nonce":2510922453}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[]} 2024-09-17T03:36:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 87 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f10f8075570 msgr2=0x7f10f8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f10f8075570 0x7f10f8077a80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f110c000c10 tx=0x7f110c006b20 comp rx=0 tx=0).stop 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 msgr2=0x7f111815df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 0x7f111815df40 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1104001250 tx=0x7f1104006c50 comp rx=0 tx=0).stop 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 shutdown_connections 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f10f8075570 0x7f10f8077a80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f111815e480 0x7f11181f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1118159ad0 0x7f111815df40 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 --2- 172.21.15.46:0/375041155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1100004810 0x7f111814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 >> 172.21.15.46:0/375041155 conn(0x7f11180b2040 msgr2=0x7f1118141e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 shutdown_connections 2024-09-17T03:36:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:56.929+0000 7f111fe00640 1 -- 172.21.15.46:0/375041155 wait complete. 2024-09-17T03:36:56.949 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:36:56.949 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:36:56.950 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:36:56.950 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:36:56.950 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:36:56.950 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:36:56.950 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:36:56.950 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:36:56.950 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:36:56.950 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:36:56.951 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:36:56.951 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:36:56.951 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:36:56.951 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:36:56.951 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:36:56.951 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:36:56.963 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:36:56.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:36:56.969 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:36:56.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:36:56.971 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:36:56.971 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:36:56.973 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:36:56.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:36:57.014 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:36:57.014 DEBUG:teuthology.parallel:result is None 2024-09-17T03:36:57.015 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:36:57.016 DEBUG:teuthology.parallel:result is None 2024-09-17T03:36:57.017 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:36:57.018 DEBUG:teuthology.parallel:result is None 2024-09-17T03:36:57.020 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:36:57.020 DEBUG:teuthology.parallel:result is None 2024-09-17T03:36:57.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:36:57.118 INFO:tasks.ceph.mds.b.smithi099.stdout:starting mds.b at 2024-09-17T03:36:57.119 INFO:tasks.ceph.mds.d.smithi099.stdout:starting mds.d at 2024-09-17T03:36:57.119 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-09-17T03:36:57.121 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:36:57.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 -- 172.21.15.46:0/3277532109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c0b5c70 msgr2=0x7fb71c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 --2- 172.21.15.46:0/3277532109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c0b5c70 0x7fb71c15bb60 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb710009840 tx=0x7fb71001c760 comp rx=0 tx=0).stop 2024-09-17T03:36:57.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 -- 172.21.15.46:0/3277532109 shutdown_connections 2024-09-17T03:36:57.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 --2- 172.21.15.46:0/3277532109 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c1527b0 0x7fb71c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 --2- 172.21.15.46:0/3277532109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c0b5c70 0x7fb71c15bb60 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 -- 172.21.15.46:0/3277532109 >> 172.21.15.46:0/3277532109 conn(0x7fb71c0b2040 msgr2=0x7fb71c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:57.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 -- 172.21.15.46:0/3277532109 shutdown_connections 2024-09-17T03:36:57.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 -- 172.21.15.46:0/3277532109 wait complete. 2024-09-17T03:36:57.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 Processor -- start 2024-09-17T03:36:57.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.189+0000 7fb723800640 1 -- start start 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 0x7fb71c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c0b5c70 0x7fb71c144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c1527b0 0x7fb71c145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c168bb0 con 0x7fb71c1527b0 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c168a30 con 0x7fb71c0b5c70 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c168d30 con 0x7fb71c0b4de0 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb721800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c1527b0 0x7fb71c145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 0x7fb71c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 0x7fb71c157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:48286/0 (socket says 172.21.15.46:48286) 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb721800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c1527b0 0x7fb71c145240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46646/0 (socket says 172.21.15.46:46646) 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 -- 172.21.15.46:0/2684573877 learned_addr learned my addr 172.21.15.46:0/2684573877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb71be00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c0b5c70 0x7fb71c144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 -- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c0b5c70 msgr2=0x7fb71c144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c0b5c70 0x7fb71c144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 -- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c1527b0 msgr2=0x7fb71c145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c1527b0 0x7fb71c145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 -- 172.21.15.46:0/2684573877 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c145ba0 con 0x7fb71c0b4de0 2024-09-17T03:36:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb720e00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 0x7fb71c157ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb71001ae10 tx=0x7fb7100a57a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:57.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100bb070 con 0x7fb71c0b4de0 2024-09-17T03:36:57.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb710005030 con 0x7fb71c0b4de0 2024-09-17T03:36:57.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100b2040 con 0x7fb71c0b4de0 2024-09-17T03:36:57.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c158190 con 0x7fb71c0b4de0 2024-09-17T03:36:57.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.193+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c158750 con 0x7fb71c0b4de0 2024-09-17T03:36:57.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.197+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c15bb60 con 0x7fb71c0b4de0 2024-09-17T03:36:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.197+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fb710005440 con 0x7fb71c0b4de0 2024-09-17T03:36:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.197+0000 7fb718c00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6fc075570 0x7fb6fc077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.197+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(135..135 src has 1..135) ==== 4971+0+0 (secure 0 0 0) 0x7fb71012cea0 con 0x7fb71c0b4de0 2024-09-17T03:36:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.197+0000 7fb71be00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6fc075570 0x7fb6fc077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.197+0000 7fb71be00640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6fc075570 0x7fb6fc077a80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb71c166280 tx=0x7fb70800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.201+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb71c075580 con 0x7fb71c0b4de0 2024-09-17T03:36:57.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.389+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb71c0b4bc0 con 0x7fb71c0b4de0 2024-09-17T03:36:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.457+0000 7fb718c00640 1 -- 172.21.15.46:0/2684573877 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v137) ==== 97+0+0 (secure 0 0 0) 0x7fb710016410 con 0x7fb71c0b4de0 2024-09-17T03:36:57.461 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6fc075570 msgr2=0x7fb6fc077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6fc075570 0x7fb6fc077a80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb71c166280 tx=0x7fb70800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 msgr2=0x7fb71c157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 0x7fb71c157ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb71001ae10 tx=0x7fb7100a57a0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 shutdown_connections 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb6fc075570 0x7fb6fc077a80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c1527b0 0x7fb71c145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c0b5c70 0x7fb71c144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 --2- 172.21.15.46:0/2684573877 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb71c0b4de0 0x7fb71c157ba0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 >> 172.21.15.46:0/2684573877 conn(0x7fb71c0b2040 msgr2=0x7fb71c151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 shutdown_connections 2024-09-17T03:36:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.461+0000 7fb723800640 1 -- 172.21.15.46:0/2684573877 wait complete. 2024-09-17T03:36:57.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:36:57.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 -- 172.21.15.46:0/2561810592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b4de0 msgr2=0x7f42980b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 --2- 172.21.15.46:0/2561810592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b4de0 0x7f42980b51f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f42980703e0 tx=0x7f429001ccc0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 -- 172.21.15.46:0/2561810592 shutdown_connections 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 --2- 172.21.15.46:0/2561810592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b4de0 0x7f42980b51f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 --2- 172.21.15.46:0/2561810592 >> v2:172.21.15.99:3300/0 conn(0x7f42981527b0 0x7f4298074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 --2- 172.21.15.46:0/2561810592 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42980b5c70 0x7f429815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 -- 172.21.15.46:0/2561810592 >> 172.21.15.46:0/2561810592 conn(0x7f42980b2040 msgr2=0x7f42980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 -- 172.21.15.46:0/2561810592 shutdown_connections 2024-09-17T03:36:57.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.641+0000 7f429f600640 1 -- 172.21.15.46:0/2561810592 wait complete. 2024-09-17T03:36:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 Processor -- start 2024-09-17T03:36:57.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 -- start start 2024-09-17T03:36:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42980b4de0 0x7f42980aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 0x7f429814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42981527b0 0x7f429814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f429815f6b0 con 0x7f42980b5c70 2024-09-17T03:36:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f429815f530 con 0x7f42981527b0 2024-09-17T03:36:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429f600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f429815f830 con 0x7f42980b4de0 2024-09-17T03:36:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 0x7f429814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 0x7f429814ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46650/0 (socket says 172.21.15.46:46650) 2024-09-17T03:36:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 -- 172.21.15.46:0/1462701927 learned_addr learned my addr 172.21.15.46:0/1462701927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429cc00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42980b4de0 0x7f42980aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f429d600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42981527b0 0x7f429814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 -- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42980b4de0 msgr2=0x7f42980aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42980b4de0 0x7f42980aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 -- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42981527b0 msgr2=0x7f429814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42981527b0 0x7f429814f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 -- 172.21.15.46:0/1462701927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42980cfc10 con 0x7f42980b5c70 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4297e00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 0x7f429814ec10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f428400c970 tx=0x7f428400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.645+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284014070 con 0x7f42980b5c70 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42840077a0 con 0x7f42980b5c70 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284007bf0 con 0x7f42980b5c70 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42981fbc00 con 0x7f42980b5c70 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42981fc1c0 con 0x7f42980b5c70 2024-09-17T03:36:57.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42980cf980 con 0x7f42980b5c70 2024-09-17T03:36:57.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f4284017070 con 0x7f42980b5c70 2024-09-17T03:36:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f4294c00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4278075570 0x7f4278077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4885+0+0 (secure 0 0 0) 0x7f4284093680 con 0x7f42980b5c70 2024-09-17T03:36:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.649+0000 7f429cc00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4278075570 0x7f4278077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.653+0000 7f429cc00640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4278075570 0x7f4278077a80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f428c0098f0 tx=0x7f428c006f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:57.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.653+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f428405fab0 con 0x7f42980b5c70 2024-09-17T03:36:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.889+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f429815fd90 con 0x7f42980b5c70 2024-09-17T03:36:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.889+0000 7f4294c00640 1 -- 172.21.15.46:0/1462701927 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v44) ==== 80+0+5304 (secure 0 0 0) 0x7f42840644e0 con 0x7f42980b5c70 2024-09-17T03:36:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUxE5CRAAEPrllSLOYLagmD52YaJz0w==", 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmLHV1FhAAtJhcJ8lQ7bUCHLesalxY8g==", 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm6VcjERAAK7OTOQ/BvG3mTJ9Y+5OT7g==", 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmlks0HhAA7c8Rs4pF/R3+zUNInrsvmg==", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBj9ehmfrisCRAAiTxzMHCrLuIVkQxkbas8yA==", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBm9ehmrY8mIBAAv9arDKb2VdCkv1pBYZwR+Q==", 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.895 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBp9ehmhszPNhAAmX5tKKVyNgGKVCuzFU6RtA==", 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.896 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBt9ehmCge9DBAA1Efdth+I13UeZ+islzSdGA==", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBz9ehm3D5vHhAAltBengM3sAO2LUpv+15idQ==", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.897 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB29ehmubjWNBAAlphJf2ZqXTf7n/maH+AyRA==", 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.898 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB69ehm2MHHEBAAnpNot6pudLJ7q8ZNhU5g+g==", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQB99ehmHsdeKBAAb24LSUyLVyZTxAcrsZFF2A==", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-09-17T03:36:57.899 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ==", 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-09-17T03:36:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmXi4pKxAAuXmg3JcRLl1AgKFUE1+IEA==", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm80w+JxAA1/faMHoiOTq0PnaC0Eexmg==", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:36:57.901 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehm5lM9LxAAB9bC3NT0Fd743JHk5pQyFQ==", 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmDFP3JRAAOguRoANFv2WyqzNGqTt+bw==", 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-09-17T03:36:57.903 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBh9ehmUALxAxAAUKUA7SCuBpX5GPtwoAGKjw==", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBg9ehmnIIuOxAA01qL0U2T2HbC1fOOWe7Qaw==", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-09-17T03:36:57.904 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-09-17T03:36:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:57.905 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-17T03:36:57.905 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4278075570 msgr2=0x7f4278077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4278075570 0x7f4278077a80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f428c0098f0 tx=0x7f428c006f90 comp rx=0 tx=0).stop 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 msgr2=0x7f429814ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 0x7f429814ec10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f428400c970 tx=0x7f428400ce30 comp rx=0 tx=0).stop 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 shutdown_connections 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4278075570 0x7f4278077a80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42981527b0 0x7f429814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42980b5c70 0x7f429814ec10 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 --2- 172.21.15.46:0/1462701927 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42980b4de0 0x7f42980aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 >> 172.21.15.46:0/1462701927 conn(0x7f42980b2040 msgr2=0x7f42980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 shutdown_connections 2024-09-17T03:36:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:57.893+0000 7f429f600640 1 -- 172.21.15.46:0/1462701927 wait complete. 2024-09-17T03:36:57.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 -- 172.21.15.46:0/2118062900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b4de0 msgr2=0x7f47e80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 --2- 172.21.15.46:0/2118062900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b4de0 0x7f47e80b51f0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009bc0 tx=0x7f47e001cae0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 -- 172.21.15.46:0/2118062900 shutdown_connections 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 --2- 172.21.15.46:0/2118062900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b4de0 0x7f47e80b51f0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 --2- 172.21.15.46:0/2118062900 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47e81527b0 0x7f47e815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 -- 172.21.15.46:0/2118062900 >> 172.21.15.46:0/2118062900 conn(0x7f47e80b2040 msgr2=0x7f47e80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 -- 172.21.15.46:0/2118062900 shutdown_connections 2024-09-17T03:36:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.073+0000 7f47ede00640 1 -- 172.21.15.46:0/2118062900 wait complete. 2024-09-17T03:36:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 Processor -- start 2024-09-17T03:36:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 -- start start 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e80b4de0 0x7f47e8156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 0x7f47e81428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47e81527b0 0x7f47e80af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8168a00 con 0x7f47e80b5c70 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8168880 con 0x7f47e80b4de0 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.077+0000 7f47ede00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f47e8168b80 con 0x7f47e81527b0 2024-09-17T03:36:58.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 0x7f47e81428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 0x7f47e81428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46664/0 (socket says 172.21.15.46:46664) 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 -- 172.21.15.46:0/918678959 learned_addr learned my addr 172.21.15.46:0/918678959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47e7400640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47e81527b0 0x7f47e80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47e6a00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e80b4de0 0x7f47e8156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 -- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47e81527b0 msgr2=0x7f47e80af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47e81527b0 0x7f47e80af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 -- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e80b4de0 msgr2=0x7f47e8156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e80b4de0 0x7f47e8156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 -- 172.21.15.46:0/918678959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e8157070 con 0x7f47e80b5c70 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47de000640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 0x7f47e81428c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f47d000e790 tx=0x7f47d000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d000eea0 con 0x7f47e80b5c70 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47d000cc60 con 0x7f47e80b5c70 2024-09-17T03:36:58.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e8157580 con 0x7f47e80b5c70 2024-09-17T03:36:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0005280 con 0x7f47e80b5c70 2024-09-17T03:36:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e8157b40 con 0x7f47e80b5c70 2024-09-17T03:36:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e81572f0 con 0x7f47e80b5c70 2024-09-17T03:36:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.081+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f47d0005420 con 0x7f47e80b5c70 2024-09-17T03:36:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.085+0000 7f47dfe00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f47c4075570 0x7f47c4077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.085+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4885+0+0 (secure 0 0 0) 0x7f47d0093c40 con 0x7f47e80b5c70 2024-09-17T03:36:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.085+0000 7f47e6a00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f47c4075570 0x7f47c4077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.085+0000 7f47e6a00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f47c4075570 0x7f47c4077a80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f47d8007d70 tx=0x7f47d8007d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:58.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.089+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f47d0060070 con 0x7f47e80b5c70 2024-09-17T03:36:58.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.265+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f47e80adcd0 con 0x7f47e80b5c70 2024-09-17T03:36:58.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.265+0000 7f47dfe00640 1 -- 172.21.15.46:0/918678959 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) ==== 84+0+65 (secure 0 0 0) 0x7f47e80adcd0 con 0x7f47e80b5c70 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:36:58.266 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f47c4075570 msgr2=0x7f47c4077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f47c4075570 0x7f47c4077a80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f47d8007d70 tx=0x7f47d8007d00 comp rx=0 tx=0).stop 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 msgr2=0x7f47e81428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 0x7f47e81428c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f47d000e790 tx=0x7f47d000ec50 comp rx=0 tx=0).stop 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 shutdown_connections 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f47c4075570 0x7f47c4077a80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f47e81527b0 0x7f47e80af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47e80b5c70 0x7f47e81428c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 --2- 172.21.15.46:0/918678959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47e80b4de0 0x7f47e8156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 >> 172.21.15.46:0/918678959 conn(0x7f47e80b2040 msgr2=0x7f47e80b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 shutdown_connections 2024-09-17T03:36:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.269+0000 7f47ede00640 1 -- 172.21.15.46:0/918678959 wait complete. 2024-09-17T03:36:58.289 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:36:58.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 -- 172.21.15.46:0/1986401312 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d24f0 msgr2=0x7f7f180d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1986401312 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d24f0 0x7f7f180d2900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009bf0 tx=0x7f7f1001cb50 comp rx=0 tx=0).stop 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 -- 172.21.15.46:0/1986401312 shutdown_connections 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1986401312 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d24f0 0x7f7f180d2900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 -- 172.21.15.46:0/1986401312 >> 172.21.15.46:0/1986401312 conn(0x7f7f18073a10 msgr2=0x7f7f1806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 -- 172.21.15.46:0/1986401312 shutdown_connections 2024-09-17T03:36:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.453+0000 7f7f1d000640 1 -- 172.21.15.46:0/1986401312 wait complete. 2024-09-17T03:36:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.457+0000 7f7f1d000640 1 Processor -- start 2024-09-17T03:36:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.457+0000 7f7f1d000640 1 -- start start 2024-09-17T03:36:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d0ce0 0x7f7f1815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 0x7f7f181565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f181625f0 0x7f7f18156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1816d600 con 0x7f7f180d24f0 2024-09-17T03:36:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1816d480 con 0x7f7f181625f0 2024-09-17T03:36:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1816d780 con 0x7f7f180d0ce0 2024-09-17T03:36:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 0x7f7f181565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 0x7f7f181565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46672/0 (socket says 172.21.15.46:46672) 2024-09-17T03:36:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 -- 172.21.15.46:0/1827908194 learned_addr learned my addr 172.21.15.46:0/1827908194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f17400640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d0ce0 0x7f7f1815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f17e00640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f181625f0 0x7f7f18156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 -- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d0ce0 msgr2=0x7f7f1815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d0ce0 0x7f7f1815df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 -- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f181625f0 msgr2=0x7f7f18156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f181625f0 0x7f7f18156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 -- 172.21.15.46:0/1827908194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f180cab00 con 0x7f7f180d24f0 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f16a00640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 0x7f7f181565e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c430 tx=0x7f7f0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04017070 con 0x7f7f180d24f0 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f040053c0 con 0x7f7f180d24f0 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f180b6290 con 0x7f7f180d24f0 2024-09-17T03:36:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04003860 con 0x7f7f180d24f0 2024-09-17T03:36:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f180b6d70 con 0x7f7f180d24f0 2024-09-17T03:36:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.461+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f180d2900 con 0x7f7f180d24f0 2024-09-17T03:36:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.465+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f7f0401a070 con 0x7f7f180d24f0 2024-09-17T03:36:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.465+0000 7f7f0f400640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ef8075570 0x7f7ef8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.465+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4885+0+0 (secure 0 0 0) 0x7f7f040925e0 con 0x7f7f180d24f0 2024-09-17T03:36:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.465+0000 7f7f17400640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ef8075570 0x7f7ef8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.465+0000 7f7f17400640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ef8075570 0x7f7ef8077a80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7f080046c0 tx=0x7f7f08007cd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:58.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.469+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f0405eb40 con 0x7f7f180d24f0 2024-09-17T03:36:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:58.645+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7f180766a0 con 0x7f7f180d24f0 2024-09-17T03:36:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.473+0000 7f7f0f400640 1 -- 172.21.15.46:0/1827908194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) ==== 138+0+0 (secure 0 0 0) 0x7f7f04063570 con 0x7f7f180d24f0 2024-09-17T03:36:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-09-17T03:36:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ef8075570 msgr2=0x7f7ef8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ef8075570 0x7f7ef8077a80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7f080046c0 tx=0x7f7f08007cd0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 msgr2=0x7f7f181565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 0x7f7f181565e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c430 tx=0x7f7f0400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 shutdown_connections 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7ef8075570 0x7f7ef8077a80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f181625f0 0x7f7f18156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180d24f0 0x7f7f181565e0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 --2- 172.21.15.46:0/1827908194 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7f180d0ce0 0x7f7f1815df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.477+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 >> 172.21.15.46:0/1827908194 conn(0x7f7f18073a10 msgr2=0x7f7f181698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.481+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 shutdown_connections 2024-09-17T03:36:59.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.481+0000 7f7f1d000640 1 -- 172.21.15.46:0/1827908194 wait complete. 2024-09-17T03:36:59.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:36:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 -- 172.21.15.46:0/1547230107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 msgr2=0x7fc5400b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 --2- 172.21.15.46:0/1547230107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc5400b51f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc5400b2ff0 tx=0x7fc52c01c7d0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 -- 172.21.15.46:0/1547230107 shutdown_connections 2024-09-17T03:36:59.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 --2- 172.21.15.46:0/1547230107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc5400b51f0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 --2- 172.21.15.46:0/1547230107 >> v2:172.21.15.99:3300/0 conn(0x7fc5400738b0 0x7fc540075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 -- 172.21.15.46:0/1547230107 >> 172.21.15.46:0/1547230107 conn(0x7fc5400b2040 msgr2=0x7fc5400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:36:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 -- 172.21.15.46:0/1547230107 shutdown_connections 2024-09-17T03:36:59.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 -- 172.21.15.46:0/1547230107 wait complete. 2024-09-17T03:36:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 Processor -- start 2024-09-17T03:36:59.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.681+0000 7fc547c00640 1 -- start start 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5400738b0 0x7fc540141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc54013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54015cc60 0x7fc5401fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc540168b20 con 0x7fc5400b4de0 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc540152a50 con 0x7fc54015cc60 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc540168ca0 con 0x7fc5400738b0 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc54013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc54013e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46682/0 (socket says 172.21.15.46:46682) 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 -- 172.21.15.46:0/2095855582 learned_addr learned my addr 172.21.15.46:0/2095855582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc545c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54015cc60 0x7fc5401fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc545200640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5400738b0 0x7fc540141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 -- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5400738b0 msgr2=0x7fc540141910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5400738b0 0x7fc540141910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 -- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54015cc60 msgr2=0x7fc5401fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:36:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54015cc60 0x7fc5401fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:36:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 -- 172.21.15.46:0/2095855582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5401fc580 con 0x7fc5400b4de0 2024-09-17T03:36:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53fe00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc54013e670 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc52c01ccb0 tx=0x7fc52c005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0b2040 con 0x7fc5400b4de0 2024-09-17T03:36:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc52c0bb070 con 0x7fc5400b4de0 2024-09-17T03:36:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c003600 con 0x7fc5400b4de0 2024-09-17T03:36:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5401fc860 con 0x7fc5400b4de0 2024-09-17T03:36:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.685+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc540157b80 con 0x7fc5400b4de0 2024-09-17T03:36:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.689+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc540075010 con 0x7fc5400b4de0 2024-09-17T03:36:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.689+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fc52c0037a0 con 0x7fc5400b4de0 2024-09-17T03:36:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.689+0000 7fc53cc00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc520075570 0x7fc520077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:36:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.689+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5255+0+0 (secure 0 0 0) 0x7fc52c12e1a0 con 0x7fc5400b4de0 2024-09-17T03:36:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.689+0000 7fc545200640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc520075570 0x7fc520077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:36:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.693+0000 7fc545200640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc520075570 0x7fc520077a80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc534009600 tx=0x7fc5340091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:36:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.693+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc52c0163f0 con 0x7fc5400b4de0 2024-09-17T03:36:59.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:36:59.869+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc5401fc310 con 0x7fc5400b4de0 2024-09-17T03:37:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.480+0000 7fc53cc00640 1 -- 172.21.15.46:0/2095855582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) ==== 170+0+0 (secure 0 0 0) 0x7fc52c0fa6e0 con 0x7fc5400b4de0 2024-09-17T03:37:00.483 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-09-17T03:37:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc520075570 msgr2=0x7fc520077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc520075570 0x7fc520077a80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc534009600 tx=0x7fc5340091e0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 msgr2=0x7fc54013e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc54013e670 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc52c01ccb0 tx=0x7fc52c005d20 comp rx=0 tx=0).stop 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 shutdown_connections 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc520075570 0x7fc520077a80 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54015cc60 0x7fc5401fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5400b4de0 0x7fc54013e670 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 --2- 172.21.15.46:0/2095855582 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc5400738b0 0x7fc540141910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 >> 172.21.15.46:0/2095855582 conn(0x7fc5400b2040 msgr2=0x7fc54015d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 shutdown_connections 2024-09-17T03:37:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.484+0000 7fc547c00640 1 -- 172.21.15.46:0/2095855582 wait complete. 2024-09-17T03:37:00.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:37:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 -- 172.21.15.46:0/32551490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40749a0 msgr2=0x7f3fd4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 --2- 172.21.15.46:0/32551490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40749a0 0x7f3fd4074db0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009870 tx=0x7f3fc401c860 comp rx=0 tx=0).stop 2024-09-17T03:37:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 -- 172.21.15.46:0/32551490 shutdown_connections 2024-09-17T03:37:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 --2- 172.21.15.46:0/32551490 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40749a0 0x7f3fd4074db0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 -- 172.21.15.46:0/32551490 >> 172.21.15.46:0/32551490 conn(0x7f3fd40b2040 msgr2=0x7f3fd40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 -- 172.21.15.46:0/32551490 shutdown_connections 2024-09-17T03:37:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 -- 172.21.15.46:0/32551490 wait complete. 2024-09-17T03:37:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 Processor -- start 2024-09-17T03:37:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.688+0000 7f3fd9400640 1 -- start start 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 0x7f3fd413c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd40b3340 0x7f3fd40aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40cefa0 0x7f3fd40abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd416b350 con 0x7f3fd40749a0 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd416b1d0 con 0x7f3fd40b3340 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd416b4d0 con 0x7f3fd40cefa0 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 0x7f3fd413c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 0x7f3fd413c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46696/0 (socket says 172.21.15.46:46696) 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 -- 172.21.15.46:0/3389089459 learned_addr learned my addr 172.21.15.46:0/3389089459 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd3200640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40cefa0 0x7f3fd40abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd40b3340 0x7f3fd40aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 -- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40cefa0 msgr2=0x7f3fd40abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40cefa0 0x7f3fd40abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 -- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd40b3340 msgr2=0x7f3fd40aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd40b3340 0x7f3fd40aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 -- 172.21.15.46:0/3389089459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd4142860 con 0x7f3fd40749a0 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd3200640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40cefa0 0x7f3fd40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd40b3340 0x7f3fd40aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd2800640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 0x7f3fd413c8e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4019470 tx=0x7f3fc4005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40b2040 con 0x7f3fd40749a0 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fc4004130 con 0x7f3fd40749a0 2024-09-17T03:37:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40044e0 con 0x7f3fd40749a0 2024-09-17T03:37:00.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd40ae040 con 0x7f3fd40749a0 2024-09-17T03:37:00.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd40ae600 con 0x7f3fd40749a0 2024-09-17T03:37:00.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.692+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd4074db0 con 0x7f3fd40749a0 2024-09-17T03:37:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.696+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f3fc40bd030 con 0x7f3fd40749a0 2024-09-17T03:37:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.696+0000 7f3fcb400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fb0075710 0x7f3fb0077c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.696+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5637+0+0 (secure 0 0 0) 0x7f3fc412da00 con 0x7f3fd40749a0 2024-09-17T03:37:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.696+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fb0075710 0x7f3fb0077c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.696+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fb0075710 0x7f3fb0077c20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4142720 tx=0x7f3fbc009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.700+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3fc40c4020 con 0x7f3fd40749a0 2024-09-17T03:37:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:00.908+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3fd4072350 con 0x7f3fd40749a0 2024-09-17T03:37:00.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:37:00.965 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:37:01.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.484+0000 7f3fcb400640 1 -- 172.21.15.46:0/3389089459 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v90) ==== 174+0+0 (secure 0 0 0) 0x7f3fc4016450 con 0x7f3fd40749a0 2024-09-17T03:37:01.485 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 20 and data pool 21 2024-09-17T03:37:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fb0075710 msgr2=0x7f3fb0077c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fb0075710 0x7f3fb0077c20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4142720 tx=0x7f3fbc009040 comp rx=0 tx=0).stop 2024-09-17T03:37:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 msgr2=0x7f3fd413c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 0x7f3fd413c8e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4019470 tx=0x7f3fc4005d20 comp rx=0 tx=0).stop 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 shutdown_connections 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3fb0075710 0x7f3fb0077c20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3fd40cefa0 0x7f3fd40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd40b3340 0x7f3fd40aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3389089459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd40749a0 0x7f3fd413c8e0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 >> 172.21.15.46:0/3389089459 conn(0x7f3fd40b2040 msgr2=0x7f3fd40cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 shutdown_connections 2024-09-17T03:37:01.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.488+0000 7f3fd9400640 1 -- 172.21.15.46:0/3389089459 wait complete. 2024-09-17T03:37:01.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:37:01.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 -- 172.21.15.46:0/3179227223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2781527b0 msgr2=0x7fb27815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 --2- 172.21.15.46:0/3179227223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2781527b0 0x7fb27815bb60 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb278074b20 tx=0x7fb26001c760 comp rx=0 tx=0).stop 2024-09-17T03:37:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 -- 172.21.15.46:0/3179227223 shutdown_connections 2024-09-17T03:37:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 --2- 172.21.15.46:0/3179227223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2781527b0 0x7fb27815bb60 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 -- 172.21.15.46:0/3179227223 >> 172.21.15.46:0/3179227223 conn(0x7fb2780b2040 msgr2=0x7fb2780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 -- 172.21.15.46:0/3179227223 shutdown_connections 2024-09-17T03:37:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.700+0000 7fb27e200640 1 -- 172.21.15.46:0/3179227223 wait complete. 2024-09-17T03:37:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 Processor -- start 2024-09-17T03:37:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 -- start start 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 0x7fb278151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2780b5c70 0x7fb27814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781527b0 0x7fb27814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb278169b60 con 0x7fb2780b4de0 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781699e0 con 0x7fb2781527b0 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb278169ce0 con 0x7fb2780b5c70 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 0x7fb278151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 0x7fb278151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46716/0 (socket says 172.21.15.46:46716) 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 -- 172.21.15.46:0/1786826492 learned_addr learned my addr 172.21.15.46:0/1786826492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277e00640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781527b0 0x7fb27814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb26fe00640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2780b5c70 0x7fb27814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 -- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2780b5c70 msgr2=0x7fb27814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2780b5c70 0x7fb27814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 -- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781527b0 msgr2=0x7fb27814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781527b0 0x7fb27814e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 -- 172.21.15.46:0/1786826492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2781f9b50 con 0x7fb2780b4de0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb26fe00640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2780b5c70 0x7fb27814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb277400640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 0x7fb278151800 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c420 tx=0x7fb26800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb2780b4de0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2680053c0 con 0x7fb2780b4de0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781f9e30 con 0x7fb2780b4de0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268003860 con 0x7fb2780b4de0 2024-09-17T03:37:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.704+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781fa3f0 con 0x7fb2780b4de0 2024-09-17T03:37:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.708+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27815bb60 con 0x7fb2780b4de0 2024-09-17T03:37:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.708+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fb26801a070 con 0x7fb2780b4de0 2024-09-17T03:37:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.708+0000 7fb26f400640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2540755a0 0x7fb254077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.708+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fb268092740 con 0x7fb2780b4de0 2024-09-17T03:37:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.708+0000 7fb26fe00640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2540755a0 0x7fb254077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.708+0000 7fb26fe00640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2540755a0 0x7fb254077ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb2600af000 tx=0x7fb26001afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.712+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb26805f260 con 0x7fb2780b4de0 2024-09-17T03:37:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.888+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb278155cc0 con 0x7fb2780b4de0 2024-09-17T03:37:01.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb26f400640 1 -- 172.21.15.46:0/1786826492 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16760 (secure 0 0 0) 0x7fb26805eb60 con 0x7fb2780b4de0 2024-09-17T03:37:01.892 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:01.892 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":142,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:37:00.912115+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:36:58.659502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T03:36:59.882404+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:37:01.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2540755a0 msgr2=0x7fb254077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2540755a0 0x7fb254077ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb2600af000 tx=0x7fb26001afa0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 msgr2=0x7fb278151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 0x7fb278151800 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c420 tx=0x7fb26800c8e0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 shutdown_connections 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb2540755a0 0x7fb254077ab0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781527b0 0x7fb27814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2780b5c70 0x7fb27814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 --2- 172.21.15.46:0/1786826492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2780b4de0 0x7fb278151800 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 >> 172.21.15.46:0/1786826492 conn(0x7fb2780b2040 msgr2=0x7fb2780cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 shutdown_connections 2024-09-17T03:37:01.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:01.892+0000 7fb27e200640 1 -- 172.21.15.46:0/1786826492 wait complete. 2024-09-17T03:37:01.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 -- 172.21.15.46:0/2445974416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200815f5c0 msgr2=0x7f200815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 --2- 172.21.15.46:0/2445974416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200815f5c0 0x7f200815f9d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f20080ced20 tx=0x7f200001caf0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 -- 172.21.15.46:0/2445974416 shutdown_connections 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 --2- 172.21.15.46:0/2445974416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200815f5c0 0x7f200815f9d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 --2- 172.21.15.46:0/2445974416 >> v2:172.21.15.99:3300/0 conn(0x7f20080d1140 0x7f20080b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 -- 172.21.15.46:0/2445974416 >> 172.21.15.46:0/2445974416 conn(0x7f20080b41f0 msgr2=0x7f20080b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 -- 172.21.15.46:0/2445974416 shutdown_connections 2024-09-17T03:37:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.072+0000 7f200e800640 1 -- 172.21.15.46:0/2445974416 wait complete. 2024-09-17T03:37:02.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 Processor -- start 2024-09-17T03:37:02.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 -- start start 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f200806dbc0 0x7f200815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 0x7f20081f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f200815f5c0 0x7f20080cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2008168ff0 con 0x7f20080d1140 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2008168e70 con 0x7f200806dbc0 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2008169170 con 0x7f200815f5c0 2024-09-17T03:37:02.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 0x7f20081f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 0x7f20081f9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46744/0 (socket says 172.21.15.46:46744) 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 -- 172.21.15.46:0/319666486 learned_addr learned my addr 172.21.15.46:0/319666486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200d400640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f200806dbc0 0x7f200815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200de00640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f200815f5c0 0x7f20080cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 -- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f200815f5c0 msgr2=0x7f20080cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f200815f5c0 0x7f20080cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 -- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f200806dbc0 msgr2=0x7f200815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f200806dbc0 0x7f200815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 -- 172.21.15.46:0/319666486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20081fce20 con 0x7f20080d1140 2024-09-17T03:37:02.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.076+0000 7f200ca00640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 0x7f20081f9930 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c430 tx=0x7f1ff400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:02.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4011790 con 0x7f20080d1140 2024-09-17T03:37:02.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ff4017070 con 0x7f20080d1140 2024-09-17T03:37:02.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20081fd100 con 0x7f20080d1140 2024-09-17T03:37:02.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4011cc0 con 0x7f20080d1140 2024-09-17T03:37:02.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081f9410 con 0x7f20080d1140 2024-09-17T03:37:02.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20080b0730 con 0x7f20080d1140 2024-09-17T03:37:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f1ff401c460 con 0x7f20080d1140 2024-09-17T03:37:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.080+0000 7f2005600640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1fe80755a0 0x7f1fe8077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.084+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f1ff4093040 con 0x7f20080d1140 2024-09-17T03:37:02.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.084+0000 7f200d400640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1fe80755a0 0x7f1fe8077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.084+0000 7f200d400640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1fe80755a0 0x7f1fe8077ab0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009b40 tx=0x7f1ffc0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:02.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.088+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1ff405f510 con 0x7f20080d1140 2024-09-17T03:37:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.264+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2008076200 con 0x7f20080d1140 2024-09-17T03:37:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.264+0000 7f2005600640 1 -- 172.21.15.46:0/319666486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16760 (secure 0 0 0) 0x7f1ff4063f40 con 0x7f20080d1140 2024-09-17T03:37:02.267 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:02.267 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":142,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:37:00.912115+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:36:58.659502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T03:36:59.882404+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1fe80755a0 msgr2=0x7f1fe8077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1fe80755a0 0x7f1fe8077ab0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009b40 tx=0x7f1ffc0091e0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 msgr2=0x7f20081f9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 0x7f20081f9930 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c430 tx=0x7f1ff400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 shutdown_connections 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1fe80755a0 0x7f1fe8077ab0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f200815f5c0 0x7f20080cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20080d1140 0x7f20081f9930 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 --2- 172.21.15.46:0/319666486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f200806dbc0 0x7f200815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 >> 172.21.15.46:0/319666486 conn(0x7f20080b41f0 msgr2=0x7f20080b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 shutdown_connections 2024-09-17T03:37:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.268+0000 7f200e800640 1 -- 172.21.15.46:0/319666486 wait complete. 2024-09-17T03:37:02.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:37:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 -- 172.21.15.46:0/394420046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 msgr2=0x7f78500b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 --2- 172.21.15.46:0/394420046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f78500b51f0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f78500703e0 tx=0x7f784801ccc0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 -- 172.21.15.46:0/394420046 shutdown_connections 2024-09-17T03:37:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 --2- 172.21.15.46:0/394420046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f78500b51f0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 --2- 172.21.15.46:0/394420046 >> v2:172.21.15.99:3300/0 conn(0x7f78501527b0 0x7f7850074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 --2- 172.21.15.46:0/394420046 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78500b5c70 0x7f785015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 -- 172.21.15.46:0/394420046 >> 172.21.15.46:0/394420046 conn(0x7f78500b2040 msgr2=0x7f78500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 -- 172.21.15.46:0/394420046 shutdown_connections 2024-09-17T03:37:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 -- 172.21.15.46:0/394420046 wait complete. 2024-09-17T03:37:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 Processor -- start 2024-09-17T03:37:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.452+0000 7f7858200640 1 -- start start 2024-09-17T03:37:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f7850157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500b5c70 0x7f78501510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78501527b0 0x7f7850156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f785015f730 con 0x7f78500b4de0 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f785015f5b0 con 0x7f78500b5c70 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f785015f8b0 con 0x7f78501527b0 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f7850157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f7850157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52250/0 (socket says 172.21.15.46:52250) 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 -- 172.21.15.46:0/820245576 learned_addr learned my addr 172.21.15.46:0/820245576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7854e00640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500b5c70 0x7f78501510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7856200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78501527b0 0x7f7850156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 -- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78501527b0 msgr2=0x7f7850156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78501527b0 0x7f7850156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 -- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500b5c70 msgr2=0x7f78501510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500b5c70 0x7f78501510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 -- 172.21.15.46:0/820245576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7850156260 con 0x7f78500b4de0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7856200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78501527b0 0x7f7850156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7855800640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f7850157f10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f784400d6a0 tx=0x7f784400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844017070 con 0x7f78500b4de0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7844004310 con 0x7f78500b4de0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501515e0 con 0x7f78500b4de0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844005280 con 0x7f78500b4de0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7850156500 con 0x7f78500b4de0 2024-09-17T03:37:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.456+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785015fdd0 con 0x7f78500b4de0 2024-09-17T03:37:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.460+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f7844005420 con 0x7f78500b4de0 2024-09-17T03:37:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.460+0000 7f784d600640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78300755a0 0x7f7830077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.460+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f7844093740 con 0x7f78500b4de0 2024-09-17T03:37:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.460+0000 7f7854e00640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78300755a0 0x7f7830077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.460+0000 7f7854e00640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78300755a0 0x7f7830077ab0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f78501523f0 tx=0x7f783c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.464+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7844020020 con 0x7f78500b4de0 2024-09-17T03:37:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:02.680+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f785015e430 con 0x7f78500b4de0 2024-09-17T03:37:03.463 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-09-17T03:37:03.464+0000 7f76a3e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:37:03.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f784d600640 1 -- 172.21.15.46:0/820245576 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) ==== 118+0+0 (secure 0 0 0) 0x7f7844060250 con 0x7f78500b4de0 2024-09-17T03:37:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78300755a0 msgr2=0x7f7830077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78300755a0 0x7f7830077ab0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f78501523f0 tx=0x7f783c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 msgr2=0x7f7850157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f7850157f10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f784400d6a0 tx=0x7f784400db60 comp rx=0 tx=0).stop 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 shutdown_connections 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f78300755a0 0x7f7830077ab0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f78501527b0 0x7f7850156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78500b5c70 0x7f78501510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 --2- 172.21.15.46:0/820245576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78500b4de0 0x7f7850157f10 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 >> 172.21.15.46:0/820245576 conn(0x7f78500b2040 msgr2=0x7f78500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 shutdown_connections 2024-09-17T03:37:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.524+0000 7f7858200640 1 -- 172.21.15.46:0/820245576 wait complete. 2024-09-17T03:37:03.552 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:37:03.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 -- 172.21.15.46:0/2628859333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 msgr2=0x7f578815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 --2- 172.21.15.46:0/2628859333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 0x7f578815bb60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5778009840 tx=0x7f577801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 -- 172.21.15.46:0/2628859333 shutdown_connections 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 --2- 172.21.15.46:0/2628859333 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57880b4de0 0x7f57880b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 --2- 172.21.15.46:0/2628859333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 0x7f578815bb60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 -- 172.21.15.46:0/2628859333 >> 172.21.15.46:0/2628859333 conn(0x7f57880b2040 msgr2=0x7f57880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 -- 172.21.15.46:0/2628859333 shutdown_connections 2024-09-17T03:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 -- 172.21.15.46:0/2628859333 wait complete. 2024-09-17T03:37:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 Processor -- start 2024-09-17T03:37:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.720+0000 7f5790800640 1 -- start start 2024-09-17T03:37:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57880b4de0 0x7f57880ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 0x7f5788151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 0x7f57880cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5788168950 con 0x7f57880b5c70 2024-09-17T03:37:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57881687d0 con 0x7f57881527b0 2024-09-17T03:37:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5788168ad0 con 0x7f57880b4de0 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 0x7f5788151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 0x7f5788151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52260/0 (socket says 172.21.15.46:52260) 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 -- 172.21.15.46:0/1633874124 learned_addr learned my addr 172.21.15.46:0/1633874124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578de00640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57880b4de0 0x7f57880ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578e800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 0x7f57880cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 -- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57880b4de0 msgr2=0x7f57880ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57880b4de0 0x7f57880ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 -- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 msgr2=0x7f57880cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 0x7f57880cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 -- 172.21.15.46:0/1633874124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5788151ab0 con 0x7f57880b5c70 2024-09-17T03:37:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f578d400640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 0x7f5788151570 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5778000c10 tx=0x7f5778005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:03.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57780b2040 con 0x7f57880b5c70 2024-09-17T03:37:03.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5778004330 con 0x7f57880b5c70 2024-09-17T03:37:03.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57880c5500 con 0x7f57880b5c70 2024-09-17T03:37:03.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778003800 con 0x7f57880b5c70 2024-09-17T03:37:03.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57880c5ac0 con 0x7f57880b5c70 2024-09-17T03:37:03.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.724+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57880b51f0 con 0x7f57880b5c70 2024-09-17T03:37:03.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.728+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f57780bd060 con 0x7f57880b5c70 2024-09-17T03:37:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.728+0000 7f5782000640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f576c075570 0x7f576c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.728+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f577812cb50 con 0x7f57880b5c70 2024-09-17T03:37:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.728+0000 7f578de00640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f576c075570 0x7f576c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.728+0000 7f578de00640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f576c075570 0x7f576c077a80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f577c0017f0 tx=0x7f577c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:03.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.732+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5788151ab0 con 0x7f57880b5c70 2024-09-17T03:37:03.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:03.944+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f57880ce9b0 con 0x7f57880b5c70 2024-09-17T03:37:04.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.528+0000 7f5782000640 1 -- 172.21.15.46:0/1633874124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) ==== 122+0+0 (secure 0 0 0) 0x7f5778016410 con 0x7f57880b5c70 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f576c075570 msgr2=0x7f576c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f576c075570 0x7f576c077a80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f577c0017f0 tx=0x7f577c009040 comp rx=0 tx=0).stop 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 msgr2=0x7f5788151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 0x7f5788151570 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5778000c10 tx=0x7f5778005db0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 shutdown_connections 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f576c075570 0x7f576c077a80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57881527b0 0x7f57880cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57880b5c70 0x7f5788151570 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 --2- 172.21.15.46:0/1633874124 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57880b4de0 0x7f57880ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 >> 172.21.15.46:0/1633874124 conn(0x7f57880b2040 msgr2=0x7f57880b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 shutdown_connections 2024-09-17T03:37:04.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.532+0000 7f5790800640 1 -- 172.21.15.46:0/1633874124 wait complete. 2024-09-17T03:37:04.555 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:37:04.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:37:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 -- 172.21.15.46:0/2500074980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56c003a10 msgr2=0x7fd56c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 --2- 172.21.15.46:0/2500074980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd56c003a10 0x7fd56c003e80 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd57800e400 tx=0x7fd57801d790 comp rx=0 tx=0).stop 2024-09-17T03:37:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58d000640 1 -- 172.21.15.46:0/2500074980 reap_dead start 2024-09-17T03:37:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 -- 172.21.15.46:0/2500074980 shutdown_connections 2024-09-17T03:37:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 -- 172.21.15.46:0/2500074980 >> 172.21.15.46:0/2500074980 conn(0x7fd5880708c0 msgr2=0x7fd588070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 -- 172.21.15.46:0/2500074980 shutdown_connections 2024-09-17T03:37:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 -- 172.21.15.46:0/2500074980 wait complete. 2024-09-17T03:37:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 Processor -- start 2024-09-17T03:37:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.716+0000 7fd58f000640 1 -- start start 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd56c003a10 0x7fd5881425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 0x7fd588142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5881fc670 0x7fd5881fca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd588168300 con 0x7fd5881f9850 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd588168180 con 0x7fd5881fc670 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd588168480 con 0x7fd56c003a10 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 0x7fd588142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 0x7fd588142ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52278/0 (socket says 172.21.15.46:52278) 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 -- 172.21.15.46:0/2059743677 learned_addr learned my addr 172.21.15.46:0/2059743677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd587e00640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd56c003a10 0x7fd5881425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58d000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5881fc670 0x7fd5881fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 -- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd56c003a10 msgr2=0x7fd5881425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd56c003a10 0x7fd5881425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 -- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5881fc670 msgr2=0x7fd5881fca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5881fc670 0x7fd5881fca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 -- 172.21.15.46:0/2059743677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5881661e0 con 0x7fd5881f9850 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd587e00640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd56c003a10 0x7fd5881425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58d000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5881fc670 0x7fd5881fca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd57f400640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 0x7fd588142ae0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd570006600 tx=0x7fd5700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570015070 con 0x7fd5881f9850 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd570011920 con 0x7fd5881f9850 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881664c0 con 0x7fd5881f9850 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570011d70 con 0x7fd5881f9850 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881431f0 con 0x7fd5881f9850 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.720+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd588166700 con 0x7fd5881f9850 2024-09-17T03:37:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.724+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fd570018070 con 0x7fd5881f9850 2024-09-17T03:37:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.724+0000 7fd584c00640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd5640755a0 0x7fd564077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.724+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7fd570094170 con 0x7fd5881f9850 2024-09-17T03:37:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.724+0000 7fd587e00640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd5640755a0 0x7fd564077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.724+0000 7fd587e00640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd5640755a0 0x7fd564077ab0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd5780af000 tx=0x7fd57801a530 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.728+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd588166700 con 0x7fd5881f9850 2024-09-17T03:37:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.940+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5881f9df0 con 0x7fd5881f9850 2024-09-17T03:37:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.940+0000 7fd584c00640 1 -- 172.21.15.46:0/2059743677 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5243 (secure 0 0 0) 0x7fd570060c00 con 0x7fd5881f9850 2024-09-17T03:37:04.941 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:04.941 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"btime":"2024-09-17T03:37:04:527000+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:37:00.912074+0000","modified":"2024-09-17T03:37:03.946655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248758":{"gid":248758,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/2038056563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2038056563},{"type":"v1","addr":"172.21.15.46:6837","nonce":2038056563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248758,"qdb_cluster":[248758]},"id":10}]} 2024-09-17T03:37:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2024-09-17T03:37:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.940+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd5640755a0 msgr2=0x7fd564077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.940+0000 7fd58f000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd5640755a0 0x7fd564077ab0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd5780af000 tx=0x7fd57801a530 comp rx=0 tx=0).stop 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.940+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 msgr2=0x7fd588142ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.940+0000 7fd58f000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 0x7fd588142ae0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd570006600 tx=0x7fd5700039d0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 shutdown_connections 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd5640755a0 0x7fd564077ab0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5881fc670 0x7fd5881fca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5881f9850 0x7fd588142ae0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 --2- 172.21.15.46:0/2059743677 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd56c003a10 0x7fd5881425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 >> 172.21.15.46:0/2059743677 conn(0x7fd5880708c0 msgr2=0x7fd58815dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 shutdown_connections 2024-09-17T03:37:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:04.944+0000 7fd58f000640 1 -- 172.21.15.46:0/2059743677 wait complete. 2024-09-17T03:37:04.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 -- 172.21.15.46:0/3131743733 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc1654a0 msgr2=0x7f01cc1678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 --2- 172.21.15.46:0/3131743733 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc1654a0 0x7f01cc1678c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009bf0 tx=0x7f01c401cbe0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 -- 172.21.15.46:0/3131743733 shutdown_connections 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 --2- 172.21.15.46:0/3131743733 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc1654a0 0x7f01cc1678c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 -- 172.21.15.46:0/3131743733 >> 172.21.15.46:0/3131743733 conn(0x7f01cc0b2040 msgr2=0x7f01cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 -- 172.21.15.46:0/3131743733 shutdown_connections 2024-09-17T03:37:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.120+0000 7f01d2200640 1 -- 172.21.15.46:0/3131743733 wait complete. 2024-09-17T03:37:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 Processor -- start 2024-09-17T03:37:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 -- start start 2024-09-17T03:37:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc0749a0 0x7f01cc167290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 0x7f01cc13c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01cc1654a0 0x7f01cc140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc16af00 con 0x7f01cc0cefa0 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc16ad80 con 0x7f01cc1654a0 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01d2200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f01cc16b080 con 0x7f01cc0749a0 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 0x7f01cc13c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01cbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01cc1654a0 0x7f01cc140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 0x7f01cc13c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52286/0 (socket says 172.21.15.46:52286) 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 -- 172.21.15.46:0/1824690861 learned_addr learned my addr 172.21.15.46:0/1824690861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01cb400640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc0749a0 0x7f01cc167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 -- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc0749a0 msgr2=0x7f01cc167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc0749a0 0x7f01cc167290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 -- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01cc1654a0 msgr2=0x7f01cc140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01cc1654a0 0x7f01cc140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.124+0000 7f01c3e00640 1 -- 172.21.15.46:0/1824690861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc0b2660 con 0x7f01cc0cefa0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01c3e00640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 0x7f01cc13c650 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c3a0 tx=0x7f01b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4017070 con 0x7f01cc0cefa0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01b4003600 con 0x7f01cc0cefa0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4003a50 con 0x7f01cc0cefa0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1f7910 con 0x7f01cc0cefa0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1f6e30 con 0x7f01cc0cefa0 2024-09-17T03:37:05.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01cc074db0 con 0x7f01cc0cefa0 2024-09-17T03:37:05.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.128+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f01b401a070 con 0x7f01cc0cefa0 2024-09-17T03:37:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.132+0000 7f01c3400640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01a8075570 0x7f01a8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.132+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f01b40926a0 con 0x7f01cc0cefa0 2024-09-17T03:37:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.132+0000 7f01cb400640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01a8075570 0x7f01a8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.132+0000 7f01cb400640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01a8075570 0x7f01a8077a80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f01bc001870 tx=0x7f01bc001380 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:05.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.132+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f01b405ebe0 con 0x7f01cc0cefa0 2024-09-17T03:37:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.308+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01cc1678c0 con 0x7f01cc0cefa0 2024-09-17T03:37:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01c3400640 1 -- 172.21.15.46:0/1824690861 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16747 (secure 0 0 0) 0x7f01b4063610 con 0x7f01cc0cefa0 2024-09-17T03:37:05.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:05.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":143,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:37:02.501342+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:36:58.659502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T03:36:59.882404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:37:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01a8075570 msgr2=0x7f01a8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01a8075570 0x7f01a8077a80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f01bc001870 tx=0x7f01bc001380 comp rx=0 tx=0).stop 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 msgr2=0x7f01cc13c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 0x7f01cc13c650 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c3a0 tx=0x7f01b400c860 comp rx=0 tx=0).stop 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 shutdown_connections 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f01a8075570 0x7f01a8077a80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01cc1654a0 0x7f01cc140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01cc0cefa0 0x7f01cc13c650 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 --2- 172.21.15.46:0/1824690861 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f01cc0749a0 0x7f01cc167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 >> 172.21.15.46:0/1824690861 conn(0x7f01cc0b2040 msgr2=0x7f01cc06e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 shutdown_connections 2024-09-17T03:37:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.312+0000 7f01d2200640 1 -- 172.21.15.46:0/1824690861 wait complete. 2024-09-17T03:37:05.335 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:37:05.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:37:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 -- 172.21.15.46:0/850104376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0157030 msgr2=0x7fd2b00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 --2- 172.21.15.46:0/850104376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0157030 0x7fd2b00b5a80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009840 tx=0x7fd2a001c820 comp rx=0 tx=0).stop 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 -- 172.21.15.46:0/850104376 shutdown_connections 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 --2- 172.21.15.46:0/850104376 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b0160d50 0x7fd2b0074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 --2- 172.21.15.46:0/850104376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0157030 0x7fd2b00b5a80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 -- 172.21.15.46:0/850104376 >> 172.21.15.46:0/850104376 conn(0x7fd2b00708c0 msgr2=0x7fd2b0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 -- 172.21.15.46:0/850104376 shutdown_connections 2024-09-17T03:37:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 -- 172.21.15.46:0/850104376 wait complete. 2024-09-17T03:37:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 Processor -- start 2024-09-17T03:37:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.496+0000 7fd2b5000640 1 -- start start 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 0x7fd2b0144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b0160d50 0x7fd2b015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0161160 0x7fd2b015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b016a7e0 con 0x7fd2b0157030 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b016a660 con 0x7fd2b0161160 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b016a960 con 0x7fd2b0160d50 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 0x7fd2b0144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 0x7fd2b0144a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52300/0 (socket says 172.21.15.46:52300) 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 -- 172.21.15.46:0/3468678142 learned_addr learned my addr 172.21.15.46:0/3468678142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2a7e00640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b0160d50 0x7fd2b015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ae800640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0161160 0x7fd2b015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 -- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b0160d50 msgr2=0x7fd2b015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b0160d50 0x7fd2b015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 -- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0161160 msgr2=0x7fd2b015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0161160 0x7fd2b015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 -- 172.21.15.46:0/3468678142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b00d1300 con 0x7fd2b0157030 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ae800640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0161160 0x7fd2b015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2ade00640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 0x7fd2b0144a50 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd2a001cd00 tx=0x7fd2a00a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0004040 con 0x7fd2b0157030 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b00d15a0 con 0x7fd2b0157030 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2a00042d0 con 0x7fd2b0157030 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0003600 con 0x7fd2b0157030 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b00d1b60 con 0x7fd2b0157030 2024-09-17T03:37:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.500+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b00d2260 con 0x7fd2b0157030 2024-09-17T03:37:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.504+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fd2a00c1020 con 0x7fd2b0157030 2024-09-17T03:37:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.504+0000 7fd2a6000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd28c075570 0x7fd28c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.504+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7fd2a012e4e0 con 0x7fd2b0157030 2024-09-17T03:37:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.504+0000 7fd2a7e00640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd28c075570 0x7fd28c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.504+0000 7fd2a7e00640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd28c075570 0x7fd28c077a80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00b1940 tx=0x7fd2980091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.508+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd2a00c8020 con 0x7fd2b0157030 2024-09-17T03:37:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.680+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2b006b540 con 0x7fd28c075570 2024-09-17T03:37:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.688+0000 7fd2a6000640 1 -- 172.21.15.46:0/3468678142 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251833 (secure 0 0 0) 0x7fd298011c00 con 0x7fd28c075570 2024-09-17T03:37:05.691 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd28c075570 msgr2=0x7fd28c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd28c075570 0x7fd28c077a80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00b1940 tx=0x7fd2980091e0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 msgr2=0x7fd2b0144a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 0x7fd2b0144a50 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd2a001cd00 tx=0x7fd2a00a6600 comp rx=0 tx=0).stop 2024-09-17T03:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 shutdown_connections 2024-09-17T03:37:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd28c075570 0x7fd28c077a80 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2b0161160 0x7fd2b015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd2b0160d50 0x7fd2b015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 --2- 172.21.15.46:0/3468678142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0157030 0x7fd2b0144a50 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 >> 172.21.15.46:0/3468678142 conn(0x7fd2b00708c0 msgr2=0x7fd2b00b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 shutdown_connections 2024-09-17T03:37:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.692+0000 7fd2b5000640 1 -- 172.21.15.46:0/3468678142 wait complete. 2024-09-17T03:37:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-09-17T03:37:05.579171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1171856,"kb_used_data":14924,"kb_used_omap":12,"kb_used_meta":1156723,"kb_avail":753802864,"statfs":{"total":773094113280,"available":771894132736,"internally_reserved":0,"allocated":15282176,"data_stored":7349196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1184484912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.512539+0000","last_change":"2024-09-17T03:37:01.509979+0000","last_active":"2024-09-17T03:37:01.512539+0000","last_peered":"2024-09-17T03:37:01.512539+0000","last_clean":"2024-09-17T03:37:01.512539+0000","last_became_active":"2024-09-17T03:37:01.508226+0000","last_became_peered":"2024-09-17T03:37:01.508226+0000","last_unstale":"2024-09-17T03:37:01.512539+0000","last_undegraded":"2024-09-17T03:37:01.512539+0000","last_fullsized":"2024-09-17T03:37:01.512539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:52.973966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531593+0000","last_change":"2024-09-17T03:37:01.525396+0000","last_active":"2024-09-17T03:37:01.531593+0000","last_peered":"2024-09-17T03:37:01.531593+0000","last_clean":"2024-09-17T03:37:01.531593+0000","last_became_active":"2024-09-17T03:37:01.525068+0000","last_became_peered":"2024-09-17T03:37:01.525068+0000","last_unstale":"2024-09-17T03:37:01.531593+0000","last_undegraded":"2024-09-17T03:37:01.531593+0000","last_fullsized":"2024-09-17T03:37:01.531593+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:13.211290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519515+0000","last_change":"2024-09-17T03:37:01.517425+0000","last_active":"2024-09-17T03:37:01.519515+0000","last_peered":"2024-09-17T03:37:01.519515+0000","last_clean":"2024-09-17T03:37:01.519515+0000","last_became_active":"2024-09-17T03:37:01.517118+0000","last_became_peered":"2024-09-17T03:37:01.517118+0000","last_unstale":"2024-09-17T03:37:01.519515+0000","last_undegraded":"2024-09-17T03:37:01.519515+0000","last_fullsized":"2024-09-17T03:37:01.519515+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528816+0000","last_change":"2024-09-17T03:37:01.524178+0000","last_active":"2024-09-17T03:37:01.528816+0000","last_peered":"2024-09-17T03:37:01.528816+0000","last_clean":"2024-09-17T03:37:01.528816+0000","last_became_active":"2024-09-17T03:37:01.523558+0000","last_became_peered":"2024-09-17T03:37:01.523558+0000","last_unstale":"2024-09-17T03:37:01.528816+0000","last_undegraded":"2024-09-17T03:37:01.528816+0000","last_fullsized":"2024-09-17T03:37:01.528816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:24.391567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498567+0000","last_change":"2024-09-17T03:37:00.498567+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498567+0000","last_undegraded":"2024-09-17T03:37:00.498567+0000","last_fullsized":"2024-09-17T03:37:00.498567+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510156+0000","last_change":"2024-09-17T03:37:01.507556+0000","last_active":"2024-09-17T03:37:01.510156+0000","last_peered":"2024-09-17T03:37:01.510156+0000","last_clean":"2024-09-17T03:37:01.510156+0000","last_became_active":"2024-09-17T03:37:01.507216+0000","last_became_peered":"2024-09-17T03:37:01.507216+0000","last_unstale":"2024-09-17T03:37:01.510156+0000","last_undegraded":"2024-09-17T03:37:01.510156+0000","last_fullsized":"2024-09-17T03:37:01.510156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:58.827646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524913+0000","last_change":"2024-09-17T03:37:01.520291+0000","last_active":"2024-09-17T03:37:01.524913+0000","last_peered":"2024-09-17T03:37:01.524913+0000","last_clean":"2024-09-17T03:37:01.524913+0000","last_became_active":"2024-09-17T03:37:01.519510+0000","last_became_peered":"2024-09-17T03:37:01.519510+0000","last_unstale":"2024-09-17T03:37:01.524913+0000","last_undegraded":"2024-09-17T03:37:01.524913+0000","last_fullsized":"2024-09-17T03:37:01.524913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:14.128722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.515271+0000","last_change":"2024-09-17T03:37:01.511915+0000","last_active":"2024-09-17T03:37:01.515271+0000","last_peered":"2024-09-17T03:37:01.515271+0000","last_clean":"2024-09-17T03:37:01.515271+0000","last_became_active":"2024-09-17T03:37:01.511588+0000","last_became_peered":"2024-09-17T03:37:01.511588+0000","last_unstale":"2024-09-17T03:37:01.515271+0000","last_undegraded":"2024-09-17T03:37:01.515271+0000","last_fullsized":"2024-09-17T03:37:01.515271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:34.703407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518221+0000","last_change":"2024-09-17T03:37:01.504394+0000","last_active":"2024-09-17T03:37:01.518221+0000","last_peered":"2024-09-17T03:37:01.518221+0000","last_clean":"2024-09-17T03:37:01.518221+0000","last_became_active":"2024-09-17T03:37:01.504016+0000","last_became_peered":"2024-09-17T03:37:01.504016+0000","last_unstale":"2024-09-17T03:37:01.518221+0000","last_undegraded":"2024-09-17T03:37:01.518221+0000","last_fullsized":"2024-09-17T03:37:01.518221+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:47.918178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526096+0000","last_change":"2024-09-17T03:37:01.521538+0000","last_active":"2024-09-17T03:37:01.526096+0000","last_peered":"2024-09-17T03:37:01.526096+0000","last_clean":"2024-09-17T03:37:01.526096+0000","last_became_active":"2024-09-17T03:37:01.521253+0000","last_became_peered":"2024-09-17T03:37:01.521253+0000","last_unstale":"2024-09-17T03:37:01.526096+0000","last_undegraded":"2024-09-17T03:37:01.526096+0000","last_fullsized":"2024-09-17T03:37:01.526096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:33.873450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509897+0000","last_change":"2024-09-17T03:37:01.507075+0000","last_active":"2024-09-17T03:37:01.509897+0000","last_peered":"2024-09-17T03:37:01.509897+0000","last_clean":"2024-09-17T03:37:01.509897+0000","last_became_active":"2024-09-17T03:37:01.506711+0000","last_became_peered":"2024-09-17T03:37:01.506711+0000","last_unstale":"2024-09-17T03:37:01.509897+0000","last_undegraded":"2024-09-17T03:37:01.509897+0000","last_fullsized":"2024-09-17T03:37:01.509897+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:56.420785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.520006+0000","last_change":"2024-09-17T03:37:01.515856+0000","last_active":"2024-09-17T03:37:01.520006+0000","last_peered":"2024-09-17T03:37:01.520006+0000","last_clean":"2024-09-17T03:37:01.520006+0000","last_became_active":"2024-09-17T03:37:01.512742+0000","last_became_peered":"2024-09-17T03:37:01.512742+0000","last_unstale":"2024-09-17T03:37:01.520006+0000","last_undegraded":"2024-09-17T03:37:01.520006+0000","last_fullsized":"2024-09-17T03:37:01.520006+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:16.486830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498137+0000","last_change":"2024-09-17T03:37:00.498137+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498137+0000","last_undegraded":"2024-09-17T03:37:00.498137+0000","last_fullsized":"2024-09-17T03:37:00.498137+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534060+0000","last_change":"2024-09-17T03:37:01.530828+0000","last_active":"2024-09-17T03:37:01.534060+0000","last_peered":"2024-09-17T03:37:01.534060+0000","last_clean":"2024-09-17T03:37:01.534060+0000","last_became_active":"2024-09-17T03:37:01.529611+0000","last_became_peered":"2024-09-17T03:37:01.529611+0000","last_unstale":"2024-09-17T03:37:01.534060+0000","last_undegraded":"2024-09-17T03:37:01.534060+0000","last_fullsized":"2024-09-17T03:37:01.534060+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526083+0000","last_change":"2024-09-17T03:37:01.524406+0000","last_active":"2024-09-17T03:37:01.526083+0000","last_peered":"2024-09-17T03:37:01.526083+0000","last_clean":"2024-09-17T03:37:01.526083+0000","last_became_active":"2024-09-17T03:37:01.524097+0000","last_became_peered":"2024-09-17T03:37:01.524097+0000","last_unstale":"2024-09-17T03:37:01.526083+0000","last_undegraded":"2024-09-17T03:37:01.526083+0000","last_fullsized":"2024-09-17T03:37:01.526083+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:47.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534103+0000","last_change":"2024-09-17T03:37:01.527309+0000","last_active":"2024-09-17T03:37:01.534103+0000","last_peered":"2024-09-17T03:37:01.534103+0000","last_clean":"2024-09-17T03:37:01.534103+0000","last_became_active":"2024-09-17T03:37:01.526391+0000","last_became_peered":"2024-09-17T03:37:01.526391+0000","last_unstale":"2024-09-17T03:37:01.534103+0000","last_undegraded":"2024-09-17T03:37:01.534103+0000","last_fullsized":"2024-09-17T03:37:01.534103+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:42.503735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536631+0000","last_change":"2024-09-17T03:37:01.531294+0000","last_active":"2024-09-17T03:37:01.536631+0000","last_peered":"2024-09-17T03:37:01.536631+0000","last_clean":"2024-09-17T03:37:01.536631+0000","last_became_active":"2024-09-17T03:37:01.529834+0000","last_became_peered":"2024-09-17T03:37:01.529834+0000","last_unstale":"2024-09-17T03:37:01.536631+0000","last_undegraded":"2024-09-17T03:37:01.536631+0000","last_fullsized":"2024-09-17T03:37:01.536631+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:55.329149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523036+0000","last_change":"2024-09-17T03:37:01.518791+0000","last_active":"2024-09-17T03:37:01.523036+0000","last_peered":"2024-09-17T03:37:01.523036+0000","last_clean":"2024-09-17T03:37:01.523036+0000","last_became_active":"2024-09-17T03:37:01.516765+0000","last_became_peered":"2024-09-17T03:37:01.516765+0000","last_unstale":"2024-09-17T03:37:01.523036+0000","last_undegraded":"2024-09-17T03:37:01.523036+0000","last_fullsized":"2024-09-17T03:37:01.523036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529870+0000","last_change":"2024-09-17T03:37:01.527808+0000","last_active":"2024-09-17T03:37:01.529870+0000","last_peered":"2024-09-17T03:37:01.529870+0000","last_clean":"2024-09-17T03:37:01.529870+0000","last_became_active":"2024-09-17T03:37:01.527469+0000","last_became_peered":"2024-09-17T03:37:01.527469+0000","last_unstale":"2024-09-17T03:37:01.529870+0000","last_undegraded":"2024-09-17T03:37:01.529870+0000","last_fullsized":"2024-09-17T03:37:01.529870+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:54.208586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534127+0000","last_change":"2024-09-17T03:37:01.530734+0000","last_active":"2024-09-17T03:37:01.534127+0000","last_peered":"2024-09-17T03:37:01.534127+0000","last_clean":"2024-09-17T03:37:01.534127+0000","last_became_active":"2024-09-17T03:37:01.530184+0000","last_became_peered":"2024-09-17T03:37:01.530184+0000","last_unstale":"2024-09-17T03:37:01.534127+0000","last_undegraded":"2024-09-17T03:37:01.534127+0000","last_fullsized":"2024-09-17T03:37:01.534127+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:48.884673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525880+0000","last_change":"2024-09-17T03:37:01.520019+0000","last_active":"2024-09-17T03:37:01.525880+0000","last_peered":"2024-09-17T03:37:01.525880+0000","last_clean":"2024-09-17T03:37:01.525880+0000","last_became_active":"2024-09-17T03:37:01.519632+0000","last_became_peered":"2024-09-17T03:37:01.519632+0000","last_unstale":"2024-09-17T03:37:01.525880+0000","last_undegraded":"2024-09-17T03:37:01.525880+0000","last_fullsized":"2024-09-17T03:37:01.525880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:57.353653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536601+0000","last_change":"2024-09-17T03:37:01.532969+0000","last_active":"2024-09-17T03:37:01.536601+0000","last_peered":"2024-09-17T03:37:01.536601+0000","last_clean":"2024-09-17T03:37:01.536601+0000","last_became_active":"2024-09-17T03:37:01.530349+0000","last_became_peered":"2024-09-17T03:37:01.530349+0000","last_unstale":"2024-09-17T03:37:01.536601+0000","last_undegraded":"2024-09-17T03:37:01.536601+0000","last_fullsized":"2024-09-17T03:37:01.536601+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530028+0000","last_change":"2024-09-17T03:37:01.527670+0000","last_active":"2024-09-17T03:37:01.530028+0000","last_peered":"2024-09-17T03:37:01.530028+0000","last_clean":"2024-09-17T03:37:01.530028+0000","last_became_active":"2024-09-17T03:37:01.527335+0000","last_became_peered":"2024-09-17T03:37:01.527335+0000","last_unstale":"2024-09-17T03:37:01.530028+0000","last_undegraded":"2024-09-17T03:37:01.530028+0000","last_fullsized":"2024-09-17T03:37:01.530028+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525874+0000","last_change":"2024-09-17T03:37:01.520830+0000","last_active":"2024-09-17T03:37:01.525874+0000","last_peered":"2024-09-17T03:37:01.525874+0000","last_clean":"2024-09-17T03:37:01.525874+0000","last_became_active":"2024-09-17T03:37:01.520514+0000","last_became_peered":"2024-09-17T03:37:01.520514+0000","last_unstale":"2024-09-17T03:37:01.525874+0000","last_undegraded":"2024-09-17T03:37:01.525874+0000","last_fullsized":"2024-09-17T03:37:01.525874+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:28.913796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533954+0000","last_change":"2024-09-17T03:37:01.526010+0000","last_active":"2024-09-17T03:37:01.533954+0000","last_peered":"2024-09-17T03:37:01.533954+0000","last_clean":"2024-09-17T03:37:01.533954+0000","last_became_active":"2024-09-17T03:37:01.524307+0000","last_became_peered":"2024-09-17T03:37:01.524307+0000","last_unstale":"2024-09-17T03:37:01.533954+0000","last_undegraded":"2024-09-17T03:37:01.533954+0000","last_fullsized":"2024-09-17T03:37:01.533954+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:17.817507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529940+0000","last_change":"2024-09-17T03:37:01.526850+0000","last_active":"2024-09-17T03:37:01.529940+0000","last_peered":"2024-09-17T03:37:01.529940+0000","last_clean":"2024-09-17T03:37:01.529940+0000","last_became_active":"2024-09-17T03:37:01.526500+0000","last_became_peered":"2024-09-17T03:37:01.526500+0000","last_unstale":"2024-09-17T03:37:01.529940+0000","last_undegraded":"2024-09-17T03:37:01.529940+0000","last_fullsized":"2024-09-17T03:37:01.529940+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:01.514192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534424+0000","last_change":"2024-09-17T03:37:01.525369+0000","last_active":"2024-09-17T03:37:01.534424+0000","last_peered":"2024-09-17T03:37:01.534424+0000","last_clean":"2024-09-17T03:37:01.534424+0000","last_became_active":"2024-09-17T03:37:01.524994+0000","last_became_peered":"2024-09-17T03:37:01.524994+0000","last_unstale":"2024-09-17T03:37:01.534424+0000","last_undegraded":"2024-09-17T03:37:01.534424+0000","last_fullsized":"2024-09-17T03:37:01.534424+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:16.958527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521738+0000","last_change":"2024-09-17T03:37:01.514688+0000","last_active":"2024-09-17T03:37:01.521738+0000","last_peered":"2024-09-17T03:37:01.521738+0000","last_clean":"2024-09-17T03:37:01.521738+0000","last_became_active":"2024-09-17T03:37:01.505442+0000","last_became_peered":"2024-09-17T03:37:01.505442+0000","last_unstale":"2024-09-17T03:37:01.521738+0000","last_undegraded":"2024-09-17T03:37:01.521738+0000","last_fullsized":"2024-09-17T03:37:01.521738+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:51.069459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510776+0000","last_change":"2024-09-17T03:37:01.509207+0000","last_active":"2024-09-17T03:37:01.510776+0000","last_peered":"2024-09-17T03:37:01.510776+0000","last_clean":"2024-09-17T03:37:01.510776+0000","last_became_active":"2024-09-17T03:37:01.508207+0000","last_became_peered":"2024-09-17T03:37:01.508207+0000","last_unstale":"2024-09-17T03:37:01.510776+0000","last_undegraded":"2024-09-17T03:37:01.510776+0000","last_fullsized":"2024-09-17T03:37:01.510776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:48.105755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528879+0000","last_change":"2024-09-17T03:37:01.524856+0000","last_active":"2024-09-17T03:37:01.528879+0000","last_peered":"2024-09-17T03:37:01.528879+0000","last_clean":"2024-09-17T03:37:01.528879+0000","last_became_active":"2024-09-17T03:37:01.522971+0000","last_became_peered":"2024-09-17T03:37:01.522971+0000","last_unstale":"2024-09-17T03:37:01.528879+0000","last_undegraded":"2024-09-17T03:37:01.528879+0000","last_fullsized":"2024-09-17T03:37:01.528879+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517595+0000","last_change":"2024-09-17T03:37:01.504583+0000","last_active":"2024-09-17T03:37:01.517595+0000","last_peered":"2024-09-17T03:37:01.517595+0000","last_clean":"2024-09-17T03:37:01.517595+0000","last_became_active":"2024-09-17T03:37:01.504221+0000","last_became_peered":"2024-09-17T03:37:01.504221+0000","last_unstale":"2024-09-17T03:37:01.517595+0000","last_undegraded":"2024-09-17T03:37:01.517595+0000","last_fullsized":"2024-09-17T03:37:01.517595+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:18.113674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534132+0000","last_change":"2024-09-17T03:37:01.527737+0000","last_active":"2024-09-17T03:37:01.534132+0000","last_peered":"2024-09-17T03:37:01.534132+0000","last_clean":"2024-09-17T03:37:01.534132+0000","last_became_active":"2024-09-17T03:37:01.526783+0000","last_became_peered":"2024-09-17T03:37:01.526783+0000","last_unstale":"2024-09-17T03:37:01.534132+0000","last_undegraded":"2024-09-17T03:37:01.534132+0000","last_fullsized":"2024-09-17T03:37:01.534132+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:28.866527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518126+0000","last_change":"2024-09-17T03:37:01.513699+0000","last_active":"2024-09-17T03:37:01.518126+0000","last_peered":"2024-09-17T03:37:01.518126+0000","last_clean":"2024-09-17T03:37:01.518126+0000","last_became_active":"2024-09-17T03:37:01.513310+0000","last_became_peered":"2024-09-17T03:37:01.513310+0000","last_unstale":"2024-09-17T03:37:01.518126+0000","last_undegraded":"2024-09-17T03:37:01.518126+0000","last_fullsized":"2024-09-17T03:37:01.518126+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"142'4","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535697+0000","last_change":"2024-09-17T03:37:00.521721+0000","last_active":"2024-09-17T03:37:01.535697+0000","last_peered":"2024-09-17T03:37:01.535697+0000","last_clean":"2024-09-17T03:37:01.535697+0000","last_became_active":"2024-09-17T03:37:00.521476+0000","last_became_peered":"2024-09-17T03:37:00.521476+0000","last_unstale":"2024-09-17T03:37:01.535697+0000","last_undegraded":"2024-09-17T03:37:01.535697+0000","last_fullsized":"2024-09-17T03:37:01.535697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:11.151395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531349+0000","last_change":"2024-09-17T03:37:01.524327+0000","last_active":"2024-09-17T03:37:01.531349+0000","last_peered":"2024-09-17T03:37:01.531349+0000","last_clean":"2024-09-17T03:37:01.531349+0000","last_became_active":"2024-09-17T03:37:01.523988+0000","last_became_peered":"2024-09-17T03:37:01.523988+0000","last_unstale":"2024-09-17T03:37:01.531349+0000","last_undegraded":"2024-09-17T03:37:01.531349+0000","last_fullsized":"2024-09-17T03:37:01.531349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:57.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.522778+0000","last_change":"2024-09-17T03:37:00.513012+0000","last_active":"2024-09-17T03:37:01.522778+0000","last_peered":"2024-09-17T03:37:01.522778+0000","last_clean":"2024-09-17T03:37:01.522778+0000","last_became_active":"2024-09-17T03:37:00.512669+0000","last_became_peered":"2024-09-17T03:37:00.512669+0000","last_unstale":"2024-09-17T03:37:01.522778+0000","last_undegraded":"2024-09-17T03:37:01.522778+0000","last_fullsized":"2024-09-17T03:37:01.522778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:04.275671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523023+0000","last_change":"2024-09-17T03:37:01.520484+0000","last_active":"2024-09-17T03:37:01.523023+0000","last_peered":"2024-09-17T03:37:01.523023+0000","last_clean":"2024-09-17T03:37:01.523023+0000","last_became_active":"2024-09-17T03:37:01.518380+0000","last_became_peered":"2024-09-17T03:37:01.518380+0000","last_unstale":"2024-09-17T03:37:01.523023+0000","last_undegraded":"2024-09-17T03:37:01.523023+0000","last_fullsized":"2024-09-17T03:37:01.523023+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:10.898889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537662+0000","last_change":"2024-09-17T03:37:00.518391+0000","last_active":"2024-09-17T03:37:01.537662+0000","last_peered":"2024-09-17T03:37:01.537662+0000","last_clean":"2024-09-17T03:37:01.537662+0000","last_became_active":"2024-09-17T03:37:00.518014+0000","last_became_peered":"2024-09-17T03:37:00.518014+0000","last_unstale":"2024-09-17T03:37:01.537662+0000","last_undegraded":"2024-09-17T03:37:01.537662+0000","last_fullsized":"2024-09-17T03:37:01.537662+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518010+0000","last_change":"2024-09-17T03:37:01.513224+0000","last_active":"2024-09-17T03:37:01.518010+0000","last_peered":"2024-09-17T03:37:01.518010+0000","last_clean":"2024-09-17T03:37:01.518010+0000","last_became_active":"2024-09-17T03:37:01.512886+0000","last_became_peered":"2024-09-17T03:37:01.512886+0000","last_unstale":"2024-09-17T03:37:01.518010+0000","last_undegraded":"2024-09-17T03:37:01.518010+0000","last_fullsized":"2024-09-17T03:37:01.518010+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:13.599854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535032+0000","last_change":"2024-09-17T03:37:00.516162+0000","last_active":"2024-09-17T03:37:01.535032+0000","last_peered":"2024-09-17T03:37:01.535032+0000","last_clean":"2024-09-17T03:37:01.535032+0000","last_became_active":"2024-09-17T03:37:00.515367+0000","last_became_peered":"2024-09-17T03:37:00.515367+0000","last_unstale":"2024-09-17T03:37:01.535032+0000","last_undegraded":"2024-09-17T03:37:01.535032+0000","last_fullsized":"2024-09-17T03:37:01.535032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:08.671535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518310+0000","last_change":"2024-09-17T03:37:01.506152+0000","last_active":"2024-09-17T03:37:01.518310+0000","last_peered":"2024-09-17T03:37:01.518310+0000","last_clean":"2024-09-17T03:37:01.518310+0000","last_became_active":"2024-09-17T03:37:01.503759+0000","last_became_peered":"2024-09-17T03:37:01.503759+0000","last_unstale":"2024-09-17T03:37:01.518310+0000","last_undegraded":"2024-09-17T03:37:01.518310+0000","last_fullsized":"2024-09-17T03:37:01.518310+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:43.445864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528132+0000","last_change":"2024-09-17T03:37:00.510847+0000","last_active":"2024-09-17T03:37:01.528132+0000","last_peered":"2024-09-17T03:37:01.528132+0000","last_clean":"2024-09-17T03:37:01.528132+0000","last_became_active":"2024-09-17T03:37:00.509550+0000","last_became_peered":"2024-09-17T03:37:00.509550+0000","last_unstale":"2024-09-17T03:37:01.528132+0000","last_undegraded":"2024-09-17T03:37:01.528132+0000","last_fullsized":"2024-09-17T03:37:01.528132+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.371788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531043+0000","last_change":"2024-09-17T03:37:01.524253+0000","last_active":"2024-09-17T03:37:01.531043+0000","last_peered":"2024-09-17T03:37:01.531043+0000","last_clean":"2024-09-17T03:37:01.531043+0000","last_became_active":"2024-09-17T03:37:01.523935+0000","last_became_peered":"2024-09-17T03:37:01.523935+0000","last_unstale":"2024-09-17T03:37:01.531043+0000","last_undegraded":"2024-09-17T03:37:01.531043+0000","last_fullsized":"2024-09-17T03:37:01.531043+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.515827+0000","last_change":"2024-09-17T03:37:00.513776+0000","last_active":"2024-09-17T03:37:00.515827+0000","last_peered":"2024-09-17T03:37:00.515827+0000","last_clean":"2024-09-17T03:37:00.515827+0000","last_became_active":"2024-09-17T03:37:00.511311+0000","last_became_peered":"2024-09-17T03:37:00.511311+0000","last_unstale":"2024-09-17T03:37:00.515827+0000","last_undegraded":"2024-09-17T03:37:00.515827+0000","last_fullsized":"2024-09-17T03:37:00.515827+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:16.989474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517432+0000","last_change":"2024-09-17T03:37:01.513641+0000","last_active":"2024-09-17T03:37:01.517432+0000","last_peered":"2024-09-17T03:37:01.517432+0000","last_clean":"2024-09-17T03:37:01.517432+0000","last_became_active":"2024-09-17T03:37:01.513325+0000","last_became_peered":"2024-09-17T03:37:01.513325+0000","last_unstale":"2024-09-17T03:37:01.517432+0000","last_undegraded":"2024-09-17T03:37:01.517432+0000","last_fullsized":"2024-09-17T03:37:01.517432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:54.606013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.516501+0000","last_change":"2024-09-17T03:37:00.513452+0000","last_active":"2024-09-17T03:37:01.516501+0000","last_peered":"2024-09-17T03:37:01.516501+0000","last_clean":"2024-09-17T03:37:01.516501+0000","last_became_active":"2024-09-17T03:37:00.513118+0000","last_became_peered":"2024-09-17T03:37:00.513118+0000","last_unstale":"2024-09-17T03:37:01.516501+0000","last_undegraded":"2024-09-17T03:37:01.516501+0000","last_fullsized":"2024-09-17T03:37:01.516501+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:53.259179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498231+0000","last_change":"2024-09-17T03:37:00.498231+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498231+0000","last_undegraded":"2024-09-17T03:37:00.498231+0000","last_fullsized":"2024-09-17T03:37:00.498231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509557+0000","last_change":"2024-09-17T03:37:00.506180+0000","last_active":"2024-09-17T03:37:01.509557+0000","last_peered":"2024-09-17T03:37:01.509557+0000","last_clean":"2024-09-17T03:37:01.509557+0000","last_became_active":"2024-09-17T03:37:00.505868+0000","last_became_peered":"2024-09-17T03:37:00.505868+0000","last_unstale":"2024-09-17T03:37:01.509557+0000","last_undegraded":"2024-09-17T03:37:01.509557+0000","last_fullsized":"2024-09-17T03:37:01.509557+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497947+0000","last_change":"2024-09-17T03:37:00.497947+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497947+0000","last_undegraded":"2024-09-17T03:37:00.497947+0000","last_fullsized":"2024-09-17T03:37:00.497947+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494787+0000","last_change":"2024-09-17T03:37:00.513939+0000","last_active":"2024-09-17T03:37:01.494787+0000","last_peered":"2024-09-17T03:37:01.494787+0000","last_clean":"2024-09-17T03:37:01.494787+0000","last_became_active":"2024-09-17T03:37:00.513531+0000","last_became_peered":"2024-09-17T03:37:00.513531+0000","last_unstale":"2024-09-17T03:37:01.494787+0000","last_undegraded":"2024-09-17T03:37:01.494787+0000","last_fullsized":"2024-09-17T03:37:01.494787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:43.986216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536554+0000","last_change":"2024-09-17T03:37:01.530837+0000","last_active":"2024-09-17T03:37:01.536554+0000","last_peered":"2024-09-17T03:37:01.536554+0000","last_clean":"2024-09-17T03:37:01.536554+0000","last_became_active":"2024-09-17T03:37:01.530166+0000","last_became_peered":"2024-09-17T03:37:01.530166+0000","last_unstale":"2024-09-17T03:37:01.536554+0000","last_undegraded":"2024-09-17T03:37:01.536554+0000","last_fullsized":"2024-09-17T03:37:01.536554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:32.600471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.498465+0000","last_change":"2024-09-17T03:37:00.515644+0000","last_active":"2024-09-17T03:37:01.498465+0000","last_peered":"2024-09-17T03:37:01.498465+0000","last_clean":"2024-09-17T03:37:01.498465+0000","last_became_active":"2024-09-17T03:37:00.515326+0000","last_became_peered":"2024-09-17T03:37:00.515326+0000","last_unstale":"2024-09-17T03:37:01.498465+0000","last_undegraded":"2024-09-17T03:37:01.498465+0000","last_fullsized":"2024-09-17T03:37:01.498465+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:05.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534454+0000","last_change":"2024-09-17T03:37:01.530851+0000","last_active":"2024-09-17T03:37:01.534454+0000","last_peered":"2024-09-17T03:37:01.534454+0000","last_clean":"2024-09-17T03:37:01.534454+0000","last_became_active":"2024-09-17T03:37:01.530410+0000","last_became_peered":"2024-09-17T03:37:01.530410+0000","last_unstale":"2024-09-17T03:37:01.534454+0000","last_undegraded":"2024-09-17T03:37:01.534454+0000","last_fullsized":"2024-09-17T03:37:01.534454+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:54.349008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"142'6","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537003+0000","last_change":"2024-09-17T03:37:00.511424+0000","last_active":"2024-09-17T03:37:01.537003+0000","last_peered":"2024-09-17T03:37:01.537003+0000","last_clean":"2024-09-17T03:37:01.537003+0000","last_became_active":"2024-09-17T03:37:00.511055+0000","last_became_peered":"2024-09-17T03:37:00.511055+0000","last_unstale":"2024-09-17T03:37:01.537003+0000","last_undegraded":"2024-09-17T03:37:01.537003+0000","last_fullsized":"2024-09-17T03:37:01.537003+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:04.537830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526765+0000","last_change":"2024-09-17T03:37:01.520532+0000","last_active":"2024-09-17T03:37:01.526765+0000","last_peered":"2024-09-17T03:37:01.526765+0000","last_clean":"2024-09-17T03:37:01.526765+0000","last_became_active":"2024-09-17T03:37:01.520261+0000","last_became_peered":"2024-09-17T03:37:01.520261+0000","last_unstale":"2024-09-17T03:37:01.526765+0000","last_undegraded":"2024-09-17T03:37:01.526765+0000","last_fullsized":"2024-09-17T03:37:01.526765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538130+0000","last_change":"2024-09-17T03:37:00.509307+0000","last_active":"2024-09-17T03:37:01.538130+0000","last_peered":"2024-09-17T03:37:01.538130+0000","last_clean":"2024-09-17T03:37:01.538130+0000","last_became_active":"2024-09-17T03:37:00.508858+0000","last_became_peered":"2024-09-17T03:37:00.508858+0000","last_unstale":"2024-09-17T03:37:01.538130+0000","last_undegraded":"2024-09-17T03:37:01.538130+0000","last_fullsized":"2024-09-17T03:37:01.538130+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:19.533265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532147+0000","last_change":"2024-09-17T03:37:01.526495+0000","last_active":"2024-09-17T03:37:01.532147+0000","last_peered":"2024-09-17T03:37:01.532147+0000","last_clean":"2024-09-17T03:37:01.532147+0000","last_became_active":"2024-09-17T03:37:01.526193+0000","last_became_peered":"2024-09-17T03:37:01.526193+0000","last_unstale":"2024-09-17T03:37:01.532147+0000","last_undegraded":"2024-09-17T03:37:01.532147+0000","last_fullsized":"2024-09-17T03:37:01.532147+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:38.826521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506839+0000","last_change":"2024-09-17T03:37:00.506435+0000","last_active":"2024-09-17T03:37:01.506839+0000","last_peered":"2024-09-17T03:37:01.506839+0000","last_clean":"2024-09-17T03:37:01.506839+0000","last_became_active":"2024-09-17T03:37:00.505499+0000","last_became_peered":"2024-09-17T03:37:00.505499+0000","last_unstale":"2024-09-17T03:37:01.506839+0000","last_undegraded":"2024-09-17T03:37:01.506839+0000","last_fullsized":"2024-09-17T03:37:01.506839+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497356+0000","last_change":"2024-09-17T03:37:00.497356+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497356+0000","last_undegraded":"2024-09-17T03:37:00.497356+0000","last_fullsized":"2024-09-17T03:37:00.497356+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.490845+0000","last_change":"2024-09-17T03:37:00.497616+0000","last_active":"2024-09-17T03:37:01.490845+0000","last_peered":"2024-09-17T03:37:01.490845+0000","last_clean":"2024-09-17T03:37:01.490845+0000","last_became_active":"2024-09-17T03:37:00.497229+0000","last_became_peered":"2024-09-17T03:37:00.497229+0000","last_unstale":"2024-09-17T03:37:01.490845+0000","last_undegraded":"2024-09-17T03:37:01.490845+0000","last_fullsized":"2024-09-17T03:37:01.490845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:08.103469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531634+0000","last_change":"2024-09-17T03:37:01.525031+0000","last_active":"2024-09-17T03:37:01.531634+0000","last_peered":"2024-09-17T03:37:01.531634+0000","last_clean":"2024-09-17T03:37:01.531634+0000","last_became_active":"2024-09-17T03:37:01.524710+0000","last_became_peered":"2024-09-17T03:37:01.524710+0000","last_unstale":"2024-09-17T03:37:01.531634+0000","last_undegraded":"2024-09-17T03:37:01.531634+0000","last_fullsized":"2024-09-17T03:37:01.531634+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"142'8","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534881+0000","last_change":"2024-09-17T03:37:00.506497+0000","last_active":"2024-09-17T03:37:01.534881+0000","last_peered":"2024-09-17T03:37:01.534881+0000","last_clean":"2024-09-17T03:37:01.534881+0000","last_became_active":"2024-09-17T03:37:00.506195+0000","last_became_peered":"2024-09-17T03:37:00.506195+0000","last_unstale":"2024-09-17T03:37:01.534881+0000","last_undegraded":"2024-09-17T03:37:01.534881+0000","last_fullsized":"2024-09-17T03:37:01.534881+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:35.347269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.511186+0000","last_change":"2024-09-17T03:37:01.510170+0000","last_active":"2024-09-17T03:37:01.511186+0000","last_peered":"2024-09-17T03:37:01.511186+0000","last_clean":"2024-09-17T03:37:01.511186+0000","last_became_active":"2024-09-17T03:37:01.509806+0000","last_became_peered":"2024-09-17T03:37:01.509806+0000","last_unstale":"2024-09-17T03:37:01.511186+0000","last_undegraded":"2024-09-17T03:37:01.511186+0000","last_fullsized":"2024-09-17T03:37:01.511186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:43.917446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.511017+0000","last_change":"2024-09-17T03:37:00.507289+0000","last_active":"2024-09-17T03:37:00.511017+0000","last_peered":"2024-09-17T03:37:00.511017+0000","last_clean":"2024-09-17T03:37:00.511017+0000","last_became_active":"2024-09-17T03:37:00.506994+0000","last_became_peered":"2024-09-17T03:37:00.506994+0000","last_unstale":"2024-09-17T03:37:00.511017+0000","last_undegraded":"2024-09-17T03:37:00.511017+0000","last_fullsized":"2024-09-17T03:37:00.511017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:13.903403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533925+0000","last_change":"2024-09-17T03:37:01.525640+0000","last_active":"2024-09-17T03:37:01.533925+0000","last_peered":"2024-09-17T03:37:01.533925+0000","last_clean":"2024-09-17T03:37:01.533925+0000","last_became_active":"2024-09-17T03:37:01.523930+0000","last_became_peered":"2024-09-17T03:37:01.523930+0000","last_unstale":"2024-09-17T03:37:01.533925+0000","last_undegraded":"2024-09-17T03:37:01.533925+0000","last_fullsized":"2024-09-17T03:37:01.533925+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519679+0000","last_change":"2024-09-17T03:37:00.514697+0000","last_active":"2024-09-17T03:37:01.519679+0000","last_peered":"2024-09-17T03:37:01.519679+0000","last_clean":"2024-09-17T03:37:01.519679+0000","last_became_active":"2024-09-17T03:37:00.513958+0000","last_became_peered":"2024-09-17T03:37:00.513958+0000","last_unstale":"2024-09-17T03:37:01.519679+0000","last_undegraded":"2024-09-17T03:37:01.519679+0000","last_fullsized":"2024-09-17T03:37:01.519679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:27.979522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529910+0000","last_change":"2024-09-17T03:37:01.521591+0000","last_active":"2024-09-17T03:37:01.529910+0000","last_peered":"2024-09-17T03:37:01.529910+0000","last_clean":"2024-09-17T03:37:01.529910+0000","last_became_active":"2024-09-17T03:37:01.521066+0000","last_became_peered":"2024-09-17T03:37:01.521066+0000","last_unstale":"2024-09-17T03:37:01.529910+0000","last_undegraded":"2024-09-17T03:37:01.529910+0000","last_fullsized":"2024-09-17T03:37:01.529910+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533162+0000","last_change":"2024-09-17T03:37:00.510701+0000","last_active":"2024-09-17T03:37:01.533162+0000","last_peered":"2024-09-17T03:37:01.533162+0000","last_clean":"2024-09-17T03:37:01.533162+0000","last_became_active":"2024-09-17T03:37:00.510374+0000","last_became_peered":"2024-09-17T03:37:00.510374+0000","last_unstale":"2024-09-17T03:37:01.533162+0000","last_undegraded":"2024-09-17T03:37:01.533162+0000","last_fullsized":"2024-09-17T03:37:01.533162+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:40.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510426+0000","last_change":"2024-09-17T03:37:01.508528+0000","last_active":"2024-09-17T03:37:01.510426+0000","last_peered":"2024-09-17T03:37:01.510426+0000","last_clean":"2024-09-17T03:37:01.510426+0000","last_became_active":"2024-09-17T03:37:01.508173+0000","last_became_peered":"2024-09-17T03:37:01.508173+0000","last_unstale":"2024-09-17T03:37:01.510426+0000","last_undegraded":"2024-09-17T03:37:01.510426+0000","last_fullsized":"2024-09-17T03:37:01.510426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:47.227568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494837+0000","last_change":"2024-09-17T03:37:00.517789+0000","last_active":"2024-09-17T03:37:01.494837+0000","last_peered":"2024-09-17T03:37:01.494837+0000","last_clean":"2024-09-17T03:37:01.494837+0000","last_became_active":"2024-09-17T03:37:00.517101+0000","last_became_peered":"2024-09-17T03:37:00.517101+0000","last_unstale":"2024-09-17T03:37:01.494837+0000","last_undegraded":"2024-09-17T03:37:01.494837+0000","last_fullsized":"2024-09-17T03:37:01.494837+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:30.163493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524630+0000","last_change":"2024-09-17T03:37:01.518635+0000","last_active":"2024-09-17T03:37:01.524630+0000","last_peered":"2024-09-17T03:37:01.524630+0000","last_clean":"2024-09-17T03:37:01.524630+0000","last_became_active":"2024-09-17T03:37:01.518369+0000","last_became_peered":"2024-09-17T03:37:01.518369+0000","last_unstale":"2024-09-17T03:37:01.524630+0000","last_undegraded":"2024-09-17T03:37:01.524630+0000","last_fullsized":"2024-09-17T03:37:01.524630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:42.751762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529734+0000","last_change":"2024-09-17T03:37:00.502280+0000","last_active":"2024-09-17T03:37:01.529734+0000","last_peered":"2024-09-17T03:37:01.529734+0000","last_clean":"2024-09-17T03:37:01.529734+0000","last_became_active":"2024-09-17T03:37:00.501924+0000","last_became_peered":"2024-09-17T03:37:00.501924+0000","last_unstale":"2024-09-17T03:37:01.529734+0000","last_undegraded":"2024-09-17T03:37:01.529734+0000","last_fullsized":"2024-09-17T03:37:01.529734+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:46.918045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524298+0000","last_change":"2024-09-17T03:37:01.523047+0000","last_active":"2024-09-17T03:37:01.524298+0000","last_peered":"2024-09-17T03:37:01.524298+0000","last_clean":"2024-09-17T03:37:01.524298+0000","last_became_active":"2024-09-17T03:37:01.522670+0000","last_became_peered":"2024-09-17T03:37:01.522670+0000","last_unstale":"2024-09-17T03:37:01.524298+0000","last_undegraded":"2024-09-17T03:37:01.524298+0000","last_fullsized":"2024-09-17T03:37:01.524298+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:03.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496644+0000","last_change":"2024-09-17T03:37:00.519370+0000","last_active":"2024-09-17T03:37:01.496644+0000","last_peered":"2024-09-17T03:37:01.496644+0000","last_clean":"2024-09-17T03:37:01.496644+0000","last_became_active":"2024-09-17T03:37:00.519058+0000","last_became_peered":"2024-09-17T03:37:00.519058+0000","last_unstale":"2024-09-17T03:37:01.496644+0000","last_undegraded":"2024-09-17T03:37:01.496644+0000","last_fullsized":"2024-09-17T03:37:01.496644+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:12.207851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530069+0000","last_change":"2024-09-17T03:37:01.528223+0000","last_active":"2024-09-17T03:37:01.530069+0000","last_peered":"2024-09-17T03:37:01.530069+0000","last_clean":"2024-09-17T03:37:01.530069+0000","last_became_active":"2024-09-17T03:37:01.527875+0000","last_became_peered":"2024-09-17T03:37:01.527875+0000","last_unstale":"2024-09-17T03:37:01.530069+0000","last_undegraded":"2024-09-17T03:37:01.530069+0000","last_fullsized":"2024-09-17T03:37:01.530069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496550+0000","last_change":"2024-09-17T03:37:00.514756+0000","last_active":"2024-09-17T03:37:01.496550+0000","last_peered":"2024-09-17T03:37:01.496550+0000","last_clean":"2024-09-17T03:37:01.496550+0000","last_became_active":"2024-09-17T03:37:00.514505+0000","last_became_peered":"2024-09-17T03:37:00.514505+0000","last_unstale":"2024-09-17T03:37:01.496550+0000","last_undegraded":"2024-09-17T03:37:01.496550+0000","last_fullsized":"2024-09-17T03:37:01.496550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524885+0000","last_change":"2024-09-17T03:37:01.519916+0000","last_active":"2024-09-17T03:37:01.524885+0000","last_peered":"2024-09-17T03:37:01.524885+0000","last_clean":"2024-09-17T03:37:01.524885+0000","last_became_active":"2024-09-17T03:37:01.519171+0000","last_became_peered":"2024-09-17T03:37:01.519171+0000","last_unstale":"2024-09-17T03:37:01.524885+0000","last_undegraded":"2024-09-17T03:37:01.524885+0000","last_fullsized":"2024-09-17T03:37:01.524885+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:18.402815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.507817+0000","last_change":"2024-09-17T03:37:00.516033+0000","last_active":"2024-09-17T03:37:01.507817+0000","last_peered":"2024-09-17T03:37:01.507817+0000","last_clean":"2024-09-17T03:37:01.507817+0000","last_became_active":"2024-09-17T03:37:00.515737+0000","last_became_peered":"2024-09-17T03:37:00.515737+0000","last_unstale":"2024-09-17T03:37:01.507817+0000","last_undegraded":"2024-09-17T03:37:01.507817+0000","last_fullsized":"2024-09-17T03:37:01.507817+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:23.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497461+0000","last_change":"2024-09-17T03:37:00.497461+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497461+0000","last_undegraded":"2024-09-17T03:37:00.497461+0000","last_fullsized":"2024-09-17T03:37:00.497461+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533095+0000","last_change":"2024-09-17T03:37:00.518901+0000","last_active":"2024-09-17T03:37:01.533095+0000","last_peered":"2024-09-17T03:37:01.533095+0000","last_clean":"2024-09-17T03:37:01.533095+0000","last_became_active":"2024-09-17T03:37:00.518609+0000","last_became_peered":"2024-09-17T03:37:00.518609+0000","last_unstale":"2024-09-17T03:37:01.533095+0000","last_undegraded":"2024-09-17T03:37:01.533095+0000","last_fullsized":"2024-09-17T03:37:01.533095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:31.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519672+0000","last_change":"2024-09-17T03:37:01.515337+0000","last_active":"2024-09-17T03:37:01.519672+0000","last_peered":"2024-09-17T03:37:01.519672+0000","last_clean":"2024-09-17T03:37:01.519672+0000","last_became_active":"2024-09-17T03:37:01.515033+0000","last_became_peered":"2024-09-17T03:37:01.515033+0000","last_unstale":"2024-09-17T03:37:01.519672+0000","last_undegraded":"2024-09-17T03:37:01.519672+0000","last_fullsized":"2024-09-17T03:37:01.519672+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494565+0000","last_change":"2024-09-17T03:37:00.509124+0000","last_active":"2024-09-17T03:37:01.494565+0000","last_peered":"2024-09-17T03:37:01.494565+0000","last_clean":"2024-09-17T03:37:01.494565+0000","last_became_active":"2024-09-17T03:37:00.508710+0000","last_became_peered":"2024-09-17T03:37:00.508710+0000","last_unstale":"2024-09-17T03:37:01.494565+0000","last_undegraded":"2024-09-17T03:37:01.494565+0000","last_fullsized":"2024-09-17T03:37:01.494565+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:09.199848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518488+0000","last_change":"2024-09-17T03:37:01.514963+0000","last_active":"2024-09-17T03:37:01.518488+0000","last_peered":"2024-09-17T03:37:01.518488+0000","last_clean":"2024-09-17T03:37:01.518488+0000","last_became_active":"2024-09-17T03:37:01.514337+0000","last_became_peered":"2024-09-17T03:37:01.514337+0000","last_unstale":"2024-09-17T03:37:01.518488+0000","last_undegraded":"2024-09-17T03:37:01.518488+0000","last_fullsized":"2024-09-17T03:37:01.518488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:17.288497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.497349+0000","last_change":"2024-09-17T03:37:00.506035+0000","last_active":"2024-09-17T03:37:01.497349+0000","last_peered":"2024-09-17T03:37:01.497349+0000","last_clean":"2024-09-17T03:37:01.497349+0000","last_became_active":"2024-09-17T03:37:00.505658+0000","last_became_peered":"2024-09-17T03:37:00.505658+0000","last_unstale":"2024-09-17T03:37:01.497349+0000","last_undegraded":"2024-09-17T03:37:01.497349+0000","last_fullsized":"2024-09-17T03:37:01.497349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:52.915238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521428+0000","last_change":"2024-09-17T03:37:01.517782+0000","last_active":"2024-09-17T03:37:01.521428+0000","last_peered":"2024-09-17T03:37:01.521428+0000","last_clean":"2024-09-17T03:37:01.521428+0000","last_became_active":"2024-09-17T03:37:01.517363+0000","last_became_peered":"2024-09-17T03:37:01.517363+0000","last_unstale":"2024-09-17T03:37:01.521428+0000","last_undegraded":"2024-09-17T03:37:01.521428+0000","last_fullsized":"2024-09-17T03:37:01.521428+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:17.301571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496098+0000","last_change":"2024-09-17T03:37:00.514913+0000","last_active":"2024-09-17T03:37:01.496098+0000","last_peered":"2024-09-17T03:37:01.496098+0000","last_clean":"2024-09-17T03:37:01.496098+0000","last_became_active":"2024-09-17T03:37:00.514588+0000","last_became_peered":"2024-09-17T03:37:00.514588+0000","last_unstale":"2024-09-17T03:37:01.496098+0000","last_undegraded":"2024-09-17T03:37:01.496098+0000","last_fullsized":"2024-09-17T03:37:01.496098+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:15.350414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532735+0000","last_change":"2024-09-17T03:37:01.523154+0000","last_active":"2024-09-17T03:37:01.532735+0000","last_peered":"2024-09-17T03:37:01.532735+0000","last_clean":"2024-09-17T03:37:01.532735+0000","last_became_active":"2024-09-17T03:37:01.522826+0000","last_became_peered":"2024-09-17T03:37:01.522826+0000","last_unstale":"2024-09-17T03:37:01.532735+0000","last_undegraded":"2024-09-17T03:37:01.532735+0000","last_fullsized":"2024-09-17T03:37:01.532735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:19.317037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.501477+0000","last_change":"2024-09-17T03:37:00.510360+0000","last_active":"2024-09-17T03:37:01.501477+0000","last_peered":"2024-09-17T03:37:01.501477+0000","last_clean":"2024-09-17T03:37:01.501477+0000","last_became_active":"2024-09-17T03:37:00.510038+0000","last_became_peered":"2024-09-17T03:37:00.510038+0000","last_unstale":"2024-09-17T03:37:01.501477+0000","last_undegraded":"2024-09-17T03:37:01.501477+0000","last_fullsized":"2024-09-17T03:37:01.501477+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:31.275321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497390+0000","last_change":"2024-09-17T03:37:00.497390+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497390+0000","last_undegraded":"2024-09-17T03:37:00.497390+0000","last_fullsized":"2024-09-17T03:37:00.497390+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496964+0000","last_change":"2024-09-17T03:37:00.518866+0000","last_active":"2024-09-17T03:37:01.496964+0000","last_peered":"2024-09-17T03:37:01.496964+0000","last_clean":"2024-09-17T03:37:01.496964+0000","last_became_active":"2024-09-17T03:37:00.518531+0000","last_became_peered":"2024-09-17T03:37:00.518531+0000","last_unstale":"2024-09-17T03:37:01.496964+0000","last_undegraded":"2024-09-17T03:37:01.496964+0000","last_fullsized":"2024-09-17T03:37:01.496964+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.823082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537156+0000","last_change":"2024-09-17T03:37:01.534797+0000","last_active":"2024-09-17T03:37:01.537156+0000","last_peered":"2024-09-17T03:37:01.537156+0000","last_clean":"2024-09-17T03:37:01.537156+0000","last_became_active":"2024-09-17T03:37:01.534442+0000","last_became_peered":"2024-09-17T03:37:01.534442+0000","last_unstale":"2024-09-17T03:37:01.537156+0000","last_undegraded":"2024-09-17T03:37:01.537156+0000","last_fullsized":"2024-09-17T03:37:01.537156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:42.572373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506749+0000","last_change":"2024-09-17T03:37:00.512368+0000","last_active":"2024-09-17T03:37:01.506749+0000","last_peered":"2024-09-17T03:37:01.506749+0000","last_clean":"2024-09-17T03:37:01.506749+0000","last_became_active":"2024-09-17T03:37:00.512041+0000","last_became_peered":"2024-09-17T03:37:00.512041+0000","last_unstale":"2024-09-17T03:37:01.506749+0000","last_undegraded":"2024-09-17T03:37:01.506749+0000","last_fullsized":"2024-09-17T03:37:01.506749+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497341+0000","last_change":"2024-09-17T03:37:00.497341+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497341+0000","last_undegraded":"2024-09-17T03:37:00.497341+0000","last_fullsized":"2024-09-17T03:37:00.497341+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.492735+0000","last_change":"2024-09-17T03:37:00.508129+0000","last_active":"2024-09-17T03:37:01.492735+0000","last_peered":"2024-09-17T03:37:01.492735+0000","last_clean":"2024-09-17T03:37:01.492735+0000","last_became_active":"2024-09-17T03:37:00.506285+0000","last_became_peered":"2024-09-17T03:37:00.506285+0000","last_unstale":"2024-09-17T03:37:01.492735+0000","last_undegraded":"2024-09-17T03:37:01.492735+0000","last_fullsized":"2024-09-17T03:37:01.492735+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:59.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521767+0000","last_change":"2024-09-17T03:37:01.516795+0000","last_active":"2024-09-17T03:37:01.521767+0000","last_peered":"2024-09-17T03:37:01.521767+0000","last_clean":"2024-09-17T03:37:01.521767+0000","last_became_active":"2024-09-17T03:37:01.516423+0000","last_became_peered":"2024-09-17T03:37:01.516423+0000","last_unstale":"2024-09-17T03:37:01.521767+0000","last_undegraded":"2024-09-17T03:37:01.521767+0000","last_fullsized":"2024-09-17T03:37:01.521767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:14.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"142'3","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538197+0000","last_change":"2024-09-17T03:37:00.515111+0000","last_active":"2024-09-17T03:37:01.538197+0000","last_peered":"2024-09-17T03:37:01.538197+0000","last_clean":"2024-09-17T03:37:01.538197+0000","last_became_active":"2024-09-17T03:37:00.514784+0000","last_became_peered":"2024-09-17T03:37:00.514784+0000","last_unstale":"2024-09-17T03:37:01.538197+0000","last_undegraded":"2024-09-17T03:37:01.538197+0000","last_fullsized":"2024-09-17T03:37:01.538197+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:41.748322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":333,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496801+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:37:01.496801+0000","last_peered":"2024-09-17T03:37:01.496801+0000","last_clean":"2024-09-17T03:37:01.496801+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:37:01.496801+0000","last_undegraded":"2024-09-17T03:37:01.496801+0000","last_fullsized":"2024-09-17T03:37:01.496801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119528,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252312,"statfs":{"total":96636764160,"available":96514367488,"internally_reserved":0,"allocated":2371584,"data_stored":1366963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149452,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222388,"statfs":{"total":96636764160,"available":96483725312,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159188,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212652,"statfs":{"total":96636764160,"available":96473755648,"internally_reserved":0,"allocated":1769472,"data_stored":775943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127244,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244596,"statfs":{"total":96636764160,"available":96506466304,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167504,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204336,"statfs":{"total":96636764160,"available":96465240064,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150032,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221808,"statfs":{"total":96636764160,"available":96483131392,"internally_reserved":0,"allocated":2351104,"data_stored":1359508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":762837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542340,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171680,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200160,"statfs":{"total":96636764160,"available":96460963840,"internally_reserved":0,"allocated":1773568,"data_stored":776525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":173996485},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:37:05.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:37:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 -- 172.21.15.46:0/1075265855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f41527b0 msgr2=0x7f04f415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 --2- 172.21.15.46:0/1075265855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f41527b0 0x7f04f415bb60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f04f40cca90 tx=0x7f04dc01c800 comp rx=0 tx=0).stop 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 -- 172.21.15.46:0/1075265855 shutdown_connections 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 --2- 172.21.15.46:0/1075265855 >> v2:172.21.15.99:3300/0 conn(0x7f04f40b4de0 0x7f04f40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 --2- 172.21.15.46:0/1075265855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f41527b0 0x7f04f415bb60 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 -- 172.21.15.46:0/1075265855 >> 172.21.15.46:0/1075265855 conn(0x7f04f40b2040 msgr2=0x7f04f40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 -- 172.21.15.46:0/1075265855 shutdown_connections 2024-09-17T03:37:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 -- 172.21.15.46:0/1075265855 wait complete. 2024-09-17T03:37:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.912+0000 7f04f8a00640 1 Processor -- start 2024-09-17T03:37:05.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- start start 2024-09-17T03:37:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04f40b4de0 0x7f04f40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 0x7f04f40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04f41527b0 0x7f04f40ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4168920 con 0x7f04f40b5c70 2024-09-17T03:37:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41687a0 con 0x7f04f41527b0 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f04f4168aa0 con 0x7f04f40b4de0 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 0x7f04f40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 0x7f04f40ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52312/0 (socket says 172.21.15.46:52312) 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 -- 172.21.15.46:0/647213423 learned_addr learned my addr 172.21.15.46:0/647213423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f1e00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04f40b4de0 0x7f04f40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f2800640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04f41527b0 0x7f04f40ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 -- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04f40b4de0 msgr2=0x7f04f40c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04f40b4de0 0x7f04f40c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 -- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04f41527b0 msgr2=0x7f04f40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04f41527b0 0x7f04f40ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 -- 172.21.15.46:0/647213423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f415b1d0 con 0x7f04f40b5c70 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ebe00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 0x7f04f40ce960 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0b0040 tx=0x7f04dc0a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0bb070 con 0x7f04f40b5c70 2024-09-17T03:37:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04dc0043b0 con 0x7f04f40b5c70 2024-09-17T03:37:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f415b4b0 con 0x7f04f40b5c70 2024-09-17T03:37:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0b2040 con 0x7f04f40b5c70 2024-09-17T03:37:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41570c0 con 0x7f04f40b5c70 2024-09-17T03:37:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.916+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f4156bd0 con 0x7f04f40b5c70 2024-09-17T03:37:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.920+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f04dc005290 con 0x7f04f40b5c70 2024-09-17T03:37:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.920+0000 7f04ea000640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04d0075570 0x7f04d0077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:05.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.920+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f04dc12cd60 con 0x7f04f40b5c70 2024-09-17T03:37:05.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.920+0000 7f04f1e00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04d0075570 0x7f04d0077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:05.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.920+0000 7f04f1e00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04d0075570 0x7f04d0077a80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f04e4001650 tx=0x7f04e4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:05.924+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f04dc0164b0 con 0x7f04f40b5c70 2024-09-17T03:37:06.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.100+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04f406b540 con 0x7f04d0075570 2024-09-17T03:37:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.104+0000 7f04ea000640 1 -- 172.21.15.46:0/647213423 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251833 (secure 0 0 0) 0x7f04e40050d0 con 0x7f04d0075570 2024-09-17T03:37:06.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:37:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04d0075570 msgr2=0x7f04d0077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04d0075570 0x7f04d0077a80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f04e4001650 tx=0x7f04e4009040 comp rx=0 tx=0).stop 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 msgr2=0x7f04f40ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 0x7f04f40ce960 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0b0040 tx=0x7f04dc0a5740 comp rx=0 tx=0).stop 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 shutdown_connections 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f04d0075570 0x7f04d0077a80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04f41527b0 0x7f04f40ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f40b5c70 0x7f04f40ce960 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 --2- 172.21.15.46:0/647213423 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04f40b4de0 0x7f04f40c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 >> 172.21.15.46:0/647213423 conn(0x7f04f40b2040 msgr2=0x7f04f41583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 shutdown_connections 2024-09-17T03:37:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.108+0000 7f04f8a00640 1 -- 172.21.15.46:0/647213423 wait complete. 2024-09-17T03:37:06.128 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-09-17T03:37:05.579171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1171856,"kb_used_data":14924,"kb_used_omap":12,"kb_used_meta":1156723,"kb_avail":753802864,"statfs":{"total":773094113280,"available":771894132736,"internally_reserved":0,"allocated":15282176,"data_stored":7349196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1184484912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.512539+0000","last_change":"2024-09-17T03:37:01.509979+0000","last_active":"2024-09-17T03:37:01.512539+0000","last_peered":"2024-09-17T03:37:01.512539+0000","last_clean":"2024-09-17T03:37:01.512539+0000","last_became_active":"2024-09-17T03:37:01.508226+0000","last_became_peered":"2024-09-17T03:37:01.508226+0000","last_unstale":"2024-09-17T03:37:01.512539+0000","last_undegraded":"2024-09-17T03:37:01.512539+0000","last_fullsized":"2024-09-17T03:37:01.512539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:52.973966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531593+0000","last_change":"2024-09-17T03:37:01.525396+0000","last_active":"2024-09-17T03:37:01.531593+0000","last_peered":"2024-09-17T03:37:01.531593+0000","last_clean":"2024-09-17T03:37:01.531593+0000","last_became_active":"2024-09-17T03:37:01.525068+0000","last_became_peered":"2024-09-17T03:37:01.525068+0000","last_unstale":"2024-09-17T03:37:01.531593+0000","last_undegraded":"2024-09-17T03:37:01.531593+0000","last_fullsized":"2024-09-17T03:37:01.531593+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:13.211290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519515+0000","last_change":"2024-09-17T03:37:01.517425+0000","last_active":"2024-09-17T03:37:01.519515+0000","last_peered":"2024-09-17T03:37:01.519515+0000","last_clean":"2024-09-17T03:37:01.519515+0000","last_became_active":"2024-09-17T03:37:01.517118+0000","last_became_peered":"2024-09-17T03:37:01.517118+0000","last_unstale":"2024-09-17T03:37:01.519515+0000","last_undegraded":"2024-09-17T03:37:01.519515+0000","last_fullsized":"2024-09-17T03:37:01.519515+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528816+0000","last_change":"2024-09-17T03:37:01.524178+0000","last_active":"2024-09-17T03:37:01.528816+0000","last_peered":"2024-09-17T03:37:01.528816+0000","last_clean":"2024-09-17T03:37:01.528816+0000","last_became_active":"2024-09-17T03:37:01.523558+0000","last_became_peered":"2024-09-17T03:37:01.523558+0000","last_unstale":"2024-09-17T03:37:01.528816+0000","last_undegraded":"2024-09-17T03:37:01.528816+0000","last_fullsized":"2024-09-17T03:37:01.528816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:24.391567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498567+0000","last_change":"2024-09-17T03:37:00.498567+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498567+0000","last_undegraded":"2024-09-17T03:37:00.498567+0000","last_fullsized":"2024-09-17T03:37:00.498567+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510156+0000","last_change":"2024-09-17T03:37:01.507556+0000","last_active":"2024-09-17T03:37:01.510156+0000","last_peered":"2024-09-17T03:37:01.510156+0000","last_clean":"2024-09-17T03:37:01.510156+0000","last_became_active":"2024-09-17T03:37:01.507216+0000","last_became_peered":"2024-09-17T03:37:01.507216+0000","last_unstale":"2024-09-17T03:37:01.510156+0000","last_undegraded":"2024-09-17T03:37:01.510156+0000","last_fullsized":"2024-09-17T03:37:01.510156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:58.827646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524913+0000","last_change":"2024-09-17T03:37:01.520291+0000","last_active":"2024-09-17T03:37:01.524913+0000","last_peered":"2024-09-17T03:37:01.524913+0000","last_clean":"2024-09-17T03:37:01.524913+0000","last_became_active":"2024-09-17T03:37:01.519510+0000","last_became_peered":"2024-09-17T03:37:01.519510+0000","last_unstale":"2024-09-17T03:37:01.524913+0000","last_undegraded":"2024-09-17T03:37:01.524913+0000","last_fullsized":"2024-09-17T03:37:01.524913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:14.128722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.515271+0000","last_change":"2024-09-17T03:37:01.511915+0000","last_active":"2024-09-17T03:37:01.515271+0000","last_peered":"2024-09-17T03:37:01.515271+0000","last_clean":"2024-09-17T03:37:01.515271+0000","last_became_active":"2024-09-17T03:37:01.511588+0000","last_became_peered":"2024-09-17T03:37:01.511588+0000","last_unstale":"2024-09-17T03:37:01.515271+0000","last_undegraded":"2024-09-17T03:37:01.515271+0000","last_fullsized":"2024-09-17T03:37:01.515271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:34.703407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518221+0000","last_change":"2024-09-17T03:37:01.504394+0000","last_active":"2024-09-17T03:37:01.518221+0000","last_peered":"2024-09-17T03:37:01.518221+0000","last_clean":"2024-09-17T03:37:01.518221+0000","last_became_active":"2024-09-17T03:37:01.504016+0000","last_became_peered":"2024-09-17T03:37:01.504016+0000","last_unstale":"2024-09-17T03:37:01.518221+0000","last_undegraded":"2024-09-17T03:37:01.518221+0000","last_fullsized":"2024-09-17T03:37:01.518221+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:47.918178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526096+0000","last_change":"2024-09-17T03:37:01.521538+0000","last_active":"2024-09-17T03:37:01.526096+0000","last_peered":"2024-09-17T03:37:01.526096+0000","last_clean":"2024-09-17T03:37:01.526096+0000","last_became_active":"2024-09-17T03:37:01.521253+0000","last_became_peered":"2024-09-17T03:37:01.521253+0000","last_unstale":"2024-09-17T03:37:01.526096+0000","last_undegraded":"2024-09-17T03:37:01.526096+0000","last_fullsized":"2024-09-17T03:37:01.526096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:33.873450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509897+0000","last_change":"2024-09-17T03:37:01.507075+0000","last_active":"2024-09-17T03:37:01.509897+0000","last_peered":"2024-09-17T03:37:01.509897+0000","last_clean":"2024-09-17T03:37:01.509897+0000","last_became_active":"2024-09-17T03:37:01.506711+0000","last_became_peered":"2024-09-17T03:37:01.506711+0000","last_unstale":"2024-09-17T03:37:01.509897+0000","last_undegraded":"2024-09-17T03:37:01.509897+0000","last_fullsized":"2024-09-17T03:37:01.509897+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:56.420785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.520006+0000","last_change":"2024-09-17T03:37:01.515856+0000","last_active":"2024-09-17T03:37:01.520006+0000","last_peered":"2024-09-17T03:37:01.520006+0000","last_clean":"2024-09-17T03:37:01.520006+0000","last_became_active":"2024-09-17T03:37:01.512742+0000","last_became_peered":"2024-09-17T03:37:01.512742+0000","last_unstale":"2024-09-17T03:37:01.520006+0000","last_undegraded":"2024-09-17T03:37:01.520006+0000","last_fullsized":"2024-09-17T03:37:01.520006+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:16.486830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498137+0000","last_change":"2024-09-17T03:37:00.498137+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498137+0000","last_undegraded":"2024-09-17T03:37:00.498137+0000","last_fullsized":"2024-09-17T03:37:00.498137+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534060+0000","last_change":"2024-09-17T03:37:01.530828+0000","last_active":"2024-09-17T03:37:01.534060+0000","last_peered":"2024-09-17T03:37:01.534060+0000","last_clean":"2024-09-17T03:37:01.534060+0000","last_became_active":"2024-09-17T03:37:01.529611+0000","last_became_peered":"2024-09-17T03:37:01.529611+0000","last_unstale":"2024-09-17T03:37:01.534060+0000","last_undegraded":"2024-09-17T03:37:01.534060+0000","last_fullsized":"2024-09-17T03:37:01.534060+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526083+0000","last_change":"2024-09-17T03:37:01.524406+0000","last_active":"2024-09-17T03:37:01.526083+0000","last_peered":"2024-09-17T03:37:01.526083+0000","last_clean":"2024-09-17T03:37:01.526083+0000","last_became_active":"2024-09-17T03:37:01.524097+0000","last_became_peered":"2024-09-17T03:37:01.524097+0000","last_unstale":"2024-09-17T03:37:01.526083+0000","last_undegraded":"2024-09-17T03:37:01.526083+0000","last_fullsized":"2024-09-17T03:37:01.526083+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:47.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534103+0000","last_change":"2024-09-17T03:37:01.527309+0000","last_active":"2024-09-17T03:37:01.534103+0000","last_peered":"2024-09-17T03:37:01.534103+0000","last_clean":"2024-09-17T03:37:01.534103+0000","last_became_active":"2024-09-17T03:37:01.526391+0000","last_became_peered":"2024-09-17T03:37:01.526391+0000","last_unstale":"2024-09-17T03:37:01.534103+0000","last_undegraded":"2024-09-17T03:37:01.534103+0000","last_fullsized":"2024-09-17T03:37:01.534103+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:42.503735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536631+0000","last_change":"2024-09-17T03:37:01.531294+0000","last_active":"2024-09-17T03:37:01.536631+0000","last_peered":"2024-09-17T03:37:01.536631+0000","last_clean":"2024-09-17T03:37:01.536631+0000","last_became_active":"2024-09-17T03:37:01.529834+0000","last_became_peered":"2024-09-17T03:37:01.529834+0000","last_unstale":"2024-09-17T03:37:01.536631+0000","last_undegraded":"2024-09-17T03:37:01.536631+0000","last_fullsized":"2024-09-17T03:37:01.536631+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:55.329149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523036+0000","last_change":"2024-09-17T03:37:01.518791+0000","last_active":"2024-09-17T03:37:01.523036+0000","last_peered":"2024-09-17T03:37:01.523036+0000","last_clean":"2024-09-17T03:37:01.523036+0000","last_became_active":"2024-09-17T03:37:01.516765+0000","last_became_peered":"2024-09-17T03:37:01.516765+0000","last_unstale":"2024-09-17T03:37:01.523036+0000","last_undegraded":"2024-09-17T03:37:01.523036+0000","last_fullsized":"2024-09-17T03:37:01.523036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529870+0000","last_change":"2024-09-17T03:37:01.527808+0000","last_active":"2024-09-17T03:37:01.529870+0000","last_peered":"2024-09-17T03:37:01.529870+0000","last_clean":"2024-09-17T03:37:01.529870+0000","last_became_active":"2024-09-17T03:37:01.527469+0000","last_became_peered":"2024-09-17T03:37:01.527469+0000","last_unstale":"2024-09-17T03:37:01.529870+0000","last_undegraded":"2024-09-17T03:37:01.529870+0000","last_fullsized":"2024-09-17T03:37:01.529870+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:54.208586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534127+0000","last_change":"2024-09-17T03:37:01.530734+0000","last_active":"2024-09-17T03:37:01.534127+0000","last_peered":"2024-09-17T03:37:01.534127+0000","last_clean":"2024-09-17T03:37:01.534127+0000","last_became_active":"2024-09-17T03:37:01.530184+0000","last_became_peered":"2024-09-17T03:37:01.530184+0000","last_unstale":"2024-09-17T03:37:01.534127+0000","last_undegraded":"2024-09-17T03:37:01.534127+0000","last_fullsized":"2024-09-17T03:37:01.534127+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:48.884673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525880+0000","last_change":"2024-09-17T03:37:01.520019+0000","last_active":"2024-09-17T03:37:01.525880+0000","last_peered":"2024-09-17T03:37:01.525880+0000","last_clean":"2024-09-17T03:37:01.525880+0000","last_became_active":"2024-09-17T03:37:01.519632+0000","last_became_peered":"2024-09-17T03:37:01.519632+0000","last_unstale":"2024-09-17T03:37:01.525880+0000","last_undegraded":"2024-09-17T03:37:01.525880+0000","last_fullsized":"2024-09-17T03:37:01.525880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:57.353653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536601+0000","last_change":"2024-09-17T03:37:01.532969+0000","last_active":"2024-09-17T03:37:01.536601+0000","last_peered":"2024-09-17T03:37:01.536601+0000","last_clean":"2024-09-17T03:37:01.536601+0000","last_became_active":"2024-09-17T03:37:01.530349+0000","last_became_peered":"2024-09-17T03:37:01.530349+0000","last_unstale":"2024-09-17T03:37:01.536601+0000","last_undegraded":"2024-09-17T03:37:01.536601+0000","last_fullsized":"2024-09-17T03:37:01.536601+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530028+0000","last_change":"2024-09-17T03:37:01.527670+0000","last_active":"2024-09-17T03:37:01.530028+0000","last_peered":"2024-09-17T03:37:01.530028+0000","last_clean":"2024-09-17T03:37:01.530028+0000","last_became_active":"2024-09-17T03:37:01.527335+0000","last_became_peered":"2024-09-17T03:37:01.527335+0000","last_unstale":"2024-09-17T03:37:01.530028+0000","last_undegraded":"2024-09-17T03:37:01.530028+0000","last_fullsized":"2024-09-17T03:37:01.530028+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525874+0000","last_change":"2024-09-17T03:37:01.520830+0000","last_active":"2024-09-17T03:37:01.525874+0000","last_peered":"2024-09-17T03:37:01.525874+0000","last_clean":"2024-09-17T03:37:01.525874+0000","last_became_active":"2024-09-17T03:37:01.520514+0000","last_became_peered":"2024-09-17T03:37:01.520514+0000","last_unstale":"2024-09-17T03:37:01.525874+0000","last_undegraded":"2024-09-17T03:37:01.525874+0000","last_fullsized":"2024-09-17T03:37:01.525874+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:28.913796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533954+0000","last_change":"2024-09-17T03:37:01.526010+0000","last_active":"2024-09-17T03:37:01.533954+0000","last_peered":"2024-09-17T03:37:01.533954+0000","last_clean":"2024-09-17T03:37:01.533954+0000","last_became_active":"2024-09-17T03:37:01.524307+0000","last_became_peered":"2024-09-17T03:37:01.524307+0000","last_unstale":"2024-09-17T03:37:01.533954+0000","last_undegraded":"2024-09-17T03:37:01.533954+0000","last_fullsized":"2024-09-17T03:37:01.533954+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:17.817507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529940+0000","last_change":"2024-09-17T03:37:01.526850+0000","last_active":"2024-09-17T03:37:01.529940+0000","last_peered":"2024-09-17T03:37:01.529940+0000","last_clean":"2024-09-17T03:37:01.529940+0000","last_became_active":"2024-09-17T03:37:01.526500+0000","last_became_peered":"2024-09-17T03:37:01.526500+0000","last_unstale":"2024-09-17T03:37:01.529940+0000","last_undegraded":"2024-09-17T03:37:01.529940+0000","last_fullsized":"2024-09-17T03:37:01.529940+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:01.514192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534424+0000","last_change":"2024-09-17T03:37:01.525369+0000","last_active":"2024-09-17T03:37:01.534424+0000","last_peered":"2024-09-17T03:37:01.534424+0000","last_clean":"2024-09-17T03:37:01.534424+0000","last_became_active":"2024-09-17T03:37:01.524994+0000","last_became_peered":"2024-09-17T03:37:01.524994+0000","last_unstale":"2024-09-17T03:37:01.534424+0000","last_undegraded":"2024-09-17T03:37:01.534424+0000","last_fullsized":"2024-09-17T03:37:01.534424+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:16.958527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521738+0000","last_change":"2024-09-17T03:37:01.514688+0000","last_active":"2024-09-17T03:37:01.521738+0000","last_peered":"2024-09-17T03:37:01.521738+0000","last_clean":"2024-09-17T03:37:01.521738+0000","last_became_active":"2024-09-17T03:37:01.505442+0000","last_became_peered":"2024-09-17T03:37:01.505442+0000","last_unstale":"2024-09-17T03:37:01.521738+0000","last_undegraded":"2024-09-17T03:37:01.521738+0000","last_fullsized":"2024-09-17T03:37:01.521738+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:51.069459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510776+0000","last_change":"2024-09-17T03:37:01.509207+0000","last_active":"2024-09-17T03:37:01.510776+0000","last_peered":"2024-09-17T03:37:01.510776+0000","last_clean":"2024-09-17T03:37:01.510776+0000","last_became_active":"2024-09-17T03:37:01.508207+0000","last_became_peered":"2024-09-17T03:37:01.508207+0000","last_unstale":"2024-09-17T03:37:01.510776+0000","last_undegraded":"2024-09-17T03:37:01.510776+0000","last_fullsized":"2024-09-17T03:37:01.510776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:48.105755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528879+0000","last_change":"2024-09-17T03:37:01.524856+0000","last_active":"2024-09-17T03:37:01.528879+0000","last_peered":"2024-09-17T03:37:01.528879+0000","last_clean":"2024-09-17T03:37:01.528879+0000","last_became_active":"2024-09-17T03:37:01.522971+0000","last_became_peered":"2024-09-17T03:37:01.522971+0000","last_unstale":"2024-09-17T03:37:01.528879+0000","last_undegraded":"2024-09-17T03:37:01.528879+0000","last_fullsized":"2024-09-17T03:37:01.528879+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517595+0000","last_change":"2024-09-17T03:37:01.504583+0000","last_active":"2024-09-17T03:37:01.517595+0000","last_peered":"2024-09-17T03:37:01.517595+0000","last_clean":"2024-09-17T03:37:01.517595+0000","last_became_active":"2024-09-17T03:37:01.504221+0000","last_became_peered":"2024-09-17T03:37:01.504221+0000","last_unstale":"2024-09-17T03:37:01.517595+0000","last_undegraded":"2024-09-17T03:37:01.517595+0000","last_fullsized":"2024-09-17T03:37:01.517595+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:18.113674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534132+0000","last_change":"2024-09-17T03:37:01.527737+0000","last_active":"2024-09-17T03:37:01.534132+0000","last_peered":"2024-09-17T03:37:01.534132+0000","last_clean":"2024-09-17T03:37:01.534132+0000","last_became_active":"2024-09-17T03:37:01.526783+0000","last_became_peered":"2024-09-17T03:37:01.526783+0000","last_unstale":"2024-09-17T03:37:01.534132+0000","last_undegraded":"2024-09-17T03:37:01.534132+0000","last_fullsized":"2024-09-17T03:37:01.534132+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:28.866527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518126+0000","last_change":"2024-09-17T03:37:01.513699+0000","last_active":"2024-09-17T03:37:01.518126+0000","last_peered":"2024-09-17T03:37:01.518126+0000","last_clean":"2024-09-17T03:37:01.518126+0000","last_became_active":"2024-09-17T03:37:01.513310+0000","last_became_peered":"2024-09-17T03:37:01.513310+0000","last_unstale":"2024-09-17T03:37:01.518126+0000","last_undegraded":"2024-09-17T03:37:01.518126+0000","last_fullsized":"2024-09-17T03:37:01.518126+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"142'4","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535697+0000","last_change":"2024-09-17T03:37:00.521721+0000","last_active":"2024-09-17T03:37:01.535697+0000","last_peered":"2024-09-17T03:37:01.535697+0000","last_clean":"2024-09-17T03:37:01.535697+0000","last_became_active":"2024-09-17T03:37:00.521476+0000","last_became_peered":"2024-09-17T03:37:00.521476+0000","last_unstale":"2024-09-17T03:37:01.535697+0000","last_undegraded":"2024-09-17T03:37:01.535697+0000","last_fullsized":"2024-09-17T03:37:01.535697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:11.151395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531349+0000","last_change":"2024-09-17T03:37:01.524327+0000","last_active":"2024-09-17T03:37:01.531349+0000","last_peered":"2024-09-17T03:37:01.531349+0000","last_clean":"2024-09-17T03:37:01.531349+0000","last_became_active":"2024-09-17T03:37:01.523988+0000","last_became_peered":"2024-09-17T03:37:01.523988+0000","last_unstale":"2024-09-17T03:37:01.531349+0000","last_undegraded":"2024-09-17T03:37:01.531349+0000","last_fullsized":"2024-09-17T03:37:01.531349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:57.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.522778+0000","last_change":"2024-09-17T03:37:00.513012+0000","last_active":"2024-09-17T03:37:01.522778+0000","last_peered":"2024-09-17T03:37:01.522778+0000","last_clean":"2024-09-17T03:37:01.522778+0000","last_became_active":"2024-09-17T03:37:00.512669+0000","last_became_peered":"2024-09-17T03:37:00.512669+0000","last_unstale":"2024-09-17T03:37:01.522778+0000","last_undegraded":"2024-09-17T03:37:01.522778+0000","last_fullsized":"2024-09-17T03:37:01.522778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:04.275671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523023+0000","last_change":"2024-09-17T03:37:01.520484+0000","last_active":"2024-09-17T03:37:01.523023+0000","last_peered":"2024-09-17T03:37:01.523023+0000","last_clean":"2024-09-17T03:37:01.523023+0000","last_became_active":"2024-09-17T03:37:01.518380+0000","last_became_peered":"2024-09-17T03:37:01.518380+0000","last_unstale":"2024-09-17T03:37:01.523023+0000","last_undegraded":"2024-09-17T03:37:01.523023+0000","last_fullsized":"2024-09-17T03:37:01.523023+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:10.898889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537662+0000","last_change":"2024-09-17T03:37:00.518391+0000","last_active":"2024-09-17T03:37:01.537662+0000","last_peered":"2024-09-17T03:37:01.537662+0000","last_clean":"2024-09-17T03:37:01.537662+0000","last_became_active":"2024-09-17T03:37:00.518014+0000","last_became_peered":"2024-09-17T03:37:00.518014+0000","last_unstale":"2024-09-17T03:37:01.537662+0000","last_undegraded":"2024-09-17T03:37:01.537662+0000","last_fullsized":"2024-09-17T03:37:01.537662+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518010+0000","last_change":"2024-09-17T03:37:01.513224+0000","last_active":"2024-09-17T03:37:01.518010+0000","last_peered":"2024-09-17T03:37:01.518010+0000","last_clean":"2024-09-17T03:37:01.518010+0000","last_became_active":"2024-09-17T03:37:01.512886+0000","last_became_peered":"2024-09-17T03:37:01.512886+0000","last_unstale":"2024-09-17T03:37:01.518010+0000","last_undegraded":"2024-09-17T03:37:01.518010+0000","last_fullsized":"2024-09-17T03:37:01.518010+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:13.599854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535032+0000","last_change":"2024-09-17T03:37:00.516162+0000","last_active":"2024-09-17T03:37:01.535032+0000","last_peered":"2024-09-17T03:37:01.535032+0000","last_clean":"2024-09-17T03:37:01.535032+0000","last_became_active":"2024-09-17T03:37:00.515367+0000","last_became_peered":"2024-09-17T03:37:00.515367+0000","last_unstale":"2024-09-17T03:37:01.535032+0000","last_undegraded":"2024-09-17T03:37:01.535032+0000","last_fullsized":"2024-09-17T03:37:01.535032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:08.671535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518310+0000","last_change":"2024-09-17T03:37:01.506152+0000","last_active":"2024-09-17T03:37:01.518310+0000","last_peered":"2024-09-17T03:37:01.518310+0000","last_clean":"2024-09-17T03:37:01.518310+0000","last_became_active":"2024-09-17T03:37:01.503759+0000","last_became_peered":"2024-09-17T03:37:01.503759+0000","last_unstale":"2024-09-17T03:37:01.518310+0000","last_undegraded":"2024-09-17T03:37:01.518310+0000","last_fullsized":"2024-09-17T03:37:01.518310+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:43.445864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528132+0000","last_change":"2024-09-17T03:37:00.510847+0000","last_active":"2024-09-17T03:37:01.528132+0000","last_peered":"2024-09-17T03:37:01.528132+0000","last_clean":"2024-09-17T03:37:01.528132+0000","last_became_active":"2024-09-17T03:37:00.509550+0000","last_became_peered":"2024-09-17T03:37:00.509550+0000","last_unstale":"2024-09-17T03:37:01.528132+0000","last_undegraded":"2024-09-17T03:37:01.528132+0000","last_fullsized":"2024-09-17T03:37:01.528132+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.371788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531043+0000","last_change":"2024-09-17T03:37:01.524253+0000","last_active":"2024-09-17T03:37:01.531043+0000","last_peered":"2024-09-17T03:37:01.531043+0000","last_clean":"2024-09-17T03:37:01.531043+0000","last_became_active":"2024-09-17T03:37:01.523935+0000","last_became_peered":"2024-09-17T03:37:01.523935+0000","last_unstale":"2024-09-17T03:37:01.531043+0000","last_undegraded":"2024-09-17T03:37:01.531043+0000","last_fullsized":"2024-09-17T03:37:01.531043+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.515827+0000","last_change":"2024-09-17T03:37:00.513776+0000","last_active":"2024-09-17T03:37:00.515827+0000","last_peered":"2024-09-17T03:37:00.515827+0000","last_clean":"2024-09-17T03:37:00.515827+0000","last_became_active":"2024-09-17T03:37:00.511311+0000","last_became_peered":"2024-09-17T03:37:00.511311+0000","last_unstale":"2024-09-17T03:37:00.515827+0000","last_undegraded":"2024-09-17T03:37:00.515827+0000","last_fullsized":"2024-09-17T03:37:00.515827+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:16.989474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517432+0000","last_change":"2024-09-17T03:37:01.513641+0000","last_active":"2024-09-17T03:37:01.517432+0000","last_peered":"2024-09-17T03:37:01.517432+0000","last_clean":"2024-09-17T03:37:01.517432+0000","last_became_active":"2024-09-17T03:37:01.513325+0000","last_became_peered":"2024-09-17T03:37:01.513325+0000","last_unstale":"2024-09-17T03:37:01.517432+0000","last_undegraded":"2024-09-17T03:37:01.517432+0000","last_fullsized":"2024-09-17T03:37:01.517432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:54.606013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.516501+0000","last_change":"2024-09-17T03:37:00.513452+0000","last_active":"2024-09-17T03:37:01.516501+0000","last_peered":"2024-09-17T03:37:01.516501+0000","last_clean":"2024-09-17T03:37:01.516501+0000","last_became_active":"2024-09-17T03:37:00.513118+0000","last_became_peered":"2024-09-17T03:37:00.513118+0000","last_unstale":"2024-09-17T03:37:01.516501+0000","last_undegraded":"2024-09-17T03:37:01.516501+0000","last_fullsized":"2024-09-17T03:37:01.516501+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:53.259179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498231+0000","last_change":"2024-09-17T03:37:00.498231+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498231+0000","last_undegraded":"2024-09-17T03:37:00.498231+0000","last_fullsized":"2024-09-17T03:37:00.498231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509557+0000","last_change":"2024-09-17T03:37:00.506180+0000","last_active":"2024-09-17T03:37:01.509557+0000","last_peered":"2024-09-17T03:37:01.509557+0000","last_clean":"2024-09-17T03:37:01.509557+0000","last_became_active":"2024-09-17T03:37:00.505868+0000","last_became_peered":"2024-09-17T03:37:00.505868+0000","last_unstale":"2024-09-17T03:37:01.509557+0000","last_undegraded":"2024-09-17T03:37:01.509557+0000","last_fullsized":"2024-09-17T03:37:01.509557+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497947+0000","last_change":"2024-09-17T03:37:00.497947+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497947+0000","last_undegraded":"2024-09-17T03:37:00.497947+0000","last_fullsized":"2024-09-17T03:37:00.497947+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494787+0000","last_change":"2024-09-17T03:37:00.513939+0000","last_active":"2024-09-17T03:37:01.494787+0000","last_peered":"2024-09-17T03:37:01.494787+0000","last_clean":"2024-09-17T03:37:01.494787+0000","last_became_active":"2024-09-17T03:37:00.513531+0000","last_became_peered":"2024-09-17T03:37:00.513531+0000","last_unstale":"2024-09-17T03:37:01.494787+0000","last_undegraded":"2024-09-17T03:37:01.494787+0000","last_fullsized":"2024-09-17T03:37:01.494787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:43.986216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536554+0000","last_change":"2024-09-17T03:37:01.530837+0000","last_active":"2024-09-17T03:37:01.536554+0000","last_peered":"2024-09-17T03:37:01.536554+0000","last_clean":"2024-09-17T03:37:01.536554+0000","last_became_active":"2024-09-17T03:37:01.530166+0000","last_became_peered":"2024-09-17T03:37:01.530166+0000","last_unstale":"2024-09-17T03:37:01.536554+0000","last_undegraded":"2024-09-17T03:37:01.536554+0000","last_fullsized":"2024-09-17T03:37:01.536554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:32.600471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.498465+0000","last_change":"2024-09-17T03:37:00.515644+0000","last_active":"2024-09-17T03:37:01.498465+0000","last_peered":"2024-09-17T03:37:01.498465+0000","last_clean":"2024-09-17T03:37:01.498465+0000","last_became_active":"2024-09-17T03:37:00.515326+0000","last_became_peered":"2024-09-17T03:37:00.515326+0000","last_unstale":"2024-09-17T03:37:01.498465+0000","last_undegraded":"2024-09-17T03:37:01.498465+0000","last_fullsized":"2024-09-17T03:37:01.498465+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:05.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534454+0000","last_change":"2024-09-17T03:37:01.530851+0000","last_active":"2024-09-17T03:37:01.534454+0000","last_peered":"2024-09-17T03:37:01.534454+0000","last_clean":"2024-09-17T03:37:01.534454+0000","last_became_active":"2024-09-17T03:37:01.530410+0000","last_became_peered":"2024-09-17T03:37:01.530410+0000","last_unstale":"2024-09-17T03:37:01.534454+0000","last_undegraded":"2024-09-17T03:37:01.534454+0000","last_fullsized":"2024-09-17T03:37:01.534454+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:54.349008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"142'6","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537003+0000","last_change":"2024-09-17T03:37:00.511424+0000","last_active":"2024-09-17T03:37:01.537003+0000","last_peered":"2024-09-17T03:37:01.537003+0000","last_clean":"2024-09-17T03:37:01.537003+0000","last_became_active":"2024-09-17T03:37:00.511055+0000","last_became_peered":"2024-09-17T03:37:00.511055+0000","last_unstale":"2024-09-17T03:37:01.537003+0000","last_undegraded":"2024-09-17T03:37:01.537003+0000","last_fullsized":"2024-09-17T03:37:01.537003+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:04.537830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526765+0000","last_change":"2024-09-17T03:37:01.520532+0000","last_active":"2024-09-17T03:37:01.526765+0000","last_peered":"2024-09-17T03:37:01.526765+0000","last_clean":"2024-09-17T03:37:01.526765+0000","last_became_active":"2024-09-17T03:37:01.520261+0000","last_became_peered":"2024-09-17T03:37:01.520261+0000","last_unstale":"2024-09-17T03:37:01.526765+0000","last_undegraded":"2024-09-17T03:37:01.526765+0000","last_fullsized":"2024-09-17T03:37:01.526765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538130+0000","last_change":"2024-09-17T03:37:00.509307+0000","last_active":"2024-09-17T03:37:01.538130+0000","last_peered":"2024-09-17T03:37:01.538130+0000","last_clean":"2024-09-17T03:37:01.538130+0000","last_became_active":"2024-09-17T03:37:00.508858+0000","last_became_peered":"2024-09-17T03:37:00.508858+0000","last_unstale":"2024-09-17T03:37:01.538130+0000","last_undegraded":"2024-09-17T03:37:01.538130+0000","last_fullsized":"2024-09-17T03:37:01.538130+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:19.533265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532147+0000","last_change":"2024-09-17T03:37:01.526495+0000","last_active":"2024-09-17T03:37:01.532147+0000","last_peered":"2024-09-17T03:37:01.532147+0000","last_clean":"2024-09-17T03:37:01.532147+0000","last_became_active":"2024-09-17T03:37:01.526193+0000","last_became_peered":"2024-09-17T03:37:01.526193+0000","last_unstale":"2024-09-17T03:37:01.532147+0000","last_undegraded":"2024-09-17T03:37:01.532147+0000","last_fullsized":"2024-09-17T03:37:01.532147+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:38.826521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506839+0000","last_change":"2024-09-17T03:37:00.506435+0000","last_active":"2024-09-17T03:37:01.506839+0000","last_peered":"2024-09-17T03:37:01.506839+0000","last_clean":"2024-09-17T03:37:01.506839+0000","last_became_active":"2024-09-17T03:37:00.505499+0000","last_became_peered":"2024-09-17T03:37:00.505499+0000","last_unstale":"2024-09-17T03:37:01.506839+0000","last_undegraded":"2024-09-17T03:37:01.506839+0000","last_fullsized":"2024-09-17T03:37:01.506839+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497356+0000","last_change":"2024-09-17T03:37:00.497356+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497356+0000","last_undegraded":"2024-09-17T03:37:00.497356+0000","last_fullsized":"2024-09-17T03:37:00.497356+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.490845+0000","last_change":"2024-09-17T03:37:00.497616+0000","last_active":"2024-09-17T03:37:01.490845+0000","last_peered":"2024-09-17T03:37:01.490845+0000","last_clean":"2024-09-17T03:37:01.490845+0000","last_became_active":"2024-09-17T03:37:00.497229+0000","last_became_peered":"2024-09-17T03:37:00.497229+0000","last_unstale":"2024-09-17T03:37:01.490845+0000","last_undegraded":"2024-09-17T03:37:01.490845+0000","last_fullsized":"2024-09-17T03:37:01.490845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:08.103469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531634+0000","last_change":"2024-09-17T03:37:01.525031+0000","last_active":"2024-09-17T03:37:01.531634+0000","last_peered":"2024-09-17T03:37:01.531634+0000","last_clean":"2024-09-17T03:37:01.531634+0000","last_became_active":"2024-09-17T03:37:01.524710+0000","last_became_peered":"2024-09-17T03:37:01.524710+0000","last_unstale":"2024-09-17T03:37:01.531634+0000","last_undegraded":"2024-09-17T03:37:01.531634+0000","last_fullsized":"2024-09-17T03:37:01.531634+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"142'8","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534881+0000","last_change":"2024-09-17T03:37:00.506497+0000","last_active":"2024-09-17T03:37:01.534881+0000","last_peered":"2024-09-17T03:37:01.534881+0000","last_clean":"2024-09-17T03:37:01.534881+0000","last_became_active":"2024-09-17T03:37:00.506195+0000","last_became_peered":"2024-09-17T03:37:00.506195+0000","last_unstale":"2024-09-17T03:37:01.534881+0000","last_undegraded":"2024-09-17T03:37:01.534881+0000","last_fullsized":"2024-09-17T03:37:01.534881+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:35.347269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.511186+0000","last_change":"2024-09-17T03:37:01.510170+0000","last_active":"2024-09-17T03:37:01.511186+0000","last_peered":"2024-09-17T03:37:01.511186+0000","last_clean":"2024-09-17T03:37:01.511186+0000","last_became_active":"2024-09-17T03:37:01.509806+0000","last_became_peered":"2024-09-17T03:37:01.509806+0000","last_unstale":"2024-09-17T03:37:01.511186+0000","last_undegraded":"2024-09-17T03:37:01.511186+0000","last_fullsized":"2024-09-17T03:37:01.511186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:43.917446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.511017+0000","last_change":"2024-09-17T03:37:00.507289+0000","last_active":"2024-09-17T03:37:00.511017+0000","last_peered":"2024-09-17T03:37:00.511017+0000","last_clean":"2024-09-17T03:37:00.511017+0000","last_became_active":"2024-09-17T03:37:00.506994+0000","last_became_peered":"2024-09-17T03:37:00.506994+0000","last_unstale":"2024-09-17T03:37:00.511017+0000","last_undegraded":"2024-09-17T03:37:00.511017+0000","last_fullsized":"2024-09-17T03:37:00.511017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:13.903403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533925+0000","last_change":"2024-09-17T03:37:01.525640+0000","last_active":"2024-09-17T03:37:01.533925+0000","last_peered":"2024-09-17T03:37:01.533925+0000","last_clean":"2024-09-17T03:37:01.533925+0000","last_became_active":"2024-09-17T03:37:01.523930+0000","last_became_peered":"2024-09-17T03:37:01.523930+0000","last_unstale":"2024-09-17T03:37:01.533925+0000","last_undegraded":"2024-09-17T03:37:01.533925+0000","last_fullsized":"2024-09-17T03:37:01.533925+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519679+0000","last_change":"2024-09-17T03:37:00.514697+0000","last_active":"2024-09-17T03:37:01.519679+0000","last_peered":"2024-09-17T03:37:01.519679+0000","last_clean":"2024-09-17T03:37:01.519679+0000","last_became_active":"2024-09-17T03:37:00.513958+0000","last_became_peered":"2024-09-17T03:37:00.513958+0000","last_unstale":"2024-09-17T03:37:01.519679+0000","last_undegraded":"2024-09-17T03:37:01.519679+0000","last_fullsized":"2024-09-17T03:37:01.519679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:27.979522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529910+0000","last_change":"2024-09-17T03:37:01.521591+0000","last_active":"2024-09-17T03:37:01.529910+0000","last_peered":"2024-09-17T03:37:01.529910+0000","last_clean":"2024-09-17T03:37:01.529910+0000","last_became_active":"2024-09-17T03:37:01.521066+0000","last_became_peered":"2024-09-17T03:37:01.521066+0000","last_unstale":"2024-09-17T03:37:01.529910+0000","last_undegraded":"2024-09-17T03:37:01.529910+0000","last_fullsized":"2024-09-17T03:37:01.529910+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533162+0000","last_change":"2024-09-17T03:37:00.510701+0000","last_active":"2024-09-17T03:37:01.533162+0000","last_peered":"2024-09-17T03:37:01.533162+0000","last_clean":"2024-09-17T03:37:01.533162+0000","last_became_active":"2024-09-17T03:37:00.510374+0000","last_became_peered":"2024-09-17T03:37:00.510374+0000","last_unstale":"2024-09-17T03:37:01.533162+0000","last_undegraded":"2024-09-17T03:37:01.533162+0000","last_fullsized":"2024-09-17T03:37:01.533162+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:40.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510426+0000","last_change":"2024-09-17T03:37:01.508528+0000","last_active":"2024-09-17T03:37:01.510426+0000","last_peered":"2024-09-17T03:37:01.510426+0000","last_clean":"2024-09-17T03:37:01.510426+0000","last_became_active":"2024-09-17T03:37:01.508173+0000","last_became_peered":"2024-09-17T03:37:01.508173+0000","last_unstale":"2024-09-17T03:37:01.510426+0000","last_undegraded":"2024-09-17T03:37:01.510426+0000","last_fullsized":"2024-09-17T03:37:01.510426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:47.227568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494837+0000","last_change":"2024-09-17T03:37:00.517789+0000","last_active":"2024-09-17T03:37:01.494837+0000","last_peered":"2024-09-17T03:37:01.494837+0000","last_clean":"2024-09-17T03:37:01.494837+0000","last_became_active":"2024-09-17T03:37:00.517101+0000","last_became_peered":"2024-09-17T03:37:00.517101+0000","last_unstale":"2024-09-17T03:37:01.494837+0000","last_undegraded":"2024-09-17T03:37:01.494837+0000","last_fullsized":"2024-09-17T03:37:01.494837+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:30.163493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524630+0000","last_change":"2024-09-17T03:37:01.518635+0000","last_active":"2024-09-17T03:37:01.524630+0000","last_peered":"2024-09-17T03:37:01.524630+0000","last_clean":"2024-09-17T03:37:01.524630+0000","last_became_active":"2024-09-17T03:37:01.518369+0000","last_became_peered":"2024-09-17T03:37:01.518369+0000","last_unstale":"2024-09-17T03:37:01.524630+0000","last_undegraded":"2024-09-17T03:37:01.524630+0000","last_fullsized":"2024-09-17T03:37:01.524630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:42.751762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529734+0000","last_change":"2024-09-17T03:37:00.502280+0000","last_active":"2024-09-17T03:37:01.529734+0000","last_peered":"2024-09-17T03:37:01.529734+0000","last_clean":"2024-09-17T03:37:01.529734+0000","last_became_active":"2024-09-17T03:37:00.501924+0000","last_became_peered":"2024-09-17T03:37:00.501924+0000","last_unstale":"2024-09-17T03:37:01.529734+0000","last_undegraded":"2024-09-17T03:37:01.529734+0000","last_fullsized":"2024-09-17T03:37:01.529734+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:46.918045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524298+0000","last_change":"2024-09-17T03:37:01.523047+0000","last_active":"2024-09-17T03:37:01.524298+0000","last_peered":"2024-09-17T03:37:01.524298+0000","last_clean":"2024-09-17T03:37:01.524298+0000","last_became_active":"2024-09-17T03:37:01.522670+0000","last_became_peered":"2024-09-17T03:37:01.522670+0000","last_unstale":"2024-09-17T03:37:01.524298+0000","last_undegraded":"2024-09-17T03:37:01.524298+0000","last_fullsized":"2024-09-17T03:37:01.524298+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:03.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496644+0000","last_change":"2024-09-17T03:37:00.519370+0000","last_active":"2024-09-17T03:37:01.496644+0000","last_peered":"2024-09-17T03:37:01.496644+0000","last_clean":"2024-09-17T03:37:01.496644+0000","last_became_active":"2024-09-17T03:37:00.519058+0000","last_became_peered":"2024-09-17T03:37:00.519058+0000","last_unstale":"2024-09-17T03:37:01.496644+0000","last_undegraded":"2024-09-17T03:37:01.496644+0000","last_fullsized":"2024-09-17T03:37:01.496644+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:12.207851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530069+0000","last_change":"2024-09-17T03:37:01.528223+0000","last_active":"2024-09-17T03:37:01.530069+0000","last_peered":"2024-09-17T03:37:01.530069+0000","last_clean":"2024-09-17T03:37:01.530069+0000","last_became_active":"2024-09-17T03:37:01.527875+0000","last_became_peered":"2024-09-17T03:37:01.527875+0000","last_unstale":"2024-09-17T03:37:01.530069+0000","last_undegraded":"2024-09-17T03:37:01.530069+0000","last_fullsized":"2024-09-17T03:37:01.530069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496550+0000","last_change":"2024-09-17T03:37:00.514756+0000","last_active":"2024-09-17T03:37:01.496550+0000","last_peered":"2024-09-17T03:37:01.496550+0000","last_clean":"2024-09-17T03:37:01.496550+0000","last_became_active":"2024-09-17T03:37:00.514505+0000","last_became_peered":"2024-09-17T03:37:00.514505+0000","last_unstale":"2024-09-17T03:37:01.496550+0000","last_undegraded":"2024-09-17T03:37:01.496550+0000","last_fullsized":"2024-09-17T03:37:01.496550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524885+0000","last_change":"2024-09-17T03:37:01.519916+0000","last_active":"2024-09-17T03:37:01.524885+0000","last_peered":"2024-09-17T03:37:01.524885+0000","last_clean":"2024-09-17T03:37:01.524885+0000","last_became_active":"2024-09-17T03:37:01.519171+0000","last_became_peered":"2024-09-17T03:37:01.519171+0000","last_unstale":"2024-09-17T03:37:01.524885+0000","last_undegraded":"2024-09-17T03:37:01.524885+0000","last_fullsized":"2024-09-17T03:37:01.524885+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:18.402815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.507817+0000","last_change":"2024-09-17T03:37:00.516033+0000","last_active":"2024-09-17T03:37:01.507817+0000","last_peered":"2024-09-17T03:37:01.507817+0000","last_clean":"2024-09-17T03:37:01.507817+0000","last_became_active":"2024-09-17T03:37:00.515737+0000","last_became_peered":"2024-09-17T03:37:00.515737+0000","last_unstale":"2024-09-17T03:37:01.507817+0000","last_undegraded":"2024-09-17T03:37:01.507817+0000","last_fullsized":"2024-09-17T03:37:01.507817+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:23.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497461+0000","last_change":"2024-09-17T03:37:00.497461+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497461+0000","last_undegraded":"2024-09-17T03:37:00.497461+0000","last_fullsized":"2024-09-17T03:37:00.497461+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533095+0000","last_change":"2024-09-17T03:37:00.518901+0000","last_active":"2024-09-17T03:37:01.533095+0000","last_peered":"2024-09-17T03:37:01.533095+0000","last_clean":"2024-09-17T03:37:01.533095+0000","last_became_active":"2024-09-17T03:37:00.518609+0000","last_became_peered":"2024-09-17T03:37:00.518609+0000","last_unstale":"2024-09-17T03:37:01.533095+0000","last_undegraded":"2024-09-17T03:37:01.533095+0000","last_fullsized":"2024-09-17T03:37:01.533095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:31.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519672+0000","last_change":"2024-09-17T03:37:01.515337+0000","last_active":"2024-09-17T03:37:01.519672+0000","last_peered":"2024-09-17T03:37:01.519672+0000","last_clean":"2024-09-17T03:37:01.519672+0000","last_became_active":"2024-09-17T03:37:01.515033+0000","last_became_peered":"2024-09-17T03:37:01.515033+0000","last_unstale":"2024-09-17T03:37:01.519672+0000","last_undegraded":"2024-09-17T03:37:01.519672+0000","last_fullsized":"2024-09-17T03:37:01.519672+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494565+0000","last_change":"2024-09-17T03:37:00.509124+0000","last_active":"2024-09-17T03:37:01.494565+0000","last_peered":"2024-09-17T03:37:01.494565+0000","last_clean":"2024-09-17T03:37:01.494565+0000","last_became_active":"2024-09-17T03:37:00.508710+0000","last_became_peered":"2024-09-17T03:37:00.508710+0000","last_unstale":"2024-09-17T03:37:01.494565+0000","last_undegraded":"2024-09-17T03:37:01.494565+0000","last_fullsized":"2024-09-17T03:37:01.494565+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:09.199848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518488+0000","last_change":"2024-09-17T03:37:01.514963+0000","last_active":"2024-09-17T03:37:01.518488+0000","last_peered":"2024-09-17T03:37:01.518488+0000","last_clean":"2024-09-17T03:37:01.518488+0000","last_became_active":"2024-09-17T03:37:01.514337+0000","last_became_peered":"2024-09-17T03:37:01.514337+0000","last_unstale":"2024-09-17T03:37:01.518488+0000","last_undegraded":"2024-09-17T03:37:01.518488+0000","last_fullsized":"2024-09-17T03:37:01.518488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:17.288497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.497349+0000","last_change":"2024-09-17T03:37:00.506035+0000","last_active":"2024-09-17T03:37:01.497349+0000","last_peered":"2024-09-17T03:37:01.497349+0000","last_clean":"2024-09-17T03:37:01.497349+0000","last_became_active":"2024-09-17T03:37:00.505658+0000","last_became_peered":"2024-09-17T03:37:00.505658+0000","last_unstale":"2024-09-17T03:37:01.497349+0000","last_undegraded":"2024-09-17T03:37:01.497349+0000","last_fullsized":"2024-09-17T03:37:01.497349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:52.915238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521428+0000","last_change":"2024-09-17T03:37:01.517782+0000","last_active":"2024-09-17T03:37:01.521428+0000","last_peered":"2024-09-17T03:37:01.521428+0000","last_clean":"2024-09-17T03:37:01.521428+0000","last_became_active":"2024-09-17T03:37:01.517363+0000","last_became_peered":"2024-09-17T03:37:01.517363+0000","last_unstale":"2024-09-17T03:37:01.521428+0000","last_undegraded":"2024-09-17T03:37:01.521428+0000","last_fullsized":"2024-09-17T03:37:01.521428+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:17.301571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496098+0000","last_change":"2024-09-17T03:37:00.514913+0000","last_active":"2024-09-17T03:37:01.496098+0000","last_peered":"2024-09-17T03:37:01.496098+0000","last_clean":"2024-09-17T03:37:01.496098+0000","last_became_active":"2024-09-17T03:37:00.514588+0000","last_became_peered":"2024-09-17T03:37:00.514588+0000","last_unstale":"2024-09-17T03:37:01.496098+0000","last_undegraded":"2024-09-17T03:37:01.496098+0000","last_fullsized":"2024-09-17T03:37:01.496098+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:15.350414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532735+0000","last_change":"2024-09-17T03:37:01.523154+0000","last_active":"2024-09-17T03:37:01.532735+0000","last_peered":"2024-09-17T03:37:01.532735+0000","last_clean":"2024-09-17T03:37:01.532735+0000","last_became_active":"2024-09-17T03:37:01.522826+0000","last_became_peered":"2024-09-17T03:37:01.522826+0000","last_unstale":"2024-09-17T03:37:01.532735+0000","last_undegraded":"2024-09-17T03:37:01.532735+0000","last_fullsized":"2024-09-17T03:37:01.532735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:19.317037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.501477+0000","last_change":"2024-09-17T03:37:00.510360+0000","last_active":"2024-09-17T03:37:01.501477+0000","last_peered":"2024-09-17T03:37:01.501477+0000","last_clean":"2024-09-17T03:37:01.501477+0000","last_became_active":"2024-09-17T03:37:00.510038+0000","last_became_peered":"2024-09-17T03:37:00.510038+0000","last_unstale":"2024-09-17T03:37:01.501477+0000","last_undegraded":"2024-09-17T03:37:01.501477+0000","last_fullsized":"2024-09-17T03:37:01.501477+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:31.275321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497390+0000","last_change":"2024-09-17T03:37:00.497390+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497390+0000","last_undegraded":"2024-09-17T03:37:00.497390+0000","last_fullsized":"2024-09-17T03:37:00.497390+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496964+0000","last_change":"2024-09-17T03:37:00.518866+0000","last_active":"2024-09-17T03:37:01.496964+0000","last_peered":"2024-09-17T03:37:01.496964+0000","last_clean":"2024-09-17T03:37:01.496964+0000","last_became_active":"2024-09-17T03:37:00.518531+0000","last_became_peered":"2024-09-17T03:37:00.518531+0000","last_unstale":"2024-09-17T03:37:01.496964+0000","last_undegraded":"2024-09-17T03:37:01.496964+0000","last_fullsized":"2024-09-17T03:37:01.496964+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.823082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537156+0000","last_change":"2024-09-17T03:37:01.534797+0000","last_active":"2024-09-17T03:37:01.537156+0000","last_peered":"2024-09-17T03:37:01.537156+0000","last_clean":"2024-09-17T03:37:01.537156+0000","last_became_active":"2024-09-17T03:37:01.534442+0000","last_became_peered":"2024-09-17T03:37:01.534442+0000","last_unstale":"2024-09-17T03:37:01.537156+0000","last_undegraded":"2024-09-17T03:37:01.537156+0000","last_fullsized":"2024-09-17T03:37:01.537156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:42.572373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506749+0000","last_change":"2024-09-17T03:37:00.512368+0000","last_active":"2024-09-17T03:37:01.506749+0000","last_peered":"2024-09-17T03:37:01.506749+0000","last_clean":"2024-09-17T03:37:01.506749+0000","last_became_active":"2024-09-17T03:37:00.512041+0000","last_became_peered":"2024-09-17T03:37:00.512041+0000","last_unstale":"2024-09-17T03:37:01.506749+0000","last_undegraded":"2024-09-17T03:37:01.506749+0000","last_fullsized":"2024-09-17T03:37:01.506749+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497341+0000","last_change":"2024-09-17T03:37:00.497341+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497341+0000","last_undegraded":"2024-09-17T03:37:00.497341+0000","last_fullsized":"2024-09-17T03:37:00.497341+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.492735+0000","last_change":"2024-09-17T03:37:00.508129+0000","last_active":"2024-09-17T03:37:01.492735+0000","last_peered":"2024-09-17T03:37:01.492735+0000","last_clean":"2024-09-17T03:37:01.492735+0000","last_became_active":"2024-09-17T03:37:00.506285+0000","last_became_peered":"2024-09-17T03:37:00.506285+0000","last_unstale":"2024-09-17T03:37:01.492735+0000","last_undegraded":"2024-09-17T03:37:01.492735+0000","last_fullsized":"2024-09-17T03:37:01.492735+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:59.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521767+0000","last_change":"2024-09-17T03:37:01.516795+0000","last_active":"2024-09-17T03:37:01.521767+0000","last_peered":"2024-09-17T03:37:01.521767+0000","last_clean":"2024-09-17T03:37:01.521767+0000","last_became_active":"2024-09-17T03:37:01.516423+0000","last_became_peered":"2024-09-17T03:37:01.516423+0000","last_unstale":"2024-09-17T03:37:01.521767+0000","last_undegraded":"2024-09-17T03:37:01.521767+0000","last_fullsized":"2024-09-17T03:37:01.521767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:14.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"142'3","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538197+0000","last_change":"2024-09-17T03:37:00.515111+0000","last_active":"2024-09-17T03:37:01.538197+0000","last_peered":"2024-09-17T03:37:01.538197+0000","last_clean":"2024-09-17T03:37:01.538197+0000","last_became_active":"2024-09-17T03:37:00.514784+0000","last_became_peered":"2024-09-17T03:37:00.514784+0000","last_unstale":"2024-09-17T03:37:01.538197+0000","last_undegraded":"2024-09-17T03:37:01.538197+0000","last_fullsized":"2024-09-17T03:37:01.538197+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:41.748322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":333,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496801+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:37:01.496801+0000","last_peered":"2024-09-17T03:37:01.496801+0000","last_clean":"2024-09-17T03:37:01.496801+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:37:01.496801+0000","last_undegraded":"2024-09-17T03:37:01.496801+0000","last_fullsized":"2024-09-17T03:37:01.496801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119528,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252312,"statfs":{"total":96636764160,"available":96514367488,"internally_reserved":0,"allocated":2371584,"data_stored":1366963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149452,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222388,"statfs":{"total":96636764160,"available":96483725312,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159188,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212652,"statfs":{"total":96636764160,"available":96473755648,"internally_reserved":0,"allocated":1769472,"data_stored":775943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127244,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244596,"statfs":{"total":96636764160,"available":96506466304,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167504,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204336,"statfs":{"total":96636764160,"available":96465240064,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150032,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221808,"statfs":{"total":96636764160,"available":96483131392,"internally_reserved":0,"allocated":2351104,"data_stored":1359508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":762837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542340,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171680,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200160,"statfs":{"total":96636764160,"available":96460963840,"internally_reserved":0,"allocated":1773568,"data_stored":776525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":173996485},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:37:06.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:37:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 -- 172.21.15.46:0/2687746063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 msgr2=0x7fe0ec15f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 --2- 172.21.15.46:0/2687746063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec15f170 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec071970 tx=0x7fe0d801c7d0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 -- 172.21.15.46:0/2687746063 shutdown_connections 2024-09-17T03:37:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 --2- 172.21.15.46:0/2687746063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec15f170 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 --2- 172.21.15.46:0/2687746063 >> v2:172.21.15.99:3300/0 conn(0x7fe0ec15c5d0 0x7fe0ec073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 -- 172.21.15.46:0/2687746063 >> 172.21.15.46:0/2687746063 conn(0x7fe0ec0b2c40 msgr2=0x7fe0ec0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 -- 172.21.15.46:0/2687746063 shutdown_connections 2024-09-17T03:37:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.332+0000 7fe0f2600640 1 -- 172.21.15.46:0/2687746063 wait complete. 2024-09-17T03:37:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 Processor -- start 2024-09-17T03:37:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 -- start start 2024-09-17T03:37:06.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec0afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0ec15c5d0 0x7fe0ec1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec15f6b0 0x7fe0ec1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec06f980 con 0x7fe0ec074300 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec0b3d50 con 0x7fe0ec15f6b0 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ec06fb00 con 0x7fe0ec15c5d0 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec0afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec0afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52328/0 (socket says 172.21.15.46:52328) 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 -- 172.21.15.46:0/1384549517 learned_addr learned my addr 172.21.15.46:0/1384549517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eaa00640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0ec15c5d0 0x7fe0ec1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0ebe00640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec15f6b0 0x7fe0ec1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 -- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0ec15c5d0 msgr2=0x7fe0ec1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0ec15c5d0 0x7fe0ec1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 -- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec15f6b0 msgr2=0x7fe0ec1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec15f6b0 0x7fe0ec1536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 -- 172.21.15.46:0/1384549517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec1563c0 con 0x7fe0ec074300 2024-09-17T03:37:06.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eaa00640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0ec15c5d0 0x7fe0ec1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:06.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0ebe00640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec15f6b0 0x7fe0ec1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:06.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0eb400640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec0afc40 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0058d0 tx=0x7fe0dc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:06.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc017070 con 0x7fe0ec074300 2024-09-17T03:37:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0dc009d20 con 0x7fe0ec074300 2024-09-17T03:37:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec1566a0 con 0x7fe0ec074300 2024-09-17T03:37:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0106c0 con 0x7fe0ec074300 2024-09-17T03:37:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.336+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec156c60 con 0x7fe0ec074300 2024-09-17T03:37:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.340+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec15f170 con 0x7fe0ec074300 2024-09-17T03:37:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.340+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fe0dc01a070 con 0x7fe0ec074300 2024-09-17T03:37:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.340+0000 7fe0e3400640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0cc075570 0x7fe0cc077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.340+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7fe0dc093610 con 0x7fe0ec074300 2024-09-17T03:37:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.340+0000 7fe0eaa00640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0cc075570 0x7fe0cc077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.340+0000 7fe0eaa00640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0cc075570 0x7fe0cc077a80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80af000 tx=0x7fe0d801b010 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.344+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe0dc060090 con 0x7fe0ec074300 2024-09-17T03:37:06.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.516+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0ec06b540 con 0x7fe0cc075570 2024-09-17T03:37:06.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.520+0000 7fe0e3400640 1 -- 172.21.15.46:0/1384549517 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251833 (secure 0 0 0) 0x7fe0d80b3040 con 0x7fe0cc075570 2024-09-17T03:37:06.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:06.527 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0cc075570 msgr2=0x7fe0cc077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0cc075570 0x7fe0cc077a80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80af000 tx=0x7fe0d801b010 comp rx=0 tx=0).stop 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 msgr2=0x7fe0ec0afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec0afc40 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0058d0 tx=0x7fe0dc00dde0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 shutdown_connections 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe0cc075570 0x7fe0cc077a80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec15f6b0 0x7fe0ec1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0ec15c5d0 0x7fe0ec1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 --2- 172.21.15.46:0/1384549517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0ec074300 0x7fe0ec0afc40 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 >> 172.21.15.46:0/1384549517 conn(0x7fe0ec0b2c40 msgr2=0x7fe0ec154800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:06.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 shutdown_connections 2024-09-17T03:37:06.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.524+0000 7fe0f2600640 1 -- 172.21.15.46:0/1384549517 wait complete. 2024-09-17T03:37:06.548 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-09-17T03:37:05.579171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1171856,"kb_used_data":14924,"kb_used_omap":12,"kb_used_meta":1156723,"kb_avail":753802864,"statfs":{"total":773094113280,"available":771894132736,"internally_reserved":0,"allocated":15282176,"data_stored":7349196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1184484912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.512539+0000","last_change":"2024-09-17T03:37:01.509979+0000","last_active":"2024-09-17T03:37:01.512539+0000","last_peered":"2024-09-17T03:37:01.512539+0000","last_clean":"2024-09-17T03:37:01.512539+0000","last_became_active":"2024-09-17T03:37:01.508226+0000","last_became_peered":"2024-09-17T03:37:01.508226+0000","last_unstale":"2024-09-17T03:37:01.512539+0000","last_undegraded":"2024-09-17T03:37:01.512539+0000","last_fullsized":"2024-09-17T03:37:01.512539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:52.973966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531593+0000","last_change":"2024-09-17T03:37:01.525396+0000","last_active":"2024-09-17T03:37:01.531593+0000","last_peered":"2024-09-17T03:37:01.531593+0000","last_clean":"2024-09-17T03:37:01.531593+0000","last_became_active":"2024-09-17T03:37:01.525068+0000","last_became_peered":"2024-09-17T03:37:01.525068+0000","last_unstale":"2024-09-17T03:37:01.531593+0000","last_undegraded":"2024-09-17T03:37:01.531593+0000","last_fullsized":"2024-09-17T03:37:01.531593+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:13.211290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519515+0000","last_change":"2024-09-17T03:37:01.517425+0000","last_active":"2024-09-17T03:37:01.519515+0000","last_peered":"2024-09-17T03:37:01.519515+0000","last_clean":"2024-09-17T03:37:01.519515+0000","last_became_active":"2024-09-17T03:37:01.517118+0000","last_became_peered":"2024-09-17T03:37:01.517118+0000","last_unstale":"2024-09-17T03:37:01.519515+0000","last_undegraded":"2024-09-17T03:37:01.519515+0000","last_fullsized":"2024-09-17T03:37:01.519515+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528816+0000","last_change":"2024-09-17T03:37:01.524178+0000","last_active":"2024-09-17T03:37:01.528816+0000","last_peered":"2024-09-17T03:37:01.528816+0000","last_clean":"2024-09-17T03:37:01.528816+0000","last_became_active":"2024-09-17T03:37:01.523558+0000","last_became_peered":"2024-09-17T03:37:01.523558+0000","last_unstale":"2024-09-17T03:37:01.528816+0000","last_undegraded":"2024-09-17T03:37:01.528816+0000","last_fullsized":"2024-09-17T03:37:01.528816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:24.391567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498567+0000","last_change":"2024-09-17T03:37:00.498567+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498567+0000","last_undegraded":"2024-09-17T03:37:00.498567+0000","last_fullsized":"2024-09-17T03:37:00.498567+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510156+0000","last_change":"2024-09-17T03:37:01.507556+0000","last_active":"2024-09-17T03:37:01.510156+0000","last_peered":"2024-09-17T03:37:01.510156+0000","last_clean":"2024-09-17T03:37:01.510156+0000","last_became_active":"2024-09-17T03:37:01.507216+0000","last_became_peered":"2024-09-17T03:37:01.507216+0000","last_unstale":"2024-09-17T03:37:01.510156+0000","last_undegraded":"2024-09-17T03:37:01.510156+0000","last_fullsized":"2024-09-17T03:37:01.510156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:58.827646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524913+0000","last_change":"2024-09-17T03:37:01.520291+0000","last_active":"2024-09-17T03:37:01.524913+0000","last_peered":"2024-09-17T03:37:01.524913+0000","last_clean":"2024-09-17T03:37:01.524913+0000","last_became_active":"2024-09-17T03:37:01.519510+0000","last_became_peered":"2024-09-17T03:37:01.519510+0000","last_unstale":"2024-09-17T03:37:01.524913+0000","last_undegraded":"2024-09-17T03:37:01.524913+0000","last_fullsized":"2024-09-17T03:37:01.524913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:14.128722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.515271+0000","last_change":"2024-09-17T03:37:01.511915+0000","last_active":"2024-09-17T03:37:01.515271+0000","last_peered":"2024-09-17T03:37:01.515271+0000","last_clean":"2024-09-17T03:37:01.515271+0000","last_became_active":"2024-09-17T03:37:01.511588+0000","last_became_peered":"2024-09-17T03:37:01.511588+0000","last_unstale":"2024-09-17T03:37:01.515271+0000","last_undegraded":"2024-09-17T03:37:01.515271+0000","last_fullsized":"2024-09-17T03:37:01.515271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:34.703407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518221+0000","last_change":"2024-09-17T03:37:01.504394+0000","last_active":"2024-09-17T03:37:01.518221+0000","last_peered":"2024-09-17T03:37:01.518221+0000","last_clean":"2024-09-17T03:37:01.518221+0000","last_became_active":"2024-09-17T03:37:01.504016+0000","last_became_peered":"2024-09-17T03:37:01.504016+0000","last_unstale":"2024-09-17T03:37:01.518221+0000","last_undegraded":"2024-09-17T03:37:01.518221+0000","last_fullsized":"2024-09-17T03:37:01.518221+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:47.918178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526096+0000","last_change":"2024-09-17T03:37:01.521538+0000","last_active":"2024-09-17T03:37:01.526096+0000","last_peered":"2024-09-17T03:37:01.526096+0000","last_clean":"2024-09-17T03:37:01.526096+0000","last_became_active":"2024-09-17T03:37:01.521253+0000","last_became_peered":"2024-09-17T03:37:01.521253+0000","last_unstale":"2024-09-17T03:37:01.526096+0000","last_undegraded":"2024-09-17T03:37:01.526096+0000","last_fullsized":"2024-09-17T03:37:01.526096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:33.873450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509897+0000","last_change":"2024-09-17T03:37:01.507075+0000","last_active":"2024-09-17T03:37:01.509897+0000","last_peered":"2024-09-17T03:37:01.509897+0000","last_clean":"2024-09-17T03:37:01.509897+0000","last_became_active":"2024-09-17T03:37:01.506711+0000","last_became_peered":"2024-09-17T03:37:01.506711+0000","last_unstale":"2024-09-17T03:37:01.509897+0000","last_undegraded":"2024-09-17T03:37:01.509897+0000","last_fullsized":"2024-09-17T03:37:01.509897+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:56.420785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.520006+0000","last_change":"2024-09-17T03:37:01.515856+0000","last_active":"2024-09-17T03:37:01.520006+0000","last_peered":"2024-09-17T03:37:01.520006+0000","last_clean":"2024-09-17T03:37:01.520006+0000","last_became_active":"2024-09-17T03:37:01.512742+0000","last_became_peered":"2024-09-17T03:37:01.512742+0000","last_unstale":"2024-09-17T03:37:01.520006+0000","last_undegraded":"2024-09-17T03:37:01.520006+0000","last_fullsized":"2024-09-17T03:37:01.520006+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:16.486830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498137+0000","last_change":"2024-09-17T03:37:00.498137+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498137+0000","last_undegraded":"2024-09-17T03:37:00.498137+0000","last_fullsized":"2024-09-17T03:37:00.498137+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534060+0000","last_change":"2024-09-17T03:37:01.530828+0000","last_active":"2024-09-17T03:37:01.534060+0000","last_peered":"2024-09-17T03:37:01.534060+0000","last_clean":"2024-09-17T03:37:01.534060+0000","last_became_active":"2024-09-17T03:37:01.529611+0000","last_became_peered":"2024-09-17T03:37:01.529611+0000","last_unstale":"2024-09-17T03:37:01.534060+0000","last_undegraded":"2024-09-17T03:37:01.534060+0000","last_fullsized":"2024-09-17T03:37:01.534060+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526083+0000","last_change":"2024-09-17T03:37:01.524406+0000","last_active":"2024-09-17T03:37:01.526083+0000","last_peered":"2024-09-17T03:37:01.526083+0000","last_clean":"2024-09-17T03:37:01.526083+0000","last_became_active":"2024-09-17T03:37:01.524097+0000","last_became_peered":"2024-09-17T03:37:01.524097+0000","last_unstale":"2024-09-17T03:37:01.526083+0000","last_undegraded":"2024-09-17T03:37:01.526083+0000","last_fullsized":"2024-09-17T03:37:01.526083+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:47.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534103+0000","last_change":"2024-09-17T03:37:01.527309+0000","last_active":"2024-09-17T03:37:01.534103+0000","last_peered":"2024-09-17T03:37:01.534103+0000","last_clean":"2024-09-17T03:37:01.534103+0000","last_became_active":"2024-09-17T03:37:01.526391+0000","last_became_peered":"2024-09-17T03:37:01.526391+0000","last_unstale":"2024-09-17T03:37:01.534103+0000","last_undegraded":"2024-09-17T03:37:01.534103+0000","last_fullsized":"2024-09-17T03:37:01.534103+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:42.503735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536631+0000","last_change":"2024-09-17T03:37:01.531294+0000","last_active":"2024-09-17T03:37:01.536631+0000","last_peered":"2024-09-17T03:37:01.536631+0000","last_clean":"2024-09-17T03:37:01.536631+0000","last_became_active":"2024-09-17T03:37:01.529834+0000","last_became_peered":"2024-09-17T03:37:01.529834+0000","last_unstale":"2024-09-17T03:37:01.536631+0000","last_undegraded":"2024-09-17T03:37:01.536631+0000","last_fullsized":"2024-09-17T03:37:01.536631+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:55.329149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523036+0000","last_change":"2024-09-17T03:37:01.518791+0000","last_active":"2024-09-17T03:37:01.523036+0000","last_peered":"2024-09-17T03:37:01.523036+0000","last_clean":"2024-09-17T03:37:01.523036+0000","last_became_active":"2024-09-17T03:37:01.516765+0000","last_became_peered":"2024-09-17T03:37:01.516765+0000","last_unstale":"2024-09-17T03:37:01.523036+0000","last_undegraded":"2024-09-17T03:37:01.523036+0000","last_fullsized":"2024-09-17T03:37:01.523036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529870+0000","last_change":"2024-09-17T03:37:01.527808+0000","last_active":"2024-09-17T03:37:01.529870+0000","last_peered":"2024-09-17T03:37:01.529870+0000","last_clean":"2024-09-17T03:37:01.529870+0000","last_became_active":"2024-09-17T03:37:01.527469+0000","last_became_peered":"2024-09-17T03:37:01.527469+0000","last_unstale":"2024-09-17T03:37:01.529870+0000","last_undegraded":"2024-09-17T03:37:01.529870+0000","last_fullsized":"2024-09-17T03:37:01.529870+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:54.208586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534127+0000","last_change":"2024-09-17T03:37:01.530734+0000","last_active":"2024-09-17T03:37:01.534127+0000","last_peered":"2024-09-17T03:37:01.534127+0000","last_clean":"2024-09-17T03:37:01.534127+0000","last_became_active":"2024-09-17T03:37:01.530184+0000","last_became_peered":"2024-09-17T03:37:01.530184+0000","last_unstale":"2024-09-17T03:37:01.534127+0000","last_undegraded":"2024-09-17T03:37:01.534127+0000","last_fullsized":"2024-09-17T03:37:01.534127+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:48.884673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525880+0000","last_change":"2024-09-17T03:37:01.520019+0000","last_active":"2024-09-17T03:37:01.525880+0000","last_peered":"2024-09-17T03:37:01.525880+0000","last_clean":"2024-09-17T03:37:01.525880+0000","last_became_active":"2024-09-17T03:37:01.519632+0000","last_became_peered":"2024-09-17T03:37:01.519632+0000","last_unstale":"2024-09-17T03:37:01.525880+0000","last_undegraded":"2024-09-17T03:37:01.525880+0000","last_fullsized":"2024-09-17T03:37:01.525880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:57.353653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536601+0000","last_change":"2024-09-17T03:37:01.532969+0000","last_active":"2024-09-17T03:37:01.536601+0000","last_peered":"2024-09-17T03:37:01.536601+0000","last_clean":"2024-09-17T03:37:01.536601+0000","last_became_active":"2024-09-17T03:37:01.530349+0000","last_became_peered":"2024-09-17T03:37:01.530349+0000","last_unstale":"2024-09-17T03:37:01.536601+0000","last_undegraded":"2024-09-17T03:37:01.536601+0000","last_fullsized":"2024-09-17T03:37:01.536601+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530028+0000","last_change":"2024-09-17T03:37:01.527670+0000","last_active":"2024-09-17T03:37:01.530028+0000","last_peered":"2024-09-17T03:37:01.530028+0000","last_clean":"2024-09-17T03:37:01.530028+0000","last_became_active":"2024-09-17T03:37:01.527335+0000","last_became_peered":"2024-09-17T03:37:01.527335+0000","last_unstale":"2024-09-17T03:37:01.530028+0000","last_undegraded":"2024-09-17T03:37:01.530028+0000","last_fullsized":"2024-09-17T03:37:01.530028+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525874+0000","last_change":"2024-09-17T03:37:01.520830+0000","last_active":"2024-09-17T03:37:01.525874+0000","last_peered":"2024-09-17T03:37:01.525874+0000","last_clean":"2024-09-17T03:37:01.525874+0000","last_became_active":"2024-09-17T03:37:01.520514+0000","last_became_peered":"2024-09-17T03:37:01.520514+0000","last_unstale":"2024-09-17T03:37:01.525874+0000","last_undegraded":"2024-09-17T03:37:01.525874+0000","last_fullsized":"2024-09-17T03:37:01.525874+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:28.913796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533954+0000","last_change":"2024-09-17T03:37:01.526010+0000","last_active":"2024-09-17T03:37:01.533954+0000","last_peered":"2024-09-17T03:37:01.533954+0000","last_clean":"2024-09-17T03:37:01.533954+0000","last_became_active":"2024-09-17T03:37:01.524307+0000","last_became_peered":"2024-09-17T03:37:01.524307+0000","last_unstale":"2024-09-17T03:37:01.533954+0000","last_undegraded":"2024-09-17T03:37:01.533954+0000","last_fullsized":"2024-09-17T03:37:01.533954+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:17.817507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529940+0000","last_change":"2024-09-17T03:37:01.526850+0000","last_active":"2024-09-17T03:37:01.529940+0000","last_peered":"2024-09-17T03:37:01.529940+0000","last_clean":"2024-09-17T03:37:01.529940+0000","last_became_active":"2024-09-17T03:37:01.526500+0000","last_became_peered":"2024-09-17T03:37:01.526500+0000","last_unstale":"2024-09-17T03:37:01.529940+0000","last_undegraded":"2024-09-17T03:37:01.529940+0000","last_fullsized":"2024-09-17T03:37:01.529940+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:01.514192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534424+0000","last_change":"2024-09-17T03:37:01.525369+0000","last_active":"2024-09-17T03:37:01.534424+0000","last_peered":"2024-09-17T03:37:01.534424+0000","last_clean":"2024-09-17T03:37:01.534424+0000","last_became_active":"2024-09-17T03:37:01.524994+0000","last_became_peered":"2024-09-17T03:37:01.524994+0000","last_unstale":"2024-09-17T03:37:01.534424+0000","last_undegraded":"2024-09-17T03:37:01.534424+0000","last_fullsized":"2024-09-17T03:37:01.534424+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:16.958527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521738+0000","last_change":"2024-09-17T03:37:01.514688+0000","last_active":"2024-09-17T03:37:01.521738+0000","last_peered":"2024-09-17T03:37:01.521738+0000","last_clean":"2024-09-17T03:37:01.521738+0000","last_became_active":"2024-09-17T03:37:01.505442+0000","last_became_peered":"2024-09-17T03:37:01.505442+0000","last_unstale":"2024-09-17T03:37:01.521738+0000","last_undegraded":"2024-09-17T03:37:01.521738+0000","last_fullsized":"2024-09-17T03:37:01.521738+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:51.069459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510776+0000","last_change":"2024-09-17T03:37:01.509207+0000","last_active":"2024-09-17T03:37:01.510776+0000","last_peered":"2024-09-17T03:37:01.510776+0000","last_clean":"2024-09-17T03:37:01.510776+0000","last_became_active":"2024-09-17T03:37:01.508207+0000","last_became_peered":"2024-09-17T03:37:01.508207+0000","last_unstale":"2024-09-17T03:37:01.510776+0000","last_undegraded":"2024-09-17T03:37:01.510776+0000","last_fullsized":"2024-09-17T03:37:01.510776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:48.105755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528879+0000","last_change":"2024-09-17T03:37:01.524856+0000","last_active":"2024-09-17T03:37:01.528879+0000","last_peered":"2024-09-17T03:37:01.528879+0000","last_clean":"2024-09-17T03:37:01.528879+0000","last_became_active":"2024-09-17T03:37:01.522971+0000","last_became_peered":"2024-09-17T03:37:01.522971+0000","last_unstale":"2024-09-17T03:37:01.528879+0000","last_undegraded":"2024-09-17T03:37:01.528879+0000","last_fullsized":"2024-09-17T03:37:01.528879+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517595+0000","last_change":"2024-09-17T03:37:01.504583+0000","last_active":"2024-09-17T03:37:01.517595+0000","last_peered":"2024-09-17T03:37:01.517595+0000","last_clean":"2024-09-17T03:37:01.517595+0000","last_became_active":"2024-09-17T03:37:01.504221+0000","last_became_peered":"2024-09-17T03:37:01.504221+0000","last_unstale":"2024-09-17T03:37:01.517595+0000","last_undegraded":"2024-09-17T03:37:01.517595+0000","last_fullsized":"2024-09-17T03:37:01.517595+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:18.113674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534132+0000","last_change":"2024-09-17T03:37:01.527737+0000","last_active":"2024-09-17T03:37:01.534132+0000","last_peered":"2024-09-17T03:37:01.534132+0000","last_clean":"2024-09-17T03:37:01.534132+0000","last_became_active":"2024-09-17T03:37:01.526783+0000","last_became_peered":"2024-09-17T03:37:01.526783+0000","last_unstale":"2024-09-17T03:37:01.534132+0000","last_undegraded":"2024-09-17T03:37:01.534132+0000","last_fullsized":"2024-09-17T03:37:01.534132+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:28.866527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518126+0000","last_change":"2024-09-17T03:37:01.513699+0000","last_active":"2024-09-17T03:37:01.518126+0000","last_peered":"2024-09-17T03:37:01.518126+0000","last_clean":"2024-09-17T03:37:01.518126+0000","last_became_active":"2024-09-17T03:37:01.513310+0000","last_became_peered":"2024-09-17T03:37:01.513310+0000","last_unstale":"2024-09-17T03:37:01.518126+0000","last_undegraded":"2024-09-17T03:37:01.518126+0000","last_fullsized":"2024-09-17T03:37:01.518126+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"142'4","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535697+0000","last_change":"2024-09-17T03:37:00.521721+0000","last_active":"2024-09-17T03:37:01.535697+0000","last_peered":"2024-09-17T03:37:01.535697+0000","last_clean":"2024-09-17T03:37:01.535697+0000","last_became_active":"2024-09-17T03:37:00.521476+0000","last_became_peered":"2024-09-17T03:37:00.521476+0000","last_unstale":"2024-09-17T03:37:01.535697+0000","last_undegraded":"2024-09-17T03:37:01.535697+0000","last_fullsized":"2024-09-17T03:37:01.535697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:11.151395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531349+0000","last_change":"2024-09-17T03:37:01.524327+0000","last_active":"2024-09-17T03:37:01.531349+0000","last_peered":"2024-09-17T03:37:01.531349+0000","last_clean":"2024-09-17T03:37:01.531349+0000","last_became_active":"2024-09-17T03:37:01.523988+0000","last_became_peered":"2024-09-17T03:37:01.523988+0000","last_unstale":"2024-09-17T03:37:01.531349+0000","last_undegraded":"2024-09-17T03:37:01.531349+0000","last_fullsized":"2024-09-17T03:37:01.531349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:57.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.522778+0000","last_change":"2024-09-17T03:37:00.513012+0000","last_active":"2024-09-17T03:37:01.522778+0000","last_peered":"2024-09-17T03:37:01.522778+0000","last_clean":"2024-09-17T03:37:01.522778+0000","last_became_active":"2024-09-17T03:37:00.512669+0000","last_became_peered":"2024-09-17T03:37:00.512669+0000","last_unstale":"2024-09-17T03:37:01.522778+0000","last_undegraded":"2024-09-17T03:37:01.522778+0000","last_fullsized":"2024-09-17T03:37:01.522778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:04.275671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523023+0000","last_change":"2024-09-17T03:37:01.520484+0000","last_active":"2024-09-17T03:37:01.523023+0000","last_peered":"2024-09-17T03:37:01.523023+0000","last_clean":"2024-09-17T03:37:01.523023+0000","last_became_active":"2024-09-17T03:37:01.518380+0000","last_became_peered":"2024-09-17T03:37:01.518380+0000","last_unstale":"2024-09-17T03:37:01.523023+0000","last_undegraded":"2024-09-17T03:37:01.523023+0000","last_fullsized":"2024-09-17T03:37:01.523023+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:10.898889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537662+0000","last_change":"2024-09-17T03:37:00.518391+0000","last_active":"2024-09-17T03:37:01.537662+0000","last_peered":"2024-09-17T03:37:01.537662+0000","last_clean":"2024-09-17T03:37:01.537662+0000","last_became_active":"2024-09-17T03:37:00.518014+0000","last_became_peered":"2024-09-17T03:37:00.518014+0000","last_unstale":"2024-09-17T03:37:01.537662+0000","last_undegraded":"2024-09-17T03:37:01.537662+0000","last_fullsized":"2024-09-17T03:37:01.537662+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518010+0000","last_change":"2024-09-17T03:37:01.513224+0000","last_active":"2024-09-17T03:37:01.518010+0000","last_peered":"2024-09-17T03:37:01.518010+0000","last_clean":"2024-09-17T03:37:01.518010+0000","last_became_active":"2024-09-17T03:37:01.512886+0000","last_became_peered":"2024-09-17T03:37:01.512886+0000","last_unstale":"2024-09-17T03:37:01.518010+0000","last_undegraded":"2024-09-17T03:37:01.518010+0000","last_fullsized":"2024-09-17T03:37:01.518010+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:13.599854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535032+0000","last_change":"2024-09-17T03:37:00.516162+0000","last_active":"2024-09-17T03:37:01.535032+0000","last_peered":"2024-09-17T03:37:01.535032+0000","last_clean":"2024-09-17T03:37:01.535032+0000","last_became_active":"2024-09-17T03:37:00.515367+0000","last_became_peered":"2024-09-17T03:37:00.515367+0000","last_unstale":"2024-09-17T03:37:01.535032+0000","last_undegraded":"2024-09-17T03:37:01.535032+0000","last_fullsized":"2024-09-17T03:37:01.535032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:08.671535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518310+0000","last_change":"2024-09-17T03:37:01.506152+0000","last_active":"2024-09-17T03:37:01.518310+0000","last_peered":"2024-09-17T03:37:01.518310+0000","last_clean":"2024-09-17T03:37:01.518310+0000","last_became_active":"2024-09-17T03:37:01.503759+0000","last_became_peered":"2024-09-17T03:37:01.503759+0000","last_unstale":"2024-09-17T03:37:01.518310+0000","last_undegraded":"2024-09-17T03:37:01.518310+0000","last_fullsized":"2024-09-17T03:37:01.518310+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:43.445864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528132+0000","last_change":"2024-09-17T03:37:00.510847+0000","last_active":"2024-09-17T03:37:01.528132+0000","last_peered":"2024-09-17T03:37:01.528132+0000","last_clean":"2024-09-17T03:37:01.528132+0000","last_became_active":"2024-09-17T03:37:00.509550+0000","last_became_peered":"2024-09-17T03:37:00.509550+0000","last_unstale":"2024-09-17T03:37:01.528132+0000","last_undegraded":"2024-09-17T03:37:01.528132+0000","last_fullsized":"2024-09-17T03:37:01.528132+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.371788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531043+0000","last_change":"2024-09-17T03:37:01.524253+0000","last_active":"2024-09-17T03:37:01.531043+0000","last_peered":"2024-09-17T03:37:01.531043+0000","last_clean":"2024-09-17T03:37:01.531043+0000","last_became_active":"2024-09-17T03:37:01.523935+0000","last_became_peered":"2024-09-17T03:37:01.523935+0000","last_unstale":"2024-09-17T03:37:01.531043+0000","last_undegraded":"2024-09-17T03:37:01.531043+0000","last_fullsized":"2024-09-17T03:37:01.531043+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.515827+0000","last_change":"2024-09-17T03:37:00.513776+0000","last_active":"2024-09-17T03:37:00.515827+0000","last_peered":"2024-09-17T03:37:00.515827+0000","last_clean":"2024-09-17T03:37:00.515827+0000","last_became_active":"2024-09-17T03:37:00.511311+0000","last_became_peered":"2024-09-17T03:37:00.511311+0000","last_unstale":"2024-09-17T03:37:00.515827+0000","last_undegraded":"2024-09-17T03:37:00.515827+0000","last_fullsized":"2024-09-17T03:37:00.515827+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:16.989474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517432+0000","last_change":"2024-09-17T03:37:01.513641+0000","last_active":"2024-09-17T03:37:01.517432+0000","last_peered":"2024-09-17T03:37:01.517432+0000","last_clean":"2024-09-17T03:37:01.517432+0000","last_became_active":"2024-09-17T03:37:01.513325+0000","last_became_peered":"2024-09-17T03:37:01.513325+0000","last_unstale":"2024-09-17T03:37:01.517432+0000","last_undegraded":"2024-09-17T03:37:01.517432+0000","last_fullsized":"2024-09-17T03:37:01.517432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:54.606013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.516501+0000","last_change":"2024-09-17T03:37:00.513452+0000","last_active":"2024-09-17T03:37:01.516501+0000","last_peered":"2024-09-17T03:37:01.516501+0000","last_clean":"2024-09-17T03:37:01.516501+0000","last_became_active":"2024-09-17T03:37:00.513118+0000","last_became_peered":"2024-09-17T03:37:00.513118+0000","last_unstale":"2024-09-17T03:37:01.516501+0000","last_undegraded":"2024-09-17T03:37:01.516501+0000","last_fullsized":"2024-09-17T03:37:01.516501+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:53.259179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498231+0000","last_change":"2024-09-17T03:37:00.498231+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498231+0000","last_undegraded":"2024-09-17T03:37:00.498231+0000","last_fullsized":"2024-09-17T03:37:00.498231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509557+0000","last_change":"2024-09-17T03:37:00.506180+0000","last_active":"2024-09-17T03:37:01.509557+0000","last_peered":"2024-09-17T03:37:01.509557+0000","last_clean":"2024-09-17T03:37:01.509557+0000","last_became_active":"2024-09-17T03:37:00.505868+0000","last_became_peered":"2024-09-17T03:37:00.505868+0000","last_unstale":"2024-09-17T03:37:01.509557+0000","last_undegraded":"2024-09-17T03:37:01.509557+0000","last_fullsized":"2024-09-17T03:37:01.509557+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497947+0000","last_change":"2024-09-17T03:37:00.497947+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497947+0000","last_undegraded":"2024-09-17T03:37:00.497947+0000","last_fullsized":"2024-09-17T03:37:00.497947+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494787+0000","last_change":"2024-09-17T03:37:00.513939+0000","last_active":"2024-09-17T03:37:01.494787+0000","last_peered":"2024-09-17T03:37:01.494787+0000","last_clean":"2024-09-17T03:37:01.494787+0000","last_became_active":"2024-09-17T03:37:00.513531+0000","last_became_peered":"2024-09-17T03:37:00.513531+0000","last_unstale":"2024-09-17T03:37:01.494787+0000","last_undegraded":"2024-09-17T03:37:01.494787+0000","last_fullsized":"2024-09-17T03:37:01.494787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:43.986216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536554+0000","last_change":"2024-09-17T03:37:01.530837+0000","last_active":"2024-09-17T03:37:01.536554+0000","last_peered":"2024-09-17T03:37:01.536554+0000","last_clean":"2024-09-17T03:37:01.536554+0000","last_became_active":"2024-09-17T03:37:01.530166+0000","last_became_peered":"2024-09-17T03:37:01.530166+0000","last_unstale":"2024-09-17T03:37:01.536554+0000","last_undegraded":"2024-09-17T03:37:01.536554+0000","last_fullsized":"2024-09-17T03:37:01.536554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:32.600471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.498465+0000","last_change":"2024-09-17T03:37:00.515644+0000","last_active":"2024-09-17T03:37:01.498465+0000","last_peered":"2024-09-17T03:37:01.498465+0000","last_clean":"2024-09-17T03:37:01.498465+0000","last_became_active":"2024-09-17T03:37:00.515326+0000","last_became_peered":"2024-09-17T03:37:00.515326+0000","last_unstale":"2024-09-17T03:37:01.498465+0000","last_undegraded":"2024-09-17T03:37:01.498465+0000","last_fullsized":"2024-09-17T03:37:01.498465+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:05.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534454+0000","last_change":"2024-09-17T03:37:01.530851+0000","last_active":"2024-09-17T03:37:01.534454+0000","last_peered":"2024-09-17T03:37:01.534454+0000","last_clean":"2024-09-17T03:37:01.534454+0000","last_became_active":"2024-09-17T03:37:01.530410+0000","last_became_peered":"2024-09-17T03:37:01.530410+0000","last_unstale":"2024-09-17T03:37:01.534454+0000","last_undegraded":"2024-09-17T03:37:01.534454+0000","last_fullsized":"2024-09-17T03:37:01.534454+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:54.349008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"142'6","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537003+0000","last_change":"2024-09-17T03:37:00.511424+0000","last_active":"2024-09-17T03:37:01.537003+0000","last_peered":"2024-09-17T03:37:01.537003+0000","last_clean":"2024-09-17T03:37:01.537003+0000","last_became_active":"2024-09-17T03:37:00.511055+0000","last_became_peered":"2024-09-17T03:37:00.511055+0000","last_unstale":"2024-09-17T03:37:01.537003+0000","last_undegraded":"2024-09-17T03:37:01.537003+0000","last_fullsized":"2024-09-17T03:37:01.537003+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:04.537830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526765+0000","last_change":"2024-09-17T03:37:01.520532+0000","last_active":"2024-09-17T03:37:01.526765+0000","last_peered":"2024-09-17T03:37:01.526765+0000","last_clean":"2024-09-17T03:37:01.526765+0000","last_became_active":"2024-09-17T03:37:01.520261+0000","last_became_peered":"2024-09-17T03:37:01.520261+0000","last_unstale":"2024-09-17T03:37:01.526765+0000","last_undegraded":"2024-09-17T03:37:01.526765+0000","last_fullsized":"2024-09-17T03:37:01.526765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538130+0000","last_change":"2024-09-17T03:37:00.509307+0000","last_active":"2024-09-17T03:37:01.538130+0000","last_peered":"2024-09-17T03:37:01.538130+0000","last_clean":"2024-09-17T03:37:01.538130+0000","last_became_active":"2024-09-17T03:37:00.508858+0000","last_became_peered":"2024-09-17T03:37:00.508858+0000","last_unstale":"2024-09-17T03:37:01.538130+0000","last_undegraded":"2024-09-17T03:37:01.538130+0000","last_fullsized":"2024-09-17T03:37:01.538130+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:19.533265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532147+0000","last_change":"2024-09-17T03:37:01.526495+0000","last_active":"2024-09-17T03:37:01.532147+0000","last_peered":"2024-09-17T03:37:01.532147+0000","last_clean":"2024-09-17T03:37:01.532147+0000","last_became_active":"2024-09-17T03:37:01.526193+0000","last_became_peered":"2024-09-17T03:37:01.526193+0000","last_unstale":"2024-09-17T03:37:01.532147+0000","last_undegraded":"2024-09-17T03:37:01.532147+0000","last_fullsized":"2024-09-17T03:37:01.532147+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:38.826521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506839+0000","last_change":"2024-09-17T03:37:00.506435+0000","last_active":"2024-09-17T03:37:01.506839+0000","last_peered":"2024-09-17T03:37:01.506839+0000","last_clean":"2024-09-17T03:37:01.506839+0000","last_became_active":"2024-09-17T03:37:00.505499+0000","last_became_peered":"2024-09-17T03:37:00.505499+0000","last_unstale":"2024-09-17T03:37:01.506839+0000","last_undegraded":"2024-09-17T03:37:01.506839+0000","last_fullsized":"2024-09-17T03:37:01.506839+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497356+0000","last_change":"2024-09-17T03:37:00.497356+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497356+0000","last_undegraded":"2024-09-17T03:37:00.497356+0000","last_fullsized":"2024-09-17T03:37:00.497356+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.490845+0000","last_change":"2024-09-17T03:37:00.497616+0000","last_active":"2024-09-17T03:37:01.490845+0000","last_peered":"2024-09-17T03:37:01.490845+0000","last_clean":"2024-09-17T03:37:01.490845+0000","last_became_active":"2024-09-17T03:37:00.497229+0000","last_became_peered":"2024-09-17T03:37:00.497229+0000","last_unstale":"2024-09-17T03:37:01.490845+0000","last_undegraded":"2024-09-17T03:37:01.490845+0000","last_fullsized":"2024-09-17T03:37:01.490845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:08.103469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531634+0000","last_change":"2024-09-17T03:37:01.525031+0000","last_active":"2024-09-17T03:37:01.531634+0000","last_peered":"2024-09-17T03:37:01.531634+0000","last_clean":"2024-09-17T03:37:01.531634+0000","last_became_active":"2024-09-17T03:37:01.524710+0000","last_became_peered":"2024-09-17T03:37:01.524710+0000","last_unstale":"2024-09-17T03:37:01.531634+0000","last_undegraded":"2024-09-17T03:37:01.531634+0000","last_fullsized":"2024-09-17T03:37:01.531634+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"142'8","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534881+0000","last_change":"2024-09-17T03:37:00.506497+0000","last_active":"2024-09-17T03:37:01.534881+0000","last_peered":"2024-09-17T03:37:01.534881+0000","last_clean":"2024-09-17T03:37:01.534881+0000","last_became_active":"2024-09-17T03:37:00.506195+0000","last_became_peered":"2024-09-17T03:37:00.506195+0000","last_unstale":"2024-09-17T03:37:01.534881+0000","last_undegraded":"2024-09-17T03:37:01.534881+0000","last_fullsized":"2024-09-17T03:37:01.534881+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:35.347269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.511186+0000","last_change":"2024-09-17T03:37:01.510170+0000","last_active":"2024-09-17T03:37:01.511186+0000","last_peered":"2024-09-17T03:37:01.511186+0000","last_clean":"2024-09-17T03:37:01.511186+0000","last_became_active":"2024-09-17T03:37:01.509806+0000","last_became_peered":"2024-09-17T03:37:01.509806+0000","last_unstale":"2024-09-17T03:37:01.511186+0000","last_undegraded":"2024-09-17T03:37:01.511186+0000","last_fullsized":"2024-09-17T03:37:01.511186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:43.917446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.511017+0000","last_change":"2024-09-17T03:37:00.507289+0000","last_active":"2024-09-17T03:37:00.511017+0000","last_peered":"2024-09-17T03:37:00.511017+0000","last_clean":"2024-09-17T03:37:00.511017+0000","last_became_active":"2024-09-17T03:37:00.506994+0000","last_became_peered":"2024-09-17T03:37:00.506994+0000","last_unstale":"2024-09-17T03:37:00.511017+0000","last_undegraded":"2024-09-17T03:37:00.511017+0000","last_fullsized":"2024-09-17T03:37:00.511017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:13.903403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533925+0000","last_change":"2024-09-17T03:37:01.525640+0000","last_active":"2024-09-17T03:37:01.533925+0000","last_peered":"2024-09-17T03:37:01.533925+0000","last_clean":"2024-09-17T03:37:01.533925+0000","last_became_active":"2024-09-17T03:37:01.523930+0000","last_became_peered":"2024-09-17T03:37:01.523930+0000","last_unstale":"2024-09-17T03:37:01.533925+0000","last_undegraded":"2024-09-17T03:37:01.533925+0000","last_fullsized":"2024-09-17T03:37:01.533925+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519679+0000","last_change":"2024-09-17T03:37:00.514697+0000","last_active":"2024-09-17T03:37:01.519679+0000","last_peered":"2024-09-17T03:37:01.519679+0000","last_clean":"2024-09-17T03:37:01.519679+0000","last_became_active":"2024-09-17T03:37:00.513958+0000","last_became_peered":"2024-09-17T03:37:00.513958+0000","last_unstale":"2024-09-17T03:37:01.519679+0000","last_undegraded":"2024-09-17T03:37:01.519679+0000","last_fullsized":"2024-09-17T03:37:01.519679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:27.979522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529910+0000","last_change":"2024-09-17T03:37:01.521591+0000","last_active":"2024-09-17T03:37:01.529910+0000","last_peered":"2024-09-17T03:37:01.529910+0000","last_clean":"2024-09-17T03:37:01.529910+0000","last_became_active":"2024-09-17T03:37:01.521066+0000","last_became_peered":"2024-09-17T03:37:01.521066+0000","last_unstale":"2024-09-17T03:37:01.529910+0000","last_undegraded":"2024-09-17T03:37:01.529910+0000","last_fullsized":"2024-09-17T03:37:01.529910+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533162+0000","last_change":"2024-09-17T03:37:00.510701+0000","last_active":"2024-09-17T03:37:01.533162+0000","last_peered":"2024-09-17T03:37:01.533162+0000","last_clean":"2024-09-17T03:37:01.533162+0000","last_became_active":"2024-09-17T03:37:00.510374+0000","last_became_peered":"2024-09-17T03:37:00.510374+0000","last_unstale":"2024-09-17T03:37:01.533162+0000","last_undegraded":"2024-09-17T03:37:01.533162+0000","last_fullsized":"2024-09-17T03:37:01.533162+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:40.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510426+0000","last_change":"2024-09-17T03:37:01.508528+0000","last_active":"2024-09-17T03:37:01.510426+0000","last_peered":"2024-09-17T03:37:01.510426+0000","last_clean":"2024-09-17T03:37:01.510426+0000","last_became_active":"2024-09-17T03:37:01.508173+0000","last_became_peered":"2024-09-17T03:37:01.508173+0000","last_unstale":"2024-09-17T03:37:01.510426+0000","last_undegraded":"2024-09-17T03:37:01.510426+0000","last_fullsized":"2024-09-17T03:37:01.510426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:47.227568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494837+0000","last_change":"2024-09-17T03:37:00.517789+0000","last_active":"2024-09-17T03:37:01.494837+0000","last_peered":"2024-09-17T03:37:01.494837+0000","last_clean":"2024-09-17T03:37:01.494837+0000","last_became_active":"2024-09-17T03:37:00.517101+0000","last_became_peered":"2024-09-17T03:37:00.517101+0000","last_unstale":"2024-09-17T03:37:01.494837+0000","last_undegraded":"2024-09-17T03:37:01.494837+0000","last_fullsized":"2024-09-17T03:37:01.494837+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:30.163493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524630+0000","last_change":"2024-09-17T03:37:01.518635+0000","last_active":"2024-09-17T03:37:01.524630+0000","last_peered":"2024-09-17T03:37:01.524630+0000","last_clean":"2024-09-17T03:37:01.524630+0000","last_became_active":"2024-09-17T03:37:01.518369+0000","last_became_peered":"2024-09-17T03:37:01.518369+0000","last_unstale":"2024-09-17T03:37:01.524630+0000","last_undegraded":"2024-09-17T03:37:01.524630+0000","last_fullsized":"2024-09-17T03:37:01.524630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:42.751762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529734+0000","last_change":"2024-09-17T03:37:00.502280+0000","last_active":"2024-09-17T03:37:01.529734+0000","last_peered":"2024-09-17T03:37:01.529734+0000","last_clean":"2024-09-17T03:37:01.529734+0000","last_became_active":"2024-09-17T03:37:00.501924+0000","last_became_peered":"2024-09-17T03:37:00.501924+0000","last_unstale":"2024-09-17T03:37:01.529734+0000","last_undegraded":"2024-09-17T03:37:01.529734+0000","last_fullsized":"2024-09-17T03:37:01.529734+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:46.918045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524298+0000","last_change":"2024-09-17T03:37:01.523047+0000","last_active":"2024-09-17T03:37:01.524298+0000","last_peered":"2024-09-17T03:37:01.524298+0000","last_clean":"2024-09-17T03:37:01.524298+0000","last_became_active":"2024-09-17T03:37:01.522670+0000","last_became_peered":"2024-09-17T03:37:01.522670+0000","last_unstale":"2024-09-17T03:37:01.524298+0000","last_undegraded":"2024-09-17T03:37:01.524298+0000","last_fullsized":"2024-09-17T03:37:01.524298+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:03.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496644+0000","last_change":"2024-09-17T03:37:00.519370+0000","last_active":"2024-09-17T03:37:01.496644+0000","last_peered":"2024-09-17T03:37:01.496644+0000","last_clean":"2024-09-17T03:37:01.496644+0000","last_became_active":"2024-09-17T03:37:00.519058+0000","last_became_peered":"2024-09-17T03:37:00.519058+0000","last_unstale":"2024-09-17T03:37:01.496644+0000","last_undegraded":"2024-09-17T03:37:01.496644+0000","last_fullsized":"2024-09-17T03:37:01.496644+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:12.207851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530069+0000","last_change":"2024-09-17T03:37:01.528223+0000","last_active":"2024-09-17T03:37:01.530069+0000","last_peered":"2024-09-17T03:37:01.530069+0000","last_clean":"2024-09-17T03:37:01.530069+0000","last_became_active":"2024-09-17T03:37:01.527875+0000","last_became_peered":"2024-09-17T03:37:01.527875+0000","last_unstale":"2024-09-17T03:37:01.530069+0000","last_undegraded":"2024-09-17T03:37:01.530069+0000","last_fullsized":"2024-09-17T03:37:01.530069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496550+0000","last_change":"2024-09-17T03:37:00.514756+0000","last_active":"2024-09-17T03:37:01.496550+0000","last_peered":"2024-09-17T03:37:01.496550+0000","last_clean":"2024-09-17T03:37:01.496550+0000","last_became_active":"2024-09-17T03:37:00.514505+0000","last_became_peered":"2024-09-17T03:37:00.514505+0000","last_unstale":"2024-09-17T03:37:01.496550+0000","last_undegraded":"2024-09-17T03:37:01.496550+0000","last_fullsized":"2024-09-17T03:37:01.496550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524885+0000","last_change":"2024-09-17T03:37:01.519916+0000","last_active":"2024-09-17T03:37:01.524885+0000","last_peered":"2024-09-17T03:37:01.524885+0000","last_clean":"2024-09-17T03:37:01.524885+0000","last_became_active":"2024-09-17T03:37:01.519171+0000","last_became_peered":"2024-09-17T03:37:01.519171+0000","last_unstale":"2024-09-17T03:37:01.524885+0000","last_undegraded":"2024-09-17T03:37:01.524885+0000","last_fullsized":"2024-09-17T03:37:01.524885+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:18.402815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.507817+0000","last_change":"2024-09-17T03:37:00.516033+0000","last_active":"2024-09-17T03:37:01.507817+0000","last_peered":"2024-09-17T03:37:01.507817+0000","last_clean":"2024-09-17T03:37:01.507817+0000","last_became_active":"2024-09-17T03:37:00.515737+0000","last_became_peered":"2024-09-17T03:37:00.515737+0000","last_unstale":"2024-09-17T03:37:01.507817+0000","last_undegraded":"2024-09-17T03:37:01.507817+0000","last_fullsized":"2024-09-17T03:37:01.507817+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:23.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497461+0000","last_change":"2024-09-17T03:37:00.497461+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497461+0000","last_undegraded":"2024-09-17T03:37:00.497461+0000","last_fullsized":"2024-09-17T03:37:00.497461+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533095+0000","last_change":"2024-09-17T03:37:00.518901+0000","last_active":"2024-09-17T03:37:01.533095+0000","last_peered":"2024-09-17T03:37:01.533095+0000","last_clean":"2024-09-17T03:37:01.533095+0000","last_became_active":"2024-09-17T03:37:00.518609+0000","last_became_peered":"2024-09-17T03:37:00.518609+0000","last_unstale":"2024-09-17T03:37:01.533095+0000","last_undegraded":"2024-09-17T03:37:01.533095+0000","last_fullsized":"2024-09-17T03:37:01.533095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:31.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519672+0000","last_change":"2024-09-17T03:37:01.515337+0000","last_active":"2024-09-17T03:37:01.519672+0000","last_peered":"2024-09-17T03:37:01.519672+0000","last_clean":"2024-09-17T03:37:01.519672+0000","last_became_active":"2024-09-17T03:37:01.515033+0000","last_became_peered":"2024-09-17T03:37:01.515033+0000","last_unstale":"2024-09-17T03:37:01.519672+0000","last_undegraded":"2024-09-17T03:37:01.519672+0000","last_fullsized":"2024-09-17T03:37:01.519672+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494565+0000","last_change":"2024-09-17T03:37:00.509124+0000","last_active":"2024-09-17T03:37:01.494565+0000","last_peered":"2024-09-17T03:37:01.494565+0000","last_clean":"2024-09-17T03:37:01.494565+0000","last_became_active":"2024-09-17T03:37:00.508710+0000","last_became_peered":"2024-09-17T03:37:00.508710+0000","last_unstale":"2024-09-17T03:37:01.494565+0000","last_undegraded":"2024-09-17T03:37:01.494565+0000","last_fullsized":"2024-09-17T03:37:01.494565+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:09.199848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518488+0000","last_change":"2024-09-17T03:37:01.514963+0000","last_active":"2024-09-17T03:37:01.518488+0000","last_peered":"2024-09-17T03:37:01.518488+0000","last_clean":"2024-09-17T03:37:01.518488+0000","last_became_active":"2024-09-17T03:37:01.514337+0000","last_became_peered":"2024-09-17T03:37:01.514337+0000","last_unstale":"2024-09-17T03:37:01.518488+0000","last_undegraded":"2024-09-17T03:37:01.518488+0000","last_fullsized":"2024-09-17T03:37:01.518488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:17.288497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.497349+0000","last_change":"2024-09-17T03:37:00.506035+0000","last_active":"2024-09-17T03:37:01.497349+0000","last_peered":"2024-09-17T03:37:01.497349+0000","last_clean":"2024-09-17T03:37:01.497349+0000","last_became_active":"2024-09-17T03:37:00.505658+0000","last_became_peered":"2024-09-17T03:37:00.505658+0000","last_unstale":"2024-09-17T03:37:01.497349+0000","last_undegraded":"2024-09-17T03:37:01.497349+0000","last_fullsized":"2024-09-17T03:37:01.497349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:52.915238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521428+0000","last_change":"2024-09-17T03:37:01.517782+0000","last_active":"2024-09-17T03:37:01.521428+0000","last_peered":"2024-09-17T03:37:01.521428+0000","last_clean":"2024-09-17T03:37:01.521428+0000","last_became_active":"2024-09-17T03:37:01.517363+0000","last_became_peered":"2024-09-17T03:37:01.517363+0000","last_unstale":"2024-09-17T03:37:01.521428+0000","last_undegraded":"2024-09-17T03:37:01.521428+0000","last_fullsized":"2024-09-17T03:37:01.521428+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:17.301571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496098+0000","last_change":"2024-09-17T03:37:00.514913+0000","last_active":"2024-09-17T03:37:01.496098+0000","last_peered":"2024-09-17T03:37:01.496098+0000","last_clean":"2024-09-17T03:37:01.496098+0000","last_became_active":"2024-09-17T03:37:00.514588+0000","last_became_peered":"2024-09-17T03:37:00.514588+0000","last_unstale":"2024-09-17T03:37:01.496098+0000","last_undegraded":"2024-09-17T03:37:01.496098+0000","last_fullsized":"2024-09-17T03:37:01.496098+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:15.350414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532735+0000","last_change":"2024-09-17T03:37:01.523154+0000","last_active":"2024-09-17T03:37:01.532735+0000","last_peered":"2024-09-17T03:37:01.532735+0000","last_clean":"2024-09-17T03:37:01.532735+0000","last_became_active":"2024-09-17T03:37:01.522826+0000","last_became_peered":"2024-09-17T03:37:01.522826+0000","last_unstale":"2024-09-17T03:37:01.532735+0000","last_undegraded":"2024-09-17T03:37:01.532735+0000","last_fullsized":"2024-09-17T03:37:01.532735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:19.317037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.501477+0000","last_change":"2024-09-17T03:37:00.510360+0000","last_active":"2024-09-17T03:37:01.501477+0000","last_peered":"2024-09-17T03:37:01.501477+0000","last_clean":"2024-09-17T03:37:01.501477+0000","last_became_active":"2024-09-17T03:37:00.510038+0000","last_became_peered":"2024-09-17T03:37:00.510038+0000","last_unstale":"2024-09-17T03:37:01.501477+0000","last_undegraded":"2024-09-17T03:37:01.501477+0000","last_fullsized":"2024-09-17T03:37:01.501477+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:31.275321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497390+0000","last_change":"2024-09-17T03:37:00.497390+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497390+0000","last_undegraded":"2024-09-17T03:37:00.497390+0000","last_fullsized":"2024-09-17T03:37:00.497390+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496964+0000","last_change":"2024-09-17T03:37:00.518866+0000","last_active":"2024-09-17T03:37:01.496964+0000","last_peered":"2024-09-17T03:37:01.496964+0000","last_clean":"2024-09-17T03:37:01.496964+0000","last_became_active":"2024-09-17T03:37:00.518531+0000","last_became_peered":"2024-09-17T03:37:00.518531+0000","last_unstale":"2024-09-17T03:37:01.496964+0000","last_undegraded":"2024-09-17T03:37:01.496964+0000","last_fullsized":"2024-09-17T03:37:01.496964+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.823082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537156+0000","last_change":"2024-09-17T03:37:01.534797+0000","last_active":"2024-09-17T03:37:01.537156+0000","last_peered":"2024-09-17T03:37:01.537156+0000","last_clean":"2024-09-17T03:37:01.537156+0000","last_became_active":"2024-09-17T03:37:01.534442+0000","last_became_peered":"2024-09-17T03:37:01.534442+0000","last_unstale":"2024-09-17T03:37:01.537156+0000","last_undegraded":"2024-09-17T03:37:01.537156+0000","last_fullsized":"2024-09-17T03:37:01.537156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:42.572373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506749+0000","last_change":"2024-09-17T03:37:00.512368+0000","last_active":"2024-09-17T03:37:01.506749+0000","last_peered":"2024-09-17T03:37:01.506749+0000","last_clean":"2024-09-17T03:37:01.506749+0000","last_became_active":"2024-09-17T03:37:00.512041+0000","last_became_peered":"2024-09-17T03:37:00.512041+0000","last_unstale":"2024-09-17T03:37:01.506749+0000","last_undegraded":"2024-09-17T03:37:01.506749+0000","last_fullsized":"2024-09-17T03:37:01.506749+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497341+0000","last_change":"2024-09-17T03:37:00.497341+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497341+0000","last_undegraded":"2024-09-17T03:37:00.497341+0000","last_fullsized":"2024-09-17T03:37:00.497341+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.492735+0000","last_change":"2024-09-17T03:37:00.508129+0000","last_active":"2024-09-17T03:37:01.492735+0000","last_peered":"2024-09-17T03:37:01.492735+0000","last_clean":"2024-09-17T03:37:01.492735+0000","last_became_active":"2024-09-17T03:37:00.506285+0000","last_became_peered":"2024-09-17T03:37:00.506285+0000","last_unstale":"2024-09-17T03:37:01.492735+0000","last_undegraded":"2024-09-17T03:37:01.492735+0000","last_fullsized":"2024-09-17T03:37:01.492735+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:59.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521767+0000","last_change":"2024-09-17T03:37:01.516795+0000","last_active":"2024-09-17T03:37:01.521767+0000","last_peered":"2024-09-17T03:37:01.521767+0000","last_clean":"2024-09-17T03:37:01.521767+0000","last_became_active":"2024-09-17T03:37:01.516423+0000","last_became_peered":"2024-09-17T03:37:01.516423+0000","last_unstale":"2024-09-17T03:37:01.521767+0000","last_undegraded":"2024-09-17T03:37:01.521767+0000","last_fullsized":"2024-09-17T03:37:01.521767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:14.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"142'3","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538197+0000","last_change":"2024-09-17T03:37:00.515111+0000","last_active":"2024-09-17T03:37:01.538197+0000","last_peered":"2024-09-17T03:37:01.538197+0000","last_clean":"2024-09-17T03:37:01.538197+0000","last_became_active":"2024-09-17T03:37:00.514784+0000","last_became_peered":"2024-09-17T03:37:00.514784+0000","last_unstale":"2024-09-17T03:37:01.538197+0000","last_undegraded":"2024-09-17T03:37:01.538197+0000","last_fullsized":"2024-09-17T03:37:01.538197+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:41.748322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":333,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496801+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:37:01.496801+0000","last_peered":"2024-09-17T03:37:01.496801+0000","last_clean":"2024-09-17T03:37:01.496801+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:37:01.496801+0000","last_undegraded":"2024-09-17T03:37:01.496801+0000","last_fullsized":"2024-09-17T03:37:01.496801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119528,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252312,"statfs":{"total":96636764160,"available":96514367488,"internally_reserved":0,"allocated":2371584,"data_stored":1366963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149452,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222388,"statfs":{"total":96636764160,"available":96483725312,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159188,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212652,"statfs":{"total":96636764160,"available":96473755648,"internally_reserved":0,"allocated":1769472,"data_stored":775943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127244,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244596,"statfs":{"total":96636764160,"available":96506466304,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167504,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204336,"statfs":{"total":96636764160,"available":96465240064,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150032,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221808,"statfs":{"total":96636764160,"available":96483131392,"internally_reserved":0,"allocated":2351104,"data_stored":1359508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":762837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542340,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171680,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200160,"statfs":{"total":96636764160,"available":96460963840,"internally_reserved":0,"allocated":1773568,"data_stored":776525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":173996485},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:37:06.552 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3b is not active+clean 2024-09-17T03:37:06.552 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.498567+0000', 'last_change': '2024-09-17T03:37:00.498567+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.498567+0000', 'last_undegraded': '2024-09-17T03:37:00.498567+0000', 'last_fullsized': '2024-09-17T03:37:00.498567+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.552 INFO:tasks.ceph.ceph_manager.ceph:PG 21.33 is not active+clean 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.498137+0000', 'last_change': '2024-09-17T03:37:00.498137+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.498137+0000', 'last_undegraded': '2024-09-17T03:37:00.498137+0000', 'last_fullsized': '2024-09-17T03:37:00.498137+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:PG 21.19 is not active+clean 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.498231+0000', 'last_change': '2024-09-17T03:37:00.498231+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.498231+0000', 'last_undegraded': '2024-09-17T03:37:00.498231+0000', 'last_fullsized': '2024-09-17T03:37:00.498231+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:PG 21.16 is not active+clean 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.497947+0000', 'last_change': '2024-09-17T03:37:00.497947+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.497947+0000', 'last_undegraded': '2024-09-17T03:37:00.497947+0000', 'last_fullsized': '2024-09-17T03:37:00.497947+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:PG 21.10 is not active+clean 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.497356+0000', 'last_change': '2024-09-17T03:37:00.497356+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.497356+0000', 'last_undegraded': '2024-09-17T03:37:00.497356+0000', 'last_fullsized': '2024-09-17T03:37:00.497356+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:PG 21.6 is not active+clean 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.497461+0000', 'last_change': '2024-09-17T03:37:00.497461+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.497461+0000', 'last_undegraded': '2024-09-17T03:37:00.497461+0000', 'last_fullsized': '2024-09-17T03:37:00.497461+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.553 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3 is not active+clean 2024-09-17T03:37:06.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.497390+0000', 'last_change': '2024-09-17T03:37:00.497390+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.497390+0000', 'last_undegraded': '2024-09-17T03:37:00.497390+0000', 'last_fullsized': '2024-09-17T03:37:00.497390+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.554 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1 is not active+clean 2024-09-17T03:37:06.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 141, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:37:00.497341+0000', 'last_change': '2024-09-17T03:37:00.497341+0000', 'last_active': '2024-09-17T03:37:00.476150+0000', 'last_peered': '2024-09-17T03:37:00.476150+0000', 'last_clean': '2024-09-17T03:37:00.476150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:37:00.497341+0000', 'last_undegraded': '2024-09-17T03:37:00.497341+0000', 'last_fullsized': '2024-09-17T03:37:00.497341+0000', 'mapping_epoch': 141, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 141, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'last_clean_scrub_stamp': '2024-09-17T03:37:00.476150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:37:06.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 -- 172.21.15.46:0/4010575667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c074b20 msgr2=0x7fab0c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 --2- 172.21.15.46:0/4010575667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c074b20 0x7fab0c15cf90 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0713e0 tx=0x7fab0401ccf0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 -- 172.21.15.46:0/4010575667 shutdown_connections 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 --2- 172.21.15.46:0/4010575667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c074b20 0x7fab0c15cf90 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 --2- 172.21.15.46:0/4010575667 >> v2:172.21.15.99:3300/0 conn(0x7fab0c157030 0x7fab0c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 -- 172.21.15.46:0/4010575667 >> 172.21.15.46:0/4010575667 conn(0x7fab0c0708c0 msgr2=0x7fab0c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:06.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 -- 172.21.15.46:0/4010575667 shutdown_connections 2024-09-17T03:37:06.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 -- 172.21.15.46:0/4010575667 wait complete. 2024-09-17T03:37:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 Processor -- start 2024-09-17T03:37:06.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.752+0000 7fab13200640 1 -- start start 2024-09-17T03:37:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fab0c157030 0x7fab0c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 0x7fab0c1fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab0c1f7100 0x7fab0c1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c16a750 con 0x7fab0c160d50 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c16a5d0 con 0x7fab0c1f7100 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c16a8d0 con 0x7fab0c157030 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 0x7fab0c1fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 0x7fab0c1fafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52344/0 (socket says 172.21.15.46:52344) 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 -- 172.21.15.46:0/3196822151 learned_addr learned my addr 172.21.15.46:0/3196822151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab0be00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fab0c157030 0x7fab0c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab11200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab0c1f7100 0x7fab0c1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 -- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fab0c157030 msgr2=0x7fab0c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fab0c157030 0x7fab0c0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 -- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab0c1f7100 msgr2=0x7fab0c1f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab0c1f7100 0x7fab0c1f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:06.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 -- 172.21.15.46:0/3196822151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c1f80d0 con 0x7fab0c160d50 2024-09-17T03:37:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab03e00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 0x7fab0c1fafc0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b3e0 tx=0x7faaf400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4015020 con 0x7fab0c160d50 2024-09-17T03:37:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf4011940 con 0x7fab0c160d50 2024-09-17T03:37:06.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4011d90 con 0x7fab0c160d50 2024-09-17T03:37:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c1f7df0 con 0x7fab0c160d50 2024-09-17T03:37:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c0b28d0 con 0x7fab0c160d50 2024-09-17T03:37:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.756+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c1f7b10 con 0x7fab0c160d50 2024-09-17T03:37:06.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.760+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7faaf4018020 con 0x7fab0c160d50 2024-09-17T03:37:06.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.760+0000 7fab08c00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faae8075570 0x7faae8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:06.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.760+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7faaf4093310 con 0x7fab0c160d50 2024-09-17T03:37:06.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.760+0000 7fab0be00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faae8075570 0x7faae8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:06.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.760+0000 7fab0be00640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faae8075570 0x7faae8077a80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7faafc0098f0 tx=0x7faafc006f90 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:06.764+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faaf401f020 con 0x7fab0c160d50 2024-09-17T03:37:07.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fab0c1508e0 con 0x7fab0c160d50 2024-09-17T03:37:07.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab08c00640 1 -- 172.21.15.46:0/3196822151 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1438 (secure 0 0 0) 0x7faaf405f740 con 0x7fab0c160d50 2024-09-17T03:37:07.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:07.001 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":984,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":143,"num_osds":8,"num_up_osds":8,"osd_up_since":1726543255,"num_in_osds":8,"osd_in_since":1726543245,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592756,"bytes_used":1199980544,"bytes_avail":771894132736,"bytes_total":773094113280,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":94,"btime":"2024-09-17T03:37:04:527000+0000","id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":248758}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-09-17T03:36:57.577784+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T03:37:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faae8075570 msgr2=0x7faae8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faae8075570 0x7faae8077a80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7faafc0098f0 tx=0x7faafc006f90 comp rx=0 tx=0).stop 2024-09-17T03:37:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 msgr2=0x7fab0c1fafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 0x7fab0c1fafc0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b3e0 tx=0x7faaf400b8a0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 shutdown_connections 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faae8075570 0x7faae8077a80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab0c1f7100 0x7fab0c1f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab0c160d50 0x7fab0c1fafc0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 --2- 172.21.15.46:0/3196822151 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fab0c157030 0x7fab0c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 >> 172.21.15.46:0/3196822151 conn(0x7fab0c0708c0 msgr2=0x7fab0c15e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 shutdown_connections 2024-09-17T03:37:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.000+0000 7fab13200640 1 -- 172.21.15.46:0/3196822151 wait complete. 2024-09-17T03:37:07.024 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:37:07.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.180+0000 7f71a1200640 1 -- 172.21.15.46:0/1510989511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b4de0 msgr2=0x7f719c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.180+0000 7f71a1200640 1 --2- 172.21.15.46:0/1510989511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b4de0 0x7f719c0b51f0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f719c071cb0 tx=0x7f719401c800 comp rx=0 tx=0).stop 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 -- 172.21.15.46:0/1510989511 shutdown_connections 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 --2- 172.21.15.46:0/1510989511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b4de0 0x7f719c0b51f0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 --2- 172.21.15.46:0/1510989511 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 0x7f719c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 --2- 172.21.15.46:0/1510989511 >> v2:172.21.15.99:3300/0 conn(0x7f719c0b5c70 0x7f719c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 -- 172.21.15.46:0/1510989511 >> 172.21.15.46:0/1510989511 conn(0x7f719c0b2040 msgr2=0x7f719c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 -- 172.21.15.46:0/1510989511 shutdown_connections 2024-09-17T03:37:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 -- 172.21.15.46:0/1510989511 wait complete. 2024-09-17T03:37:07.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 Processor -- start 2024-09-17T03:37:07.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 -- start start 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.184+0000 7f71a1200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719c0b4de0 0x7f719c0af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 0x7f719c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 0x7f719c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f719c15f730 con 0x7f719c0b5c70 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f719c15f5b0 con 0x7f719c0b4de0 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f719c15f8b0 con 0x7f719c1527b0 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 0x7f719c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 0x7f719c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52354/0 (socket says 172.21.15.46:52354) 2024-09-17T03:37:07.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 -- 172.21.15.46:0/4205612375 learned_addr learned my addr 172.21.15.46:0/4205612375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7199e00640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719c0b4de0 0x7f719c0af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f719a800640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 0x7f719c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 -- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 msgr2=0x7f719c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 0x7f719c0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 -- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719c0b4de0 msgr2=0x7f719c0af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719c0b4de0 0x7f719c0af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 -- 172.21.15.46:0/4205612375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c1fa050 con 0x7f719c0b5c70 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f719a800640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 0x7f719c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7191400640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 0x7f719c158e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f718400da10 tx=0x7f718400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184017070 con 0x7f719c0b5c70 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7184004b30 con 0x7f719c0b5c70 2024-09-17T03:37:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c1fa330 con 0x7f719c0b5c70 2024-09-17T03:37:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718400be60 con 0x7f719c0b5c70 2024-09-17T03:37:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1fa8f0 con 0x7f719c0b5c70 2024-09-17T03:37:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c1f9dc0 con 0x7f719c0b5c70 2024-09-17T03:37:07.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.188+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f7184003980 con 0x7f719c0b5c70 2024-09-17T03:37:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.192+0000 7f7192a00640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7178075570 0x7f7178077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.192+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f7184093ae0 con 0x7f719c0b5c70 2024-09-17T03:37:07.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.192+0000 7f7199e00640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7178075570 0x7f7178077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:07.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.192+0000 7f7199e00640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7178075570 0x7f7178077a80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c005d60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:07.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.196+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7184020020 con 0x7f719c0b5c70 2024-09-17T03:37:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.364+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f719c151ff0 con 0x7f7178075570 2024-09-17T03:37:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.372+0000 7f7192a00640 1 -- 172.21.15.46:0/4205612375 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251833 (secure 0 0 0) 0x7f718c00faa0 con 0x7f7178075570 2024-09-17T03:37:07.374 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:07.376 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7178075570 msgr2=0x7f7178077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7178075570 0x7f7178077a80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f718c005fd0 tx=0x7f718c005d60 comp rx=0 tx=0).stop 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 msgr2=0x7f719c158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 0x7f719c158e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f718400da10 tx=0x7f718400ded0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 shutdown_connections 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7178075570 0x7f7178077a80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f719c1527b0 0x7f719c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719c0b5c70 0x7f719c158e00 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 --2- 172.21.15.46:0/4205612375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f719c0b4de0 0x7f719c0af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 >> 172.21.15.46:0/4205612375 conn(0x7f719c0b2040 msgr2=0x7f719c15b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 shutdown_connections 2024-09-17T03:37:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:07.376+0000 7f71a1200640 1 -- 172.21.15.46:0/4205612375 wait complete. 2024-09-17T03:37:07.396 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-09-17T03:37:05.579171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1171856,"kb_used_data":14924,"kb_used_omap":12,"kb_used_meta":1156723,"kb_avail":753802864,"statfs":{"total":773094113280,"available":771894132736,"internally_reserved":0,"allocated":15282176,"data_stored":7349196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1184484912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.512539+0000","last_change":"2024-09-17T03:37:01.509979+0000","last_active":"2024-09-17T03:37:01.512539+0000","last_peered":"2024-09-17T03:37:01.512539+0000","last_clean":"2024-09-17T03:37:01.512539+0000","last_became_active":"2024-09-17T03:37:01.508226+0000","last_became_peered":"2024-09-17T03:37:01.508226+0000","last_unstale":"2024-09-17T03:37:01.512539+0000","last_undegraded":"2024-09-17T03:37:01.512539+0000","last_fullsized":"2024-09-17T03:37:01.512539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:52.973966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531593+0000","last_change":"2024-09-17T03:37:01.525396+0000","last_active":"2024-09-17T03:37:01.531593+0000","last_peered":"2024-09-17T03:37:01.531593+0000","last_clean":"2024-09-17T03:37:01.531593+0000","last_became_active":"2024-09-17T03:37:01.525068+0000","last_became_peered":"2024-09-17T03:37:01.525068+0000","last_unstale":"2024-09-17T03:37:01.531593+0000","last_undegraded":"2024-09-17T03:37:01.531593+0000","last_fullsized":"2024-09-17T03:37:01.531593+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:13.211290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519515+0000","last_change":"2024-09-17T03:37:01.517425+0000","last_active":"2024-09-17T03:37:01.519515+0000","last_peered":"2024-09-17T03:37:01.519515+0000","last_clean":"2024-09-17T03:37:01.519515+0000","last_became_active":"2024-09-17T03:37:01.517118+0000","last_became_peered":"2024-09-17T03:37:01.517118+0000","last_unstale":"2024-09-17T03:37:01.519515+0000","last_undegraded":"2024-09-17T03:37:01.519515+0000","last_fullsized":"2024-09-17T03:37:01.519515+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528816+0000","last_change":"2024-09-17T03:37:01.524178+0000","last_active":"2024-09-17T03:37:01.528816+0000","last_peered":"2024-09-17T03:37:01.528816+0000","last_clean":"2024-09-17T03:37:01.528816+0000","last_became_active":"2024-09-17T03:37:01.523558+0000","last_became_peered":"2024-09-17T03:37:01.523558+0000","last_unstale":"2024-09-17T03:37:01.528816+0000","last_undegraded":"2024-09-17T03:37:01.528816+0000","last_fullsized":"2024-09-17T03:37:01.528816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:24.391567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498567+0000","last_change":"2024-09-17T03:37:00.498567+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498567+0000","last_undegraded":"2024-09-17T03:37:00.498567+0000","last_fullsized":"2024-09-17T03:37:00.498567+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510156+0000","last_change":"2024-09-17T03:37:01.507556+0000","last_active":"2024-09-17T03:37:01.510156+0000","last_peered":"2024-09-17T03:37:01.510156+0000","last_clean":"2024-09-17T03:37:01.510156+0000","last_became_active":"2024-09-17T03:37:01.507216+0000","last_became_peered":"2024-09-17T03:37:01.507216+0000","last_unstale":"2024-09-17T03:37:01.510156+0000","last_undegraded":"2024-09-17T03:37:01.510156+0000","last_fullsized":"2024-09-17T03:37:01.510156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:58.827646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524913+0000","last_change":"2024-09-17T03:37:01.520291+0000","last_active":"2024-09-17T03:37:01.524913+0000","last_peered":"2024-09-17T03:37:01.524913+0000","last_clean":"2024-09-17T03:37:01.524913+0000","last_became_active":"2024-09-17T03:37:01.519510+0000","last_became_peered":"2024-09-17T03:37:01.519510+0000","last_unstale":"2024-09-17T03:37:01.524913+0000","last_undegraded":"2024-09-17T03:37:01.524913+0000","last_fullsized":"2024-09-17T03:37:01.524913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:14.128722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.515271+0000","last_change":"2024-09-17T03:37:01.511915+0000","last_active":"2024-09-17T03:37:01.515271+0000","last_peered":"2024-09-17T03:37:01.515271+0000","last_clean":"2024-09-17T03:37:01.515271+0000","last_became_active":"2024-09-17T03:37:01.511588+0000","last_became_peered":"2024-09-17T03:37:01.511588+0000","last_unstale":"2024-09-17T03:37:01.515271+0000","last_undegraded":"2024-09-17T03:37:01.515271+0000","last_fullsized":"2024-09-17T03:37:01.515271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:34.703407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518221+0000","last_change":"2024-09-17T03:37:01.504394+0000","last_active":"2024-09-17T03:37:01.518221+0000","last_peered":"2024-09-17T03:37:01.518221+0000","last_clean":"2024-09-17T03:37:01.518221+0000","last_became_active":"2024-09-17T03:37:01.504016+0000","last_became_peered":"2024-09-17T03:37:01.504016+0000","last_unstale":"2024-09-17T03:37:01.518221+0000","last_undegraded":"2024-09-17T03:37:01.518221+0000","last_fullsized":"2024-09-17T03:37:01.518221+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:47.918178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526096+0000","last_change":"2024-09-17T03:37:01.521538+0000","last_active":"2024-09-17T03:37:01.526096+0000","last_peered":"2024-09-17T03:37:01.526096+0000","last_clean":"2024-09-17T03:37:01.526096+0000","last_became_active":"2024-09-17T03:37:01.521253+0000","last_became_peered":"2024-09-17T03:37:01.521253+0000","last_unstale":"2024-09-17T03:37:01.526096+0000","last_undegraded":"2024-09-17T03:37:01.526096+0000","last_fullsized":"2024-09-17T03:37:01.526096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:33.873450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509897+0000","last_change":"2024-09-17T03:37:01.507075+0000","last_active":"2024-09-17T03:37:01.509897+0000","last_peered":"2024-09-17T03:37:01.509897+0000","last_clean":"2024-09-17T03:37:01.509897+0000","last_became_active":"2024-09-17T03:37:01.506711+0000","last_became_peered":"2024-09-17T03:37:01.506711+0000","last_unstale":"2024-09-17T03:37:01.509897+0000","last_undegraded":"2024-09-17T03:37:01.509897+0000","last_fullsized":"2024-09-17T03:37:01.509897+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:56.420785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.520006+0000","last_change":"2024-09-17T03:37:01.515856+0000","last_active":"2024-09-17T03:37:01.520006+0000","last_peered":"2024-09-17T03:37:01.520006+0000","last_clean":"2024-09-17T03:37:01.520006+0000","last_became_active":"2024-09-17T03:37:01.512742+0000","last_became_peered":"2024-09-17T03:37:01.512742+0000","last_unstale":"2024-09-17T03:37:01.520006+0000","last_undegraded":"2024-09-17T03:37:01.520006+0000","last_fullsized":"2024-09-17T03:37:01.520006+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:16.486830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498137+0000","last_change":"2024-09-17T03:37:00.498137+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498137+0000","last_undegraded":"2024-09-17T03:37:00.498137+0000","last_fullsized":"2024-09-17T03:37:00.498137+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534060+0000","last_change":"2024-09-17T03:37:01.530828+0000","last_active":"2024-09-17T03:37:01.534060+0000","last_peered":"2024-09-17T03:37:01.534060+0000","last_clean":"2024-09-17T03:37:01.534060+0000","last_became_active":"2024-09-17T03:37:01.529611+0000","last_became_peered":"2024-09-17T03:37:01.529611+0000","last_unstale":"2024-09-17T03:37:01.534060+0000","last_undegraded":"2024-09-17T03:37:01.534060+0000","last_fullsized":"2024-09-17T03:37:01.534060+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526083+0000","last_change":"2024-09-17T03:37:01.524406+0000","last_active":"2024-09-17T03:37:01.526083+0000","last_peered":"2024-09-17T03:37:01.526083+0000","last_clean":"2024-09-17T03:37:01.526083+0000","last_became_active":"2024-09-17T03:37:01.524097+0000","last_became_peered":"2024-09-17T03:37:01.524097+0000","last_unstale":"2024-09-17T03:37:01.526083+0000","last_undegraded":"2024-09-17T03:37:01.526083+0000","last_fullsized":"2024-09-17T03:37:01.526083+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:47.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534103+0000","last_change":"2024-09-17T03:37:01.527309+0000","last_active":"2024-09-17T03:37:01.534103+0000","last_peered":"2024-09-17T03:37:01.534103+0000","last_clean":"2024-09-17T03:37:01.534103+0000","last_became_active":"2024-09-17T03:37:01.526391+0000","last_became_peered":"2024-09-17T03:37:01.526391+0000","last_unstale":"2024-09-17T03:37:01.534103+0000","last_undegraded":"2024-09-17T03:37:01.534103+0000","last_fullsized":"2024-09-17T03:37:01.534103+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:42.503735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536631+0000","last_change":"2024-09-17T03:37:01.531294+0000","last_active":"2024-09-17T03:37:01.536631+0000","last_peered":"2024-09-17T03:37:01.536631+0000","last_clean":"2024-09-17T03:37:01.536631+0000","last_became_active":"2024-09-17T03:37:01.529834+0000","last_became_peered":"2024-09-17T03:37:01.529834+0000","last_unstale":"2024-09-17T03:37:01.536631+0000","last_undegraded":"2024-09-17T03:37:01.536631+0000","last_fullsized":"2024-09-17T03:37:01.536631+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:55.329149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523036+0000","last_change":"2024-09-17T03:37:01.518791+0000","last_active":"2024-09-17T03:37:01.523036+0000","last_peered":"2024-09-17T03:37:01.523036+0000","last_clean":"2024-09-17T03:37:01.523036+0000","last_became_active":"2024-09-17T03:37:01.516765+0000","last_became_peered":"2024-09-17T03:37:01.516765+0000","last_unstale":"2024-09-17T03:37:01.523036+0000","last_undegraded":"2024-09-17T03:37:01.523036+0000","last_fullsized":"2024-09-17T03:37:01.523036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529870+0000","last_change":"2024-09-17T03:37:01.527808+0000","last_active":"2024-09-17T03:37:01.529870+0000","last_peered":"2024-09-17T03:37:01.529870+0000","last_clean":"2024-09-17T03:37:01.529870+0000","last_became_active":"2024-09-17T03:37:01.527469+0000","last_became_peered":"2024-09-17T03:37:01.527469+0000","last_unstale":"2024-09-17T03:37:01.529870+0000","last_undegraded":"2024-09-17T03:37:01.529870+0000","last_fullsized":"2024-09-17T03:37:01.529870+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:54.208586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534127+0000","last_change":"2024-09-17T03:37:01.530734+0000","last_active":"2024-09-17T03:37:01.534127+0000","last_peered":"2024-09-17T03:37:01.534127+0000","last_clean":"2024-09-17T03:37:01.534127+0000","last_became_active":"2024-09-17T03:37:01.530184+0000","last_became_peered":"2024-09-17T03:37:01.530184+0000","last_unstale":"2024-09-17T03:37:01.534127+0000","last_undegraded":"2024-09-17T03:37:01.534127+0000","last_fullsized":"2024-09-17T03:37:01.534127+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:48.884673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525880+0000","last_change":"2024-09-17T03:37:01.520019+0000","last_active":"2024-09-17T03:37:01.525880+0000","last_peered":"2024-09-17T03:37:01.525880+0000","last_clean":"2024-09-17T03:37:01.525880+0000","last_became_active":"2024-09-17T03:37:01.519632+0000","last_became_peered":"2024-09-17T03:37:01.519632+0000","last_unstale":"2024-09-17T03:37:01.525880+0000","last_undegraded":"2024-09-17T03:37:01.525880+0000","last_fullsized":"2024-09-17T03:37:01.525880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:57.353653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536601+0000","last_change":"2024-09-17T03:37:01.532969+0000","last_active":"2024-09-17T03:37:01.536601+0000","last_peered":"2024-09-17T03:37:01.536601+0000","last_clean":"2024-09-17T03:37:01.536601+0000","last_became_active":"2024-09-17T03:37:01.530349+0000","last_became_peered":"2024-09-17T03:37:01.530349+0000","last_unstale":"2024-09-17T03:37:01.536601+0000","last_undegraded":"2024-09-17T03:37:01.536601+0000","last_fullsized":"2024-09-17T03:37:01.536601+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530028+0000","last_change":"2024-09-17T03:37:01.527670+0000","last_active":"2024-09-17T03:37:01.530028+0000","last_peered":"2024-09-17T03:37:01.530028+0000","last_clean":"2024-09-17T03:37:01.530028+0000","last_became_active":"2024-09-17T03:37:01.527335+0000","last_became_peered":"2024-09-17T03:37:01.527335+0000","last_unstale":"2024-09-17T03:37:01.530028+0000","last_undegraded":"2024-09-17T03:37:01.530028+0000","last_fullsized":"2024-09-17T03:37:01.530028+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:42.382487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.525874+0000","last_change":"2024-09-17T03:37:01.520830+0000","last_active":"2024-09-17T03:37:01.525874+0000","last_peered":"2024-09-17T03:37:01.525874+0000","last_clean":"2024-09-17T03:37:01.525874+0000","last_became_active":"2024-09-17T03:37:01.520514+0000","last_became_peered":"2024-09-17T03:37:01.520514+0000","last_unstale":"2024-09-17T03:37:01.525874+0000","last_undegraded":"2024-09-17T03:37:01.525874+0000","last_fullsized":"2024-09-17T03:37:01.525874+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:28.913796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533954+0000","last_change":"2024-09-17T03:37:01.526010+0000","last_active":"2024-09-17T03:37:01.533954+0000","last_peered":"2024-09-17T03:37:01.533954+0000","last_clean":"2024-09-17T03:37:01.533954+0000","last_became_active":"2024-09-17T03:37:01.524307+0000","last_became_peered":"2024-09-17T03:37:01.524307+0000","last_unstale":"2024-09-17T03:37:01.533954+0000","last_undegraded":"2024-09-17T03:37:01.533954+0000","last_fullsized":"2024-09-17T03:37:01.533954+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:17.817507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529940+0000","last_change":"2024-09-17T03:37:01.526850+0000","last_active":"2024-09-17T03:37:01.529940+0000","last_peered":"2024-09-17T03:37:01.529940+0000","last_clean":"2024-09-17T03:37:01.529940+0000","last_became_active":"2024-09-17T03:37:01.526500+0000","last_became_peered":"2024-09-17T03:37:01.526500+0000","last_unstale":"2024-09-17T03:37:01.529940+0000","last_undegraded":"2024-09-17T03:37:01.529940+0000","last_fullsized":"2024-09-17T03:37:01.529940+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:01.514192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534424+0000","last_change":"2024-09-17T03:37:01.525369+0000","last_active":"2024-09-17T03:37:01.534424+0000","last_peered":"2024-09-17T03:37:01.534424+0000","last_clean":"2024-09-17T03:37:01.534424+0000","last_became_active":"2024-09-17T03:37:01.524994+0000","last_became_peered":"2024-09-17T03:37:01.524994+0000","last_unstale":"2024-09-17T03:37:01.534424+0000","last_undegraded":"2024-09-17T03:37:01.534424+0000","last_fullsized":"2024-09-17T03:37:01.534424+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:16.958527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521738+0000","last_change":"2024-09-17T03:37:01.514688+0000","last_active":"2024-09-17T03:37:01.521738+0000","last_peered":"2024-09-17T03:37:01.521738+0000","last_clean":"2024-09-17T03:37:01.521738+0000","last_became_active":"2024-09-17T03:37:01.505442+0000","last_became_peered":"2024-09-17T03:37:01.505442+0000","last_unstale":"2024-09-17T03:37:01.521738+0000","last_undegraded":"2024-09-17T03:37:01.521738+0000","last_fullsized":"2024-09-17T03:37:01.521738+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:51.069459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510776+0000","last_change":"2024-09-17T03:37:01.509207+0000","last_active":"2024-09-17T03:37:01.510776+0000","last_peered":"2024-09-17T03:37:01.510776+0000","last_clean":"2024-09-17T03:37:01.510776+0000","last_became_active":"2024-09-17T03:37:01.508207+0000","last_became_peered":"2024-09-17T03:37:01.508207+0000","last_unstale":"2024-09-17T03:37:01.510776+0000","last_undegraded":"2024-09-17T03:37:01.510776+0000","last_fullsized":"2024-09-17T03:37:01.510776+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:48.105755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528879+0000","last_change":"2024-09-17T03:37:01.524856+0000","last_active":"2024-09-17T03:37:01.528879+0000","last_peered":"2024-09-17T03:37:01.528879+0000","last_clean":"2024-09-17T03:37:01.528879+0000","last_became_active":"2024-09-17T03:37:01.522971+0000","last_became_peered":"2024-09-17T03:37:01.522971+0000","last_unstale":"2024-09-17T03:37:01.528879+0000","last_undegraded":"2024-09-17T03:37:01.528879+0000","last_fullsized":"2024-09-17T03:37:01.528879+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.014998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517595+0000","last_change":"2024-09-17T03:37:01.504583+0000","last_active":"2024-09-17T03:37:01.517595+0000","last_peered":"2024-09-17T03:37:01.517595+0000","last_clean":"2024-09-17T03:37:01.517595+0000","last_became_active":"2024-09-17T03:37:01.504221+0000","last_became_peered":"2024-09-17T03:37:01.504221+0000","last_unstale":"2024-09-17T03:37:01.517595+0000","last_undegraded":"2024-09-17T03:37:01.517595+0000","last_fullsized":"2024-09-17T03:37:01.517595+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:18.113674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534132+0000","last_change":"2024-09-17T03:37:01.527737+0000","last_active":"2024-09-17T03:37:01.534132+0000","last_peered":"2024-09-17T03:37:01.534132+0000","last_clean":"2024-09-17T03:37:01.534132+0000","last_became_active":"2024-09-17T03:37:01.526783+0000","last_became_peered":"2024-09-17T03:37:01.526783+0000","last_unstale":"2024-09-17T03:37:01.534132+0000","last_undegraded":"2024-09-17T03:37:01.534132+0000","last_fullsized":"2024-09-17T03:37:01.534132+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:28.866527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518126+0000","last_change":"2024-09-17T03:37:01.513699+0000","last_active":"2024-09-17T03:37:01.518126+0000","last_peered":"2024-09-17T03:37:01.518126+0000","last_clean":"2024-09-17T03:37:01.518126+0000","last_became_active":"2024-09-17T03:37:01.513310+0000","last_became_peered":"2024-09-17T03:37:01.513310+0000","last_unstale":"2024-09-17T03:37:01.518126+0000","last_undegraded":"2024-09-17T03:37:01.518126+0000","last_fullsized":"2024-09-17T03:37:01.518126+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"142'4","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535697+0000","last_change":"2024-09-17T03:37:00.521721+0000","last_active":"2024-09-17T03:37:01.535697+0000","last_peered":"2024-09-17T03:37:01.535697+0000","last_clean":"2024-09-17T03:37:01.535697+0000","last_became_active":"2024-09-17T03:37:00.521476+0000","last_became_peered":"2024-09-17T03:37:00.521476+0000","last_unstale":"2024-09-17T03:37:01.535697+0000","last_undegraded":"2024-09-17T03:37:01.535697+0000","last_fullsized":"2024-09-17T03:37:01.535697+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:11.151395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531349+0000","last_change":"2024-09-17T03:37:01.524327+0000","last_active":"2024-09-17T03:37:01.531349+0000","last_peered":"2024-09-17T03:37:01.531349+0000","last_clean":"2024-09-17T03:37:01.531349+0000","last_became_active":"2024-09-17T03:37:01.523988+0000","last_became_peered":"2024-09-17T03:37:01.523988+0000","last_unstale":"2024-09-17T03:37:01.531349+0000","last_undegraded":"2024-09-17T03:37:01.531349+0000","last_fullsized":"2024-09-17T03:37:01.531349+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:57.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.522778+0000","last_change":"2024-09-17T03:37:00.513012+0000","last_active":"2024-09-17T03:37:01.522778+0000","last_peered":"2024-09-17T03:37:01.522778+0000","last_clean":"2024-09-17T03:37:01.522778+0000","last_became_active":"2024-09-17T03:37:00.512669+0000","last_became_peered":"2024-09-17T03:37:00.512669+0000","last_unstale":"2024-09-17T03:37:01.522778+0000","last_undegraded":"2024-09-17T03:37:01.522778+0000","last_fullsized":"2024-09-17T03:37:01.522778+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:04.275671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.523023+0000","last_change":"2024-09-17T03:37:01.520484+0000","last_active":"2024-09-17T03:37:01.523023+0000","last_peered":"2024-09-17T03:37:01.523023+0000","last_clean":"2024-09-17T03:37:01.523023+0000","last_became_active":"2024-09-17T03:37:01.518380+0000","last_became_peered":"2024-09-17T03:37:01.518380+0000","last_unstale":"2024-09-17T03:37:01.523023+0000","last_undegraded":"2024-09-17T03:37:01.523023+0000","last_fullsized":"2024-09-17T03:37:01.523023+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:10.898889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537662+0000","last_change":"2024-09-17T03:37:00.518391+0000","last_active":"2024-09-17T03:37:01.537662+0000","last_peered":"2024-09-17T03:37:01.537662+0000","last_clean":"2024-09-17T03:37:01.537662+0000","last_became_active":"2024-09-17T03:37:00.518014+0000","last_became_peered":"2024-09-17T03:37:00.518014+0000","last_unstale":"2024-09-17T03:37:01.537662+0000","last_undegraded":"2024-09-17T03:37:01.537662+0000","last_fullsized":"2024-09-17T03:37:01.537662+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518010+0000","last_change":"2024-09-17T03:37:01.513224+0000","last_active":"2024-09-17T03:37:01.518010+0000","last_peered":"2024-09-17T03:37:01.518010+0000","last_clean":"2024-09-17T03:37:01.518010+0000","last_became_active":"2024-09-17T03:37:01.512886+0000","last_became_peered":"2024-09-17T03:37:01.512886+0000","last_unstale":"2024-09-17T03:37:01.518010+0000","last_undegraded":"2024-09-17T03:37:01.518010+0000","last_fullsized":"2024-09-17T03:37:01.518010+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:13.599854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.535032+0000","last_change":"2024-09-17T03:37:00.516162+0000","last_active":"2024-09-17T03:37:01.535032+0000","last_peered":"2024-09-17T03:37:01.535032+0000","last_clean":"2024-09-17T03:37:01.535032+0000","last_became_active":"2024-09-17T03:37:00.515367+0000","last_became_peered":"2024-09-17T03:37:00.515367+0000","last_unstale":"2024-09-17T03:37:01.535032+0000","last_undegraded":"2024-09-17T03:37:01.535032+0000","last_fullsized":"2024-09-17T03:37:01.535032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:08.671535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518310+0000","last_change":"2024-09-17T03:37:01.506152+0000","last_active":"2024-09-17T03:37:01.518310+0000","last_peered":"2024-09-17T03:37:01.518310+0000","last_clean":"2024-09-17T03:37:01.518310+0000","last_became_active":"2024-09-17T03:37:01.503759+0000","last_became_peered":"2024-09-17T03:37:01.503759+0000","last_unstale":"2024-09-17T03:37:01.518310+0000","last_undegraded":"2024-09-17T03:37:01.518310+0000","last_fullsized":"2024-09-17T03:37:01.518310+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:43.445864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.528132+0000","last_change":"2024-09-17T03:37:00.510847+0000","last_active":"2024-09-17T03:37:01.528132+0000","last_peered":"2024-09-17T03:37:01.528132+0000","last_clean":"2024-09-17T03:37:01.528132+0000","last_became_active":"2024-09-17T03:37:00.509550+0000","last_became_peered":"2024-09-17T03:37:00.509550+0000","last_unstale":"2024-09-17T03:37:01.528132+0000","last_undegraded":"2024-09-17T03:37:01.528132+0000","last_fullsized":"2024-09-17T03:37:01.528132+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.371788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531043+0000","last_change":"2024-09-17T03:37:01.524253+0000","last_active":"2024-09-17T03:37:01.531043+0000","last_peered":"2024-09-17T03:37:01.531043+0000","last_clean":"2024-09-17T03:37:01.531043+0000","last_became_active":"2024-09-17T03:37:01.523935+0000","last_became_peered":"2024-09-17T03:37:01.523935+0000","last_unstale":"2024-09-17T03:37:01.531043+0000","last_undegraded":"2024-09-17T03:37:01.531043+0000","last_fullsized":"2024-09-17T03:37:01.531043+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.515827+0000","last_change":"2024-09-17T03:37:00.513776+0000","last_active":"2024-09-17T03:37:00.515827+0000","last_peered":"2024-09-17T03:37:00.515827+0000","last_clean":"2024-09-17T03:37:00.515827+0000","last_became_active":"2024-09-17T03:37:00.511311+0000","last_became_peered":"2024-09-17T03:37:00.511311+0000","last_unstale":"2024-09-17T03:37:00.515827+0000","last_undegraded":"2024-09-17T03:37:00.515827+0000","last_fullsized":"2024-09-17T03:37:00.515827+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:16.989474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.517432+0000","last_change":"2024-09-17T03:37:01.513641+0000","last_active":"2024-09-17T03:37:01.517432+0000","last_peered":"2024-09-17T03:37:01.517432+0000","last_clean":"2024-09-17T03:37:01.517432+0000","last_became_active":"2024-09-17T03:37:01.513325+0000","last_became_peered":"2024-09-17T03:37:01.513325+0000","last_unstale":"2024-09-17T03:37:01.517432+0000","last_undegraded":"2024-09-17T03:37:01.517432+0000","last_fullsized":"2024-09-17T03:37:01.517432+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:54.606013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.516501+0000","last_change":"2024-09-17T03:37:00.513452+0000","last_active":"2024-09-17T03:37:01.516501+0000","last_peered":"2024-09-17T03:37:01.516501+0000","last_clean":"2024-09-17T03:37:01.516501+0000","last_became_active":"2024-09-17T03:37:00.513118+0000","last_became_peered":"2024-09-17T03:37:00.513118+0000","last_unstale":"2024-09-17T03:37:01.516501+0000","last_undegraded":"2024-09-17T03:37:01.516501+0000","last_fullsized":"2024-09-17T03:37:01.516501+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:53.259179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.498231+0000","last_change":"2024-09-17T03:37:00.498231+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.498231+0000","last_undegraded":"2024-09-17T03:37:00.498231+0000","last_fullsized":"2024-09-17T03:37:00.498231+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.509557+0000","last_change":"2024-09-17T03:37:00.506180+0000","last_active":"2024-09-17T03:37:01.509557+0000","last_peered":"2024-09-17T03:37:01.509557+0000","last_clean":"2024-09-17T03:37:01.509557+0000","last_became_active":"2024-09-17T03:37:00.505868+0000","last_became_peered":"2024-09-17T03:37:00.505868+0000","last_unstale":"2024-09-17T03:37:01.509557+0000","last_undegraded":"2024-09-17T03:37:01.509557+0000","last_fullsized":"2024-09-17T03:37:01.509557+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497947+0000","last_change":"2024-09-17T03:37:00.497947+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497947+0000","last_undegraded":"2024-09-17T03:37:00.497947+0000","last_fullsized":"2024-09-17T03:37:00.497947+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494787+0000","last_change":"2024-09-17T03:37:00.513939+0000","last_active":"2024-09-17T03:37:01.494787+0000","last_peered":"2024-09-17T03:37:01.494787+0000","last_clean":"2024-09-17T03:37:01.494787+0000","last_became_active":"2024-09-17T03:37:00.513531+0000","last_became_peered":"2024-09-17T03:37:00.513531+0000","last_unstale":"2024-09-17T03:37:01.494787+0000","last_undegraded":"2024-09-17T03:37:01.494787+0000","last_fullsized":"2024-09-17T03:37:01.494787+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:43.986216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.536554+0000","last_change":"2024-09-17T03:37:01.530837+0000","last_active":"2024-09-17T03:37:01.536554+0000","last_peered":"2024-09-17T03:37:01.536554+0000","last_clean":"2024-09-17T03:37:01.536554+0000","last_became_active":"2024-09-17T03:37:01.530166+0000","last_became_peered":"2024-09-17T03:37:01.530166+0000","last_unstale":"2024-09-17T03:37:01.536554+0000","last_undegraded":"2024-09-17T03:37:01.536554+0000","last_fullsized":"2024-09-17T03:37:01.536554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:32.600471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.498465+0000","last_change":"2024-09-17T03:37:00.515644+0000","last_active":"2024-09-17T03:37:01.498465+0000","last_peered":"2024-09-17T03:37:01.498465+0000","last_clean":"2024-09-17T03:37:01.498465+0000","last_became_active":"2024-09-17T03:37:00.515326+0000","last_became_peered":"2024-09-17T03:37:00.515326+0000","last_unstale":"2024-09-17T03:37:01.498465+0000","last_undegraded":"2024-09-17T03:37:01.498465+0000","last_fullsized":"2024-09-17T03:37:01.498465+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:05.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534454+0000","last_change":"2024-09-17T03:37:01.530851+0000","last_active":"2024-09-17T03:37:01.534454+0000","last_peered":"2024-09-17T03:37:01.534454+0000","last_clean":"2024-09-17T03:37:01.534454+0000","last_became_active":"2024-09-17T03:37:01.530410+0000","last_became_peered":"2024-09-17T03:37:01.530410+0000","last_unstale":"2024-09-17T03:37:01.534454+0000","last_undegraded":"2024-09-17T03:37:01.534454+0000","last_fullsized":"2024-09-17T03:37:01.534454+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:54.349008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"142'6","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537003+0000","last_change":"2024-09-17T03:37:00.511424+0000","last_active":"2024-09-17T03:37:01.537003+0000","last_peered":"2024-09-17T03:37:01.537003+0000","last_clean":"2024-09-17T03:37:01.537003+0000","last_became_active":"2024-09-17T03:37:00.511055+0000","last_became_peered":"2024-09-17T03:37:00.511055+0000","last_unstale":"2024-09-17T03:37:01.537003+0000","last_undegraded":"2024-09-17T03:37:01.537003+0000","last_fullsized":"2024-09-17T03:37:01.537003+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:04.537830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.526765+0000","last_change":"2024-09-17T03:37:01.520532+0000","last_active":"2024-09-17T03:37:01.526765+0000","last_peered":"2024-09-17T03:37:01.526765+0000","last_clean":"2024-09-17T03:37:01.526765+0000","last_became_active":"2024-09-17T03:37:01.520261+0000","last_became_peered":"2024-09-17T03:37:01.520261+0000","last_unstale":"2024-09-17T03:37:01.526765+0000","last_undegraded":"2024-09-17T03:37:01.526765+0000","last_fullsized":"2024-09-17T03:37:01.526765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538130+0000","last_change":"2024-09-17T03:37:00.509307+0000","last_active":"2024-09-17T03:37:01.538130+0000","last_peered":"2024-09-17T03:37:01.538130+0000","last_clean":"2024-09-17T03:37:01.538130+0000","last_became_active":"2024-09-17T03:37:00.508858+0000","last_became_peered":"2024-09-17T03:37:00.508858+0000","last_unstale":"2024-09-17T03:37:01.538130+0000","last_undegraded":"2024-09-17T03:37:01.538130+0000","last_fullsized":"2024-09-17T03:37:01.538130+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:19.533265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532147+0000","last_change":"2024-09-17T03:37:01.526495+0000","last_active":"2024-09-17T03:37:01.532147+0000","last_peered":"2024-09-17T03:37:01.532147+0000","last_clean":"2024-09-17T03:37:01.532147+0000","last_became_active":"2024-09-17T03:37:01.526193+0000","last_became_peered":"2024-09-17T03:37:01.526193+0000","last_unstale":"2024-09-17T03:37:01.532147+0000","last_undegraded":"2024-09-17T03:37:01.532147+0000","last_fullsized":"2024-09-17T03:37:01.532147+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:38.826521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506839+0000","last_change":"2024-09-17T03:37:00.506435+0000","last_active":"2024-09-17T03:37:01.506839+0000","last_peered":"2024-09-17T03:37:01.506839+0000","last_clean":"2024-09-17T03:37:01.506839+0000","last_became_active":"2024-09-17T03:37:00.505499+0000","last_became_peered":"2024-09-17T03:37:00.505499+0000","last_unstale":"2024-09-17T03:37:01.506839+0000","last_undegraded":"2024-09-17T03:37:01.506839+0000","last_fullsized":"2024-09-17T03:37:01.506839+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497356+0000","last_change":"2024-09-17T03:37:00.497356+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497356+0000","last_undegraded":"2024-09-17T03:37:00.497356+0000","last_fullsized":"2024-09-17T03:37:00.497356+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.490845+0000","last_change":"2024-09-17T03:37:00.497616+0000","last_active":"2024-09-17T03:37:01.490845+0000","last_peered":"2024-09-17T03:37:01.490845+0000","last_clean":"2024-09-17T03:37:01.490845+0000","last_became_active":"2024-09-17T03:37:00.497229+0000","last_became_peered":"2024-09-17T03:37:00.497229+0000","last_unstale":"2024-09-17T03:37:01.490845+0000","last_undegraded":"2024-09-17T03:37:01.490845+0000","last_fullsized":"2024-09-17T03:37:01.490845+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:08.103469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.531634+0000","last_change":"2024-09-17T03:37:01.525031+0000","last_active":"2024-09-17T03:37:01.531634+0000","last_peered":"2024-09-17T03:37:01.531634+0000","last_clean":"2024-09-17T03:37:01.531634+0000","last_became_active":"2024-09-17T03:37:01.524710+0000","last_became_peered":"2024-09-17T03:37:01.524710+0000","last_unstale":"2024-09-17T03:37:01.531634+0000","last_undegraded":"2024-09-17T03:37:01.531634+0000","last_fullsized":"2024-09-17T03:37:01.531634+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:59.085601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"142'8","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.534881+0000","last_change":"2024-09-17T03:37:00.506497+0000","last_active":"2024-09-17T03:37:01.534881+0000","last_peered":"2024-09-17T03:37:01.534881+0000","last_clean":"2024-09-17T03:37:01.534881+0000","last_became_active":"2024-09-17T03:37:00.506195+0000","last_became_peered":"2024-09-17T03:37:00.506195+0000","last_unstale":"2024-09-17T03:37:01.534881+0000","last_undegraded":"2024-09-17T03:37:01.534881+0000","last_fullsized":"2024-09-17T03:37:01.534881+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:35.347269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.511186+0000","last_change":"2024-09-17T03:37:01.510170+0000","last_active":"2024-09-17T03:37:01.511186+0000","last_peered":"2024-09-17T03:37:01.511186+0000","last_clean":"2024-09-17T03:37:01.511186+0000","last_became_active":"2024-09-17T03:37:01.509806+0000","last_became_peered":"2024-09-17T03:37:01.509806+0000","last_unstale":"2024-09-17T03:37:01.511186+0000","last_undegraded":"2024-09-17T03:37:01.511186+0000","last_fullsized":"2024-09-17T03:37:01.511186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:43.917446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-17T03:37:00.511017+0000","last_change":"2024-09-17T03:37:00.507289+0000","last_active":"2024-09-17T03:37:00.511017+0000","last_peered":"2024-09-17T03:37:00.511017+0000","last_clean":"2024-09-17T03:37:00.511017+0000","last_became_active":"2024-09-17T03:37:00.506994+0000","last_became_peered":"2024-09-17T03:37:00.506994+0000","last_unstale":"2024-09-17T03:37:00.511017+0000","last_undegraded":"2024-09-17T03:37:00.511017+0000","last_fullsized":"2024-09-17T03:37:00.511017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:13.903403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533925+0000","last_change":"2024-09-17T03:37:01.525640+0000","last_active":"2024-09-17T03:37:01.533925+0000","last_peered":"2024-09-17T03:37:01.533925+0000","last_clean":"2024-09-17T03:37:01.533925+0000","last_became_active":"2024-09-17T03:37:01.523930+0000","last_became_peered":"2024-09-17T03:37:01.523930+0000","last_unstale":"2024-09-17T03:37:01.533925+0000","last_undegraded":"2024-09-17T03:37:01.533925+0000","last_fullsized":"2024-09-17T03:37:01.533925+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:17.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519679+0000","last_change":"2024-09-17T03:37:00.514697+0000","last_active":"2024-09-17T03:37:01.519679+0000","last_peered":"2024-09-17T03:37:01.519679+0000","last_clean":"2024-09-17T03:37:01.519679+0000","last_became_active":"2024-09-17T03:37:00.513958+0000","last_became_peered":"2024-09-17T03:37:00.513958+0000","last_unstale":"2024-09-17T03:37:01.519679+0000","last_undegraded":"2024-09-17T03:37:01.519679+0000","last_fullsized":"2024-09-17T03:37:01.519679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:27.979522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529910+0000","last_change":"2024-09-17T03:37:01.521591+0000","last_active":"2024-09-17T03:37:01.529910+0000","last_peered":"2024-09-17T03:37:01.529910+0000","last_clean":"2024-09-17T03:37:01.529910+0000","last_became_active":"2024-09-17T03:37:01.521066+0000","last_became_peered":"2024-09-17T03:37:01.521066+0000","last_unstale":"2024-09-17T03:37:01.529910+0000","last_undegraded":"2024-09-17T03:37:01.529910+0000","last_fullsized":"2024-09-17T03:37:01.529910+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:30.334793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"142'1","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533162+0000","last_change":"2024-09-17T03:37:00.510701+0000","last_active":"2024-09-17T03:37:01.533162+0000","last_peered":"2024-09-17T03:37:01.533162+0000","last_clean":"2024-09-17T03:37:01.533162+0000","last_became_active":"2024-09-17T03:37:00.510374+0000","last_became_peered":"2024-09-17T03:37:00.510374+0000","last_unstale":"2024-09-17T03:37:01.533162+0000","last_undegraded":"2024-09-17T03:37:01.533162+0000","last_fullsized":"2024-09-17T03:37:01.533162+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:40.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.510426+0000","last_change":"2024-09-17T03:37:01.508528+0000","last_active":"2024-09-17T03:37:01.510426+0000","last_peered":"2024-09-17T03:37:01.510426+0000","last_clean":"2024-09-17T03:37:01.510426+0000","last_became_active":"2024-09-17T03:37:01.508173+0000","last_became_peered":"2024-09-17T03:37:01.508173+0000","last_unstale":"2024-09-17T03:37:01.510426+0000","last_undegraded":"2024-09-17T03:37:01.510426+0000","last_fullsized":"2024-09-17T03:37:01.510426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:47.227568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494837+0000","last_change":"2024-09-17T03:37:00.517789+0000","last_active":"2024-09-17T03:37:01.494837+0000","last_peered":"2024-09-17T03:37:01.494837+0000","last_clean":"2024-09-17T03:37:01.494837+0000","last_became_active":"2024-09-17T03:37:00.517101+0000","last_became_peered":"2024-09-17T03:37:00.517101+0000","last_unstale":"2024-09-17T03:37:01.494837+0000","last_undegraded":"2024-09-17T03:37:01.494837+0000","last_fullsized":"2024-09-17T03:37:01.494837+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:30.163493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524630+0000","last_change":"2024-09-17T03:37:01.518635+0000","last_active":"2024-09-17T03:37:01.524630+0000","last_peered":"2024-09-17T03:37:01.524630+0000","last_clean":"2024-09-17T03:37:01.524630+0000","last_became_active":"2024-09-17T03:37:01.518369+0000","last_became_peered":"2024-09-17T03:37:01.518369+0000","last_unstale":"2024-09-17T03:37:01.524630+0000","last_undegraded":"2024-09-17T03:37:01.524630+0000","last_fullsized":"2024-09-17T03:37:01.524630+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:42.751762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.529734+0000","last_change":"2024-09-17T03:37:00.502280+0000","last_active":"2024-09-17T03:37:01.529734+0000","last_peered":"2024-09-17T03:37:01.529734+0000","last_clean":"2024-09-17T03:37:01.529734+0000","last_became_active":"2024-09-17T03:37:00.501924+0000","last_became_peered":"2024-09-17T03:37:00.501924+0000","last_unstale":"2024-09-17T03:37:01.529734+0000","last_undegraded":"2024-09-17T03:37:01.529734+0000","last_fullsized":"2024-09-17T03:37:01.529734+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:46.918045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524298+0000","last_change":"2024-09-17T03:37:01.523047+0000","last_active":"2024-09-17T03:37:01.524298+0000","last_peered":"2024-09-17T03:37:01.524298+0000","last_clean":"2024-09-17T03:37:01.524298+0000","last_became_active":"2024-09-17T03:37:01.522670+0000","last_became_peered":"2024-09-17T03:37:01.522670+0000","last_unstale":"2024-09-17T03:37:01.524298+0000","last_undegraded":"2024-09-17T03:37:01.524298+0000","last_fullsized":"2024-09-17T03:37:01.524298+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:03.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496644+0000","last_change":"2024-09-17T03:37:00.519370+0000","last_active":"2024-09-17T03:37:01.496644+0000","last_peered":"2024-09-17T03:37:01.496644+0000","last_clean":"2024-09-17T03:37:01.496644+0000","last_became_active":"2024-09-17T03:37:00.519058+0000","last_became_peered":"2024-09-17T03:37:00.519058+0000","last_unstale":"2024-09-17T03:37:01.496644+0000","last_undegraded":"2024-09-17T03:37:01.496644+0000","last_fullsized":"2024-09-17T03:37:01.496644+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:12.207851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.530069+0000","last_change":"2024-09-17T03:37:01.528223+0000","last_active":"2024-09-17T03:37:01.530069+0000","last_peered":"2024-09-17T03:37:01.530069+0000","last_clean":"2024-09-17T03:37:01.530069+0000","last_became_active":"2024-09-17T03:37:01.527875+0000","last_became_peered":"2024-09-17T03:37:01.527875+0000","last_unstale":"2024-09-17T03:37:01.530069+0000","last_undegraded":"2024-09-17T03:37:01.530069+0000","last_fullsized":"2024-09-17T03:37:01.530069+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:42.158840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496550+0000","last_change":"2024-09-17T03:37:00.514756+0000","last_active":"2024-09-17T03:37:01.496550+0000","last_peered":"2024-09-17T03:37:01.496550+0000","last_clean":"2024-09-17T03:37:01.496550+0000","last_became_active":"2024-09-17T03:37:00.514505+0000","last_became_peered":"2024-09-17T03:37:00.514505+0000","last_unstale":"2024-09-17T03:37:01.496550+0000","last_undegraded":"2024-09-17T03:37:01.496550+0000","last_fullsized":"2024-09-17T03:37:01.496550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.524885+0000","last_change":"2024-09-17T03:37:01.519916+0000","last_active":"2024-09-17T03:37:01.524885+0000","last_peered":"2024-09-17T03:37:01.524885+0000","last_clean":"2024-09-17T03:37:01.524885+0000","last_became_active":"2024-09-17T03:37:01.519171+0000","last_became_peered":"2024-09-17T03:37:01.519171+0000","last_unstale":"2024-09-17T03:37:01.524885+0000","last_undegraded":"2024-09-17T03:37:01.524885+0000","last_fullsized":"2024-09-17T03:37:01.524885+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:18.402815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.507817+0000","last_change":"2024-09-17T03:37:00.516033+0000","last_active":"2024-09-17T03:37:01.507817+0000","last_peered":"2024-09-17T03:37:01.507817+0000","last_clean":"2024-09-17T03:37:01.507817+0000","last_became_active":"2024-09-17T03:37:00.515737+0000","last_became_peered":"2024-09-17T03:37:00.515737+0000","last_unstale":"2024-09-17T03:37:01.507817+0000","last_undegraded":"2024-09-17T03:37:01.507817+0000","last_fullsized":"2024-09-17T03:37:01.507817+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:23.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497461+0000","last_change":"2024-09-17T03:37:00.497461+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497461+0000","last_undegraded":"2024-09-17T03:37:00.497461+0000","last_fullsized":"2024-09-17T03:37:00.497461+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"142'2","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.533095+0000","last_change":"2024-09-17T03:37:00.518901+0000","last_active":"2024-09-17T03:37:01.533095+0000","last_peered":"2024-09-17T03:37:01.533095+0000","last_clean":"2024-09-17T03:37:01.533095+0000","last_became_active":"2024-09-17T03:37:00.518609+0000","last_became_peered":"2024-09-17T03:37:00.518609+0000","last_unstale":"2024-09-17T03:37:01.533095+0000","last_undegraded":"2024-09-17T03:37:01.533095+0000","last_fullsized":"2024-09-17T03:37:01.533095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:31.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.519672+0000","last_change":"2024-09-17T03:37:01.515337+0000","last_active":"2024-09-17T03:37:01.519672+0000","last_peered":"2024-09-17T03:37:01.519672+0000","last_clean":"2024-09-17T03:37:01.519672+0000","last_became_active":"2024-09-17T03:37:01.515033+0000","last_became_peered":"2024-09-17T03:37:01.515033+0000","last_unstale":"2024-09-17T03:37:01.519672+0000","last_undegraded":"2024-09-17T03:37:01.519672+0000","last_fullsized":"2024-09-17T03:37:01.519672+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:14.851758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.494565+0000","last_change":"2024-09-17T03:37:00.509124+0000","last_active":"2024-09-17T03:37:01.494565+0000","last_peered":"2024-09-17T03:37:01.494565+0000","last_clean":"2024-09-17T03:37:01.494565+0000","last_became_active":"2024-09-17T03:37:00.508710+0000","last_became_peered":"2024-09-17T03:37:00.508710+0000","last_unstale":"2024-09-17T03:37:01.494565+0000","last_undegraded":"2024-09-17T03:37:01.494565+0000","last_fullsized":"2024-09-17T03:37:01.494565+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:09.199848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.518488+0000","last_change":"2024-09-17T03:37:01.514963+0000","last_active":"2024-09-17T03:37:01.518488+0000","last_peered":"2024-09-17T03:37:01.518488+0000","last_clean":"2024-09-17T03:37:01.518488+0000","last_became_active":"2024-09-17T03:37:01.514337+0000","last_became_peered":"2024-09-17T03:37:01.514337+0000","last_unstale":"2024-09-17T03:37:01.518488+0000","last_undegraded":"2024-09-17T03:37:01.518488+0000","last_fullsized":"2024-09-17T03:37:01.518488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:55:17.288497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.497349+0000","last_change":"2024-09-17T03:37:00.506035+0000","last_active":"2024-09-17T03:37:01.497349+0000","last_peered":"2024-09-17T03:37:01.497349+0000","last_clean":"2024-09-17T03:37:01.497349+0000","last_became_active":"2024-09-17T03:37:00.505658+0000","last_became_peered":"2024-09-17T03:37:00.505658+0000","last_unstale":"2024-09-17T03:37:01.497349+0000","last_undegraded":"2024-09-17T03:37:01.497349+0000","last_fullsized":"2024-09-17T03:37:01.497349+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:52.915238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521428+0000","last_change":"2024-09-17T03:37:01.517782+0000","last_active":"2024-09-17T03:37:01.521428+0000","last_peered":"2024-09-17T03:37:01.521428+0000","last_clean":"2024-09-17T03:37:01.521428+0000","last_became_active":"2024-09-17T03:37:01.517363+0000","last_became_peered":"2024-09-17T03:37:01.517363+0000","last_unstale":"2024-09-17T03:37:01.521428+0000","last_undegraded":"2024-09-17T03:37:01.521428+0000","last_fullsized":"2024-09-17T03:37:01.521428+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:17.301571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496098+0000","last_change":"2024-09-17T03:37:00.514913+0000","last_active":"2024-09-17T03:37:01.496098+0000","last_peered":"2024-09-17T03:37:01.496098+0000","last_clean":"2024-09-17T03:37:01.496098+0000","last_became_active":"2024-09-17T03:37:00.514588+0000","last_became_peered":"2024-09-17T03:37:00.514588+0000","last_unstale":"2024-09-17T03:37:01.496098+0000","last_undegraded":"2024-09-17T03:37:01.496098+0000","last_fullsized":"2024-09-17T03:37:01.496098+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:15.350414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.532735+0000","last_change":"2024-09-17T03:37:01.523154+0000","last_active":"2024-09-17T03:37:01.532735+0000","last_peered":"2024-09-17T03:37:01.532735+0000","last_clean":"2024-09-17T03:37:01.532735+0000","last_became_active":"2024-09-17T03:37:01.522826+0000","last_became_peered":"2024-09-17T03:37:01.522826+0000","last_unstale":"2024-09-17T03:37:01.532735+0000","last_undegraded":"2024-09-17T03:37:01.532735+0000","last_fullsized":"2024-09-17T03:37:01.532735+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:19.317037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.501477+0000","last_change":"2024-09-17T03:37:00.510360+0000","last_active":"2024-09-17T03:37:01.501477+0000","last_peered":"2024-09-17T03:37:01.501477+0000","last_clean":"2024-09-17T03:37:01.501477+0000","last_became_active":"2024-09-17T03:37:00.510038+0000","last_became_peered":"2024-09-17T03:37:00.510038+0000","last_unstale":"2024-09-17T03:37:01.501477+0000","last_undegraded":"2024-09-17T03:37:01.501477+0000","last_fullsized":"2024-09-17T03:37:01.501477+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:31.275321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497390+0000","last_change":"2024-09-17T03:37:00.497390+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497390+0000","last_undegraded":"2024-09-17T03:37:00.497390+0000","last_fullsized":"2024-09-17T03:37:00.497390+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496964+0000","last_change":"2024-09-17T03:37:00.518866+0000","last_active":"2024-09-17T03:37:01.496964+0000","last_peered":"2024-09-17T03:37:01.496964+0000","last_clean":"2024-09-17T03:37:01.496964+0000","last_became_active":"2024-09-17T03:37:00.518531+0000","last_became_peered":"2024-09-17T03:37:00.518531+0000","last_unstale":"2024-09-17T03:37:01.496964+0000","last_undegraded":"2024-09-17T03:37:01.496964+0000","last_fullsized":"2024-09-17T03:37:01.496964+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.823082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.537156+0000","last_change":"2024-09-17T03:37:01.534797+0000","last_active":"2024-09-17T03:37:01.537156+0000","last_peered":"2024-09-17T03:37:01.537156+0000","last_clean":"2024-09-17T03:37:01.537156+0000","last_became_active":"2024-09-17T03:37:01.534442+0000","last_became_peered":"2024-09-17T03:37:01.534442+0000","last_unstale":"2024-09-17T03:37:01.537156+0000","last_undegraded":"2024-09-17T03:37:01.537156+0000","last_fullsized":"2024-09-17T03:37:01.537156+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:42.572373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.506749+0000","last_change":"2024-09-17T03:37:00.512368+0000","last_active":"2024-09-17T03:37:01.506749+0000","last_peered":"2024-09-17T03:37:01.506749+0000","last_clean":"2024-09-17T03:37:01.506749+0000","last_became_active":"2024-09-17T03:37:00.512041+0000","last_became_peered":"2024-09-17T03:37:00.512041+0000","last_unstale":"2024-09-17T03:37:01.506749+0000","last_undegraded":"2024-09-17T03:37:01.506749+0000","last_fullsized":"2024-09-17T03:37:01.506749+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":141,"state":"creating+peering","last_fresh":"2024-09-17T03:37:00.497341+0000","last_change":"2024-09-17T03:37:00.497341+0000","last_active":"2024-09-17T03:37:00.476150+0000","last_peered":"2024-09-17T03:37:00.476150+0000","last_clean":"2024-09-17T03:37:00.476150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:37:00.497341+0000","last_undegraded":"2024-09-17T03:37:00.497341+0000","last_fullsized":"2024-09-17T03:37:00.497341+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.492735+0000","last_change":"2024-09-17T03:37:00.508129+0000","last_active":"2024-09-17T03:37:01.492735+0000","last_peered":"2024-09-17T03:37:01.492735+0000","last_clean":"2024-09-17T03:37:01.492735+0000","last_became_active":"2024-09-17T03:37:00.506285+0000","last_became_peered":"2024-09-17T03:37:00.506285+0000","last_unstale":"2024-09-17T03:37:01.492735+0000","last_undegraded":"2024-09-17T03:37:01.492735+0000","last_fullsized":"2024-09-17T03:37:01.492735+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:59.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.521767+0000","last_change":"2024-09-17T03:37:01.516795+0000","last_active":"2024-09-17T03:37:01.521767+0000","last_peered":"2024-09-17T03:37:01.521767+0000","last_clean":"2024-09-17T03:37:01.521767+0000","last_became_active":"2024-09-17T03:37:01.516423+0000","last_became_peered":"2024-09-17T03:37:01.516423+0000","last_unstale":"2024-09-17T03:37:01.521767+0000","last_undegraded":"2024-09-17T03:37:01.521767+0000","last_fullsized":"2024-09-17T03:37:01.521767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:14.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"142'3","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.538197+0000","last_change":"2024-09-17T03:37:00.515111+0000","last_active":"2024-09-17T03:37:01.538197+0000","last_peered":"2024-09-17T03:37:01.538197+0000","last_clean":"2024-09-17T03:37:01.538197+0000","last_became_active":"2024-09-17T03:37:00.514784+0000","last_became_peered":"2024-09-17T03:37:00.514784+0000","last_unstale":"2024-09-17T03:37:01.538197+0000","last_undegraded":"2024-09-17T03:37:01.538197+0000","last_fullsized":"2024-09-17T03:37:01.538197+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:41.748322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":333,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-17T03:37:01.496801+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:37:01.496801+0000","last_peered":"2024-09-17T03:37:01.496801+0000","last_clean":"2024-09-17T03:37:01.496801+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:37:01.496801+0000","last_undegraded":"2024-09-17T03:37:01.496801+0000","last_fullsized":"2024-09-17T03:37:01.496801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119528,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252312,"statfs":{"total":96636764160,"available":96514367488,"internally_reserved":0,"allocated":2371584,"data_stored":1366963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509636,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149452,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222388,"statfs":{"total":96636764160,"available":96483725312,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509636,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159188,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212652,"statfs":{"total":96636764160,"available":96473755648,"internally_reserved":0,"allocated":1769472,"data_stored":775943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509637,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127244,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244596,"statfs":{"total":96636764160,"available":96506466304,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167504,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204336,"statfs":{"total":96636764160,"available":96465240064,"internally_reserved":0,"allocated":1757184,"data_stored":769140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150032,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221808,"statfs":{"total":96636764160,"available":96483131392,"internally_reserved":0,"allocated":2351104,"data_stored":1359508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127228,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244612,"statfs":{"total":96636764160,"available":96506482688,"internally_reserved":0,"allocated":1744896,"data_stored":762837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542340,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171680,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200160,"statfs":{"total":96636764160,"available":96460963840,"internally_reserved":0,"allocated":1773568,"data_stored":776525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":173996485},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:37:10.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:37:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 -- 172.21.15.46:0/3714148898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75940b5c70 msgr2=0x7f759415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 --2- 172.21.15.46:0/3714148898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75940b5c70 0x7f759415bb60 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7584009840 tx=0x7f758401c760 comp rx=0 tx=0).stop 2024-09-17T03:37:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 -- 172.21.15.46:0/3714148898 shutdown_connections 2024-09-17T03:37:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 --2- 172.21.15.46:0/3714148898 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75941527b0 0x7f7594074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 --2- 172.21.15.46:0/3714148898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75940b5c70 0x7f759415bb60 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 -- 172.21.15.46:0/3714148898 >> 172.21.15.46:0/3714148898 conn(0x7f75940b2040 msgr2=0x7f75940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 -- 172.21.15.46:0/3714148898 shutdown_connections 2024-09-17T03:37:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.576+0000 7f7598e00640 1 -- 172.21.15.46:0/3714148898 wait complete. 2024-09-17T03:37:10.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 Processor -- start 2024-09-17T03:37:10.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 -- start start 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75940b4de0 0x7f7594154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75940b5c70 0x7f7594153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 0x7f759415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7594168bd0 con 0x7f75941527b0 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7594168a50 con 0x7f75940b5c70 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7594168d50 con 0x7f75940b4de0 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 0x7f759415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 0x7f759415a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52372/0 (socket says 172.21.15.46:52372) 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 -- 172.21.15.46:0/3048434147 learned_addr learned my addr 172.21.15.46:0/3048434147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7591e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75940b4de0 0x7f7594154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f758be00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75940b5c70 0x7f7594153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 -- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75940b4de0 msgr2=0x7f7594154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75940b4de0 0x7f7594154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 -- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75940b5c70 msgr2=0x7f7594153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75940b5c70 0x7f7594153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 -- 172.21.15.46:0/3048434147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75941573b0 con 0x7f75941527b0 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7592800640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 0x7f759415a8a0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f758c00c760 tx=0x7f758c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:10.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c003db0 con 0x7f75941527b0 2024-09-17T03:37:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f758c00ce10 con 0x7f75941527b0 2024-09-17T03:37:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c0052d0 con 0x7f75941527b0 2024-09-17T03:37:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7594157690 con 0x7f75941527b0 2024-09-17T03:37:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.580+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7594157c50 con 0x7f75941527b0 2024-09-17T03:37:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.584+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759415bb60 con 0x7f75941527b0 2024-09-17T03:37:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.584+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f758c005470 con 0x7f75941527b0 2024-09-17T03:37:10.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.584+0000 7f758a000640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7570075570 0x7f7570077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:10.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.584+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f758c093000 con 0x7f75941527b0 2024-09-17T03:37:10.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.584+0000 7f7591e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7570075570 0x7f7570077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:10.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.584+0000 7f7591e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7570075570 0x7f7570077a80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f75840af000 tx=0x7f758401afa0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:10.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.588+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f758c05f560 con 0x7f75941527b0 2024-09-17T03:37:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.760+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f759406b540 con 0x7f7570075570 2024-09-17T03:37:10.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.764+0000 7f758a000640 1 -- 172.21.15.46:0/3048434147 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f75840040c0 con 0x7f7570075570 2024-09-17T03:37:10.768 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:10.770 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7570075570 msgr2=0x7f7570077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7570075570 0x7f7570077a80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f75840af000 tx=0x7f758401afa0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 msgr2=0x7f759415a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 0x7f759415a8a0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f758c00c760 tx=0x7f758c00cc20 comp rx=0 tx=0).stop 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 shutdown_connections 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7570075570 0x7f7570077a80 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75941527b0 0x7f759415a8a0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75940b5c70 0x7f7594153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 --2- 172.21.15.46:0/3048434147 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75940b4de0 0x7f7594154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:10.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 >> 172.21.15.46:0/3048434147 conn(0x7f75940b2040 msgr2=0x7f75940b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 shutdown_connections 2024-09-17T03:37:10.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:10.768+0000 7f7598e00640 1 -- 172.21.15.46:0/3048434147 wait complete. 2024-09-17T03:37:10.790 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2024-09-17T03:37:09.580277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1172056,"kb_used_data":15144,"kb_used_omap":12,"kb_used_meta":1156723,"kb_avail":753802664,"statfs":{"total":773094113280,"available":771893927936,"internally_reserved":0,"allocated":15507456,"data_stored":7443724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1184484912},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.091956"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760653+0000","last_change":"2024-09-17T03:37:01.509979+0000","last_active":"2024-09-17T03:37:02.760653+0000","last_peered":"2024-09-17T03:37:02.760653+0000","last_clean":"2024-09-17T03:37:02.760653+0000","last_became_active":"2024-09-17T03:37:01.508226+0000","last_became_peered":"2024-09-17T03:37:01.508226+0000","last_unstale":"2024-09-17T03:37:02.760653+0000","last_undegraded":"2024-09-17T03:37:02.760653+0000","last_fullsized":"2024-09-17T03:37:02.760653+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:33.705150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517446+0000","last_change":"2024-09-17T03:37:01.525396+0000","last_active":"2024-09-17T03:37:02.517446+0000","last_peered":"2024-09-17T03:37:02.517446+0000","last_clean":"2024-09-17T03:37:02.517446+0000","last_became_active":"2024-09-17T03:37:01.525068+0000","last_became_peered":"2024-09-17T03:37:01.525068+0000","last_unstale":"2024-09-17T03:37:02.517446+0000","last_undegraded":"2024-09-17T03:37:02.517446+0000","last_fullsized":"2024-09-17T03:37:02.517446+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.621910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.767260+0000","last_change":"2024-09-17T03:37:01.517425+0000","last_active":"2024-09-17T03:37:02.767260+0000","last_peered":"2024-09-17T03:37:02.767260+0000","last_clean":"2024-09-17T03:37:02.767260+0000","last_became_active":"2024-09-17T03:37:01.517118+0000","last_became_peered":"2024-09-17T03:37:01.517118+0000","last_unstale":"2024-09-17T03:37:02.767260+0000","last_undegraded":"2024-09-17T03:37:02.767260+0000","last_fullsized":"2024-09-17T03:37:02.767260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:49:08.878323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766455+0000","last_change":"2024-09-17T03:37:01.524178+0000","last_active":"2024-09-17T03:37:02.766455+0000","last_peered":"2024-09-17T03:37:02.766455+0000","last_clean":"2024-09-17T03:37:02.766455+0000","last_became_active":"2024-09-17T03:37:01.523558+0000","last_became_peered":"2024-09-17T03:37:01.523558+0000","last_unstale":"2024-09-17T03:37:02.766455+0000","last_undegraded":"2024-09-17T03:37:02.766455+0000","last_fullsized":"2024-09-17T03:37:02.766455+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:23.958037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765111+0000","last_change":"2024-09-17T03:37:01.514529+0000","last_active":"2024-09-17T03:37:02.765111+0000","last_peered":"2024-09-17T03:37:02.765111+0000","last_clean":"2024-09-17T03:37:02.765111+0000","last_became_active":"2024-09-17T03:37:01.511805+0000","last_became_peered":"2024-09-17T03:37:01.511805+0000","last_unstale":"2024-09-17T03:37:02.765111+0000","last_undegraded":"2024-09-17T03:37:02.765111+0000","last_fullsized":"2024-09-17T03:37:02.765111+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:34.741549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759902+0000","last_change":"2024-09-17T03:37:01.507556+0000","last_active":"2024-09-17T03:37:02.759902+0000","last_peered":"2024-09-17T03:37:02.759902+0000","last_clean":"2024-09-17T03:37:02.759902+0000","last_became_active":"2024-09-17T03:37:01.507216+0000","last_became_peered":"2024-09-17T03:37:01.507216+0000","last_unstale":"2024-09-17T03:37:02.759902+0000","last_undegraded":"2024-09-17T03:37:02.759902+0000","last_fullsized":"2024-09-17T03:37:02.759902+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:18.819552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.513991+0000","last_change":"2024-09-17T03:37:01.520291+0000","last_active":"2024-09-17T03:37:02.513991+0000","last_peered":"2024-09-17T03:37:02.513991+0000","last_clean":"2024-09-17T03:37:02.513991+0000","last_became_active":"2024-09-17T03:37:01.519510+0000","last_became_peered":"2024-09-17T03:37:01.519510+0000","last_unstale":"2024-09-17T03:37:02.513991+0000","last_undegraded":"2024-09-17T03:37:02.513991+0000","last_fullsized":"2024-09-17T03:37:02.513991+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:14.991178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759359+0000","last_change":"2024-09-17T03:37:01.511915+0000","last_active":"2024-09-17T03:37:02.759359+0000","last_peered":"2024-09-17T03:37:02.759359+0000","last_clean":"2024-09-17T03:37:02.759359+0000","last_became_active":"2024-09-17T03:37:01.511588+0000","last_became_peered":"2024-09-17T03:37:01.511588+0000","last_unstale":"2024-09-17T03:37:02.759359+0000","last_undegraded":"2024-09-17T03:37:02.759359+0000","last_fullsized":"2024-09-17T03:37:02.759359+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:59.643927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517696+0000","last_change":"2024-09-17T03:37:01.504394+0000","last_active":"2024-09-17T03:37:02.517696+0000","last_peered":"2024-09-17T03:37:02.517696+0000","last_clean":"2024-09-17T03:37:02.517696+0000","last_became_active":"2024-09-17T03:37:01.504016+0000","last_became_peered":"2024-09-17T03:37:01.504016+0000","last_unstale":"2024-09-17T03:37:02.517696+0000","last_undegraded":"2024-09-17T03:37:02.517696+0000","last_fullsized":"2024-09-17T03:37:02.517696+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:16.486830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525452+0000","last_change":"2024-09-17T03:37:01.521538+0000","last_active":"2024-09-17T03:37:02.525452+0000","last_peered":"2024-09-17T03:37:02.525452+0000","last_clean":"2024-09-17T03:37:02.525452+0000","last_became_active":"2024-09-17T03:37:01.521253+0000","last_became_peered":"2024-09-17T03:37:01.521253+0000","last_unstale":"2024-09-17T03:37:02.525452+0000","last_undegraded":"2024-09-17T03:37:02.525452+0000","last_fullsized":"2024-09-17T03:37:02.525452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:02.452357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759707+0000","last_change":"2024-09-17T03:37:01.507075+0000","last_active":"2024-09-17T03:37:02.759707+0000","last_peered":"2024-09-17T03:37:02.759707+0000","last_clean":"2024-09-17T03:37:02.759707+0000","last_became_active":"2024-09-17T03:37:01.506711+0000","last_became_peered":"2024-09-17T03:37:01.506711+0000","last_unstale":"2024-09-17T03:37:02.759707+0000","last_undegraded":"2024-09-17T03:37:02.759707+0000","last_fullsized":"2024-09-17T03:37:02.759707+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:59.707095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525097+0000","last_change":"2024-09-17T03:37:01.515856+0000","last_active":"2024-09-17T03:37:02.525097+0000","last_peered":"2024-09-17T03:37:02.525097+0000","last_clean":"2024-09-17T03:37:02.525097+0000","last_became_active":"2024-09-17T03:37:01.512742+0000","last_became_peered":"2024-09-17T03:37:01.512742+0000","last_unstale":"2024-09-17T03:37:02.525097+0000","last_undegraded":"2024-09-17T03:37:02.525097+0000","last_fullsized":"2024-09-17T03:37:02.525097+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:59.565093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766114+0000","last_change":"2024-09-17T03:37:01.514924+0000","last_active":"2024-09-17T03:37:02.766114+0000","last_peered":"2024-09-17T03:37:02.766114+0000","last_clean":"2024-09-17T03:37:02.766114+0000","last_became_active":"2024-09-17T03:37:01.512655+0000","last_became_peered":"2024-09-17T03:37:01.512655+0000","last_unstale":"2024-09-17T03:37:02.766114+0000","last_undegraded":"2024-09-17T03:37:02.766114+0000","last_fullsized":"2024-09-17T03:37:02.766114+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:26.998995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765343+0000","last_change":"2024-09-17T03:37:01.530828+0000","last_active":"2024-09-17T03:37:02.765343+0000","last_peered":"2024-09-17T03:37:02.765343+0000","last_clean":"2024-09-17T03:37:02.765343+0000","last_became_active":"2024-09-17T03:37:01.529611+0000","last_became_peered":"2024-09-17T03:37:01.529611+0000","last_unstale":"2024-09-17T03:37:02.765343+0000","last_undegraded":"2024-09-17T03:37:02.765343+0000","last_fullsized":"2024-09-17T03:37:02.765343+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:50.342144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524524+0000","last_change":"2024-09-17T03:37:01.524406+0000","last_active":"2024-09-17T03:37:02.524524+0000","last_peered":"2024-09-17T03:37:02.524524+0000","last_clean":"2024-09-17T03:37:02.524524+0000","last_became_active":"2024-09-17T03:37:01.524097+0000","last_became_peered":"2024-09-17T03:37:01.524097+0000","last_unstale":"2024-09-17T03:37:02.524524+0000","last_undegraded":"2024-09-17T03:37:02.524524+0000","last_fullsized":"2024-09-17T03:37:02.524524+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:05.754961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760014+0000","last_change":"2024-09-17T03:37:01.527309+0000","last_active":"2024-09-17T03:37:02.760014+0000","last_peered":"2024-09-17T03:37:02.760014+0000","last_clean":"2024-09-17T03:37:02.760014+0000","last_became_active":"2024-09-17T03:37:01.526391+0000","last_became_peered":"2024-09-17T03:37:01.526391+0000","last_unstale":"2024-09-17T03:37:02.760014+0000","last_undegraded":"2024-09-17T03:37:02.760014+0000","last_fullsized":"2024-09-17T03:37:02.760014+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:01.548147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525756+0000","last_change":"2024-09-17T03:37:01.531294+0000","last_active":"2024-09-17T03:37:02.525756+0000","last_peered":"2024-09-17T03:37:02.525756+0000","last_clean":"2024-09-17T03:37:02.525756+0000","last_became_active":"2024-09-17T03:37:01.529834+0000","last_became_peered":"2024-09-17T03:37:01.529834+0000","last_unstale":"2024-09-17T03:37:02.525756+0000","last_undegraded":"2024-09-17T03:37:02.525756+0000","last_fullsized":"2024-09-17T03:37:02.525756+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:21.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.767352+0000","last_change":"2024-09-17T03:37:01.518791+0000","last_active":"2024-09-17T03:37:02.767352+0000","last_peered":"2024-09-17T03:37:02.767352+0000","last_clean":"2024-09-17T03:37:02.767352+0000","last_became_active":"2024-09-17T03:37:01.516765+0000","last_became_peered":"2024-09-17T03:37:01.516765+0000","last_unstale":"2024-09-17T03:37:02.767352+0000","last_undegraded":"2024-09-17T03:37:02.767352+0000","last_fullsized":"2024-09-17T03:37:02.767352+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:34.776126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760236+0000","last_change":"2024-09-17T03:37:01.527808+0000","last_active":"2024-09-17T03:37:02.760236+0000","last_peered":"2024-09-17T03:37:02.760236+0000","last_clean":"2024-09-17T03:37:02.760236+0000","last_became_active":"2024-09-17T03:37:01.527469+0000","last_became_peered":"2024-09-17T03:37:01.527469+0000","last_unstale":"2024-09-17T03:37:02.760236+0000","last_undegraded":"2024-09-17T03:37:02.760236+0000","last_fullsized":"2024-09-17T03:37:02.760236+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:23.239976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759655+0000","last_change":"2024-09-17T03:37:01.530734+0000","last_active":"2024-09-17T03:37:02.759655+0000","last_peered":"2024-09-17T03:37:02.759655+0000","last_clean":"2024-09-17T03:37:02.759655+0000","last_became_active":"2024-09-17T03:37:01.530184+0000","last_became_peered":"2024-09-17T03:37:01.530184+0000","last_unstale":"2024-09-17T03:37:02.759655+0000","last_undegraded":"2024-09-17T03:37:02.759655+0000","last_fullsized":"2024-09-17T03:37:02.759655+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:54.363430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525096+0000","last_change":"2024-09-17T03:37:01.520019+0000","last_active":"2024-09-17T03:37:02.525096+0000","last_peered":"2024-09-17T03:37:02.525096+0000","last_clean":"2024-09-17T03:37:02.525096+0000","last_became_active":"2024-09-17T03:37:01.519632+0000","last_became_peered":"2024-09-17T03:37:01.519632+0000","last_unstale":"2024-09-17T03:37:02.525096+0000","last_undegraded":"2024-09-17T03:37:02.525096+0000","last_fullsized":"2024-09-17T03:37:02.525096+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:04.878961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765599+0000","last_change":"2024-09-17T03:37:01.532969+0000","last_active":"2024-09-17T03:37:02.765599+0000","last_peered":"2024-09-17T03:37:02.765599+0000","last_clean":"2024-09-17T03:37:02.765599+0000","last_became_active":"2024-09-17T03:37:01.530349+0000","last_became_peered":"2024-09-17T03:37:01.530349+0000","last_unstale":"2024-09-17T03:37:02.765599+0000","last_undegraded":"2024-09-17T03:37:02.765599+0000","last_fullsized":"2024-09-17T03:37:02.765599+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:27.313800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766072+0000","last_change":"2024-09-17T03:37:01.527670+0000","last_active":"2024-09-17T03:37:02.766072+0000","last_peered":"2024-09-17T03:37:02.766072+0000","last_clean":"2024-09-17T03:37:02.766072+0000","last_became_active":"2024-09-17T03:37:01.527335+0000","last_became_peered":"2024-09-17T03:37:01.527335+0000","last_unstale":"2024-09-17T03:37:02.766072+0000","last_undegraded":"2024-09-17T03:37:02.766072+0000","last_fullsized":"2024-09-17T03:37:02.766072+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:35.170952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524551+0000","last_change":"2024-09-17T03:37:01.520830+0000","last_active":"2024-09-17T03:37:02.524551+0000","last_peered":"2024-09-17T03:37:02.524551+0000","last_clean":"2024-09-17T03:37:02.524551+0000","last_became_active":"2024-09-17T03:37:01.520514+0000","last_became_peered":"2024-09-17T03:37:01.520514+0000","last_unstale":"2024-09-17T03:37:02.524551+0000","last_undegraded":"2024-09-17T03:37:02.524551+0000","last_fullsized":"2024-09-17T03:37:02.524551+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:20.922765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760441+0000","last_change":"2024-09-17T03:37:01.526010+0000","last_active":"2024-09-17T03:37:02.760441+0000","last_peered":"2024-09-17T03:37:02.760441+0000","last_clean":"2024-09-17T03:37:02.760441+0000","last_became_active":"2024-09-17T03:37:01.524307+0000","last_became_peered":"2024-09-17T03:37:01.524307+0000","last_unstale":"2024-09-17T03:37:02.760441+0000","last_undegraded":"2024-09-17T03:37:02.760441+0000","last_fullsized":"2024-09-17T03:37:02.760441+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:07.938497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760427+0000","last_change":"2024-09-17T03:37:01.526850+0000","last_active":"2024-09-17T03:37:02.760427+0000","last_peered":"2024-09-17T03:37:02.760427+0000","last_clean":"2024-09-17T03:37:02.760427+0000","last_became_active":"2024-09-17T03:37:01.526500+0000","last_became_peered":"2024-09-17T03:37:01.526500+0000","last_unstale":"2024-09-17T03:37:02.760427+0000","last_undegraded":"2024-09-17T03:37:02.760427+0000","last_fullsized":"2024-09-17T03:37:02.760427+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:52.130908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760311+0000","last_change":"2024-09-17T03:37:01.525369+0000","last_active":"2024-09-17T03:37:02.760311+0000","last_peered":"2024-09-17T03:37:02.760311+0000","last_clean":"2024-09-17T03:37:02.760311+0000","last_became_active":"2024-09-17T03:37:01.524994+0000","last_became_peered":"2024-09-17T03:37:01.524994+0000","last_unstale":"2024-09-17T03:37:02.760311+0000","last_undegraded":"2024-09-17T03:37:02.760311+0000","last_fullsized":"2024-09-17T03:37:02.760311+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:19.012348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514218+0000","last_change":"2024-09-17T03:37:01.514688+0000","last_active":"2024-09-17T03:37:02.514218+0000","last_peered":"2024-09-17T03:37:02.514218+0000","last_clean":"2024-09-17T03:37:02.514218+0000","last_became_active":"2024-09-17T03:37:01.505442+0000","last_became_peered":"2024-09-17T03:37:01.505442+0000","last_unstale":"2024-09-17T03:37:02.514218+0000","last_undegraded":"2024-09-17T03:37:02.514218+0000","last_fullsized":"2024-09-17T03:37:02.514218+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:04.099119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759921+0000","last_change":"2024-09-17T03:37:01.509207+0000","last_active":"2024-09-17T03:37:02.759921+0000","last_peered":"2024-09-17T03:37:02.759921+0000","last_clean":"2024-09-17T03:37:02.759921+0000","last_became_active":"2024-09-17T03:37:01.508207+0000","last_became_peered":"2024-09-17T03:37:01.508207+0000","last_unstale":"2024-09-17T03:37:02.759921+0000","last_undegraded":"2024-09-17T03:37:02.759921+0000","last_fullsized":"2024-09-17T03:37:02.759921+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:26.772137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765761+0000","last_change":"2024-09-17T03:37:01.524856+0000","last_active":"2024-09-17T03:37:02.765761+0000","last_peered":"2024-09-17T03:37:02.765761+0000","last_clean":"2024-09-17T03:37:02.765761+0000","last_became_active":"2024-09-17T03:37:01.522971+0000","last_became_peered":"2024-09-17T03:37:01.522971+0000","last_unstale":"2024-09-17T03:37:02.765761+0000","last_undegraded":"2024-09-17T03:37:02.765761+0000","last_fullsized":"2024-09-17T03:37:02.765761+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:00.644966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517033+0000","last_change":"2024-09-17T03:37:01.504583+0000","last_active":"2024-09-17T03:37:02.517033+0000","last_peered":"2024-09-17T03:37:02.517033+0000","last_clean":"2024-09-17T03:37:02.517033+0000","last_became_active":"2024-09-17T03:37:01.504221+0000","last_became_peered":"2024-09-17T03:37:01.504221+0000","last_unstale":"2024-09-17T03:37:02.517033+0000","last_undegraded":"2024-09-17T03:37:02.517033+0000","last_fullsized":"2024-09-17T03:37:02.517033+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:32.163982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759300+0000","last_change":"2024-09-17T03:37:01.527737+0000","last_active":"2024-09-17T03:37:02.759300+0000","last_peered":"2024-09-17T03:37:02.759300+0000","last_clean":"2024-09-17T03:37:02.759300+0000","last_became_active":"2024-09-17T03:37:01.526783+0000","last_became_peered":"2024-09-17T03:37:01.526783+0000","last_unstale":"2024-09-17T03:37:02.759300+0000","last_undegraded":"2024-09-17T03:37:02.759300+0000","last_fullsized":"2024-09-17T03:37:02.759300+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:45.219664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525503+0000","last_change":"2024-09-17T03:37:01.513699+0000","last_active":"2024-09-17T03:37:02.525503+0000","last_peered":"2024-09-17T03:37:02.525503+0000","last_clean":"2024-09-17T03:37:02.525503+0000","last_became_active":"2024-09-17T03:37:01.513310+0000","last_became_peered":"2024-09-17T03:37:01.513310+0000","last_unstale":"2024-09-17T03:37:02.525503+0000","last_undegraded":"2024-09-17T03:37:02.525503+0000","last_fullsized":"2024-09-17T03:37:02.525503+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:30.615444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"142'4","reported_seq":19,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525529+0000","last_change":"2024-09-17T03:37:00.521721+0000","last_active":"2024-09-17T03:37:02.525529+0000","last_peered":"2024-09-17T03:37:02.525529+0000","last_clean":"2024-09-17T03:37:02.525529+0000","last_became_active":"2024-09-17T03:37:00.521476+0000","last_became_peered":"2024-09-17T03:37:00.521476+0000","last_unstale":"2024-09-17T03:37:02.525529+0000","last_undegraded":"2024-09-17T03:37:02.525529+0000","last_fullsized":"2024-09-17T03:37:02.525529+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:11.151395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517715+0000","last_change":"2024-09-17T03:37:01.524327+0000","last_active":"2024-09-17T03:37:02.517715+0000","last_peered":"2024-09-17T03:37:02.517715+0000","last_clean":"2024-09-17T03:37:02.517715+0000","last_became_active":"2024-09-17T03:37:01.523988+0000","last_became_peered":"2024-09-17T03:37:01.523988+0000","last_unstale":"2024-09-17T03:37:02.517715+0000","last_undegraded":"2024-09-17T03:37:02.517715+0000","last_fullsized":"2024-09-17T03:37:02.517715+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:57.353653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"142'1","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517513+0000","last_change":"2024-09-17T03:37:00.513012+0000","last_active":"2024-09-17T03:37:02.517513+0000","last_peered":"2024-09-17T03:37:02.517513+0000","last_clean":"2024-09-17T03:37:02.517513+0000","last_became_active":"2024-09-17T03:37:00.512669+0000","last_became_peered":"2024-09-17T03:37:00.512669+0000","last_unstale":"2024-09-17T03:37:02.517513+0000","last_undegraded":"2024-09-17T03:37:02.517513+0000","last_fullsized":"2024-09-17T03:37:02.517513+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:04.275671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760153+0000","last_change":"2024-09-17T03:37:01.520484+0000","last_active":"2024-09-17T03:37:02.760153+0000","last_peered":"2024-09-17T03:37:02.760153+0000","last_clean":"2024-09-17T03:37:02.760153+0000","last_became_active":"2024-09-17T03:37:01.518380+0000","last_became_peered":"2024-09-17T03:37:01.518380+0000","last_unstale":"2024-09-17T03:37:02.760153+0000","last_undegraded":"2024-09-17T03:37:02.760153+0000","last_fullsized":"2024-09-17T03:37:02.760153+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:17.039352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"142'2","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765805+0000","last_change":"2024-09-17T03:37:00.518391+0000","last_active":"2024-09-17T03:37:02.765805+0000","last_peered":"2024-09-17T03:37:02.765805+0000","last_clean":"2024-09-17T03:37:02.765805+0000","last_became_active":"2024-09-17T03:37:00.518014+0000","last_became_peered":"2024-09-17T03:37:00.518014+0000","last_unstale":"2024-09-17T03:37:02.765805+0000","last_undegraded":"2024-09-17T03:37:02.765805+0000","last_fullsized":"2024-09-17T03:37:02.765805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525289+0000","last_change":"2024-09-17T03:37:01.513224+0000","last_active":"2024-09-17T03:37:02.525289+0000","last_peered":"2024-09-17T03:37:02.525289+0000","last_clean":"2024-09-17T03:37:02.525289+0000","last_became_active":"2024-09-17T03:37:01.512886+0000","last_became_peered":"2024-09-17T03:37:01.512886+0000","last_unstale":"2024-09-17T03:37:02.525289+0000","last_undegraded":"2024-09-17T03:37:02.525289+0000","last_fullsized":"2024-09-17T03:37:02.525289+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:22.660997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"142'2","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766430+0000","last_change":"2024-09-17T03:37:00.516162+0000","last_active":"2024-09-17T03:37:02.766430+0000","last_peered":"2024-09-17T03:37:02.766430+0000","last_clean":"2024-09-17T03:37:02.766430+0000","last_became_active":"2024-09-17T03:37:00.515367+0000","last_became_peered":"2024-09-17T03:37:00.515367+0000","last_unstale":"2024-09-17T03:37:02.766430+0000","last_undegraded":"2024-09-17T03:37:02.766430+0000","last_fullsized":"2024-09-17T03:37:02.766430+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:08.671535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517188+0000","last_change":"2024-09-17T03:37:01.506152+0000","last_active":"2024-09-17T03:37:02.517188+0000","last_peered":"2024-09-17T03:37:02.517188+0000","last_clean":"2024-09-17T03:37:02.517188+0000","last_became_active":"2024-09-17T03:37:01.503759+0000","last_became_peered":"2024-09-17T03:37:01.503759+0000","last_unstale":"2024-09-17T03:37:02.517188+0000","last_undegraded":"2024-09-17T03:37:02.517188+0000","last_fullsized":"2024-09-17T03:37:02.517188+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:06.555973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"142'1","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759240+0000","last_change":"2024-09-17T03:37:00.510847+0000","last_active":"2024-09-17T03:37:02.759240+0000","last_peered":"2024-09-17T03:37:02.759240+0000","last_clean":"2024-09-17T03:37:02.759240+0000","last_became_active":"2024-09-17T03:37:00.509550+0000","last_became_peered":"2024-09-17T03:37:00.509550+0000","last_unstale":"2024-09-17T03:37:02.759240+0000","last_undegraded":"2024-09-17T03:37:02.759240+0000","last_fullsized":"2024-09-17T03:37:02.759240+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.371788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765606+0000","last_change":"2024-09-17T03:37:01.524253+0000","last_active":"2024-09-17T03:37:02.765606+0000","last_peered":"2024-09-17T03:37:02.765606+0000","last_clean":"2024-09-17T03:37:02.765606+0000","last_became_active":"2024-09-17T03:37:01.523935+0000","last_became_peered":"2024-09-17T03:37:01.523935+0000","last_unstale":"2024-09-17T03:37:02.765606+0000","last_undegraded":"2024-09-17T03:37:02.765606+0000","last_fullsized":"2024-09-17T03:37:02.765606+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:43.974951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"142'1","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765982+0000","last_change":"2024-09-17T03:37:00.513776+0000","last_active":"2024-09-17T03:37:02.765982+0000","last_peered":"2024-09-17T03:37:02.765982+0000","last_clean":"2024-09-17T03:37:02.765982+0000","last_became_active":"2024-09-17T03:37:00.511311+0000","last_became_peered":"2024-09-17T03:37:00.511311+0000","last_unstale":"2024-09-17T03:37:02.765982+0000","last_undegraded":"2024-09-17T03:37:02.765982+0000","last_fullsized":"2024-09-17T03:37:02.765982+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:43.334054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.516898+0000","last_change":"2024-09-17T03:37:01.513641+0000","last_active":"2024-09-17T03:37:02.516898+0000","last_peered":"2024-09-17T03:37:02.516898+0000","last_clean":"2024-09-17T03:37:02.516898+0000","last_became_active":"2024-09-17T03:37:01.513325+0000","last_became_peered":"2024-09-17T03:37:01.513325+0000","last_unstale":"2024-09-17T03:37:02.516898+0000","last_undegraded":"2024-09-17T03:37:02.516898+0000","last_fullsized":"2024-09-17T03:37:02.516898+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:12.154835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759305+0000","last_change":"2024-09-17T03:37:00.513452+0000","last_active":"2024-09-17T03:37:02.759305+0000","last_peered":"2024-09-17T03:37:02.759305+0000","last_clean":"2024-09-17T03:37:02.759305+0000","last_became_active":"2024-09-17T03:37:00.513118+0000","last_became_peered":"2024-09-17T03:37:00.513118+0000","last_unstale":"2024-09-17T03:37:02.759305+0000","last_undegraded":"2024-09-17T03:37:02.759305+0000","last_fullsized":"2024-09-17T03:37:02.759305+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:53.259179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765426+0000","last_change":"2024-09-17T03:37:01.516740+0000","last_active":"2024-09-17T03:37:02.765426+0000","last_peered":"2024-09-17T03:37:02.765426+0000","last_clean":"2024-09-17T03:37:02.765426+0000","last_became_active":"2024-09-17T03:37:01.512184+0000","last_became_peered":"2024-09-17T03:37:01.512184+0000","last_unstale":"2024-09-17T03:37:02.765426+0000","last_undegraded":"2024-09-17T03:37:02.765426+0000","last_fullsized":"2024-09-17T03:37:02.765426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:20.665884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765201+0000","last_change":"2024-09-17T03:37:00.506180+0000","last_active":"2024-09-17T03:37:02.765201+0000","last_peered":"2024-09-17T03:37:02.765201+0000","last_clean":"2024-09-17T03:37:02.765201+0000","last_became_active":"2024-09-17T03:37:00.505868+0000","last_became_peered":"2024-09-17T03:37:00.505868+0000","last_unstale":"2024-09-17T03:37:02.765201+0000","last_undegraded":"2024-09-17T03:37:02.765201+0000","last_fullsized":"2024-09-17T03:37:02.765201+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766251+0000","last_change":"2024-09-17T03:37:01.532772+0000","last_active":"2024-09-17T03:37:02.766251+0000","last_peered":"2024-09-17T03:37:02.766251+0000","last_clean":"2024-09-17T03:37:02.766251+0000","last_became_active":"2024-09-17T03:37:01.532431+0000","last_became_peered":"2024-09-17T03:37:01.532431+0000","last_unstale":"2024-09-17T03:37:02.766251+0000","last_undegraded":"2024-09-17T03:37:02.766251+0000","last_fullsized":"2024-09-17T03:37:02.766251+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:58.151196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517813+0000","last_change":"2024-09-17T03:37:00.513939+0000","last_active":"2024-09-17T03:37:02.517813+0000","last_peered":"2024-09-17T03:37:02.517813+0000","last_clean":"2024-09-17T03:37:02.517813+0000","last_became_active":"2024-09-17T03:37:00.513531+0000","last_became_peered":"2024-09-17T03:37:00.513531+0000","last_unstale":"2024-09-17T03:37:02.517813+0000","last_undegraded":"2024-09-17T03:37:02.517813+0000","last_fullsized":"2024-09-17T03:37:02.517813+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:43.986216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525539+0000","last_change":"2024-09-17T03:37:01.530837+0000","last_active":"2024-09-17T03:37:02.525539+0000","last_peered":"2024-09-17T03:37:02.525539+0000","last_clean":"2024-09-17T03:37:02.525539+0000","last_became_active":"2024-09-17T03:37:01.530166+0000","last_became_peered":"2024-09-17T03:37:01.530166+0000","last_unstale":"2024-09-17T03:37:02.525539+0000","last_undegraded":"2024-09-17T03:37:02.525539+0000","last_fullsized":"2024-09-17T03:37:02.525539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:21.459330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525817+0000","last_change":"2024-09-17T03:37:00.515644+0000","last_active":"2024-09-17T03:37:02.525817+0000","last_peered":"2024-09-17T03:37:02.525817+0000","last_clean":"2024-09-17T03:37:02.525817+0000","last_became_active":"2024-09-17T03:37:00.515326+0000","last_became_peered":"2024-09-17T03:37:00.515326+0000","last_unstale":"2024-09-17T03:37:02.525817+0000","last_undegraded":"2024-09-17T03:37:02.525817+0000","last_fullsized":"2024-09-17T03:37:02.525817+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:05.552534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759955+0000","last_change":"2024-09-17T03:37:01.530851+0000","last_active":"2024-09-17T03:37:02.759955+0000","last_peered":"2024-09-17T03:37:02.759955+0000","last_clean":"2024-09-17T03:37:02.759955+0000","last_became_active":"2024-09-17T03:37:01.530410+0000","last_became_peered":"2024-09-17T03:37:01.530410+0000","last_unstale":"2024-09-17T03:37:02.759955+0000","last_undegraded":"2024-09-17T03:37:02.759955+0000","last_fullsized":"2024-09-17T03:37:02.759955+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:03.678740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"142'6","reported_seq":21,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514395+0000","last_change":"2024-09-17T03:37:00.511424+0000","last_active":"2024-09-17T03:37:02.514395+0000","last_peered":"2024-09-17T03:37:02.514395+0000","last_clean":"2024-09-17T03:37:02.514395+0000","last_became_active":"2024-09-17T03:37:00.511055+0000","last_became_peered":"2024-09-17T03:37:00.511055+0000","last_unstale":"2024-09-17T03:37:02.514395+0000","last_undegraded":"2024-09-17T03:37:02.514395+0000","last_fullsized":"2024-09-17T03:37:02.514395+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:04.537830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766802+0000","last_change":"2024-09-17T03:37:01.520532+0000","last_active":"2024-09-17T03:37:02.766802+0000","last_peered":"2024-09-17T03:37:02.766802+0000","last_clean":"2024-09-17T03:37:02.766802+0000","last_became_active":"2024-09-17T03:37:01.520261+0000","last_became_peered":"2024-09-17T03:37:01.520261+0000","last_unstale":"2024-09-17T03:37:02.766802+0000","last_undegraded":"2024-09-17T03:37:02.766802+0000","last_fullsized":"2024-09-17T03:37:02.766802+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:38.692340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"142'2","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766926+0000","last_change":"2024-09-17T03:37:00.509307+0000","last_active":"2024-09-17T03:37:02.766926+0000","last_peered":"2024-09-17T03:37:02.766926+0000","last_clean":"2024-09-17T03:37:02.766926+0000","last_became_active":"2024-09-17T03:37:00.508858+0000","last_became_peered":"2024-09-17T03:37:00.508858+0000","last_unstale":"2024-09-17T03:37:02.766926+0000","last_undegraded":"2024-09-17T03:37:02.766926+0000","last_fullsized":"2024-09-17T03:37:02.766926+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:19.533265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517218+0000","last_change":"2024-09-17T03:37:01.526495+0000","last_active":"2024-09-17T03:37:02.517218+0000","last_peered":"2024-09-17T03:37:02.517218+0000","last_clean":"2024-09-17T03:37:02.517218+0000","last_became_active":"2024-09-17T03:37:01.526193+0000","last_became_peered":"2024-09-17T03:37:01.526193+0000","last_unstale":"2024-09-17T03:37:02.517218+0000","last_undegraded":"2024-09-17T03:37:02.517218+0000","last_fullsized":"2024-09-17T03:37:02.517218+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:43.917446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766768+0000","last_change":"2024-09-17T03:37:00.506435+0000","last_active":"2024-09-17T03:37:02.766768+0000","last_peered":"2024-09-17T03:37:02.766768+0000","last_clean":"2024-09-17T03:37:02.766768+0000","last_became_active":"2024-09-17T03:37:00.505499+0000","last_became_peered":"2024-09-17T03:37:00.505499+0000","last_unstale":"2024-09-17T03:37:02.766768+0000","last_undegraded":"2024-09-17T03:37:02.766768+0000","last_fullsized":"2024-09-17T03:37:02.766768+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:13.938141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765485+0000","last_change":"2024-09-17T03:37:01.527237+0000","last_active":"2024-09-17T03:37:02.765485+0000","last_peered":"2024-09-17T03:37:02.765485+0000","last_clean":"2024-09-17T03:37:02.765485+0000","last_became_active":"2024-09-17T03:37:01.523682+0000","last_became_peered":"2024-09-17T03:37:01.523682+0000","last_unstale":"2024-09-17T03:37:02.765485+0000","last_undegraded":"2024-09-17T03:37:02.765485+0000","last_fullsized":"2024-09-17T03:37:02.765485+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:28.117302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759368+0000","last_change":"2024-09-17T03:37:00.497616+0000","last_active":"2024-09-17T03:37:02.759368+0000","last_peered":"2024-09-17T03:37:02.759368+0000","last_clean":"2024-09-17T03:37:02.759368+0000","last_became_active":"2024-09-17T03:37:00.497229+0000","last_became_peered":"2024-09-17T03:37:00.497229+0000","last_unstale":"2024-09-17T03:37:02.759368+0000","last_undegraded":"2024-09-17T03:37:02.759368+0000","last_fullsized":"2024-09-17T03:37:02.759368+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:08.103469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765906+0000","last_change":"2024-09-17T03:37:01.525031+0000","last_active":"2024-09-17T03:37:02.765906+0000","last_peered":"2024-09-17T03:37:02.765906+0000","last_clean":"2024-09-17T03:37:02.765906+0000","last_became_active":"2024-09-17T03:37:01.524710+0000","last_became_peered":"2024-09-17T03:37:01.524710+0000","last_unstale":"2024-09-17T03:37:02.765906+0000","last_undegraded":"2024-09-17T03:37:02.765906+0000","last_fullsized":"2024-09-17T03:37:02.765906+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:12.689461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"142'8","reported_seq":23,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759503+0000","last_change":"2024-09-17T03:37:00.506497+0000","last_active":"2024-09-17T03:37:02.759503+0000","last_peered":"2024-09-17T03:37:02.759503+0000","last_clean":"2024-09-17T03:37:02.759503+0000","last_became_active":"2024-09-17T03:37:00.506195+0000","last_became_peered":"2024-09-17T03:37:00.506195+0000","last_unstale":"2024-09-17T03:37:02.759503+0000","last_undegraded":"2024-09-17T03:37:02.759503+0000","last_fullsized":"2024-09-17T03:37:02.759503+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:35.347269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525768+0000","last_change":"2024-09-17T03:37:01.510170+0000","last_active":"2024-09-17T03:37:02.525768+0000","last_peered":"2024-09-17T03:37:02.525768+0000","last_clean":"2024-09-17T03:37:02.525768+0000","last_became_active":"2024-09-17T03:37:01.509806+0000","last_became_peered":"2024-09-17T03:37:01.509806+0000","last_unstale":"2024-09-17T03:37:02.525768+0000","last_undegraded":"2024-09-17T03:37:02.525768+0000","last_fullsized":"2024-09-17T03:37:02.525768+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:49.896996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"142'1","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766126+0000","last_change":"2024-09-17T03:37:00.507289+0000","last_active":"2024-09-17T03:37:02.766126+0000","last_peered":"2024-09-17T03:37:02.766126+0000","last_clean":"2024-09-17T03:37:02.766126+0000","last_became_active":"2024-09-17T03:37:00.506994+0000","last_became_peered":"2024-09-17T03:37:00.506994+0000","last_unstale":"2024-09-17T03:37:02.766126+0000","last_undegraded":"2024-09-17T03:37:02.766126+0000","last_fullsized":"2024-09-17T03:37:02.766126+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:43.484274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759821+0000","last_change":"2024-09-17T03:37:01.525640+0000","last_active":"2024-09-17T03:37:02.759821+0000","last_peered":"2024-09-17T03:37:02.759821+0000","last_clean":"2024-09-17T03:37:02.759821+0000","last_became_active":"2024-09-17T03:37:01.523930+0000","last_became_peered":"2024-09-17T03:37:01.523930+0000","last_unstale":"2024-09-17T03:37:02.759821+0000","last_undegraded":"2024-09-17T03:37:02.759821+0000","last_fullsized":"2024-09-17T03:37:02.759821+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:49.522956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"142'2","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517543+0000","last_change":"2024-09-17T03:37:00.514697+0000","last_active":"2024-09-17T03:37:02.517543+0000","last_peered":"2024-09-17T03:37:02.517543+0000","last_clean":"2024-09-17T03:37:02.517543+0000","last_became_active":"2024-09-17T03:37:00.513958+0000","last_became_peered":"2024-09-17T03:37:00.513958+0000","last_unstale":"2024-09-17T03:37:02.517543+0000","last_undegraded":"2024-09-17T03:37:02.517543+0000","last_fullsized":"2024-09-17T03:37:02.517543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:27.979522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.767120+0000","last_change":"2024-09-17T03:37:01.521591+0000","last_active":"2024-09-17T03:37:02.767120+0000","last_peered":"2024-09-17T03:37:02.767120+0000","last_clean":"2024-09-17T03:37:02.767120+0000","last_became_active":"2024-09-17T03:37:01.521066+0000","last_became_peered":"2024-09-17T03:37:01.521066+0000","last_unstale":"2024-09-17T03:37:02.767120+0000","last_undegraded":"2024-09-17T03:37:02.767120+0000","last_fullsized":"2024-09-17T03:37:02.767120+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:57.066769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"142'1","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760156+0000","last_change":"2024-09-17T03:37:00.510701+0000","last_active":"2024-09-17T03:37:02.760156+0000","last_peered":"2024-09-17T03:37:02.760156+0000","last_clean":"2024-09-17T03:37:02.760156+0000","last_became_active":"2024-09-17T03:37:00.510374+0000","last_became_peered":"2024-09-17T03:37:00.510374+0000","last_unstale":"2024-09-17T03:37:02.760156+0000","last_undegraded":"2024-09-17T03:37:02.760156+0000","last_fullsized":"2024-09-17T03:37:02.760156+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:40.356225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759958+0000","last_change":"2024-09-17T03:37:01.508528+0000","last_active":"2024-09-17T03:37:02.759958+0000","last_peered":"2024-09-17T03:37:02.759958+0000","last_clean":"2024-09-17T03:37:02.759958+0000","last_became_active":"2024-09-17T03:37:01.508173+0000","last_became_peered":"2024-09-17T03:37:01.508173+0000","last_unstale":"2024-09-17T03:37:02.759958+0000","last_undegraded":"2024-09-17T03:37:02.759958+0000","last_fullsized":"2024-09-17T03:37:02.759958+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:20.214645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517344+0000","last_change":"2024-09-17T03:37:00.517789+0000","last_active":"2024-09-17T03:37:02.517344+0000","last_peered":"2024-09-17T03:37:02.517344+0000","last_clean":"2024-09-17T03:37:02.517344+0000","last_became_active":"2024-09-17T03:37:00.517101+0000","last_became_peered":"2024-09-17T03:37:00.517101+0000","last_unstale":"2024-09-17T03:37:02.517344+0000","last_undegraded":"2024-09-17T03:37:02.517344+0000","last_fullsized":"2024-09-17T03:37:02.517344+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:30.163493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765913+0000","last_change":"2024-09-17T03:37:01.518635+0000","last_active":"2024-09-17T03:37:02.765913+0000","last_peered":"2024-09-17T03:37:02.765913+0000","last_clean":"2024-09-17T03:37:02.765913+0000","last_became_active":"2024-09-17T03:37:01.518369+0000","last_became_peered":"2024-09-17T03:37:01.518369+0000","last_unstale":"2024-09-17T03:37:02.765913+0000","last_undegraded":"2024-09-17T03:37:02.765913+0000","last_fullsized":"2024-09-17T03:37:02.765913+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:16.913516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"142'2","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514119+0000","last_change":"2024-09-17T03:37:00.502280+0000","last_active":"2024-09-17T03:37:02.514119+0000","last_peered":"2024-09-17T03:37:02.514119+0000","last_clean":"2024-09-17T03:37:02.514119+0000","last_became_active":"2024-09-17T03:37:00.501924+0000","last_became_peered":"2024-09-17T03:37:00.501924+0000","last_unstale":"2024-09-17T03:37:02.514119+0000","last_undegraded":"2024-09-17T03:37:02.514119+0000","last_fullsized":"2024-09-17T03:37:02.514119+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:46.918045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514102+0000","last_change":"2024-09-17T03:37:01.523047+0000","last_active":"2024-09-17T03:37:02.514102+0000","last_peered":"2024-09-17T03:37:02.514102+0000","last_clean":"2024-09-17T03:37:02.514102+0000","last_became_active":"2024-09-17T03:37:01.522670+0000","last_became_peered":"2024-09-17T03:37:01.522670+0000","last_unstale":"2024-09-17T03:37:02.514102+0000","last_undegraded":"2024-09-17T03:37:02.514102+0000","last_fullsized":"2024-09-17T03:37:02.514102+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:28.003293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524631+0000","last_change":"2024-09-17T03:37:00.519370+0000","last_active":"2024-09-17T03:37:02.524631+0000","last_peered":"2024-09-17T03:37:02.524631+0000","last_clean":"2024-09-17T03:37:02.524631+0000","last_became_active":"2024-09-17T03:37:00.519058+0000","last_became_peered":"2024-09-17T03:37:00.519058+0000","last_unstale":"2024-09-17T03:37:02.524631+0000","last_undegraded":"2024-09-17T03:37:02.524631+0000","last_fullsized":"2024-09-17T03:37:02.524631+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:12.207851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.764963+0000","last_change":"2024-09-17T03:37:01.528223+0000","last_active":"2024-09-17T03:37:02.764963+0000","last_peered":"2024-09-17T03:37:02.764963+0000","last_clean":"2024-09-17T03:37:02.764963+0000","last_became_active":"2024-09-17T03:37:01.527875+0000","last_became_peered":"2024-09-17T03:37:01.527875+0000","last_unstale":"2024-09-17T03:37:02.764963+0000","last_undegraded":"2024-09-17T03:37:02.764963+0000","last_fullsized":"2024-09-17T03:37:02.764963+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.126646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524485+0000","last_change":"2024-09-17T03:37:00.514756+0000","last_active":"2024-09-17T03:37:02.524485+0000","last_peered":"2024-09-17T03:37:02.524485+0000","last_clean":"2024-09-17T03:37:02.524485+0000","last_became_active":"2024-09-17T03:37:00.514505+0000","last_became_peered":"2024-09-17T03:37:00.514505+0000","last_unstale":"2024-09-17T03:37:02.524485+0000","last_undegraded":"2024-09-17T03:37:02.524485+0000","last_fullsized":"2024-09-17T03:37:02.524485+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:56.546368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514019+0000","last_change":"2024-09-17T03:37:01.519916+0000","last_active":"2024-09-17T03:37:02.514019+0000","last_peered":"2024-09-17T03:37:02.514019+0000","last_clean":"2024-09-17T03:37:02.514019+0000","last_became_active":"2024-09-17T03:37:01.519171+0000","last_became_peered":"2024-09-17T03:37:01.519171+0000","last_unstale":"2024-09-17T03:37:02.514019+0000","last_undegraded":"2024-09-17T03:37:02.514019+0000","last_fullsized":"2024-09-17T03:37:02.514019+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:11.883491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765710+0000","last_change":"2024-09-17T03:37:00.516033+0000","last_active":"2024-09-17T03:37:02.765710+0000","last_peered":"2024-09-17T03:37:02.765710+0000","last_clean":"2024-09-17T03:37:02.765710+0000","last_became_active":"2024-09-17T03:37:00.515737+0000","last_became_peered":"2024-09-17T03:37:00.515737+0000","last_unstale":"2024-09-17T03:37:02.765710+0000","last_undegraded":"2024-09-17T03:37:02.765710+0000","last_fullsized":"2024-09-17T03:37:02.765710+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:23.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766296+0000","last_change":"2024-09-17T03:37:01.511753+0000","last_active":"2024-09-17T03:37:02.766296+0000","last_peered":"2024-09-17T03:37:02.766296+0000","last_clean":"2024-09-17T03:37:02.766296+0000","last_became_active":"2024-09-17T03:37:01.511356+0000","last_became_peered":"2024-09-17T03:37:01.511356+0000","last_unstale":"2024-09-17T03:37:02.766296+0000","last_undegraded":"2024-09-17T03:37:02.766296+0000","last_fullsized":"2024-09-17T03:37:02.766296+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:12.259479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"142'2","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.525703+0000","last_change":"2024-09-17T03:37:00.518901+0000","last_active":"2024-09-17T03:37:02.525703+0000","last_peered":"2024-09-17T03:37:02.525703+0000","last_clean":"2024-09-17T03:37:02.525703+0000","last_became_active":"2024-09-17T03:37:00.518609+0000","last_became_peered":"2024-09-17T03:37:00.518609+0000","last_unstale":"2024-09-17T03:37:02.525703+0000","last_undegraded":"2024-09-17T03:37:02.525703+0000","last_fullsized":"2024-09-17T03:37:02.525703+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:31.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.767381+0000","last_change":"2024-09-17T03:37:01.515337+0000","last_active":"2024-09-17T03:37:02.767381+0000","last_peered":"2024-09-17T03:37:02.767381+0000","last_clean":"2024-09-17T03:37:02.767381+0000","last_became_active":"2024-09-17T03:37:01.515033+0000","last_became_peered":"2024-09-17T03:37:01.515033+0000","last_unstale":"2024-09-17T03:37:02.767381+0000","last_undegraded":"2024-09-17T03:37:02.767381+0000","last_fullsized":"2024-09-17T03:37:02.767381+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:20.705501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"143'1","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760675+0000","last_change":"2024-09-17T03:37:00.509124+0000","last_active":"2024-09-17T03:37:02.760675+0000","last_peered":"2024-09-17T03:37:02.760675+0000","last_clean":"2024-09-17T03:37:02.760675+0000","last_became_active":"2024-09-17T03:37:00.508710+0000","last_became_peered":"2024-09-17T03:37:00.508710+0000","last_unstale":"2024-09-17T03:37:02.760675+0000","last_undegraded":"2024-09-17T03:37:02.760675+0000","last_fullsized":"2024-09-17T03:37:02.760675+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:09.199848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.517337+0000","last_change":"2024-09-17T03:37:01.514963+0000","last_active":"2024-09-17T03:37:02.517337+0000","last_peered":"2024-09-17T03:37:02.517337+0000","last_clean":"2024-09-17T03:37:02.517337+0000","last_became_active":"2024-09-17T03:37:01.514337+0000","last_became_peered":"2024-09-17T03:37:01.514337+0000","last_unstale":"2024-09-17T03:37:02.517337+0000","last_undegraded":"2024-09-17T03:37:02.517337+0000","last_fullsized":"2024-09-17T03:37:02.517337+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:13.599854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760168+0000","last_change":"2024-09-17T03:37:00.506035+0000","last_active":"2024-09-17T03:37:02.760168+0000","last_peered":"2024-09-17T03:37:02.760168+0000","last_clean":"2024-09-17T03:37:02.760168+0000","last_became_active":"2024-09-17T03:37:00.505658+0000","last_became_peered":"2024-09-17T03:37:00.505658+0000","last_unstale":"2024-09-17T03:37:02.760168+0000","last_undegraded":"2024-09-17T03:37:02.760168+0000","last_fullsized":"2024-09-17T03:37:02.760168+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:52.915238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759996+0000","last_change":"2024-09-17T03:37:01.517782+0000","last_active":"2024-09-17T03:37:02.759996+0000","last_peered":"2024-09-17T03:37:02.759996+0000","last_clean":"2024-09-17T03:37:02.759996+0000","last_became_active":"2024-09-17T03:37:01.517363+0000","last_became_peered":"2024-09-17T03:37:01.517363+0000","last_unstale":"2024-09-17T03:37:02.759996+0000","last_undegraded":"2024-09-17T03:37:02.759996+0000","last_fullsized":"2024-09-17T03:37:02.759996+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:08.783680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.759759+0000","last_change":"2024-09-17T03:37:00.514913+0000","last_active":"2024-09-17T03:37:02.759759+0000","last_peered":"2024-09-17T03:37:02.759759+0000","last_clean":"2024-09-17T03:37:02.759759+0000","last_became_active":"2024-09-17T03:37:00.514588+0000","last_became_peered":"2024-09-17T03:37:00.514588+0000","last_unstale":"2024-09-17T03:37:02.759759+0000","last_undegraded":"2024-09-17T03:37:02.759759+0000","last_fullsized":"2024-09-17T03:37:02.759759+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:15.350414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.760224+0000","last_change":"2024-09-17T03:37:01.523154+0000","last_active":"2024-09-17T03:37:02.760224+0000","last_peered":"2024-09-17T03:37:02.760224+0000","last_clean":"2024-09-17T03:37:02.760224+0000","last_became_active":"2024-09-17T03:37:01.522826+0000","last_became_peered":"2024-09-17T03:37:01.522826+0000","last_unstale":"2024-09-17T03:37:02.760224+0000","last_undegraded":"2024-09-17T03:37:02.760224+0000","last_fullsized":"2024-09-17T03:37:02.760224+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:54.416293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514273+0000","last_change":"2024-09-17T03:37:00.510360+0000","last_active":"2024-09-17T03:37:02.514273+0000","last_peered":"2024-09-17T03:37:02.514273+0000","last_clean":"2024-09-17T03:37:02.514273+0000","last_became_active":"2024-09-17T03:37:00.510038+0000","last_became_peered":"2024-09-17T03:37:00.510038+0000","last_unstale":"2024-09-17T03:37:02.514273+0000","last_undegraded":"2024-09-17T03:37:02.514273+0000","last_fullsized":"2024-09-17T03:37:02.514273+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:31.275321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.766379+0000","last_change":"2024-09-17T03:37:01.516358+0000","last_active":"2024-09-17T03:37:02.766379+0000","last_peered":"2024-09-17T03:37:02.766379+0000","last_clean":"2024-09-17T03:37:02.766379+0000","last_became_active":"2024-09-17T03:37:01.516041+0000","last_became_peered":"2024-09-17T03:37:01.516041+0000","last_unstale":"2024-09-17T03:37:02.766379+0000","last_undegraded":"2024-09-17T03:37:02.766379+0000","last_fullsized":"2024-09-17T03:37:02.766379+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:05.911471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524693+0000","last_change":"2024-09-17T03:37:00.518866+0000","last_active":"2024-09-17T03:37:02.524693+0000","last_peered":"2024-09-17T03:37:02.524693+0000","last_clean":"2024-09-17T03:37:02.524693+0000","last_became_active":"2024-09-17T03:37:00.518531+0000","last_became_peered":"2024-09-17T03:37:00.518531+0000","last_unstale":"2024-09-17T03:37:02.524693+0000","last_undegraded":"2024-09-17T03:37:02.524693+0000","last_fullsized":"2024-09-17T03:37:02.524693+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:37.823082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524684+0000","last_change":"2024-09-17T03:37:01.534797+0000","last_active":"2024-09-17T03:37:02.524684+0000","last_peered":"2024-09-17T03:37:02.524684+0000","last_clean":"2024-09-17T03:37:02.524684+0000","last_became_active":"2024-09-17T03:37:01.534442+0000","last_became_peered":"2024-09-17T03:37:01.534442+0000","last_unstale":"2024-09-17T03:37:02.524684+0000","last_undegraded":"2024-09-17T03:37:02.524684+0000","last_fullsized":"2024-09-17T03:37:02.524684+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:25.987864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765884+0000","last_change":"2024-09-17T03:37:00.512368+0000","last_active":"2024-09-17T03:37:02.765884+0000","last_peered":"2024-09-17T03:37:02.765884+0000","last_clean":"2024-09-17T03:37:02.765884+0000","last_became_active":"2024-09-17T03:37:00.512041+0000","last_became_peered":"2024-09-17T03:37:00.512041+0000","last_unstale":"2024-09-17T03:37:02.765884+0000","last_undegraded":"2024-09-17T03:37:02.765884+0000","last_fullsized":"2024-09-17T03:37:02.765884+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:07.507903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.765453+0000","last_change":"2024-09-17T03:37:01.527568+0000","last_active":"2024-09-17T03:37:02.765453+0000","last_peered":"2024-09-17T03:37:02.765453+0000","last_clean":"2024-09-17T03:37:02.765453+0000","last_became_active":"2024-09-17T03:37:01.527256+0000","last_became_peered":"2024-09-17T03:37:01.527256+0000","last_unstale":"2024-09-17T03:37:02.765453+0000","last_undegraded":"2024-09-17T03:37:02.765453+0000","last_fullsized":"2024-09-17T03:37:02.765453+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:31.322497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.516891+0000","last_change":"2024-09-17T03:37:00.508129+0000","last_active":"2024-09-17T03:37:02.516891+0000","last_peered":"2024-09-17T03:37:02.516891+0000","last_clean":"2024-09-17T03:37:02.516891+0000","last_became_active":"2024-09-17T03:37:00.506285+0000","last_became_peered":"2024-09-17T03:37:00.506285+0000","last_unstale":"2024-09-17T03:37:02.516891+0000","last_undegraded":"2024-09-17T03:37:02.516891+0000","last_fullsized":"2024-09-17T03:37:02.516891+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:59.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.514363+0000","last_change":"2024-09-17T03:37:01.516795+0000","last_active":"2024-09-17T03:37:02.514363+0000","last_peered":"2024-09-17T03:37:02.514363+0000","last_clean":"2024-09-17T03:37:02.514363+0000","last_became_active":"2024-09-17T03:37:01.516423+0000","last_became_peered":"2024-09-17T03:37:01.516423+0000","last_unstale":"2024-09-17T03:37:02.514363+0000","last_undegraded":"2024-09-17T03:37:02.514363+0000","last_fullsized":"2024-09-17T03:37:02.514363+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:37:00.476150+0000","last_clean_scrub_stamp":"2024-09-17T03:37:00.476150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:29.674109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"142'3","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.767226+0000","last_change":"2024-09-17T03:37:00.515111+0000","last_active":"2024-09-17T03:37:02.767226+0000","last_peered":"2024-09-17T03:37:02.767226+0000","last_clean":"2024-09-17T03:37:02.767226+0000","last_became_active":"2024-09-17T03:37:00.514784+0000","last_became_peered":"2024-09-17T03:37:00.514784+0000","last_unstale":"2024-09-17T03:37:02.767226+0000","last_undegraded":"2024-09-17T03:37:02.767226+0000","last_fullsized":"2024-09-17T03:37:02.767226+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:36:59.472710+0000","last_clean_scrub_stamp":"2024-09-17T03:36:59.472710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:41.748322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":335,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T03:37:02.524853+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:37:02.524853+0000","last_peered":"2024-09-17T03:37:02.524853+0000","last_clean":"2024-09-17T03:37:02.524853+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:37:02.524853+0000","last_undegraded":"2024-09-17T03:37:02.524853+0000","last_fullsized":"2024-09-17T03:37:02.524853+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509637,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119532,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252308,"statfs":{"total":96636764160,"available":96514363392,"internally_reserved":0,"allocated":2383872,"data_stored":1373291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":119994826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149484,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222356,"statfs":{"total":96636764160,"available":96483692544,"internally_reserved":0,"allocated":1794048,"data_stored":782417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159204,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212636,"statfs":{"total":96636764160,"available":96473739264,"internally_reserved":0,"allocated":1781760,"data_stored":782271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509638,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127272,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244568,"statfs":{"total":96636764160,"available":96506437632,"internally_reserved":0,"allocated":1785856,"data_stored":782361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509637,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167540,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204300,"statfs":{"total":96636764160,"available":96465203200,"internally_reserved":0,"allocated":1798144,"data_stored":783953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150072,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221768,"statfs":{"total":96636764160,"available":96483090432,"internally_reserved":0,"allocated":2392064,"data_stored":1374285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127268,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244572,"statfs":{"total":96636764160,"available":96506441728,"internally_reserved":0,"allocated":1785856,"data_stored":782293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542341,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171684,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200156,"statfs":{"total":96636764160,"available":96460959744,"internally_reserved":0,"allocated":1785856,"data_stored":782853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":173996485},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:37:10.797 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:37:10.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 -- 172.21.15.46:0/3544993889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b4de0 msgr2=0x7f2ffc0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 --2- 172.21.15.46:0/3544993889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b4de0 0x7f2ffc0b51f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009bc0 tx=0x7f2ff401cae0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 -- 172.21.15.46:0/3544993889 shutdown_connections 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 --2- 172.21.15.46:0/3544993889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b4de0 0x7f2ffc0b51f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 --2- 172.21.15.46:0/3544993889 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc1527b0 0x7f2ffc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 -- 172.21.15.46:0/3544993889 >> 172.21.15.46:0/3544993889 conn(0x7f2ffc0b2040 msgr2=0x7f2ffc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 -- 172.21.15.46:0/3544993889 shutdown_connections 2024-09-17T03:37:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.000+0000 7f3001000640 1 -- 172.21.15.46:0/3544993889 wait complete. 2024-09-17T03:37:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 Processor -- start 2024-09-17T03:37:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 -- start start 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc0b4de0 0x7f2ffc1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 0x7f2ffc14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ffc1527b0 0x7f2ffc157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc168ab0 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc168930 con 0x7f2ffc1527b0 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc168c30 con 0x7f2ffc0b4de0 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 0x7f2ffc14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 0x7f2ffc14f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52390/0 (socket says 172.21.15.46:52390) 2024-09-17T03:37:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 -- 172.21.15.46:0/4067427458 learned_addr learned my addr 172.21.15.46:0/4067427458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff9e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc0b4de0 0x7f2ffc1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ffa800640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ffc1527b0 0x7f2ffc157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 -- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc0b4de0 msgr2=0x7f2ffc1f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc0b4de0 0x7f2ffc1f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 -- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ffc1527b0 msgr2=0x7f2ffc157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ffc1527b0 0x7f2ffc157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 -- 172.21.15.46:0/4067427458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc156380 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff9e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc0b4de0 0x7f2ffc1f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff3e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 0x7f2ffc14f060 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d6d0 tx=0x7f2fe400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4017070 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe4005310 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4009820 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc156660 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.004+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc156c20 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.008+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc0b51f0 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.008+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f2fe40099c0 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.008+0000 7f2ff2000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2fd8075570 0x7f2fd8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.008+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f2fe40934b0 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.008+0000 7f2ff9e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2fd8075570 0x7f2fd8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.012+0000 7f2ff9e00640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2fd8075570 0x7f2fd8077a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2fec000f70 tx=0x7f2fec009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:11.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.012+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2fe405ffc0 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.248+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2ffc0b2910 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.252+0000 7f2ff2000640 1 -- 172.21.15.46:0/4067427458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v45) ==== 190+0+147 (secure 0 0 0) 0x7f2fe405f8c0 con 0x7f2ffc0b5c70 2024-09-17T03:37:11.255 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-17T03:37:11.255 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBh9ehmIhANIxAAz8O/d34pds9rVZXnbieqZQ== 2024-09-17T03:37:11.255 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-09-17T03:37:11.255 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-09-17T03:37:11.255 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:37:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-09-17T03:37:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2fd8075570 msgr2=0x7f2fd8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2fd8075570 0x7f2fd8077a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2fec000f70 tx=0x7f2fec009040 comp rx=0 tx=0).stop 2024-09-17T03:37:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 msgr2=0x7f2ffc14f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 0x7f2ffc14f060 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d6d0 tx=0x7f2fe400db90 comp rx=0 tx=0).stop 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 shutdown_connections 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2fd8075570 0x7f2fd8077a80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ffc1527b0 0x7f2ffc157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ffc0b5c70 0x7f2ffc14f060 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 --2- 172.21.15.46:0/4067427458 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ffc0b4de0 0x7f2ffc1f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 >> 172.21.15.46:0/4067427458 conn(0x7f2ffc0b2040 msgr2=0x7f2ffc154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 shutdown_connections 2024-09-17T03:37:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.256+0000 7f3001000640 1 -- 172.21.15.46:0/4067427458 wait complete. 2024-09-17T03:37:11.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 -- 172.21.15.46:0/1597327907 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d01527b0 msgr2=0x7f42d015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1597327907 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d01527b0 0x7f42d015bb60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b801c820 comp rx=0 tx=0).stop 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 -- 172.21.15.46:0/1597327907 shutdown_connections 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1597327907 >> v2:172.21.15.99:3300/0 conn(0x7f42d00b4de0 0x7f42d00b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1597327907 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d01527b0 0x7f42d015bb60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 -- 172.21.15.46:0/1597327907 >> 172.21.15.46:0/1597327907 conn(0x7f42d00b2040 msgr2=0x7f42d00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 -- 172.21.15.46:0/1597327907 shutdown_connections 2024-09-17T03:37:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.436+0000 7f42d5e00640 1 -- 172.21.15.46:0/1597327907 wait complete. 2024-09-17T03:37:11.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 Processor -- start 2024-09-17T03:37:11.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 -- start start 2024-09-17T03:37:11.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 0x7f42d0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d00b5c70 0x7f42d00c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d01527b0 0x7f42d01fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0168a60 con 0x7f42d00b4de0 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42d01688e0 con 0x7f42d01527b0 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f42d0168be0 con 0x7f42d00b5c70 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 0x7f42d0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 0x7f42d0157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52396/0 (socket says 172.21.15.46:52396) 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d01527b0 0x7f42d01fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 -- 172.21.15.46:0/1707273205 learned_addr learned my addr 172.21.15.46:0/1707273205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42c7e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d00b5c70 0x7f42d00c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 -- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d00b5c70 msgr2=0x7f42d00c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d00b5c70 0x7f42d00c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 -- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d01527b0 msgr2=0x7f42d01fcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d01527b0 0x7f42d01fcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 -- 172.21.15.46:0/1707273205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d01551e0 con 0x7f42d00b4de0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42cea00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 0x7f42d0157ac0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f42c000c400 tx=0x7f42c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0017070 con 0x7f42d00b4de0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42c0003600 con 0x7f42d00b4de0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d0156d00 con 0x7f42d00b4de0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c0003a50 con 0x7f42d00b4de0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.440+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01572c0 con 0x7f42d00b4de0 2024-09-17T03:37:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.444+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d00b51f0 con 0x7f42d00b4de0 2024-09-17T03:37:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.444+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f42c001a070 con 0x7f42d00b4de0 2024-09-17T03:37:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.444+0000 7f42c7400640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f42ac0755a0 0x7f42ac077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.444+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f42c00929f0 con 0x7f42d00b4de0 2024-09-17T03:37:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.444+0000 7f42c7e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f42ac0755a0 0x7f42ac077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.444+0000 7f42c7e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f42ac0755a0 0x7f42ac077ab0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f42b80af000 tx=0x7f42b801b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.448+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f42c0021020 con 0x7f42d00b4de0 2024-09-17T03:37:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.656+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42d00cfa60 con 0x7f42d00b4de0 2024-09-17T03:37:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.656+0000 7f42c7400640 1 -- 172.21.15.46:0/1707273205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5243 (secure 0 0 0) 0x7f42c0062490 con 0x7f42d00b4de0 2024-09-17T03:37:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"btime":"2024-09-17T03:37:04:527000+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:37:00.912074+0000","modified":"2024-09-17T03:37:03.946655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248758":{"gid":248758,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/2038056563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2038056563},{"type":"v1","addr":"172.21.15.46:6837","nonce":2038056563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248758,"qdb_cluster":[248758]},"id":10}]} 2024-09-17T03:37:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2024-09-17T03:37:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f42ac0755a0 msgr2=0x7f42ac077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f42ac0755a0 0x7f42ac077ab0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f42b80af000 tx=0x7f42b801b060 comp rx=0 tx=0).stop 2024-09-17T03:37:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 msgr2=0x7f42d0157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 0x7f42d0157ac0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f42c000c400 tx=0x7f42c000c8c0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 shutdown_connections 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f42ac0755a0 0x7f42ac077ab0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42d01527b0 0x7f42d01fcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f42d00b5c70 0x7f42d00c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 --2- 172.21.15.46:0/1707273205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d00b4de0 0x7f42d0157ac0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 >> 172.21.15.46:0/1707273205 conn(0x7f42d00b2040 msgr2=0x7f42d015a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 shutdown_connections 2024-09-17T03:37:11.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:11.660+0000 7f42d5e00640 1 -- 172.21.15.46:0/1707273205 wait complete. 2024-09-17T03:37:12.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:37:00.912074+0000', 'modified': '2024-09-17T03:37:03.946655+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 248758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_248758': {'gid': 248758, 'name': 'a', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6837/2038056563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6836', 'nonce': 2038056563}, {'type': 'v1', 'addr': '172.21.15.46:6837', 'nonce': 2038056563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 248758, 'qdb_cluster': [248758]} 2024-09-17T03:37:12.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:37:12.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T03:37:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129f000640 1 -- 172.21.15.46:0/50363582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c004810 msgr2=0x7f127c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129f000640 1 --2- 172.21.15.46:0/50363582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f127c004810 0x7f127c004c20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1298071fd0 tx=0x7f128001c850 comp rx=0 tx=0).stop 2024-09-17T03:37:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129d000640 1 -- 172.21.15.46:0/50363582 reap_dead start 2024-09-17T03:37:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129f000640 1 -- 172.21.15.46:0/50363582 shutdown_connections 2024-09-17T03:37:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129f000640 1 -- 172.21.15.46:0/50363582 >> 172.21.15.46:0/50363582 conn(0x7f12980b2040 msgr2=0x7f12980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129f000640 1 -- 172.21.15.46:0/50363582 shutdown_connections 2024-09-17T03:37:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.252+0000 7f129f000640 1 -- 172.21.15.46:0/50363582 wait complete. 2024-09-17T03:37:12.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 Processor -- start 2024-09-17T03:37:12.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 -- start start 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c004810 0x7f1298158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 0x7f1298154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12980b1330 0x7f12980b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1298167bc0 con 0x7f129814eda0 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1298167a40 con 0x7f12980b1330 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f129f000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1298167d40 con 0x7f127c004810 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f128f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 0x7f1298154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f128f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 0x7f1298154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52402/0 (socket says 172.21.15.46:52402) 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.256+0000 7f128f400640 1 -- 172.21.15.46:0/102977756 learned_addr learned my addr 172.21.15.46:0/102977756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:12.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f1297e00640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c004810 0x7f1298158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f129d000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12980b1330 0x7f12980b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f128f400640 1 -- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c004810 msgr2=0x7f1298158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f128f400640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c004810 0x7f1298158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f128f400640 1 -- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12980b1330 msgr2=0x7f12980b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f128f400640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12980b1330 0x7f12980b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f128f400640 1 -- 172.21.15.46:0/102977756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12980b1c80 con 0x7f129814eda0 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f129d000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12980b1330 0x7f12980b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:12.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f128f400640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 0x7f1298154700 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1280005040 tx=0x7f1280003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f1294c00640 1 -- 172.21.15.46:0/102977756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280001480 con 0x7f129814eda0 2024-09-17T03:37:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f1294c00640 1 -- 172.21.15.46:0/102977756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12800b4030 con 0x7f129814eda0 2024-09-17T03:37:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f1294c00640 1 -- 172.21.15.46:0/102977756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12800068d0 con 0x7f129814eda0 2024-09-17T03:37:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12980ce6e0 con 0x7f129814eda0 2024-09-17T03:37:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12980ce980 con 0x7f129814eda0 2024-09-17T03:37:12.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.260+0000 7f1294c00640 1 -- 172.21.15.46:0/102977756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f12800a5b30 con 0x7f129814eda0 2024-09-17T03:37:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f1294c00640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1274075570 0x7f1274077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f1294c00640 1 -- 172.21.15.46:0/102977756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f128012d580 con 0x7f129814eda0 2024-09-17T03:37:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f1297e00640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1274075570 0x7f1274077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 Processor -- start 2024-09-17T03:37:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 -- start start 2024-09-17T03:37:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f1297e00640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1274075570 0x7f1274077a80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1288009b40 tx=0x7f12880091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b32b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b32bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b32c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56287b200410 con 0x56287b1c3ca0 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56287b13db40 con 0x56287adea460 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56287b12cf70 con 0x56287b1da950 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b32c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f1297e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b32b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b32c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52404/0 (socket says 172.21.15.46:52404) 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 -- 172.21.15.46:0/3574995426 learned_addr learned my addr 172.21.15.46:0/3574995426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f128f400640 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b32bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 -- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 msgr2=0x56287b32bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b32bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 -- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 msgr2=0x56287b32b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b32b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 -- 172.21.15.46:0/3574995426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287b0f45b0 con 0x56287b1c3ca0 2024-09-17T03:37:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f129d000640 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b32c120 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f12980b33f0 tx=0x7f1290003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419c5ead7ba7708d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f128cc00640 1 -- 172.21.15.46:0/3574995426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290006810 con 0x56287b1c3ca0 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f128cc00640 1 -- 172.21.15.46:0/3574995426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1290016070 con 0x56287b1c3ca0 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.264+0000 7f128cc00640 1 -- 172.21.15.46:0/3574995426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290012640 con 0x56287b1c3ca0 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- 172.21.15.46:0/3574995426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 msgr2=0x56287b32c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b32c120 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f12980b33f0 tx=0x7f1290003d50 comp rx=0 tx=0).stop 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- 172.21.15.46:0/3574995426 shutdown_connections 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b32c120 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b32bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- 172.21.15.46:0/3574995426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b32b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- 172.21.15.46:0/3574995426 >> 172.21.15.46:0/3574995426 conn(0x56287b2b0d90 msgr2=0x56287b2e0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- 172.21.15.46:0/3574995426 shutdown_connections 2024-09-17T03:37:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- 172.21.15.46:0/3574995426 wait complete. 2024-09-17T03:37:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- start start 2024-09-17T03:37:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 Processor -- start 2024-09-17T03:37:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287adea460 0x56287b332920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b332e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1da950 0x56287b3333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56287b0c75f0 con 0x56287b1c3ca0 2024-09-17T03:37:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56287b130d80 con 0x56287b1da950 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56287b0d3520 con 0x56287adea460 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f12a6707000 10 client.0 _ll_register_callbacks cb 0x56287b0d7830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f128f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b332e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f128f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b332e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52418/0 (socket says 172.21.15.46:52418) 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f128f400640 1 -- 172.21.15.46:0/2951544989 learned_addr learned my addr 172.21.15.46:0/2951544989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f1297e00640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287adea460 0x56287b332920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.268+0000 7f129d000640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1da950 0x56287b3333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f128f400640 1 -- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287adea460 msgr2=0x56287b332920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f128f400640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287adea460 0x56287b332920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f128f400640 1 -- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1da950 msgr2=0x56287b3333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f128f400640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1da950 0x56287b3333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f128f400640 1 -- 172.21.15.46:0/2951544989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287b0cb190 con 0x56287b1c3ca0 2024-09-17T03:37:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f128f400640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b332e60 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1280098300 tx=0x7f12800f95b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69be1e5b5a7f4da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 1 -- 172.21.15.46:0/2951544989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12800be050 con 0x56287b1c3ca0 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 1 -- 172.21.15.46:0/2951544989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f1280102fa0 con 0x56287b1c3ca0 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56287b0c5110 con 0x56287b1c3ca0 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 1 -- 172.21.15.46:0/2951544989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f12800fa1a0 con 0x56287b1c3ca0 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 10 client.249360 fetch_fsmap learned FSMap version 94 2024-09-17T03:37:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x56287b1b1f10 con 0x56287b1c3ca0 2024-09-17T03:37:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 1 -- 172.21.15.46:0/2951544989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56287b0c5110 con 0x56287b1c3ca0 2024-09-17T03:37:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 1 -- 172.21.15.46:0/2951544989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) ==== 3502+0+0 (secure 0 0 0) 0x7f128011b080 con 0x56287b1c3ca0 2024-09-17T03:37:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 10 client.249360 fetch_fsmap finished waiting for FSMap version 94 2024-09-17T03:37:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 10 client.249360 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:37:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 20 client.249360 populate_metadata read hostname 'smithi046' 2024-09-17T03:37:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adcf7d0 0x56287b33cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 4 client.249360 mds_command: new command op to 248758 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 --> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56287b13fa90 con 0x56287adcf7d0 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1297e00640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adcf7d0 0x56287b33cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 10 client.249360 ms_handle_connect on v2:172.21.15.46:6836/2038056563 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1297e00640 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adcf7d0 0x56287b33cd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8275b6a6f28050fd server_cookie=43ef29620550689c in_seq=0 out_seq=0 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 1 -- 172.21.15.46:0/2951544989 <== mds.0 v2:172.21.15.46:6836/2038056563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56287b13fa90 con 0x56287adcf7d0 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 10 client.249360 handle_command_reply: tid=0 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 client.249360 shutdown 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 msgr2=0x56287b332e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b332e60 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1280098300 tx=0x7f12800f95b0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 shutdown_connections 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1da950 0x56287b3333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1c3ca0 0x56287b332e60 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adcf7d0 0x56287b33cd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 --2- 172.21.15.46:0/2951544989 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287adea460 0x56287b332920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f1285600640 0 client.249360 ms_handle_reset on v2:172.21.15.46:6836/2038056563 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 >> 172.21.15.46:0/2951544989 conn(0x56287b2b0d90 msgr2=0x56287b2e7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 shutdown_connections 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.272+0000 7f12a6707000 1 -- 172.21.15.46:0/2951544989 wait complete. 2024-09-17T03:37:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.276+0000 7f12a6707000 20 client.249360 trim_cache size 0 max 16384 2024-09-17T03:37:12.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.292+0000 7f12a6707000 1 Processor -- start 2024-09-17T03:37:12.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.292+0000 7f12a6707000 1 -- start start 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 0x56287b39d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1c3ca0 0x56287b39db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b39e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1298205be0 con 0x56287adea460 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1298204c20 con 0x56287b1c3ca0 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f12981fcc30 con 0x56287b1da950 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 0x56287b39d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 0x56287b39d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52432/0 (socket says 172.21.15.46:52432) 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 -- 172.21.15.46:0/685513161 learned_addr learned my addr 172.21.15.46:0/685513161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f128f400640 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1c3ca0 0x56287b39db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f129d000640 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b39e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 -- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 msgr2=0x56287b39e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b39e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 -- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1c3ca0 msgr2=0x56287b39db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1c3ca0 0x56287b39db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 -- 172.21.15.46:0/685513161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287b188db0 con 0x56287adea460 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1297e00640 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 0x56287b39d620 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x56287b137930 tx=0x7f128801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261035454bf25f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1287e00640 1 -- 172.21.15.46:0/685513161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288003d10 con 0x56287adea460 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1287e00640 1 -- 172.21.15.46:0/685513161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1288005020 con 0x56287adea460 2024-09-17T03:37:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f1287e00640 1 -- 172.21.15.46:0/685513161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288005470 con 0x56287adea460 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- 172.21.15.46:0/685513161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 msgr2=0x56287b39d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 0x56287b39d620 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x56287b137930 tx=0x7f128801b000 comp rx=0 tx=0).stop 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- 172.21.15.46:0/685513161 shutdown_connections 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1da950 0x56287b39e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287b1c3ca0 0x56287b39db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 --2- 172.21.15.46:0/685513161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287adea460 0x56287b39d620 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- 172.21.15.46:0/685513161 >> 172.21.15.46:0/685513161 conn(0x56287b2b0d90 msgr2=0x56287b329620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- 172.21.15.46:0/685513161 shutdown_connections 2024-09-17T03:37:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- 172.21.15.46:0/685513161 wait complete. 2024-09-17T03:37:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 -- start start 2024-09-17T03:37:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.296+0000 7f12a6707000 1 Processor -- start 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b3a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1c3ca0 0x56287b3a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 0x56287b3a2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56287b130d80 con 0x56287b1da950 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12982045d0 con 0x56287adea460 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x56287b0c75f0 con 0x56287b1c3ca0 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 10 client.0 _ll_register_callbacks cb 0x7f12800f93f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 0x56287b3a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 0x56287b3a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52440/0 (socket says 172.21.15.46:52440) 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 -- 172.21.15.46:0/1234120505 learned_addr learned my addr 172.21.15.46:0/1234120505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1297e00640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b3a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f128f400640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1c3ca0 0x56287b3a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 -- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1c3ca0 msgr2=0x56287b3a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1c3ca0 0x56287b3a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 -- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 msgr2=0x56287b3a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b3a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 -- 172.21.15.46:0/1234120505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287b0f3ab0 con 0x56287b1da950 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f129d000640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 0x56287b3a2020 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1298070710 tx=0x7f1290001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cedd84572a4ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1285600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-09-17T03:37:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1285600640 1 -- 172.21.15.46:0/1234120505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129001c880 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1285600640 1 -- 172.21.15.46:0/1234120505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f129001ca20 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1288005020 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1285600640 1 -- 172.21.15.46:0/1234120505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f1290031050 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 10 client.249366 fetch_fsmap learned FSMap version 94 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x56287b0d2070 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1285600640 1 -- 172.21.15.46:0/1234120505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1288005020 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1285600640 1 -- 172.21.15.46:0/1234120505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) ==== 3502+0+0 (secure 0 0 0) 0x7f1290034020 con 0x56287b1da950 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 10 client.249366 fetch_fsmap finished waiting for FSMap version 94 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 10 client.249366 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 20 client.249366 populate_metadata read hostname 'smithi046' 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adbd830 0x56287b35e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 4 client.249366 mds_command: new command op to 248758 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:37:12.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 --> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] -- command(tid 0: {"prefix": "status"}) -- 0x56287b13b540 con 0x56287adbd830 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.300+0000 7f1297e00640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adbd830 0x56287b35e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f1297e00640 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adbd830 0x56287b35e820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a00d0272ec120b7 server_cookie=f5a20788eab5dbb in_seq=0 out_seq=0 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f1285600640 10 client.249366 ms_handle_connect on v2:172.21.15.46:6836/2038056563 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f1285600640 1 -- 172.21.15.46:0/1234120505 <== mds.0 v2:172.21.15.46:6836/2038056563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x56287b13b540 con 0x56287adbd830 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f1285600640 10 client.249366 handle_command_reply: tid=0 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 client.249366 shutdown 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 msgr2=0x56287b3a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 0x56287b3a2020 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1298070710 tx=0x7f1290001a70 comp rx=0 tx=0).stop 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 shutdown_connections 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56287b1da950 0x56287b3a2020 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.46:6836/2038056563,v1:172.21.15.46:6837/2038056563] conn(0x56287adbd830 0x56287b35e820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x56287b1c3ca0 0x56287b3a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 --2- 172.21.15.46:0/1234120505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56287adea460 0x56287b3a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f1285600640 0 client.249366 ms_handle_reset on v2:172.21.15.46:6836/2038056563 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 >> 172.21.15.46:0/1234120505 conn(0x56287b2b0d90 msgr2=0x56287b33c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 shutdown_connections 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 1 -- 172.21.15.46:0/1234120505 wait complete. 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f12a6707000 20 client.249366 trim_cache size 0 max 16384 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 248758, 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-09-17T03:37:12.308 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.817870737, 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 94, 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 143, 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 143, 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.177023896 2024-09-17T03:37:12.309 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:37:12.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1274075570 msgr2=0x7f1274077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1274075570 0x7f1274077a80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1288009b40 tx=0x7f12880091e0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 msgr2=0x7f1298154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 0x7f1298154700 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1280005040 tx=0x7f1280003c60 comp rx=0 tx=0).stop 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 shutdown_connections 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1274075570 0x7f1274077a80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12980b1330 0x7f12980b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129814eda0 0x7f1298154700 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 --2- 172.21.15.46:0/102977756 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f127c004810 0x7f1298158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 >> 172.21.15.46:0/102977756 conn(0x7f12980b2040 msgr2=0x7f12980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 shutdown_connections 2024-09-17T03:37:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.304+0000 7f129f000640 1 -- 172.21.15.46:0/102977756 wait complete. 2024-09-17T03:37:12.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:37:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 -- 172.21.15.46:0/479030398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941815efc0 msgr2=0x7f9418070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 --2- 172.21.15.46:0/479030398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941815efc0 0x7f9418070910 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9408009840 tx=0x7f940801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 -- 172.21.15.46:0/479030398 shutdown_connections 2024-09-17T03:37:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 --2- 172.21.15.46:0/479030398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941815efc0 0x7f9418070910 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 -- 172.21.15.46:0/479030398 >> 172.21.15.46:0/479030398 conn(0x7f94180b35f0 msgr2=0x7f94180b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 -- 172.21.15.46:0/479030398 shutdown_connections 2024-09-17T03:37:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.480+0000 7f941ec00640 1 -- 172.21.15.46:0/479030398 wait complete. 2024-09-17T03:37:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 Processor -- start 2024-09-17T03:37:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 -- start start 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94180b0810 0x7f94180b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f941815efc0 0x7f9418161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 0x7f94181617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f941816daf0 con 0x7f94180b1bf0 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f941816d970 con 0x7f94180b0810 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941ec00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f941816dc70 con 0x7f941815efc0 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 0x7f94181617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 0x7f94181617c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56094/0 (socket says 172.21.15.46:56094) 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.484+0000 7f941cc00640 1 -- 172.21.15.46:0/1526419267 learned_addr learned my addr 172.21.15.46:0/1526419267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f9417e00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94180b0810 0x7f94180b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f940fe00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f941815efc0 0x7f9418161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941cc00640 1 -- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f941815efc0 msgr2=0x7f9418161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941cc00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f941815efc0 0x7f9418161280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941cc00640 1 -- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94180b0810 msgr2=0x7f94180b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941cc00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94180b0810 0x7f94180b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941cc00640 1 -- 172.21.15.46:0/1526419267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181fdc30 con 0x7f94180b1bf0 2024-09-17T03:37:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941cc00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 0x7f94181617c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f941000cb10 tx=0x7f941000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410016020 con 0x7f94180b1bf0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f941000ce50 con 0x7f94180b1bf0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941000ad60 con 0x7f94180b1bf0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181fce70 con 0x7f94180b1bf0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181fd430 con 0x7f94180b1bf0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941814d4f0 con 0x7f94180b1bf0 2024-09-17T03:37:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.488+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f9410019020 con 0x7f94180b1bf0 2024-09-17T03:37:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.492+0000 7f9414c00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93f4075570 0x7f93f4077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.492+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f94100930f0 con 0x7f94180b1bf0 2024-09-17T03:37:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.492+0000 7f9417e00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93f4075570 0x7f93f4077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.492+0000 7f9417e00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93f4075570 0x7f93f4077a80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f94080af000 tx=0x7f940801b030 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:12.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.496+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9410020030 con 0x7f94180b1bf0 2024-09-17T03:37:12.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.704+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f941806e5e0 con 0x7f94180b1bf0 2024-09-17T03:37:12.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f9414c00640 1 -- 172.21.15.46:0/1526419267 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) ==== 53+0+73 (secure 0 0 0) 0x7f941005f650 con 0x7f94180b1bf0 2024-09-17T03:37:12.708 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93f4075570 msgr2=0x7f93f4077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93f4075570 0x7f93f4077a80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f94080af000 tx=0x7f940801b030 comp rx=0 tx=0).stop 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 msgr2=0x7f94181617c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 0x7f94181617c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f941000cb10 tx=0x7f941000cc00 comp rx=0 tx=0).stop 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 shutdown_connections 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f93f4075570 0x7f93f4077a80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94180b1bf0 0x7f94181617c0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f941815efc0 0x7f9418161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 --2- 172.21.15.46:0/1526419267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94180b0810 0x7f94180b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:12.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 >> 172.21.15.46:0/1526419267 conn(0x7f94180b35f0 msgr2=0x7f9418075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:12.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 shutdown_connections 2024-09-17T03:37:12.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12.708+0000 7f941ec00640 1 -- 172.21.15.46:0/1526419267 wait complete. 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:37:12.731 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:37:12.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:12.732 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-17T03:37:12.741 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:37:12.741 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41456sec preferred_lft 41456sec 2024-09-17T03:37:12.742 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-17T03:37:12.744 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.744 INFO:teuthology.orchestra.run.smithi046.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:37:12.744 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:35:77:27:69 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.745 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T03:37:12.746 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:37:12.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:12.747 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:37:12.813 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-17T03:37:12.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:12.815 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-17T03:37:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-17T03:37:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:37:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T03:37:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T03:37:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T03:37:12.869 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:37:12.870 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:37:12.870 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-17T03:37:12.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T03:37:12.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T03:37:12.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T03:37:12.870 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-17T03:37:12.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:37:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:37:13.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:13.007 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:37:13.016 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:37:13.016 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:37:13.017 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:37:13.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:13.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-17T03:37:13.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:37:13.084 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:37:13.139 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:37:13.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:37:13.140 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:37:13.195 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-17T03:37:13.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:13.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:37:13.300 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:37:13.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:37:13.313 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-17T03:37:13.313 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-17T03:37:13.313 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.99:3300,172.21.15.99:3301" 2024-09-17T03:37:13.313 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@6c25af6b-3689-4426-acb2-6ebe6de6efd5.cephfs=/ 2024-09-17T03:37:13.313 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.46:3300/172.21.15.99:3300/172.21.15.99:3301" will pass to kernel 2024-09-17T03:37:13.317 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T03:37:13.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:13.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:37:13.337 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T03:37:13.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:37:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:37:13.445 INFO:teuthology.orchestra.run.smithi046.stdout:client248884 2024-09-17T03:37:13.445 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:37:13.445 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:37:13.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T03:37:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 -- 172.21.15.46:0/1390346442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 msgr2=0x7f63a80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 --2- 172.21.15.46:0/1390346442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a80b51f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f63a8071cb0 tx=0x7f63a001cae0 comp rx=0 tx=0).stop 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 -- 172.21.15.46:0/1390346442 shutdown_connections 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 --2- 172.21.15.46:0/1390346442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a80b51f0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 --2- 172.21.15.46:0/1390346442 >> v2:172.21.15.99:3300/0 conn(0x7f63a80b5c70 0x7f63a815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 -- 172.21.15.46:0/1390346442 >> 172.21.15.46:0/1390346442 conn(0x7f63a80b2040 msgr2=0x7f63a80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 -- 172.21.15.46:0/1390346442 shutdown_connections 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.660+0000 7f63aee00640 1 -- 172.21.15.46:0/1390346442 wait complete. 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 Processor -- start 2024-09-17T03:37:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 -- start start 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a8149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63a80b5c70 0x7f63a8165af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81527b0 0x7f63a814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81689a0 con 0x7f63a80b4de0 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8168820 con 0x7f63a81527b0 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63aee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f63a8168b20 con 0x7f63a80b5c70 2024-09-17T03:37:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a8149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a8149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56108/0 (socket says 172.21.15.46:56108) 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63ace00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81527b0 0x7f63a814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f63a7e00640 1 -- 172.21.15.46:0/2106447006 learned_addr learned my addr 172.21.15.46:0/2106447006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.664+0000 7f639fe00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63a80b5c70 0x7f63a8165af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a7e00640 1 -- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63a80b5c70 msgr2=0x7f63a8165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a7e00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63a80b5c70 0x7f63a8165af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a7e00640 1 -- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81527b0 msgr2=0x7f63a814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a7e00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81527b0 0x7f63a814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a7e00640 1 -- 172.21.15.46:0/2106447006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a81fbf50 con 0x7f63a80b4de0 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63ace00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81527b0 0x7f63a814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a7e00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a8149940 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f639800b440 tx=0x7f639800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398015020 con 0x7f63a80b4de0 2024-09-17T03:37:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63980119a0 con 0x7f63a80b4de0 2024-09-17T03:37:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398011df0 con 0x7f63a80b4de0 2024-09-17T03:37:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a81fc230 con 0x7f63a80b4de0 2024-09-17T03:37:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81fc7f0 con 0x7f63a80b4de0 2024-09-17T03:37:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a80b51f0 con 0x7f63a80b4de0 2024-09-17T03:37:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f6398018020 con 0x7f63a80b4de0 2024-09-17T03:37:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.668+0000 7f63a4c00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6384075570 0x7f6384077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.672+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f6398093400 con 0x7f63a80b4de0 2024-09-17T03:37:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.672+0000 7f639fe00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6384075570 0x7f6384077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.672+0000 7f639fe00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6384075570 0x7f6384077a80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f63a814a5e0 tx=0x7f6390009060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.676+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f639801f020 con 0x7f63a80b4de0 2024-09-17T03:37:13.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:13.884+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f63a8149e80 con 0x7f63a80b4de0 2024-09-17T03:37:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.568+0000 7f63a4c00640 1 -- 172.21.15.46:0/2106447006 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v95) ==== 136+0+0 (secure 0 0 0) 0x7f639800d040 con 0x7f63a80b4de0 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6384075570 msgr2=0x7f6384077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6384075570 0x7f6384077a80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f63a814a5e0 tx=0x7f6390009060 comp rx=0 tx=0).stop 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 msgr2=0x7f63a8149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a8149940 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f639800b440 tx=0x7f639800b900 comp rx=0 tx=0).stop 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 shutdown_connections 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6384075570 0x7f6384077a80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81527b0 0x7f63a814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f63a80b5c70 0x7f63a8165af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 --2- 172.21.15.46:0/2106447006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a80b4de0 0x7f63a8149940 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 >> 172.21.15.46:0/2106447006 conn(0x7f63a80b2040 msgr2=0x7f63a814d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 shutdown_connections 2024-09-17T03:37:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.572+0000 7f63aee00640 1 -- 172.21.15.46:0/2106447006 wait complete. 2024-09-17T03:37:14.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:37:14.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 -- 172.21.15.46:0/1068170241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b4de0 msgr2=0x7f3a880b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/1068170241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b4de0 0x7f3a880b51f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009c80 tx=0x7f3a8001cce0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 -- 172.21.15.46:0/1068170241 shutdown_connections 2024-09-17T03:37:14.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/1068170241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b4de0 0x7f3a880b51f0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/1068170241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a881527b0 0x7f3a8815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 -- 172.21.15.46:0/1068170241 >> 172.21.15.46:0/1068170241 conn(0x7f3a880b2040 msgr2=0x7f3a880b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 -- 172.21.15.46:0/1068170241 shutdown_connections 2024-09-17T03:37:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.760+0000 7f3a8ca00640 1 -- 172.21.15.46:0/1068170241 wait complete. 2024-09-17T03:37:14.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 Processor -- start 2024-09-17T03:37:14.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 -- start start 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a880b4de0 0x7f3a880ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 0x7f3a881581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a881527b0 0x7f3a88158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88168ab0 con 0x7f3a880b5c70 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88168930 con 0x7f3a881527b0 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3a88168c30 con 0x7f3a880b4de0 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 0x7f3a881581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 0x7f3a881581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56120/0 (socket says 172.21.15.46:56120) 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 -- 172.21.15.46:0/3909953626 learned_addr learned my addr 172.21.15.46:0/3909953626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a85e00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a880b4de0 0x7f3a880ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:14.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a86800640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a881527b0 0x7f3a88158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 -- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a880b4de0 msgr2=0x7f3a880ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a880b4de0 0x7f3a880ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 -- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a881527b0 msgr2=0x7f3a88158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a881527b0 0x7f3a88158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 -- 172.21.15.46:0/3909953626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a880ab920 con 0x7f3a880b5c70 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a86800640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a881527b0 0x7f3a88158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a85e00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a880b4de0 0x7f3a880ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7fe00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 0x7f3a881581d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000d6d0 tx=0x7f3a7000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70017070 con 0x7f3a880b5c70 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a70004310 con 0x7f3a880b5c70 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a880abc00 con 0x7f3a880b5c70 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70005280 con 0x7f3a880b5c70 2024-09-17T03:37:14.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.764+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a880ac1c0 con 0x7f3a880b5c70 2024-09-17T03:37:14.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.768+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a880b51f0 con 0x7f3a880b5c70 2024-09-17T03:37:14.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.768+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f3a70005420 con 0x7f3a880b5c70 2024-09-17T03:37:14.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.768+0000 7f3a7e000640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a640755a0 0x7f3a64077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:14.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.768+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f3a70093750 con 0x7f3a880b5c70 2024-09-17T03:37:14.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.768+0000 7f3a85e00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a640755a0 0x7f3a64077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:14.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.768+0000 7f3a85e00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a640755a0 0x7f3a64077ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3a780016b0 tx=0x7f3a78009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:14.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.772+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a70020020 con 0x7f3a880b5c70 2024-09-17T03:37:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.972+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3a8806b540 con 0x7f3a640755a0 2024-09-17T03:37:14.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.972+0000 7f3a7e000640 1 -- 172.21.15.46:0/3909953626 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3a78001c90 con 0x7f3a640755a0 2024-09-17T03:37:14.975 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:37:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:37:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:37:14.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:37:14.975 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:37:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a640755a0 msgr2=0x7f3a64077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a640755a0 0x7f3a64077ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3a780016b0 tx=0x7f3a78009040 comp rx=0 tx=0).stop 2024-09-17T03:37:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 msgr2=0x7f3a881581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 0x7f3a881581d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000d6d0 tx=0x7f3a7000db90 comp rx=0 tx=0).stop 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 shutdown_connections 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3a640755a0 0x7f3a64077ab0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a881527b0 0x7f3a88158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a880b5c70 0x7f3a881581d0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 --2- 172.21.15.46:0/3909953626 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3a880b4de0 0x7f3a880ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 >> 172.21.15.46:0/3909953626 conn(0x7f3a880b2040 msgr2=0x7f3a88152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 shutdown_connections 2024-09-17T03:37:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:14.976+0000 7f3a8ca00640 1 -- 172.21.15.46:0/3909953626 wait complete. 2024-09-17T03:37:14.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:37:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3492437318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 msgr2=0x7fa8cc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3492437318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 0x7fa8cc15bb60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009840 tx=0x7fa8bc01c680 comp rx=0 tx=0).stop 2024-09-17T03:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3492437318 shutdown_connections 2024-09-17T03:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3492437318 >> v2:172.21.15.99:3300/0 conn(0x7fa8cc0b4de0 0x7fa8cc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3492437318 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 0x7fa8cc15bb60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3492437318 >> 172.21.15.46:0/3492437318 conn(0x7fa8cc0b2040 msgr2=0x7fa8cc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3492437318 shutdown_connections 2024-09-17T03:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3492437318 wait complete. 2024-09-17T03:37:15.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 Processor -- start 2024-09-17T03:37:15.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.160+0000 7fa8d3a00640 1 -- start start 2024-09-17T03:37:15.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8cc0b4de0 0x7fa8cc149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 0x7fa8cc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 0x7fa8cc156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc168a60 con 0x7fa8cc1527b0 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1688e0 con 0x7fa8cc0b4de0 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc168be0 con 0x7fa8cc0b5c70 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 0x7fa8cc156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 0x7fa8cc156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56124/0 (socket says 172.21.15.46:56124) 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 -- 172.21.15.46:0/3487750672 learned_addr learned my addr 172.21.15.46:0/3487750672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1000640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8cc0b4de0 0x7fa8cc149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8c3e00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 0x7fa8cc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 -- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 msgr2=0x7fa8cc1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 0x7fa8cc1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 -- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8cc0b4de0 msgr2=0x7fa8cc149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8cc0b4de0 0x7fa8cc149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 -- 172.21.15.46:0/3487750672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc149e80 con 0x7fa8cc1527b0 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1000640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8cc0b4de0 0x7fa8cc149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d1a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 0x7fa8cc156ae0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ec70 tx=0x7fa8c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003b80 con 0x7fa8cc1527b0 2024-09-17T03:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc0cf410 con 0x7fa8cc1527b0 2024-09-17T03:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8c4003d20 con 0x7fa8cc1527b0 2024-09-17T03:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004280 con 0x7fa8cc1527b0 2024-09-17T03:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc0cf9d0 con 0x7fa8cc1527b0 2024-09-17T03:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.164+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc0cfbd0 con 0x7fa8cc1527b0 2024-09-17T03:37:15.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.168+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fa8c4004420 con 0x7fa8cc1527b0 2024-09-17T03:37:15.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.168+0000 7fa8c9600640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8a8075570 0x7fa8a8077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.168+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7fa8c40943f0 con 0x7fa8cc1527b0 2024-09-17T03:37:15.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.168+0000 7fa8d1000640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8a8075570 0x7fa8a8077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.168+0000 7fa8d1000640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8a8075570 0x7fa8a8077a80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0ae000 tx=0x7fa8bc01aec0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:15.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.172+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8cc0cfbd0 con 0x7fa8cc1527b0 2024-09-17T03:37:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.348+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa8cc0b51f0 con 0x7fa8cc1527b0 2024-09-17T03:37:15.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.348+0000 7fa8c9600640 1 -- 172.21.15.46:0/3487750672 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) ==== 122+0+0 (secure 0 0 0) 0x7fa8c4060f80 con 0x7fa8cc1527b0 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.348+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8a8075570 msgr2=0x7fa8a8077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.348+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8a8075570 0x7fa8a8077a80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0ae000 tx=0x7fa8bc01aec0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.348+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 msgr2=0x7fa8cc156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.348+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 0x7fa8cc156ae0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ec70 tx=0x7fa8c400c6a0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 shutdown_connections 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8a8075570 0x7fa8a8077a80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8cc1527b0 0x7fa8cc156ae0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa8cc0b5c70 0x7fa8cc1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 --2- 172.21.15.46:0/3487750672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8cc0b4de0 0x7fa8cc149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 >> 172.21.15.46:0/3487750672 conn(0x7fa8cc0b2040 msgr2=0x7fa8cc154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 shutdown_connections 2024-09-17T03:37:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.352+0000 7fa8d3a00640 1 -- 172.21.15.46:0/3487750672 wait complete. 2024-09-17T03:37:15.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T03:37:15.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 -- 172.21.15.46:0/3270466388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808074950 msgr2=0x7f08080b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 --2- 172.21.15.46:0/3270466388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808074950 0x7f08080b5260 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0800009c80 tx=0x7f080001cb30 comp rx=0 tx=0).stop 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 -- 172.21.15.46:0/3270466388 shutdown_connections 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 --2- 172.21.15.46:0/3270466388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0808074950 0x7f08080b5260 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 --2- 172.21.15.46:0/3270466388 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f08081536f0 0x7f080815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 -- 172.21.15.46:0/3270466388 >> 172.21.15.46:0/3270466388 conn(0x7f08080b4560 msgr2=0x7f08080d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 -- 172.21.15.46:0/3270466388 shutdown_connections 2024-09-17T03:37:15.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.532+0000 7f080cc00640 1 -- 172.21.15.46:0/3270466388 wait complete. 2024-09-17T03:37:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 Processor -- start 2024-09-17T03:37:15.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 -- start start 2024-09-17T03:37:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 0x7f080814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0808160060 0x7f080814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080806e1d0 0x7f08081fc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0808169e60 con 0x7f08081536f0 2024-09-17T03:37:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0808169ce0 con 0x7f080806e1d0 2024-09-17T03:37:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0808169fe0 con 0x7f0808160060 2024-09-17T03:37:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 0x7f080814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 0x7f080814fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56140/0 (socket says 172.21.15.46:56140) 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 -- 172.21.15.46:0/972910142 learned_addr learned my addr 172.21.15.46:0/972910142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807e00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080806e1d0 0x7f08081fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 -- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0808160060 msgr2=0x7f080814a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f07fea00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0808160060 0x7f080814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0808160060 0x7f080814a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 -- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080806e1d0 msgr2=0x7f08081fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080806e1d0 0x7f08081fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 -- 172.21.15.46:0/972910142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080806e770 con 0x7f08081536f0 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807e00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080806e1d0 0x7f08081fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f07fea00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0808160060 0x7f080814a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f0807400640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 0x7f080814fd80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d720 tx=0x7f07f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8017070 con 0x7f08081536f0 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07f8009d20 con 0x7f08081536f0 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080806ea50 con 0x7f08081536f0 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.536+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8005280 con 0x7f08081536f0 2024-09-17T03:37:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.540+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081fcb40 con 0x7f08081536f0 2024-09-17T03:37:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.540+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08080b5260 con 0x7f08081536f0 2024-09-17T03:37:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.540+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f07f8005420 con 0x7f08081536f0 2024-09-17T03:37:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.540+0000 7f07ffe00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07e4075570 0x7f07e4077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.540+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f07f80935c0 con 0x7f08081536f0 2024-09-17T03:37:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.540+0000 7f07fea00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07e4075570 0x7f07e4077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.544+0000 7f07fea00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07e4075570 0x7f07e4077a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f080815bbd0 tx=0x7f07f0008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.544+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07f8060040 con 0x7f08081536f0 2024-09-17T03:37:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f08080b6c00 con 0x7f08081536f0 2024-09-17T03:37:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f07ffe00640 1 -- 172.21.15.46:0/972910142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v58) ==== 116+0+2 (secure 0 0 0) 0x7f07f805f940 con 0x7f08081536f0 2024-09-17T03:37:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:4 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07e4075570 msgr2=0x7f07e4077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07e4075570 0x7f07e4077a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f080815bbd0 tx=0x7f07f0008040 comp rx=0 tx=0).stop 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 msgr2=0x7f080814fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 0x7f080814fd80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d720 tx=0x7f07f800dbe0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 shutdown_connections 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07e4075570 0x7f07e4077a80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080806e1d0 0x7f08081fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0808160060 0x7f080814a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.724+0000 7f080cc00640 1 --2- 172.21.15.46:0/972910142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08081536f0 0x7f080814fd80 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.728+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 >> 172.21.15.46:0/972910142 conn(0x7f08080b4560 msgr2=0x7f08080b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.728+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 shutdown_connections 2024-09-17T03:37:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.728+0000 7f080cc00640 1 -- 172.21.15.46:0/972910142 wait complete. 2024-09-17T03:37:15.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:37:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 -- 172.21.15.46:0/3850696651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 msgr2=0x7feb6c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3850696651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb6c004c20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7feb700016f0 tx=0x7feb7001c720 comp rx=0 tx=0).stop 2024-09-17T03:37:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 -- 172.21.15.46:0/3850696651 shutdown_connections 2024-09-17T03:37:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3850696651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb6c004c20 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 -- 172.21.15.46:0/3850696651 >> 172.21.15.46:0/3850696651 conn(0x7feb8406cdf0 msgr2=0x7feb8406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 -- 172.21.15.46:0/3850696651 shutdown_connections 2024-09-17T03:37:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.908+0000 7feb8ac00640 1 -- 172.21.15.46:0/3850696651 wait complete. 2024-09-17T03:37:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 Processor -- start 2024-09-17T03:37:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 -- start start 2024-09-17T03:37:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb84145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb8414a060 0x7feb84153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84153fb0 0x7feb84143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb840b37c0 con 0x7feb6c004810 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb840b3640 con 0x7feb84153fb0 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8ac00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7feb840b3940 con 0x7feb8414a060 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb89800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb84145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb89800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb84145f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56166/0 (socket says 172.21.15.46:56166) 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb89800640 1 -- 172.21.15.46:0/3161858453 learned_addr learned my addr 172.21.15.46:0/3161858453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb88e00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb8414a060 0x7feb84153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.912+0000 7feb8a200640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84153fb0 0x7feb84143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb89800640 1 -- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb8414a060 msgr2=0x7feb84153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb89800640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb8414a060 0x7feb84153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb89800640 1 -- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84153fb0 msgr2=0x7feb84143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb89800640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84153fb0 0x7feb84143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb89800640 1 -- 172.21.15.46:0/3161858453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb841464b0 con 0x7feb6c004810 2024-09-17T03:37:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb89800640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb84145f70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7feb78011a10 tx=0x7feb78011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78004460 con 0x7feb6c004810 2024-09-17T03:37:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb7800f9a0 con 0x7feb6c004810 2024-09-17T03:37:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7800fdb0 con 0x7feb6c004810 2024-09-17T03:37:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb841469a0 con 0x7feb6c004810 2024-09-17T03:37:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841440d0 con 0x7feb6c004810 2024-09-17T03:37:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb84072030 con 0x7feb6c004810 2024-09-17T03:37:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.916+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7feb78005020 con 0x7feb6c004810 2024-09-17T03:37:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.920+0000 7feb81600640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb64075570 0x7feb64077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.920+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7feb78094090 con 0x7feb6c004810 2024-09-17T03:37:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.920+0000 7feb88e00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb64075570 0x7feb64077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:15.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.920+0000 7feb88e00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb64075570 0x7feb64077a80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7feb700016c0 tx=0x7feb700b1040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:15.924+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb7809f080 con 0x7feb6c004810 2024-09-17T03:37:16.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.100+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7feb84146ec0 con 0x7feb6c004810 2024-09-17T03:37:16.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.104+0000 7feb81600640 1 -- 172.21.15.46:0/3161858453 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v59)=0 v59) ==== 131+0+0 (secure 0 0 0) 0x7feb78060580 con 0x7feb6c004810 2024-09-17T03:37:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb64075570 msgr2=0x7feb64077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb64075570 0x7feb64077a80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7feb700016c0 tx=0x7feb700b1040 comp rx=0 tx=0).stop 2024-09-17T03:37:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 msgr2=0x7feb84145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb84145f70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7feb78011a10 tx=0x7feb78011ed0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 shutdown_connections 2024-09-17T03:37:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feb64075570 0x7feb64077a80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84153fb0 0x7feb84143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7feb8414a060 0x7feb84153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 --2- 172.21.15.46:0/3161858453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c004810 0x7feb84145f70 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 >> 172.21.15.46:0/3161858453 conn(0x7feb8406cdf0 msgr2=0x7feb840d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 shutdown_connections 2024-09-17T03:37:16.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.108+0000 7feb8ac00640 1 -- 172.21.15.46:0/3161858453 wait complete. 2024-09-17T03:37:16.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.128+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.128+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.128+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.128+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.128+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-17T03:37:16.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 -- 172.21.15.46:0/528991509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40749a0 msgr2=0x7f35a4074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 --2- 172.21.15.46:0/528991509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40749a0 0x7f35a4074db0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3594009840 tx=0x7f359401c820 comp rx=0 tx=0).stop 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 -- 172.21.15.46:0/528991509 shutdown_connections 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 --2- 172.21.15.46:0/528991509 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40b3340 0x7f35a40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 --2- 172.21.15.46:0/528991509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40749a0 0x7f35a4074db0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 -- 172.21.15.46:0/528991509 >> 172.21.15.46:0/528991509 conn(0x7f35a40b2040 msgr2=0x7f35a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 -- 172.21.15.46:0/528991509 shutdown_connections 2024-09-17T03:37:16.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 -- 172.21.15.46:0/528991509 wait complete. 2024-09-17T03:37:16.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 Processor -- start 2024-09-17T03:37:16.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.296+0000 7f35a9800640 1 -- start start 2024-09-17T03:37:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40749a0 0x7f35a4145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 0x7f35a415fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a40cefa0 0x7f35a40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4169e10 con 0x7f35a40b3340 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4169c90 con 0x7f35a40cefa0 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f35a4169f90 con 0x7f35a40749a0 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 0x7f35a415fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 0x7f35a415fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56204/0 (socket says 172.21.15.46:56204) 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 -- 172.21.15.46:0/86605005 learned_addr learned my addr 172.21.15.46:0/86605005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a2a00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40749a0 0x7f35a4145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a3400640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a40cefa0 0x7f35a40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 -- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40749a0 msgr2=0x7f35a4145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40749a0 0x7f35a4145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 -- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a40cefa0 msgr2=0x7f35a40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a40cefa0 0x7f35a40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 -- 172.21.15.46:0/86605005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a415ff60 con 0x7f35a40b3340 2024-09-17T03:37:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a3400640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a40cefa0 0x7f35a40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a2a00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40749a0 0x7f35a4145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359be00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 0x7f35a415fa20 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c3d0 tx=0x7f358c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c017070 con 0x7f35a40b3340 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f358c003600 con 0x7f35a40b3340 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c003a50 con 0x7f35a40b3340 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a41f90a0 con 0x7f35a40b3340 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a41f9660 con 0x7f35a40b3340 2024-09-17T03:37:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.300+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a41f8e10 con 0x7f35a40b3340 2024-09-17T03:37:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.304+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f358c01a070 con 0x7f35a40b3340 2024-09-17T03:37:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.304+0000 7f359aa00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3580075570 0x7f3580077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.304+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f358c092710 con 0x7f35a40b3340 2024-09-17T03:37:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.304+0000 7f35a2a00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3580075570 0x7f3580077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.304+0000 7f35a2a00640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3580075570 0x7f3580077a80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f35940af000 tx=0x7f359401b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.308+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f35a415ff60 con 0x7f35a40b3340 2024-09-17T03:37:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.488+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f35a4074db0 con 0x7f35a40b3340 2024-09-17T03:37:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f359aa00640 1 -- 172.21.15.46:0/86605005 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v60)=0 v60) ==== 136+0+0 (secure 0 0 0) 0x7f358c05f190 con 0x7f35a40b3340 2024-09-17T03:37:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3580075570 msgr2=0x7f3580077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3580075570 0x7f3580077a80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f35940af000 tx=0x7f359401b060 comp rx=0 tx=0).stop 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 msgr2=0x7f35a415fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 0x7f35a415fa20 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c3d0 tx=0x7f358c00c890 comp rx=0 tx=0).stop 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 shutdown_connections 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3580075570 0x7f3580077a80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35a40cefa0 0x7f35a40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a40b3340 0x7f35a415fa20 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 --2- 172.21.15.46:0/86605005 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f35a40749a0 0x7f35a4145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 >> 172.21.15.46:0/86605005 conn(0x7f35a40b2040 msgr2=0x7f35a4142200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 shutdown_connections 2024-09-17T03:37:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.492+0000 7f35a9800640 1 -- 172.21.15.46:0/86605005 wait complete. 2024-09-17T03:37:16.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 -- 172.21.15.46:0/3143538561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844005360 msgr2=0x7fd84400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 --2- 172.21.15.46:0/3143538561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844005360 0x7fd84400b9d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c5c0 tx=0x7fd85801fa40 comp rx=0 tx=0).stop 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 -- 172.21.15.46:0/3143538561 shutdown_connections 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 --2- 172.21.15.46:0/3143538561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd844005360 0x7fd84400b9d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 -- 172.21.15.46:0/3143538561 >> 172.21.15.46:0/3143538561 conn(0x7fd8600b4560 msgr2=0x7fd8600d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 -- 172.21.15.46:0/3143538561 shutdown_connections 2024-09-17T03:37:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 -- 172.21.15.46:0/3143538561 wait complete. 2024-09-17T03:37:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 Processor -- start 2024-09-17T03:37:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.684+0000 7fd85fe00640 1 -- start start 2024-09-17T03:37:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd844005360 0x7fd8600b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8601529d0 0x7fd8600b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 0x7fd8601e9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8600743c0 con 0x7fd8601e9040 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd860074240 con 0x7fd8601529d0 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd860074540 con 0x7fd844005360 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 0x7fd8601e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 0x7fd8601e9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56230/0 (socket says 172.21.15.46:56230) 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 -- 172.21.15.46:0/4091509473 learned_addr learned my addr 172.21.15.46:0/4091509473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85ea00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd844005360 0x7fd8600b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd856000640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8601529d0 0x7fd8600b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 -- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd844005360 msgr2=0x7fd8600b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd844005360 0x7fd8600b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 -- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8601529d0 msgr2=0x7fd8600b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8601529d0 0x7fd8600b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 -- 172.21.15.46:0/4091509473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8600b15b0 con 0x7fd8601e9040 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd856000640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8601529d0 0x7fd8600b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85f400640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 0x7fd8601e9450 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd8580202e0 tx=0x7fd858002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858003c40 con 0x7fd8601e9040 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8580a9720 con 0x7fd8601e9040 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8601e9990 con 0x7fd8601e9040 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580a9b70 con 0x7fd8601e9040 2024-09-17T03:37:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8600af730 con 0x7fd8601e9040 2024-09-17T03:37:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.688+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd860074aa0 con 0x7fd8601e9040 2024-09-17T03:37:16.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.692+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7fd8580a9d10 con 0x7fd8601e9040 2024-09-17T03:37:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.692+0000 7fd857400640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd83c075570 0x7fd83c077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.692+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7fd858130b90 con 0x7fd8601e9040 2024-09-17T03:37:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.692+0000 7fd85ea00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd83c075570 0x7fd83c077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.692+0000 7fd85ea00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd83c075570 0x7fd83c077a80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd8500063b0 tx=0x7fd850000bd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.696+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd8580c2020 con 0x7fd8601e9040 2024-09-17T03:37:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.916+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fd8601532a0 con 0x7fd83c075570 2024-09-17T03:37:16.925 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.924+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.925 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.924+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.925 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.924+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.925 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.924+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.925 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:16.924+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.956+0000 7fd857400640 1 -- 172.21.15.46:0/4091509473 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8601532a0 con 0x7fd83c075570 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd83c075570 msgr2=0x7fd83c077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd83c075570 0x7fd83c077a80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd8500063b0 tx=0x7fd850000bd0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 msgr2=0x7fd8601e9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 0x7fd8601e9450 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd8580202e0 tx=0x7fd858002080 comp rx=0 tx=0).stop 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 shutdown_connections 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd83c075570 0x7fd83c077a80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8601e9040 0x7fd8601e9450 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8601529d0 0x7fd8600b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 --2- 172.21.15.46:0/4091509473 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd844005360 0x7fd8600b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 >> 172.21.15.46:0/4091509473 conn(0x7fd8600b4560 msgr2=0x7fd8600b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 shutdown_connections 2024-09-17T03:37:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:16.960+0000 7fd85fe00640 1 -- 172.21.15.46:0/4091509473 wait complete. 2024-09-17T03:37:16.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:37:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 -- 172.21.15.46:0/4154569207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8074300 msgr2=0x7f2ed815f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 --2- 172.21.15.46:0/4154569207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8074300 0x7f2ed815f170 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8071970 tx=0x7f2ec401c890 comp rx=0 tx=0).stop 2024-09-17T03:37:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 -- 172.21.15.46:0/4154569207 shutdown_connections 2024-09-17T03:37:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 --2- 172.21.15.46:0/4154569207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed8074300 0x7f2ed815f170 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 --2- 172.21.15.46:0/4154569207 >> v2:172.21.15.99:3300/0 conn(0x7f2ed815c5d0 0x7f2ed8073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 -- 172.21.15.46:0/4154569207 >> 172.21.15.46:0/4154569207 conn(0x7f2ed80b2c40 msgr2=0x7f2ed80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:17.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 -- 172.21.15.46:0/4154569207 shutdown_connections 2024-09-17T03:37:17.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 -- 172.21.15.46:0/4154569207 wait complete. 2024-09-17T03:37:17.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 Processor -- start 2024-09-17T03:37:17.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.144+0000 7f2edce00640 1 -- start start 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ed8074300 0x7f2ed8156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed815c5d0 0x7f2ed80afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 0x7f2ed815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed806f980 con 0x7f2ed815f6b0 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80b3d50 con 0x7f2ed815c5d0 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed806fb00 con 0x7f2ed8074300 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 0x7f2ed815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 0x7f2ed815a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56270/0 (socket says 172.21.15.46:56270) 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 -- 172.21.15.46:0/677104974 learned_addr learned my addr 172.21.15.46:0/677104974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:17.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed5e00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ed8074300 0x7f2ed8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed5400640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed815c5d0 0x7f2ed80afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 -- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ed8074300 msgr2=0x7f2ed8156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ed8074300 0x7f2ed8156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 -- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed815c5d0 msgr2=0x7f2ed80afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed815c5d0 0x7f2ed80afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 -- 172.21.15.46:0/677104974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80b0110 con 0x7f2ed815f6b0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ed6800640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 0x7f2ed815a220 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c730 tx=0x7f2ed000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0003b80 con 0x7f2ed815f6b0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ed0003d20 con 0x7f2ed815f6b0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed81516b0 con 0x7f2ed815f6b0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000aab0 con 0x7f2ed815f6b0 2024-09-17T03:37:17.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed8151c70 con 0x7f2ed815f6b0 2024-09-17T03:37:17.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.148+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed8157090 con 0x7f2ed815f6b0 2024-09-17T03:37:17.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.152+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f2ed000ac50 con 0x7f2ed815f6b0 2024-09-17T03:37:17.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.152+0000 7f2ece000640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2eb80755a0 0x7f2eb8077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.152+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f2ed0093010 con 0x7f2ed815f6b0 2024-09-17T03:37:17.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.152+0000 7f2ed5e00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2eb80755a0 0x7f2eb8077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.152+0000 7f2ed5e00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2eb80755a0 0x7f2eb8077ab0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009b40 tx=0x7f2ec80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.156+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ed005f4d0 con 0x7f2ed815f6b0 2024-09-17T03:37:17.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.360+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2ed806b540 con 0x7f2eb80755a0 2024-09-17T03:37:17.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2ece000640 1 -- 172.21.15.46:0/677104974 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2ec8011be0 con 0x7f2eb80755a0 2024-09-17T03:37:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2eb80755a0 msgr2=0x7f2eb8077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2eb80755a0 0x7f2eb8077ab0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009b40 tx=0x7f2ec80091e0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 msgr2=0x7f2ed815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 0x7f2ed815a220 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c730 tx=0x7f2ed000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 shutdown_connections 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2eb80755a0 0x7f2eb8077ab0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed815f6b0 0x7f2ed815a220 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed815c5d0 0x7f2ed80afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 --2- 172.21.15.46:0/677104974 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2ed8074300 0x7f2ed8156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 >> 172.21.15.46:0/677104974 conn(0x7f2ed80b2c40 msgr2=0x7f2ed8154a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 shutdown_connections 2024-09-17T03:37:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.364+0000 7f2edce00640 1 -- 172.21.15.46:0/677104974 wait complete. 2024-09-17T03:37:17.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:37:17.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 -- 172.21.15.46:0/3592763895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 msgr2=0x7f07f40b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 --2- 172.21.15.46:0/3592763895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f40b5a80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009900 tx=0x7f07e801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 -- 172.21.15.46:0/3592763895 shutdown_connections 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 --2- 172.21.15.46:0/3592763895 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4074b20 0x7f07f415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 --2- 172.21.15.46:0/3592763895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f40b5a80 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 -- 172.21.15.46:0/3592763895 >> 172.21.15.46:0/3592763895 conn(0x7f07f40708c0 msgr2=0x7f07f4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 -- 172.21.15.46:0/3592763895 shutdown_connections 2024-09-17T03:37:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 -- 172.21.15.46:0/3592763895 wait complete. 2024-09-17T03:37:17.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.552+0000 7f07fc200640 1 Processor -- start 2024-09-17T03:37:17.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- start start 2024-09-17T03:37:17.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f413dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4160d50 0x7f07f415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f414e5c0 0x7f07f415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07f416a7c0 con 0x7f07f4157030 2024-09-17T03:37:17.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07f416a640 con 0x7f07f414e5c0 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f07f416a940 con 0x7f07f4160d50 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f413dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f413dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56290/0 (socket says 172.21.15.46:56290) 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 -- 172.21.15.46:0/2497352273 learned_addr learned my addr 172.21.15.46:0/2497352273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fa200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f414e5c0 0x7f07f415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f8e00640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4160d50 0x7f07f415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 -- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4160d50 msgr2=0x7f07f415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4160d50 0x7f07f415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 -- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f414e5c0 msgr2=0x7f07f415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f414e5c0 0x7f07f415c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 -- 172.21.15.46:0/2497352273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f40b5050 con 0x7f07f4157030 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f8e00640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4160d50 0x7f07f415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f9800640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f413dea0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f07e80b0040 tx=0x7f07e80a5520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:17.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e80bb070 con 0x7f07f4157030 2024-09-17T03:37:17.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07e80043b0 con 0x7f07f4157030 2024-09-17T03:37:17.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f40b1eb0 con 0x7f07f4157030 2024-09-17T03:37:17.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e80b2040 con 0x7f07f4157030 2024-09-17T03:37:17.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f40b2470 con 0x7f07f4157030 2024-09-17T03:37:17.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.556+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f40d2260 con 0x7f07f4157030 2024-09-17T03:37:17.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.560+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) ==== 100447+0+0 (secure 0 0 0) 0x7f07e80be070 con 0x7f07f4157030 2024-09-17T03:37:17.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.560+0000 7f07f1600640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07d4075570 0x7f07d4077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:17.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.560+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f07e812cdb0 con 0x7f07f4157030 2024-09-17T03:37:17.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.560+0000 7f07f8e00640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07d4075570 0x7f07d4077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:17.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.564+0000 7f07f8e00640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07d4075570 0x7f07d4077a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f07f4156400 tx=0x7f07e0008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.564+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f07e80164d0 con 0x7f07f4157030 2024-09-17T03:37:17.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.764+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f07f406b540 con 0x7f07d4075570 2024-09-17T03:37:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.768+0000 7f07f1600640 1 -- 172.21.15.46:0/2497352273 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f07e000a440 con 0x7f07d4075570 2024-09-17T03:37:17.771 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07d4075570 msgr2=0x7f07d4077a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07d4075570 0x7f07d4077a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f07f4156400 tx=0x7f07e0008040 comp rx=0 tx=0).stop 2024-09-17T03:37:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 msgr2=0x7f07f413dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f413dea0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f07e80b0040 tx=0x7f07e80a5520 comp rx=0 tx=0).stop 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 shutdown_connections 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f07d4075570 0x7f07d4077a80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f414e5c0 0x7f07f415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f07f4160d50 0x7f07f415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 --2- 172.21.15.46:0/2497352273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f4157030 0x7f07f413dea0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 >> 172.21.15.46:0/2497352273 conn(0x7f07f40708c0 msgr2=0x7f07f414f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 shutdown_connections 2024-09-17T03:37:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:17.772+0000 7f07fc200640 1 -- 172.21.15.46:0/2497352273 wait complete. 2024-09-17T03:37:17.794 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:17.794 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:37:17.794 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:37:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:17.803 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:37:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:37:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:37:26.404 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.54329 s, 126 MB/s 2024-09-17T03:37:26.405 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:37:26.405 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:37:26.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:26.415 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:37:31.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:37:31.146 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:37:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:37:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:37:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.19955 s, 131 MB/s 2024-09-17T03:37:34.665 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:37:34.666 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:37:34.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:34.676 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T03:37:42.791 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records in 2024-09-17T03:37:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:1024+0 records out 2024-09-17T03:37:42.792 INFO:teuthology.orchestra.run.smithi046.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.05896 s, 133 MB/s 2024-09-17T03:37:42.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T03:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 -- 172.21.15.46:0/825788081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 msgr2=0x7f803415eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 --2- 172.21.15.46:0/825788081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f803415eca0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f801c009900 tx=0x7f801c01c820 comp rx=0 tx=0).stop 2024-09-17T03:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 -- 172.21.15.46:0/825788081 shutdown_connections 2024-09-17T03:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 --2- 172.21.15.46:0/825788081 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80340cf280 0x7f80340cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 --2- 172.21.15.46:0/825788081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f803415eca0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:42.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 -- 172.21.15.46:0/825788081 >> 172.21.15.46:0/825788081 conn(0x7f80340b2040 msgr2=0x7f80340b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 -- 172.21.15.46:0/825788081 shutdown_connections 2024-09-17T03:37:42.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.956+0000 7f8039000640 1 -- 172.21.15.46:0/825788081 wait complete. 2024-09-17T03:37:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 Processor -- start 2024-09-17T03:37:42.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 -- start start 2024-09-17T03:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80340cf280 0x7f8034141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803413e790 0x7f8034139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f8034139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80340b54b0 con 0x7f803415e890 2024-09-17T03:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80340b5330 con 0x7f803413e790 2024-09-17T03:37:42.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8039000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f80340b5630 con 0x7f80340cf280 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f8034139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f8034139ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45662/0 (socket says 172.21.15.46:45662) 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 -- 172.21.15.46:0/3644633345 learned_addr learned my addr 172.21.15.46:0/3644633345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8029400640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803413e790 0x7f8034139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8031e00640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80340cf280 0x7f8034141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 -- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80340cf280 msgr2=0x7f8034141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80340cf280 0x7f8034141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 -- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803413e790 msgr2=0x7f8034139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803413e790 0x7f8034139990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.960+0000 7f8032800640 1 -- 172.21.15.46:0/3644633345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80341388f0 con 0x7f803415e890 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f8032800640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f8034139ed0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f802c00eb70 tx=0x7f802c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004d30 con 0x7f803415e890 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f802c00b570 con 0x7f803415e890 2024-09-17T03:37:42.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8034138bd0 con 0x7f803415e890 2024-09-17T03:37:42.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c00b9c0 con 0x7f803415e890 2024-09-17T03:37:42.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8034139190 con 0x7f803415e890 2024-09-17T03:37:42.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80340cf690 con 0x7f803415e890 2024-09-17T03:37:42.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) ==== 100549+0+0 (secure 0 0 0) 0x7f802c00bb60 con 0x7f803415e890 2024-09-17T03:37:42.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f802aa00640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8010075690 0x7f8010077ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:42.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.964+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f802c094070 con 0x7f803415e890 2024-09-17T03:37:42.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.968+0000 7f8031e00640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8010075690 0x7f8010077ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:42.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.968+0000 7f8031e00640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8010075690 0x7f8010077ba0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8024007fd0 tx=0x7f8024009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:42.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:42.972+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f802c0604a0 con 0x7f803415e890 2024-09-17T03:37:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.172+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f803406b540 con 0x7f8010075690 2024-09-17T03:37:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.184+0000 7f802aa00640 1 -- 172.21.15.46:0/3644633345 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8024001d20 con 0x7f8010075690 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8010075690 msgr2=0x7f8010077ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8010075690 0x7f8010077ba0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8024007fd0 tx=0x7f8024009040 comp rx=0 tx=0).stop 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 msgr2=0x7f8034139ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f8034139ed0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f802c00eb70 tx=0x7f802c00c7a0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 shutdown_connections 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8010075690 0x7f8010077ba0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f803415e890 0x7f8034139ed0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803413e790 0x7f8034139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 --2- 172.21.15.46:0/3644633345 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f80340cf280 0x7f8034141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 >> 172.21.15.46:0/3644633345 conn(0x7f80340b2040 msgr2=0x7f803413db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:43.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 shutdown_connections 2024-09-17T03:37:43.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.188+0000 7f8039000640 1 -- 172.21.15.46:0/3644633345 wait complete. 2024-09-17T03:37:43.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T03:37:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 -- 172.21.15.46:0/3225886844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e50160470 msgr2=0x7f6e50074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/3225886844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e50160470 0x7f6e50074780 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6e500b4f70 tx=0x7f6e4801ccf0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 -- 172.21.15.46:0/3225886844 shutdown_connections 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/3225886844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e50160470 0x7f6e50074780 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/3225886844 >> v2:172.21.15.99:3300/0 conn(0x7f6e50160060 0x7f6e50074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/3225886844 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e501536f0 0x7f6e5015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 -- 172.21.15.46:0/3225886844 >> 172.21.15.46:0/3225886844 conn(0x7f6e500b4560 msgr2=0x7f6e500d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 -- 172.21.15.46:0/3225886844 shutdown_connections 2024-09-17T03:37:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.372+0000 7f6e4fe00640 1 -- 172.21.15.46:0/3225886844 wait complete. 2024-09-17T03:37:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 Processor -- start 2024-09-17T03:37:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 -- start start 2024-09-17T03:37:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 0x7f6e5014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e50160060 0x7f6e5013eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e50160470 0x7f6e501f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50071d80 con 0x7f6e501536f0 2024-09-17T03:37:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50071c00 con 0x7f6e50160060 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4fe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6e50071f00 con 0x7f6e50160470 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 0x7f6e5014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 0x7f6e5014f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45688/0 (socket says 172.21.15.46:45688) 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.376+0000 7f6e4ea00640 1 -- 172.21.15.46:0/1222531673 learned_addr learned my addr 172.21.15.46:0/1222531673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e47e00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e50160060 0x7f6e5013eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4f400640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e50160470 0x7f6e501f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4ea00640 1 -- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e50160470 msgr2=0x7f6e501f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4ea00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e50160470 0x7f6e501f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4ea00640 1 -- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e50160060 msgr2=0x7f6e5013eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4ea00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e50160060 0x7f6e5013eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4ea00640 1 -- 172.21.15.46:0/1222531673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e501578e0 con 0x7f6e501536f0 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4f400640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e50160470 0x7f6e501f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4ea00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 0x7f6e5014f520 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005950 tx=0x7f6e4000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:43.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40007bb0 con 0x7f6e501536f0 2024-09-17T03:37:43.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e40007e40 con 0x7f6e501536f0 2024-09-17T03:37:43.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e50157bc0 con 0x7f6e501536f0 2024-09-17T03:37:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000f690 con 0x7f6e501536f0 2024-09-17T03:37:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501f64d0 con 0x7f6e501536f0 2024-09-17T03:37:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e50072420 con 0x7f6e501536f0 2024-09-17T03:37:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.380+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) ==== 100549+0+0 (secure 0 0 0) 0x7f6e4000f830 con 0x7f6e501536f0 2024-09-17T03:37:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.384+0000 7f6e46a00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e2c0756c0 0x7f6e2c077bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.384+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f6e40093bf0 con 0x7f6e501536f0 2024-09-17T03:37:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.384+0000 7f6e47e00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e2c0756c0 0x7f6e2c077bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.384+0000 7f6e47e00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e2c0756c0 0x7f6e2c077bd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6e38005fd0 tx=0x7f6e38005d10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.388+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6e40014030 con 0x7f6e501536f0 2024-09-17T03:37:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.588+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6e501fa960 con 0x7f6e2c0756c0 2024-09-17T03:37:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e46a00640 1 -- 172.21.15.46:0/1222531673 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6e501fa960 con 0x7f6e2c0756c0 2024-09-17T03:37:43.596 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e2c0756c0 msgr2=0x7f6e2c077bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e2c0756c0 0x7f6e2c077bd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6e38005fd0 tx=0x7f6e38005d10 comp rx=0 tx=0).stop 2024-09-17T03:37:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 msgr2=0x7f6e5014f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 0x7f6e5014f520 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005950 tx=0x7f6e4000cdd0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 shutdown_connections 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6e2c0756c0 0x7f6e2c077bd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6e50160470 0x7f6e501f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e50160060 0x7f6e5013eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 --2- 172.21.15.46:0/1222531673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e501536f0 0x7f6e5014f520 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 >> 172.21.15.46:0/1222531673 conn(0x7f6e500b4560 msgr2=0x7f6e500d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 shutdown_connections 2024-09-17T03:37:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:43.596+0000 7f6e4fe00640 1 -- 172.21.15.46:0/1222531673 wait complete. 2024-09-17T03:37:43.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:43.618 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:43.618 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:37:43.627 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:43.628 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:37:43.628 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:43.629 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:37:45.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:45.630 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:45.630 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:37:45.641 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:45.642 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T03:37:45.642 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:45.642 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T03:37:47.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:37:47.644 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:47.644 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:37:47.655 INFO:teuthology.orchestra.run.smithi046.stdout:# file: volumes/_nogroup/sv1/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:37:47.655 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T03:37:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:37:47.656 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T03:37:47.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone63430 2024-09-17T03:37:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 -- 172.21.15.46:0/1924419556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 msgr2=0x7ff93806dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 --2- 172.21.15.46:0/1924419556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff93806dfd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff9380ced20 tx=0x7ff92401c830 comp rx=0 tx=0).stop 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 -- 172.21.15.46:0/1924419556 shutdown_connections 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 --2- 172.21.15.46:0/1924419556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff93806dfd0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 --2- 172.21.15.46:0/1924419556 >> v2:172.21.15.99:3300/0 conn(0x7ff9380d1140 0x7ff9380b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 -- 172.21.15.46:0/1924419556 >> 172.21.15.46:0/1924419556 conn(0x7ff9380b41f0 msgr2=0x7ff9380b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 -- 172.21.15.46:0/1924419556 shutdown_connections 2024-09-17T03:37:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 -- 172.21.15.46:0/1924419556 wait complete. 2024-09-17T03:37:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 Processor -- start 2024-09-17T03:37:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.880+0000 7ff93ee00640 1 -- start start 2024-09-17T03:37:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff9380d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9380d1140 0x7ff9380d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff93815f5c0 0x7ff93815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9380d0f30 con 0x7ff93806dbc0 2024-09-17T03:37:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9380d0db0 con 0x7ff9380d1140 2024-09-17T03:37:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff938168c60 con 0x7ff93815f5c0 2024-09-17T03:37:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff9380d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff9380d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45698/0 (socket says 172.21.15.46:45698) 2024-09-17T03:37:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 -- 172.21.15.46:0/774308592 learned_addr learned my addr 172.21.15.46:0/774308592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93d000640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9380d1140 0x7ff9380d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:47.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93e400640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff93815f5c0 0x7ff93815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 -- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff93815f5c0 msgr2=0x7ff93815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff93815f5c0 0x7ff93815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 -- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9380d1140 msgr2=0x7ff9380d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9380d1140 0x7ff9380d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 -- 172.21.15.46:0/774308592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9381596f0 con 0x7ff93806dbc0 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93e400640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff93815f5c0 0x7ff93815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93da00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff9380d17a0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00eb00 tx=0x7ff92c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c004b30 con 0x7ff93806dbc0 2024-09-17T03:37:47.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff92c004cf0 con 0x7ff93806dbc0 2024-09-17T03:37:47.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c005280 con 0x7ff93806dbc0 2024-09-17T03:37:47.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93815eb60 con 0x7ff93806dbc0 2024-09-17T03:37:47.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.884+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff938159c70 con 0x7ff93806dbc0 2024-09-17T03:37:47.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.888+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9380d05b0 con 0x7ff93806dbc0 2024-09-17T03:37:47.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.888+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) ==== 100549+0+0 (secure 0 0 0) 0x7ff92c005420 con 0x7ff93806dbc0 2024-09-17T03:37:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.888+0000 7ff935600640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff9180756c0 0x7ff918077bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.888+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7ff92c094110 con 0x7ff93806dbc0 2024-09-17T03:37:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.888+0000 7ff93d000640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff9180756c0 0x7ff918077bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:47.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.888+0000 7ff93d000640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff9180756c0 0x7ff918077bd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff9240af000 tx=0x7ff92401b070 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:47.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:47.892+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff92c060c20 con 0x7ff93806dbc0 2024-09-17T03:37:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.096+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone63430", "target": ["mon-mgr", ""]}) -- 0x7ff93806b540 con 0x7ff9180756c0 2024-09-17T03:37:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff935600640 1 -- 172.21.15.46:0/774308592 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff9240040c0 con 0x7ff9180756c0 2024-09-17T03:37:48.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff9180756c0 msgr2=0x7ff918077bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:48.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff9180756c0 0x7ff918077bd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff9240af000 tx=0x7ff92401b070 comp rx=0 tx=0).stop 2024-09-17T03:37:48.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 msgr2=0x7ff9380d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff9380d17a0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00eb00 tx=0x7ff92c00c2d0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 shutdown_connections 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff9180756c0 0x7ff918077bd0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff93815f5c0 0x7ff93815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9380d1140 0x7ff9380d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 --2- 172.21.15.46:0/774308592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff93806dbc0 0x7ff9380d17a0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 >> 172.21.15.46:0/774308592 conn(0x7ff9380b41f0 msgr2=0x7ff938164d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 shutdown_connections 2024-09-17T03:37:48.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.156+0000 7ff93ee00640 1 -- 172.21.15.46:0/774308592 wait complete. 2024-09-17T03:37:48.160 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.160+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.160 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.160+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.161 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.160+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.161 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.160+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.161 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.160+0000 7fbe53600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.179 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.176+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.179 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.176+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.179 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.176+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.180 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.176+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.180 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:48.176+0000 7fbe54a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:48.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6331 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47f800640 1 -- 172.21.15.46:0/1439157607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 msgr2=0x7fc460004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47f800640 1 --2- 172.21.15.46:0/1439157607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc460004c20 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fc46400bbe0 tx=0x7fc46401ce80 comp rx=0 tx=0).stop 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47d800640 1 -- 172.21.15.46:0/1439157607 reap_dead start 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47f800640 1 -- 172.21.15.46:0/1439157607 shutdown_connections 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47f800640 1 -- 172.21.15.46:0/1439157607 >> 172.21.15.46:0/1439157607 conn(0x7fc4780b2040 msgr2=0x7fc4780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47f800640 1 -- 172.21.15.46:0/1439157607 shutdown_connections 2024-09-17T03:37:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.356+0000 7fc47f800640 1 -- 172.21.15.46:0/1439157607 wait complete. 2024-09-17T03:37:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 Processor -- start 2024-09-17T03:37:48.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 -- start start 2024-09-17T03:37:48.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc4781e3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4781e3cc0 0x7fc4781f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4781f5450 0x7fc4781f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc47815d070 con 0x7fc460004810 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc478152a50 con 0x7fc4781e3cc0 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47f800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc47815d1f0 con 0x7fc4781f5450 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc4781e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc4781e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45732/0 (socket says 172.21.15.46:45732) 2024-09-17T03:37:48.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 -- 172.21.15.46:0/2131566886 learned_addr learned my addr 172.21.15.46:0/2131566886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 -- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4781f5450 msgr2=0x7fc4781f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47d800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4781f5450 0x7fc4781f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc477e00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4781e3cc0 0x7fc4781f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4781f5450 0x7fc4781f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 -- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4781e3cc0 msgr2=0x7fc4781f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4781e3cc0 0x7fc4781f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47ce00640 1 -- 172.21.15.46:0/2131566886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4781f5da0 con 0x7fc460004810 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc47d800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4781f5450 0x7fc4781f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.360+0000 7fc477e00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4781e3cc0 0x7fc4781f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc47ce00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc4781e3780 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fc46c011620 tx=0x7fc46c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c017070 con 0x7fc460004810 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc46c00fb60 con 0x7fc460004810 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4781e44f0 con 0x7fc460004810 2024-09-17T03:37:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c013450 con 0x7fc460004810 2024-09-17T03:37:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4781e4ab0 con 0x7fc460004810 2024-09-17T03:37:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47815d730 con 0x7fc460004810 2024-09-17T03:37:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.364+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) ==== 100549+0+0 (secure 0 0 0) 0x7fc46c0135f0 con 0x7fc460004810 2024-09-17T03:37:48.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.368+0000 7fc474c00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc458075690 0x7fc458077ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:48.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.368+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7fc46c097430 con 0x7fc460004810 2024-09-17T03:37:48.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.368+0000 7fc477e00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc458075690 0x7fc458077ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:48.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.368+0000 7fc477e00640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc458075690 0x7fc458077ba0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc4640b0000 tx=0x7fc464019c20 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:48.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.372+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4780b5730 con 0x7fc460004810 2024-09-17T03:37:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:48.576+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6331", "target": ["mon-mgr", ""]}) -- 0x7fc47806b540 con 0x7fc458075690 2024-09-17T03:37:49.162 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:49.160+0000 7fbdbae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:49.163 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:49.160+0000 7fbdbae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:49.163 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:49.160+0000 7fbdbae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:49.163 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:49.160+0000 7fbdbae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:49.163 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:49.160+0000 7fbdbae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:49.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.252+0000 7fc474c00640 1 -- 172.21.15.46:0/2131566886 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4640051c0 con 0x7fc458075690 2024-09-17T03:37:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc458075690 msgr2=0x7fc458077ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc458075690 0x7fc458077ba0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc4640b0000 tx=0x7fc464019c20 comp rx=0 tx=0).stop 2024-09-17T03:37:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 msgr2=0x7fc4781e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc4781e3780 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fc46c011620 tx=0x7fc46c011ae0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 shutdown_connections 2024-09-17T03:37:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc458075690 0x7fc458077ba0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc4781f5450 0x7fc4781f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4781e3cc0 0x7fc4781f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 --2- 172.21.15.46:0/2131566886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460004810 0x7fc4781e3780 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 >> 172.21.15.46:0/2131566886 conn(0x7fc4780b2040 msgr2=0x7fc4780b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 shutdown_connections 2024-09-17T03:37:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.256+0000 7fc47f800640 1 -- 172.21.15.46:0/2131566886 wait complete. 2024-09-17T03:37:49.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone59742 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 -- 172.21.15.46:0/2564306341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b01674a0 msgr2=0x7f48b01698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 --2- 172.21.15.46:0/2564306341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b01674a0 0x7f48b01698c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009bf0 tx=0x7f48a801cb10 comp rx=0 tx=0).stop 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 -- 172.21.15.46:0/2564306341 shutdown_connections 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 --2- 172.21.15.46:0/2564306341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b01674a0 0x7f48b01698c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 -- 172.21.15.46:0/2564306341 >> 172.21.15.46:0/2564306341 conn(0x7f48b006cdf0 msgr2=0x7f48b006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 -- 172.21.15.46:0/2564306341 shutdown_connections 2024-09-17T03:37:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.628+0000 7f48b6e00640 1 -- 172.21.15.46:0/2564306341 wait complete. 2024-09-17T03:37:49.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.632+0000 7f48b6e00640 1 Processor -- start 2024-09-17T03:37:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.632+0000 7f48b6e00640 1 -- start start 2024-09-17T03:37:49.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48b00aedf0 0x7f48b014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b00b1dc0 0x7f48b0159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b01674a0 0x7f48b015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48b016ceb0 con 0x7f48b01674a0 2024-09-17T03:37:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48b016cd30 con 0x7f48b00b1dc0 2024-09-17T03:37:49.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f48b016d030 con 0x7f48b00aedf0 2024-09-17T03:37:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b01674a0 0x7f48b015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b01674a0 0x7f48b015c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45758/0 (socket says 172.21.15.46:45758) 2024-09-17T03:37:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6400640 1 -- 172.21.15.46:0/3717843484 learned_addr learned my addr 172.21.15.46:0/3717843484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48b00aedf0 0x7f48b014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:49.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5000640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b00b1dc0 0x7f48b0159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 -- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b00b1dc0 msgr2=0x7f48b0159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b00b1dc0 0x7f48b0159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 -- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b01674a0 msgr2=0x7f48b015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b01674a0 0x7f48b015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 -- 172.21.15.46:0/3717843484 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b0159020 con 0x7f48b00aedf0 2024-09-17T03:37:49.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b5a00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48b00aedf0 0x7f48b014daa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c3f0 tx=0x7f48a400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4011790 con 0x7f48b00aedf0 2024-09-17T03:37:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48a4017070 con 0x7f48b00aedf0 2024-09-17T03:37:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4011c60 con 0x7f48b00aedf0 2024-09-17T03:37:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.636+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b014e320 con 0x7f48b00aedf0 2024-09-17T03:37:49.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.640+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b01595c0 con 0x7f48b00aedf0 2024-09-17T03:37:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.640+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b00af200 con 0x7f48b00aedf0 2024-09-17T03:37:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.640+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 54) ==== 100549+0+0 (secure 0 0 0) 0x7f48a401c460 con 0x7f48b00aedf0 2024-09-17T03:37:49.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.644+0000 7f48ad600640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48900756c0 0x7f4890077bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.644+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(143..143 src has 1..143) ==== 5737+0+0 (secure 0 0 0) 0x7f48a4093090 con 0x7f48b00aedf0 2024-09-17T03:37:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.644+0000 7f48b5000640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48900756c0 0x7f4890077bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:49.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.644+0000 7f48b5000640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48900756c0 0x7f4890077bd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f489c000f70 tx=0x7f489c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:49.652+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48a405fb10 con 0x7f48b00aedf0 2024-09-17T03:37:50.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:50.156+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone59742", "target": ["mon-mgr", ""]}) -- 0x7f48b006b540 con 0x7f48900756c0 2024-09-17T03:37:50.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:50.320+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mgrmap(e 55) ==== 100702+0+0 (secure 0 0 0) 0x7f48a405b680 con 0x7f48b00aedf0 2024-09-17T03:37:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.300+0000 7f48ad600640 1 -- 172.21.15.46:0/3717843484 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f489c0012b0 con 0x7f48900756c0 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48900756c0 msgr2=0x7f4890077bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48900756c0 0x7f4890077bd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f489c000f70 tx=0x7f489c009040 comp rx=0 tx=0).stop 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48b00aedf0 msgr2=0x7f48b014daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48b00aedf0 0x7f48b014daa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c3f0 tx=0x7f48a400c8b0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 shutdown_connections 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f48900756c0 0x7f4890077bd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b01674a0 0x7f48b015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48b00b1dc0 0x7f48b0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 --2- 172.21.15.46:0/3717843484 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f48b00aedf0 0x7f48b014daa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 >> 172.21.15.46:0/3717843484 conn(0x7f48b006cdf0 msgr2=0x7f48b00b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 shutdown_connections 2024-09-17T03:37:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.304+0000 7f48b6e00640 1 -- 172.21.15.46:0/3717843484 wait complete. 2024-09-17T03:37:51.310 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:51.308+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:51.310 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:51.308+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:51.310 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:51.308+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:51.310 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:51.308+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:51.310 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:51.308+0000 7fbe54000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:51.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone1673 2024-09-17T03:37:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 -- 172.21.15.46:0/3916617327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006cdf0 msgr2=0x7f151006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 --2- 172.21.15.46:0/3916617327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006cdf0 0x7f151006d200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1500009c80 tx=0x7f150001cd10 comp rx=0 tx=0).stop 2024-09-17T03:37:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 -- 172.21.15.46:0/3916617327 shutdown_connections 2024-09-17T03:37:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 --2- 172.21.15.46:0/3916617327 >> v2:172.21.15.99:3300/0 conn(0x7f151006fc90 0x7f15100700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 --2- 172.21.15.46:0/3916617327 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006cdf0 0x7f151006d200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 -- 172.21.15.46:0/3916617327 >> 172.21.15.46:0/3916617327 conn(0x7f15100b35f0 msgr2=0x7f15100b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 -- 172.21.15.46:0/3916617327 shutdown_connections 2024-09-17T03:37:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.800+0000 7f1516400640 1 -- 172.21.15.46:0/3916617327 wait complete. 2024-09-17T03:37:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.808+0000 7f1516400640 1 Processor -- start 2024-09-17T03:37:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.808+0000 7f1516400640 1 -- start start 2024-09-17T03:37:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f1516400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 0x7f1510164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:51.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f1516400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006fc90 0x7f1510149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f1516400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1510159130 0x7f151014c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f1516400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1510168e40 con 0x7f1510159130 2024-09-17T03:37:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f1516400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1510168cc0 con 0x7f151006cdf0 2024-09-17T03:37:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f1516400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1510168fc0 con 0x7f151006fc90 2024-09-17T03:37:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 0x7f1510164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:51.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 0x7f1510164680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:56404/0 (socket says 172.21.15.46:56404) 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 -- 172.21.15.46:0/4232317183 learned_addr learned my addr 172.21.15.46:0/4232317183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150ea00640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006fc90 0x7f1510149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 -- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006fc90 msgr2=0x7f1510149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006fc90 0x7f1510149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 -- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1510159130 msgr2=0x7f151014c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1510159130 0x7f151014c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.812+0000 7f150f400640 1 -- 172.21.15.46:0/4232317183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151014b780 con 0x7f151006cdf0 2024-09-17T03:37:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.816+0000 7f150f400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 0x7f1510164680 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b3f0 tx=0x7f14fc00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.816+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc015020 con 0x7f151006cdf0 2024-09-17T03:37:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.816+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14fc011940 con 0x7f151006cdf0 2024-09-17T03:37:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.816+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011d90 con 0x7f151006cdf0 2024-09-17T03:37:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.816+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151014ba20 con 0x7f151006cdf0 2024-09-17T03:37:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.816+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151015a660 con 0x7f151006cdf0 2024-09-17T03:37:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.824+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151006d200 con 0x7f151006cdf0 2024-09-17T03:37:51.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.824+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 55) ==== 100702+0+0 (secure 0 0 0) 0x7f14fc018020 con 0x7f151006cdf0 2024-09-17T03:37:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.824+0000 7f1507400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14f0075910 0x7f14f0077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.824+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f14fc093430 con 0x7f151006cdf0 2024-09-17T03:37:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.828+0000 7f150ea00640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14f0075910 0x7f14f0077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.828+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14fc01f020 con 0x7f151006cdf0 2024-09-17T03:37:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:51.832+0000 7f150ea00640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14f0075910 0x7f14f0077e20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f15000af000 tx=0x7f150001b550 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:52.316+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone1673", "target": ["mon-mgr", ""]}) -- 0x7f151006b540 con 0x7f14f0075910 2024-09-17T03:37:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:52.408+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 56) ==== 100753+0+0 (secure 0 0 0) 0x7f14fc05baa0 con 0x7f151006cdf0 2024-09-17T03:37:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.180+0000 7f1507400640 1 -- 172.21.15.46:0/4232317183 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1500004a50 con 0x7f14f0075910 2024-09-17T03:37:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14f0075910 msgr2=0x7f14f0077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14f0075910 0x7f14f0077e20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f15000af000 tx=0x7f150001b550 comp rx=0 tx=0).stop 2024-09-17T03:37:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 msgr2=0x7f1510164680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 0x7f1510164680 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b3f0 tx=0x7f14fc00b8b0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 shutdown_connections 2024-09-17T03:37:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f14f0075910 0x7f14f0077e20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1510159130 0x7f151014c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f151006fc90 0x7f1510149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 --2- 172.21.15.46:0/4232317183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151006cdf0 0x7f1510164680 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.188+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 >> 172.21.15.46:0/4232317183 conn(0x7f15100b35f0 msgr2=0x7f1510068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:53.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.192+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 shutdown_connections 2024-09-17T03:37:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.192+0000 7f1516400640 1 -- 172.21.15.46:0/4232317183 wait complete. 2024-09-17T03:37:53.281 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:53.276+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:53.281 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:53.276+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:53.281 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:53.276+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:53.281 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:53.276+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:53.281 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:37:53.276+0000 7fbe55400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:37:53.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone88214 2024-09-17T03:37:53.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.904+0000 7f34cec00640 1 -- 172.21.15.46:0/1175878077 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34b0003c00 msgr2=0x7f34b0004050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:53.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.904+0000 7f34cec00640 1 --2- 172.21.15.46:0/1175878077 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34b0003c00 0x7f34b0004050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34bc006f70 tx=0x7f34bc01cee0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.908+0000 7f34cec00640 1 -- 172.21.15.46:0/1175878077 shutdown_connections 2024-09-17T03:37:53.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.908+0000 7f34cec00640 1 --2- 172.21.15.46:0/1175878077 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34b0003c00 0x7f34b0004050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.908+0000 7f34cec00640 1 -- 172.21.15.46:0/1175878077 >> 172.21.15.46:0/1175878077 conn(0x7f34c806cdf0 msgr2=0x7f34c806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:53.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.912+0000 7f34cec00640 1 -- 172.21.15.46:0/1175878077 shutdown_connections 2024-09-17T03:37:53.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.912+0000 7f34cec00640 1 -- 172.21.15.46:0/1175878077 wait complete. 2024-09-17T03:37:53.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.920+0000 7f34cec00640 1 Processor -- start 2024-09-17T03:37:53.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.920+0000 7f34cec00640 1 -- start start 2024-09-17T03:37:53.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.924+0000 7f34cec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b0003c00 0x7f34c81ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:53.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.928+0000 7f34cec00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 0x7f34c8205f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:53.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.928+0000 7f34cec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c80d0520 0x7f34c80d0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:53.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.928+0000 7f34cce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 0x7f34c8205f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:53.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.928+0000 7f34cce00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 0x7f34c8205f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:55484/0 (socket says 172.21.15.46:55484) 2024-09-17T03:37:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34cd800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b0003c00 0x7f34c81ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:53.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34cd800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b0003c00 0x7f34c81ff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:42936/0 (socket says 172.21.15.46:42936) 2024-09-17T03:37:53.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34cce00640 1 -- 172.21.15.46:0/1407002470 learned_addr learned my addr 172.21.15.46:0/1407002470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34c80b4cf0 con 0x7f34c80d0520 2024-09-17T03:37:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34ce200640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c80d0520 0x7f34c80d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34c80b4b70 con 0x7f34b0003c00 2024-09-17T03:37:53.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.932+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f34c80b4e70 con 0x7f34c8205b20 2024-09-17T03:37:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.936+0000 7f34cce00640 1 -- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b0003c00 msgr2=0x7f34c81ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.936+0000 7f34cce00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b0003c00 0x7f34c81ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.936+0000 7f34cce00640 1 -- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c80d0520 msgr2=0x7f34c80d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.936+0000 7f34cce00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c80d0520 0x7f34c80d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:53.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.936+0000 7f34cce00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c80d0e70 con 0x7f34c8205b20 2024-09-17T03:37:53.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.944+0000 7f34cce00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 0x7f34c8205f30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34c0010b00 tx=0x7f34c000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.944+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0017070 con 0x7f34c8205b20 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.944+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34c0015070 con 0x7f34c8205b20 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.944+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0003d10 con 0x7f34c8205b20 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.944+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c82068a0 con 0x7f34c8205b20 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.944+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c8207160 con 0x7f34c8205b20 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.948+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 56) ==== 100753+0+0 (secure 0 0 0) 0x7f34c0004030 con 0x7f34c8205b20 2024-09-17T03:37:53.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.952+0000 7f34c5600640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34a8075920 0x7f34a8077e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:53.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.952+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a40052a0 con 0x7f34c8205b20 2024-09-17T03:37:53.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.956+0000 7f34cd800640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34a8075920 0x7f34a8077e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:53.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.956+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f34c005c6b0 con 0x7f34c8205b20 2024-09-17T03:37:53.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.964+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34c80d0e70 con 0x7f34c8205b20 2024-09-17T03:37:53.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:53.968+0000 7f34cd800640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34a8075920 0x7f34a8077e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0ac000 tx=0x7f34bc005040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:54.436+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mgrmap(e 57) ==== 100804+0+0 (secure 0 0 0) 0x7f34c0004030 con 0x7f34c8205b20 2024-09-17T03:37:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:54.560+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone88214", "target": ["mon-mgr", ""]}) -- 0x7f34a4000c90 con 0x7f34a8075920 2024-09-17T03:37:55.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.652+0000 7f34c5600640 1 -- 172.21.15.46:0/1407002470 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34a4000c90 con 0x7f34a8075920 2024-09-17T03:37:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34a8075920 msgr2=0x7f34a8077e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34a8075920 0x7f34a8077e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0ac000 tx=0x7f34bc005040 comp rx=0 tx=0).stop 2024-09-17T03:37:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 msgr2=0x7f34c8205f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:55.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 0x7f34c8205f30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f34c0010b00 tx=0x7f34c000e350 comp rx=0 tx=0).stop 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 shutdown_connections 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f34a8075920 0x7f34a8077e30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34c80d0520 0x7f34c80d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f34c8205b20 0x7f34c8205f30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 --2- 172.21.15.46:0/1407002470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34b0003c00 0x7f34c81ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 >> 172.21.15.46:0/1407002470 conn(0x7f34c806cdf0 msgr2=0x7f34c806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 shutdown_connections 2024-09-17T03:37:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:55.660+0000 7f34cec00640 1 -- 172.21.15.46:0/1407002470 wait complete. 2024-09-17T03:37:55.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99875 2024-09-17T03:37:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.316+0000 7fa975200640 1 -- 172.21.15.46:0/581171293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97015ec80 msgr2=0x7fa97015f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:56.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.316+0000 7fa975200640 1 --2- 172.21.15.46:0/581171293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97015ec80 0x7fa97015f090 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa968009c80 tx=0x7fa96801cd30 comp rx=0 tx=0).stop 2024-09-17T03:37:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.316+0000 7fa975200640 1 -- 172.21.15.46:0/581171293 shutdown_connections 2024-09-17T03:37:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.316+0000 7fa975200640 1 --2- 172.21.15.46:0/581171293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97015ec80 0x7fa97015f090 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.316+0000 7fa975200640 1 --2- 172.21.15.46:0/581171293 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa97006fd70 0x7fa970070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.316+0000 7fa975200640 1 -- 172.21.15.46:0/581171293 >> 172.21.15.46:0/581171293 conn(0x7fa97006e4a0 msgr2=0x7fa97006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.320+0000 7fa975200640 1 -- 172.21.15.46:0/581171293 shutdown_connections 2024-09-17T03:37:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.320+0000 7fa975200640 1 -- 172.21.15.46:0/581171293 wait complete. 2024-09-17T03:37:56.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.324+0000 7fa975200640 1 Processor -- start 2024-09-17T03:37:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.324+0000 7fa975200640 1 -- start start 2024-09-17T03:37:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa975200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97006fd70 0x7fa970150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa975200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9700d0330 0x7fa97014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa975200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97015ec80 0x7fa9701611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa975200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa97016c100 con 0x7fa97006fd70 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa975200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa97016bf80 con 0x7fa97015ec80 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa975200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa97016c280 con 0x7fa9700d0330 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa96f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9700d0330 0x7fa97014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa96f400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9700d0330 0x7fa97014ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:55498/0 (socket says 172.21.15.46:55498) 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa96f400640 1 -- 172.21.15.46:0/663345407 learned_addr learned my addr 172.21.15.46:0/663345407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa96fe00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97015ec80 0x7fa9701611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.332+0000 7fa967e00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97006fd70 0x7fa970150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.336+0000 7fa96fe00640 1 -- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9700d0330 msgr2=0x7fa97014ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.336+0000 7fa96fe00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9700d0330 0x7fa97014ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.336+0000 7fa96fe00640 1 -- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97006fd70 msgr2=0x7fa970150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.336+0000 7fa96fe00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97006fd70 0x7fa970150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:56.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.336+0000 7fa96fe00640 1 -- 172.21.15.46:0/663345407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97006f190 con 0x7fa97015ec80 2024-09-17T03:37:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.336+0000 7fa96fe00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97015ec80 0x7fa9701611f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa9680ac000 tx=0x7fa9680a5bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.340+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9680b2040 con 0x7fa97015ec80 2024-09-17T03:37:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.340+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa968003a70 con 0x7fa97015ec80 2024-09-17T03:37:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.340+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968004030 con 0x7fa97015ec80 2024-09-17T03:37:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.340+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97015c970 con 0x7fa97015ec80 2024-09-17T03:37:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.340+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9700b1c40 con 0x7fa97015ec80 2024-09-17T03:37:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.344+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9480052a0 con 0x7fa97015ec80 2024-09-17T03:37:56.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.348+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 57) ==== 100804+0+0 (secure 0 0 0) 0x7fa9680bd020 con 0x7fa97015ec80 2024-09-17T03:37:56.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.352+0000 7fa967400640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa94c075a30 0x7fa94c077f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:56.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.352+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7fa96812dc00 con 0x7fa97015ec80 2024-09-17T03:37:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.356+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa9680c4020 con 0x7fa97015ec80 2024-09-17T03:37:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.360+0000 7fa967e00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa94c075a30 0x7fa94c077f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.400+0000 7fa967e00640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa94c075a30 0x7fa94c077f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b7f0 tx=0x7fa960008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:56.952+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99875", "target": ["mon-mgr", ""]}) -- 0x7fa948000c90 con 0x7fa94c075a30 2024-09-17T03:37:57.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.948+0000 7fa967400640 1 -- 172.21.15.46:0/663345407 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa948000c90 con 0x7fa94c075a30 2024-09-17T03:37:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa94c075a30 msgr2=0x7fa94c077f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa94c075a30 0x7fa94c077f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b7f0 tx=0x7fa960008040 comp rx=0 tx=0).stop 2024-09-17T03:37:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97015ec80 msgr2=0x7fa9701611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97015ec80 0x7fa9701611f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa9680ac000 tx=0x7fa9680a5bf0 comp rx=0 tx=0).stop 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 shutdown_connections 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa94c075a30 0x7fa94c077f40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97015ec80 0x7fa9701611f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9700d0330 0x7fa97014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 --2- 172.21.15.46:0/663345407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa97006fd70 0x7fa970150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.960+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 >> 172.21.15.46:0/663345407 conn(0x7fa97006e4a0 msgr2=0x7fa970155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.964+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 shutdown_connections 2024-09-17T03:37:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:57.964+0000 7fa975200640 1 -- 172.21.15.46:0/663345407 wait complete. 2024-09-17T03:37:58.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone17366 2024-09-17T03:37:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26ee400640 1 -- 172.21.15.46:0/299609383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26d0004d50 msgr2=0x7f26d0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26ee400640 1 --2- 172.21.15.46:0/299609383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26d0004d50 0x7f26d0005160 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009ee0 tx=0x7f26e001d610 comp rx=0 tx=0).stop 2024-09-17T03:37:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26eda00640 1 -- 172.21.15.46:0/299609383 reap_dead start 2024-09-17T03:37:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26ee400640 1 -- 172.21.15.46:0/299609383 shutdown_connections 2024-09-17T03:37:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26ee400640 1 -- 172.21.15.46:0/299609383 >> 172.21.15.46:0/299609383 conn(0x7f26e8073a10 msgr2=0x7f26e806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:37:58.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26ee400640 1 -- 172.21.15.46:0/299609383 shutdown_connections 2024-09-17T03:37:58.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.712+0000 7f26ee400640 1 -- 172.21.15.46:0/299609383 wait complete. 2024-09-17T03:37:58.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.720+0000 7f26ee400640 1 Processor -- start 2024-09-17T03:37:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.720+0000 7f26ee400640 1 -- start start 2024-09-17T03:37:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.724+0000 7f26ee400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80b4610 0x7f26e81532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.724+0000 7f26ee400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8142430 0x7f26e8142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.724+0000 7f26ee400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26e8142d80 0x7f26e81fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.724+0000 7f26ee400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80b3760 con 0x7f26e80b4610 2024-09-17T03:37:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.724+0000 7f26ee400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80b35e0 con 0x7f26e8142430 2024-09-17T03:37:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.724+0000 7f26ee400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f26e80b38e0 con 0x7f26e8142d80 2024-09-17T03:37:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80b4610 0x7f26e81532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80b4610 0x7f26e81532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42426/0 (socket says 172.21.15.46:42426) 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26ed000640 1 -- 172.21.15.46:0/2615935865 learned_addr learned my addr 172.21.15.46:0/2615935865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26e8142d80 0x7f26e81fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26e7e00640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8142430 0x7f26e8142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 -- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8142430 msgr2=0x7f26e8142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8142430 0x7f26e8142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 -- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80b4610 msgr2=0x7f26e81532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80b4610 0x7f26e81532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:37:58.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 -- 172.21.15.46:0/2615935865 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e81537f0 con 0x7f26e8142d80 2024-09-17T03:37:58.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.728+0000 7f26eda00640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26e8142d80 0x7f26e81fd680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f26e00ac000 tx=0x7f26e0009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.732+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00ad070 con 0x7f26e8142d80 2024-09-17T03:37:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.736+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26e00a7ab0 con 0x7f26e8142d80 2024-09-17T03:37:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.736+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00ad070 con 0x7f26e8142d80 2024-09-17T03:37:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.736+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81fb040 con 0x7f26e8142d80 2024-09-17T03:37:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.736+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81fa560 con 0x7f26e8142d80 2024-09-17T03:37:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.744+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e80d10f0 con 0x7f26e8142d80 2024-09-17T03:37:58.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.764+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 57) ==== 100804+0+0 (secure 0 0 0) 0x7f26e00b0070 con 0x7f26e8142d80 2024-09-17T03:37:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.768+0000 7f26e4c00640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26c8075b60 0x7f26c8078070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:37:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.768+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f26e012d1e0 con 0x7f26e8142d80 2024-09-17T03:37:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.768+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26e81537f0 con 0x7f26e8142d80 2024-09-17T03:37:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.800+0000 7f26ed000640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26c8075b60 0x7f26c8078070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:37:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:58.820+0000 7f26ed000640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26c8075b60 0x7f26c8078070 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f26e80733e0 tx=0x7f26dc0093d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:37:59.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:37:59.416+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone17366", "target": ["mon-mgr", ""]}) -- 0x7f26e806b540 con 0x7f26c8075b60 2024-09-17T03:38:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.284+0000 7f26e4c00640 1 -- 172.21.15.46:0/2615935865 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26dc009c40 con 0x7f26c8075b60 2024-09-17T03:38:00.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.292+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26c8075b60 msgr2=0x7f26c8078070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26c8075b60 0x7f26c8078070 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f26e80733e0 tx=0x7f26dc0093d0 comp rx=0 tx=0).stop 2024-09-17T03:38:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26e8142d80 msgr2=0x7f26e81fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26e8142d80 0x7f26e81fd680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f26e00ac000 tx=0x7f26e0009480 comp rx=0 tx=0).stop 2024-09-17T03:38:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 shutdown_connections 2024-09-17T03:38:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f26c8075b60 0x7f26c8078070 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f26e8142d80 0x7f26e81fd680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26e8142430 0x7f26e8142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 --2- 172.21.15.46:0/2615935865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80b4610 0x7f26e81532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 >> 172.21.15.46:0/2615935865 conn(0x7f26e8073a10 msgr2=0x7f26e8162a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:00.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 shutdown_connections 2024-09-17T03:38:00.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:00.296+0000 7f26ee400640 1 -- 172.21.15.46:0/2615935865 wait complete. 2024-09-17T03:38:01.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:38:01.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:38:01.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone63430 2024-09-17T03:38:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.948+0000 7f2064000640 1 -- 172.21.15.46:0/2012303386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c0b5520 msgr2=0x7f205c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:01.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.948+0000 7f2064000640 1 --2- 172.21.15.46:0/2012303386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c0b5520 0x7f205c15b730 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f205c06de90 tx=0x7f204801c770 comp rx=0 tx=0).stop 2024-09-17T03:38:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.952+0000 7f2064000640 1 -- 172.21.15.46:0/2012303386 shutdown_connections 2024-09-17T03:38:01.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.956+0000 7f2064000640 1 --2- 172.21.15.46:0/2012303386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c0b5520 0x7f205c15b730 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.956+0000 7f2064000640 1 --2- 172.21.15.46:0/2012303386 >> v2:172.21.15.99:3300/0 conn(0x7f205c0b2070 0x7f205c0b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:01.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.956+0000 7f2064000640 1 --2- 172.21.15.46:0/2012303386 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c070290 0x7f205c15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.956+0000 7f2064000640 1 -- 172.21.15.46:0/2012303386 >> 172.21.15.46:0/2012303386 conn(0x7f205c06e2d0 msgr2=0x7f205c06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.956+0000 7f2064000640 1 -- 172.21.15.46:0/2012303386 shutdown_connections 2024-09-17T03:38:01.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.960+0000 7f2064000640 1 -- 172.21.15.46:0/2012303386 wait complete. 2024-09-17T03:38:01.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.972+0000 7f2064000640 1 Processor -- start 2024-09-17T03:38:01.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.972+0000 7f2064000640 1 -- start start 2024-09-17T03:38:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2064000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c070290 0x7f205c143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2064000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c0b2070 0x7f205c1fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2064000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 0x7f205c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2064000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f205c06cf70 con 0x7f205c070290 2024-09-17T03:38:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2064000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f205c06cdf0 con 0x7f205c0b5520 2024-09-17T03:38:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2064000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f205c06d0f0 con 0x7f205c0b2070 2024-09-17T03:38:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2062000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 0x7f205c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2062000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 0x7f205c150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43004/0 (socket says 172.21.15.46:43004) 2024-09-17T03:38:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2062000640 1 -- 172.21.15.46:0/2144491094 learned_addr learned my addr 172.21.15.46:0/2144491094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:01.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.976+0000 7f2062000640 1 -- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c0b2070 msgr2=0x7f205c1fc680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:38:01.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2061600640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c070290 0x7f205c143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2060c00640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c0b2070 0x7f205c1fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2062000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c0b2070 0x7f205c1fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2062000640 1 -- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c070290 msgr2=0x7f205c143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2062000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c070290 0x7f205c143410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:01.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2062000640 1 -- 172.21.15.46:0/2144491094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c143950 con 0x7f205c0b5520 2024-09-17T03:38:01.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2061600640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c070290 0x7f205c143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:01.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.980+0000 7f2060c00640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c0b2070 0x7f205c1fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.984+0000 7f2062000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 0x7f205c150b90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f205400b970 tx=0x7f205400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:01.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.992+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054019070 con 0x7f205c0b5520 2024-09-17T03:38:01.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.992+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f205400cc50 con 0x7f205c0b5520 2024-09-17T03:38:01.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.992+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20540146a0 con 0x7f205c0b5520 2024-09-17T03:38:01.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.992+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c142370 con 0x7f205c0b5520 2024-09-17T03:38:01.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.992+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c142930 con 0x7f205c0b5520 2024-09-17T03:38:01.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.996+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c1420e0 con 0x7f205c0b5520 2024-09-17T03:38:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:01.996+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 57) ==== 100804+0+0 (secure 0 0 0) 0x7f205401c070 con 0x7f205c0b5520 2024-09-17T03:38:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:02.000+0000 7f2059600640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f203c075960 0x7f203c077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:02.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:02.000+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f20540932e0 con 0x7f205c0b5520 2024-09-17T03:38:02.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:02.004+0000 7f2061600640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f203c075960 0x7f203c077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:02.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:02.008+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f205405fe00 con 0x7f205c0b5520 2024-09-17T03:38:02.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:02.008+0000 7f2061600640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f203c075960 0x7f203c077e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f205c06df10 tx=0x7f20500091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:02.692+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone63430", "target": ["mon-mgr", ""]}) -- 0x7f205c06b540 con 0x7f203c075960 2024-09-17T03:38:02.771 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:02.768+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:02.771 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:02.768+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:02.771 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:02.768+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:02.772 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:02.768+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:02.772 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:02.768+0000 7fbe4dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.068+0000 7f2059600640 1 -- 172.21.15.46:0/2144491094 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2050010ab0 con 0x7f203c075960 2024-09-17T03:38:04.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f203c075960 msgr2=0x7f203c077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:04.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f203c075960 0x7f203c077e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f205c06df10 tx=0x7f20500091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 msgr2=0x7f205c150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:04.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 0x7f205c150b90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f205400b970 tx=0x7f205400be30 comp rx=0 tx=0).stop 2024-09-17T03:38:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 shutdown_connections 2024-09-17T03:38:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f203c075960 0x7f203c077e70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f205c0b5520 0x7f205c150b90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f205c0b2070 0x7f205c1fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 --2- 172.21.15.46:0/2144491094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f205c070290 0x7f205c143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.080+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 >> 172.21.15.46:0/2144491094 conn(0x7f205c06e2d0 msgr2=0x7f205c15e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:04.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.084+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 shutdown_connections 2024-09-17T03:38:04.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.084+0000 7f2064000640 1 -- 172.21.15.46:0/2144491094 wait complete. 2024-09-17T03:38:04.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6331 2024-09-17T03:38:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.604+0000 7f54f6000640 1 -- 172.21.15.46:0/733660521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f00cd970 msgr2=0x7f54f00b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.604+0000 7f54f6000640 1 --2- 172.21.15.46:0/733660521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f00cd970 0x7f54f00b3340 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0093a0 tx=0x7f54dc0207d0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.608+0000 7f54f6000640 1 -- 172.21.15.46:0/733660521 shutdown_connections 2024-09-17T03:38:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.608+0000 7f54f6000640 1 --2- 172.21.15.46:0/733660521 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54f00cdd80 0x7f54f00b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.608+0000 7f54f6000640 1 --2- 172.21.15.46:0/733660521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f00cd970 0x7f54f00b3340 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.608+0000 7f54f6000640 1 -- 172.21.15.46:0/733660521 >> 172.21.15.46:0/733660521 conn(0x7f54f00b2040 msgr2=0x7f54f00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.608+0000 7f54f6000640 1 -- 172.21.15.46:0/733660521 shutdown_connections 2024-09-17T03:38:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.608+0000 7f54f6000640 1 -- 172.21.15.46:0/733660521 wait complete. 2024-09-17T03:38:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.612+0000 7f54f6000640 1 Processor -- start 2024-09-17T03:38:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.612+0000 7f54f6000640 1 -- start start 2024-09-17T03:38:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f00cd970 0x7f54f0149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54f6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f00cdd80 0x7f54f0155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54f6000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54f015aa50 0x7f54f0209070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54f6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0168190 con 0x7f54f00cd970 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54f6000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0168010 con 0x7f54f00cdd80 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54f6000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f54f0168310 con 0x7f54f015aa50 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54eea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f00cdd80 0x7f54f0155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54eea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f00cdd80 0x7f54f0155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:42710/0 (socket says 172.21.15.46:42710) 2024-09-17T03:38:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54eea00640 1 -- 172.21.15.46:0/1805540477 learned_addr learned my addr 172.21.15.46:0/1805540477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54efe00640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54f015aa50 0x7f54f0209070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54efe00640 1 -- 172.21.15.46:0/1805540477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f00cdd80 msgr2=0x7f54f0155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54efe00640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f00cdd80 0x7f54f0155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.616+0000 7f54efe00640 1 -- 172.21.15.46:0/1805540477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f00cd970 msgr2=0x7f54f0149d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:38:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.624+0000 7f54efe00640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f00cd970 0x7f54f0149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.624+0000 7f54efe00640 1 -- 172.21.15.46:0/1805540477 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f0156a00 con 0x7f54f015aa50 2024-09-17T03:38:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.628+0000 7f54efe00640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f54f015aa50 0x7f54f0209070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f54e80079f0 tx=0x7f54e8007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.628+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8017030 con 0x7f54f015aa50 2024-09-17T03:38:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.632+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e8004b30 con 0x7f54f015aa50 2024-09-17T03:38:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.632+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e800d9d0 con 0x7f54f015aa50 2024-09-17T03:38:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.632+0000 7f54f6000640 1 -- 172.21.15.46:0/1805540477 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f0156ce0 con 0x7f54f015aa50 2024-09-17T03:38:04.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.632+0000 7f54f6000640 1 -- 172.21.15.46:0/1805540477 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01572a0 con 0x7f54f015aa50 2024-09-17T03:38:04.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.636+0000 7f54f6000640 1 -- 172.21.15.46:0/1805540477 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f0156750 con 0x7f54f015aa50 2024-09-17T03:38:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.640+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 57) ==== 100804+0+0 (secure 0 0 0) 0x7f54e801a070 con 0x7f54f015aa50 2024-09-17T03:38:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.644+0000 7f54e7400640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54d0075960 0x7f54d0077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.644+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f54e8093840 con 0x7f54f015aa50 2024-09-17T03:38:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.648+0000 7f54ef400640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54d0075960 0x7f54d0077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.648+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f54e80603e0 con 0x7f54f015aa50 2024-09-17T03:38:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.652+0000 7f54ef400640 1 --2- 172.21.15.46:0/1805540477 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f54d0075960 0x7f54d0077e70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009670 tx=0x7f54e0006bd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:04.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:04.756+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f54e8012070 con 0x7f54f015aa50 2024-09-17T03:38:05.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:05.200+0000 7f54f6000640 1 -- 172.21.15.46:0/1805540477 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6331", "target": ["mon-mgr", ""]}) -- 0x7f54f006b540 con 0x7f54d0075960 2024-09-17T03:38:06.188 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:06.188+0000 7fbe4dc00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T03:38:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T03:38:06.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.188+0000 7f54e7400640 1 -- 172.21.15.46:0/1805540477 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f54e0013430 con 0x7f54d0075960 2024-09-17T03:38:06.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T03:38:06.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone59742 2024-09-17T03:38:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.648+0000 7fecb6a00640 1 -- 172.21.15.46:0/1228653463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8006c60 con 0x7fec9c003a10 2024-09-17T03:38:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.648+0000 7fecbf000640 1 -- 172.21.15.46:0/1228653463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 msgr2=0x7fec9c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.648+0000 7fecbf000640 1 --2- 172.21.15.46:0/1228653463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 0x7fec9c003e80 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7feca800c310 tx=0x7feca801cf70 comp rx=0 tx=0).stop 2024-09-17T03:38:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.648+0000 7fecbd000640 1 -- 172.21.15.46:0/1228653463 reap_dead start 2024-09-17T03:38:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.648+0000 7fecbf000640 1 -- 172.21.15.46:0/1228653463 shutdown_connections 2024-09-17T03:38:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.648+0000 7fecbf000640 1 -- 172.21.15.46:0/1228653463 >> 172.21.15.46:0/1228653463 conn(0x7fecb806e1d0 msgr2=0x7fecb806e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.652+0000 7fecbf000640 1 -- 172.21.15.46:0/1228653463 shutdown_connections 2024-09-17T03:38:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.652+0000 7fecbf000640 1 -- 172.21.15.46:0/1228653463 wait complete. 2024-09-17T03:38:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.656+0000 7fecbf000640 1 Processor -- start 2024-09-17T03:38:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.656+0000 7fecbf000640 1 -- start start 2024-09-17T03:38:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbf000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 0x7fecb8073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbf000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecb814c9a0 0x7fecb814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbf000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 0x7fecb8200180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbf000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80cc850 con 0x7fec9c003a10 2024-09-17T03:38:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbf000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80cc6d0 con 0x7fecb814c3c0 2024-09-17T03:38:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbf000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80cc9d0 con 0x7fecb814c9a0 2024-09-17T03:38:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbd000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 0x7fecb8200180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbd000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 0x7fecb8200180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:42726/0 (socket says 172.21.15.46:42726) 2024-09-17T03:38:06.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.664+0000 7fecbd000640 1 -- 172.21.15.46:0/568970845 learned_addr learned my addr 172.21.15.46:0/568970845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecb7400640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecb814c9a0 0x7fecb814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecb7e00640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 0x7fecb8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbd000640 1 -- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecb814c9a0 msgr2=0x7fecb814bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbd000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecb814c9a0 0x7fecb814bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbd000640 1 -- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 msgr2=0x7fecb8073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbd000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 0x7fecb8073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbd000640 1 -- 172.21.15.46:0/568970845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb81508f0 con 0x7fecb814c3c0 2024-09-17T03:38:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbd000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 0x7fecb8200180 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fecb80ce800 tx=0x7fecb000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0004b30 con 0x7fecb814c3c0 2024-09-17T03:38:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb0004dc0 con 0x7fecb814c3c0 2024-09-17T03:38:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000cca0 con 0x7fecb814c3c0 2024-09-17T03:38:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.668+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb814c1b0 con 0x7fecb814c3c0 2024-09-17T03:38:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.672+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb8160350 con 0x7fecb814c3c0 2024-09-17T03:38:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.676+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb80ccea0 con 0x7fecb814c3c0 2024-09-17T03:38:06.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.688+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7fecb001b030 con 0x7fecb814c3c0 2024-09-17T03:38:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.688+0000 7fecafe00640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fec98075a20 0x7fec98077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.692+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7fecb0093830 con 0x7fecb814c3c0 2024-09-17T03:38:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.692+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fecb0022060 con 0x7fecb814c3c0 2024-09-17T03:38:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.692+0000 7fecb7e00640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fec98075a20 0x7fec98077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:06.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:06.744+0000 7fecb7e00640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fec98075a20 0x7fec98077f30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7feca80ac000 tx=0x7feca8019db0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:07.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:07.316+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone59742", "target": ["mon-mgr", ""]}) -- 0x7fecb806b540 con 0x7fec98075a20 2024-09-17T03:38:08.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.424+0000 7fecafe00640 1 -- 172.21.15.46:0/568970845 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feca8001660 con 0x7fec98075a20 2024-09-17T03:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fec98075a20 msgr2=0x7fec98077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fec98075a20 0x7fec98077f30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7feca80ac000 tx=0x7feca8019db0 comp rx=0 tx=0).stop 2024-09-17T03:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 msgr2=0x7fecb8200180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 0x7fecb8200180 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fecb80ce800 tx=0x7fecb000ed80 comp rx=0 tx=0).stop 2024-09-17T03:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 shutdown_connections 2024-09-17T03:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fec98075a20 0x7fec98077f30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb814c3c0 0x7fecb8200180 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fecb814c9a0 0x7fecb814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 --2- 172.21.15.46:0/568970845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec9c003a10 0x7fecb8073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.448+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 >> 172.21.15.46:0/568970845 conn(0x7fecb806e1d0 msgr2=0x7fecb8152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.452+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 shutdown_connections 2024-09-17T03:38:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:08.452+0000 7fecbf000640 1 -- 172.21.15.46:0/568970845 wait complete. 2024-09-17T03:38:08.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1673 2024-09-17T03:38:09.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.296+0000 7f3f76c00640 1 -- 172.21.15.46:0/3204562320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f68036a20 msgr2=0x7f3f68009bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:09.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.296+0000 7f3f76c00640 1 --2- 172.21.15.46:0/3204562320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f68036a20 0x7f3f68009bb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3f70000b70 tx=0x7f3f700988f0 comp rx=0 tx=0).stop 2024-09-17T03:38:09.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.296+0000 7f3f76c00640 1 -- 172.21.15.46:0/3204562320 shutdown_connections 2024-09-17T03:38:09.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.296+0000 7f3f76c00640 1 --2- 172.21.15.46:0/3204562320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f68036a20 0x7f3f68009bb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:09.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.296+0000 7f3f76c00640 1 -- 172.21.15.46:0/3204562320 >> 172.21.15.46:0/3204562320 conn(0x7f3f680513f0 msgr2=0x7f3f68051830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.300+0000 7f3f76c00640 1 -- 172.21.15.46:0/3204562320 shutdown_connections 2024-09-17T03:38:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.300+0000 7f3f76c00640 1 -- 172.21.15.46:0/3204562320 wait complete. 2024-09-17T03:38:09.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.308+0000 7f3f76c00640 1 Processor -- start 2024-09-17T03:38:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.312+0000 7f3f76c00640 1 -- start start 2024-09-17T03:38:09.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.312+0000 7f3f76c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f680062e0 0x7f3f680ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:09.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.312+0000 7f3f76c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 0x7f3f680de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f6fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f680062e0 0x7f3f680ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f6fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f680062e0 0x7f3f680ddb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:42748/0 (socket says 172.21.15.46:42748) 2024-09-17T03:38:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 0x7f3f680de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f6f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 0x7f3f680de1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43170/0 (socket says 172.21.15.46:43170) 2024-09-17T03:38:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f6fe00640 1 -- 172.21.15.46:0/2165578169 learned_addr learned my addr 172.21.15.46:0/2165578169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:09.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f680e2950 0x7f3f68027710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f680ed5f0 con 0x7f3f6800adc0 2024-09-17T03:38:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f680076c0 con 0x7f3f680062e0 2024-09-17T03:38:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3f680ed770 con 0x7f3f680e2950 2024-09-17T03:38:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.316+0000 7f3f74c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f680e2950 0x7f3f68027710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.324+0000 7f3f6f400640 1 -- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f680e2950 msgr2=0x7f3f68027710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.324+0000 7f3f6f400640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f680e2950 0x7f3f68027710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.324+0000 7f3f6f400640 1 -- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f680062e0 msgr2=0x7f3f680ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.324+0000 7f3f6f400640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f680062e0 0x7f3f680ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:09.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.324+0000 7f3f6f400640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f680e3130 con 0x7f3f6800adc0 2024-09-17T03:38:09.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.324+0000 7f3f6f400640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 0x7f3f680de1c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b3e0 tx=0x7f3f6400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.328+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64015020 con 0x7f3f6800adc0 2024-09-17T03:38:09.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.332+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f680e3410 con 0x7f3f6800adc0 2024-09-17T03:38:09.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.332+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f64011920 con 0x7f3f6800adc0 2024-09-17T03:38:09.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.336+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f68026f10 con 0x7f3f6800adc0 2024-09-17T03:38:09.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.336+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64011db0 con 0x7f3f6800adc0 2024-09-17T03:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.336+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4c0052a0 con 0x7f3f6800adc0 2024-09-17T03:38:09.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.344+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f3f64018020 con 0x7f3f6800adc0 2024-09-17T03:38:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.348+0000 7f3f63e00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f500759f0 0x7f3f50077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:09.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.348+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f3f64093600 con 0x7f3f6800adc0 2024-09-17T03:38:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.368+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f6401f020 con 0x7f3f6800adc0 2024-09-17T03:38:09.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.368+0000 7f3f6fe00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f500759f0 0x7f3f50077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:09.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:09.388+0000 7f3f6fe00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f500759f0 0x7f3f50077f00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c007ac0 tx=0x7f3f5c007a10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:10.052+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone1673", "target": ["mon-mgr", ""]}) -- 0x7f3f4c000bd0 con 0x7f3f500759f0 2024-09-17T03:38:11.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.848+0000 7f3f63e00640 1 -- 172.21.15.46:0/2165578169 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f4c000bd0 con 0x7f3f500759f0 2024-09-17T03:38:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f500759f0 msgr2=0x7f3f50077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:11.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f500759f0 0x7f3f50077f00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c007ac0 tx=0x7f3f5c007a10 comp rx=0 tx=0).stop 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 msgr2=0x7f3f680de1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 0x7f3f680de1c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b3e0 tx=0x7f3f6400b8a0 comp rx=0 tx=0).stop 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 shutdown_connections 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3f500759f0 0x7f3f50077f00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3f680e2950 0x7f3f68027710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f6800adc0 0x7f3f680de1c0 secure :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b3e0 tx=0x7f3f6400b8a0 comp rx=0 tx=0).stop 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 --2- 172.21.15.46:0/2165578169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f680062e0 0x7f3f680ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.860+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 >> 172.21.15.46:0/2165578169 conn(0x7f3f680513f0 msgr2=0x7f3f68004950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.864+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 shutdown_connections 2024-09-17T03:38:11.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:11.864+0000 7f3f76c00640 1 -- 172.21.15.46:0/2165578169 wait complete. 2024-09-17T03:38:11.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone88214 2024-09-17T03:38:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.464+0000 7f861be00640 1 -- 172.21.15.46:0/2795859393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 msgr2=0x7f86140d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.464+0000 7f861be00640 1 --2- 172.21.15.46:0/2795859393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86140d1430 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f86140d2110 tx=0x7f860801c6e0 comp rx=0 tx=0).stop 2024-09-17T03:38:12.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.464+0000 7f861be00640 1 -- 172.21.15.46:0/2795859393 shutdown_connections 2024-09-17T03:38:12.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.464+0000 7f861be00640 1 --2- 172.21.15.46:0/2795859393 >> v2:172.21.15.46:3300/0 conn(0x7f86140d2300 0x7f86140d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:12.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.464+0000 7f861be00640 1 --2- 172.21.15.46:0/2795859393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86140d1430 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.464+0000 7f861be00640 1 -- 172.21.15.46:0/2795859393 >> 172.21.15.46:0/2795859393 conn(0x7f8614073da0 msgr2=0x7f86140b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:12.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.468+0000 7f861be00640 1 -- 172.21.15.46:0/2795859393 shutdown_connections 2024-09-17T03:38:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.472+0000 7f861be00640 1 -- 172.21.15.46:0/2795859393 wait complete. 2024-09-17T03:38:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.476+0000 7f861be00640 1 Processor -- start 2024-09-17T03:38:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.480+0000 7f861be00640 1 -- start start 2024-09-17T03:38:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.480+0000 7f861be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86141fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f861be00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 0x7f8614150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f861be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86140d3e70 0x7f8614071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8619400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86141fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8618a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 0x7f8614150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8618a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 0x7f8614150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3301/0 says I am v2:172.21.15.46:52734/0 (socket says 172.21.15.46:52734) 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8619400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86141fd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:50702/0 (socket says 172.21.15.46:50702) 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8618a00640 1 -- 172.21.15.46:0/2561860074 learned_addr learned my addr 172.21.15.46:0/2561860074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f861406f4d0 con 0x7f86140d3e70 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f861406f350 con 0x7f86140d1020 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f861406f650 con 0x7f86140d2300 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8619e00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86140d3e70 0x7f8614071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8618a00640 1 -- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 msgr2=0x7f86141fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8618a00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86141fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.484+0000 7f8618a00640 1 -- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86140d3e70 msgr2=0x7f8614071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f8618a00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86140d3e70 0x7f8614071fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f8618a00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86141543d0 con 0x7f86140d2300 2024-09-17T03:38:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f8618a00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 0x7f8614150190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f86080ac040 tx=0x7f86080aa4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608005380 con 0x7f86140d2300 2024-09-17T03:38:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86080dc070 con 0x7f86140d2300 2024-09-17T03:38:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86141546b0 con 0x7f86140d2300 2024-09-17T03:38:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8614154c70 con 0x7f86140d2300 2024-09-17T03:38:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.488+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080aa990 con 0x7f86140d2300 2024-09-17T03:38:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.492+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f00052a0 con 0x7f86140d2300 2024-09-17T03:38:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.492+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f86080aab30 con 0x7f86140d2300 2024-09-17T03:38:12.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.496+0000 7f8611600640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85f8075a20 0x7f85f8077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.496+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f860814fcd0 con 0x7f86140d2300 2024-09-17T03:38:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.500+0000 7f8619400640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85f8075a20 0x7f85f8077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.512+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f86080163f0 con 0x7f86140d2300 2024-09-17T03:38:12.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:12.556+0000 7f8619400640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85f8075a20 0x7f85f8077f30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8604007d70 tx=0x7f8604007d00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.076+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone88214", "target": ["mon-mgr", ""]}) -- 0x7f85f0000bd0 con 0x7f85f8075a20 2024-09-17T03:38:13.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.940+0000 7f8611600640 1 -- 172.21.15.46:0/2561860074 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85f0000bd0 con 0x7f85f8075a20 2024-09-17T03:38:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85f8075a20 msgr2=0x7f85f8077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85f8075a20 0x7f85f8077f30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8604007d70 tx=0x7f8604007d00 comp rx=0 tx=0).stop 2024-09-17T03:38:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 msgr2=0x7f8614150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 0x7f8614150190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f86080ac040 tx=0x7f86080aa4d0 comp rx=0 tx=0).stop 2024-09-17T03:38:13.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 shutdown_connections 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f85f8075a20 0x7f85f8077f30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86140d3e70 0x7f8614071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f86140d2300 0x7f8614150190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 --2- 172.21.15.46:0/2561860074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86140d1020 0x7f86141fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 >> 172.21.15.46:0/2561860074 conn(0x7f8614073da0 msgr2=0x7f86140b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 shutdown_connections 2024-09-17T03:38:13.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:13.948+0000 7f861be00640 1 -- 172.21.15.46:0/2561860074 wait complete. 2024-09-17T03:38:14.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99875 2024-09-17T03:38:14.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 -- 172.21.15.46:0/3512235408 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc0cf620 msgr2=0x7f40dc15b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:14.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 --2- 172.21.15.46:0/3512235408 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc0cf620 0x7f40dc15b2d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f40dc075770 tx=0x7f40d401cd80 comp rx=0 tx=0).stop 2024-09-17T03:38:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 -- 172.21.15.46:0/3512235408 shutdown_connections 2024-09-17T03:38:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 --2- 172.21.15.46:0/3512235408 >> v2:172.21.15.99:3300/0 conn(0x7f40dc15b810 0x7f40dc15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 --2- 172.21.15.46:0/3512235408 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc0cf620 0x7f40dc15b2d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 --2- 172.21.15.46:0/3512235408 >> v2:172.21.15.46:3300/0 conn(0x7f40dc15eeb0 0x7f40dc0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 -- 172.21.15.46:0/3512235408 >> 172.21.15.46:0/3512235408 conn(0x7f40dc0b2040 msgr2=0x7f40dc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 -- 172.21.15.46:0/3512235408 shutdown_connections 2024-09-17T03:38:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.428+0000 7f40e1e00640 1 -- 172.21.15.46:0/3512235408 wait complete. 2024-09-17T03:38:14.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.432+0000 7f40e1e00640 1 Processor -- start 2024-09-17T03:38:14.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.432+0000 7f40e1e00640 1 -- start start 2024-09-17T03:38:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 0x7f40dc1f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40e1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc15b810 0x7f40dc15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40e1e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc15eeb0 0x7f40dc165e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40e1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc0b4de0 con 0x7f40dc0cf620 2024-09-17T03:38:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40e1e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc15e400 con 0x7f40dc15b810 2024-09-17T03:38:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40e1e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc0b4f60 con 0x7f40dc15eeb0 2024-09-17T03:38:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 0x7f40dc1f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 0x7f40dc1f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35154/0 (socket says 172.21.15.46:35154) 2024-09-17T03:38:14.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40daa00640 1 -- 172.21.15.46:0/1235733697 learned_addr learned my addr 172.21.15.46:0/1235733697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40da000640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc15b810 0x7f40dc15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.436+0000 7f40db400640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc15eeb0 0x7f40dc165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40daa00640 1 -- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc15eeb0 msgr2=0x7f40dc165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40daa00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc15eeb0 0x7f40dc165e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40daa00640 1 -- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc15b810 msgr2=0x7f40dc15fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40daa00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc15b810 0x7f40dc15fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40daa00640 1 -- 172.21.15.46:0/1235733697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc1f85d0 con 0x7f40dc0cf620 2024-09-17T03:38:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40daa00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 0x7f40dc1f6e30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0058d0 tx=0x7f40cc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc017070 con 0x7f40dc0cf620 2024-09-17T03:38:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1f88b0 con 0x7f40dc0cf620 2024-09-17T03:38:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40cc009d20 con 0x7f40dc0cf620 2024-09-17T03:38:14.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc1f8e70 con 0x7f40dc0cf620 2024-09-17T03:38:14.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.440+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0106c0 con 0x7f40dc0cf620 2024-09-17T03:38:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.448+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b40052a0 con 0x7f40dc0cf620 2024-09-17T03:38:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.452+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f40cc01a070 con 0x7f40dc0cf620 2024-09-17T03:38:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.452+0000 7f40d3400640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40bc0759f0 0x7f40bc077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.452+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f40cc093510 con 0x7f40dc0cf620 2024-09-17T03:38:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.460+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f40cc05f9d0 con 0x7f40dc0cf620 2024-09-17T03:38:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.460+0000 7f40da000640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40bc0759f0 0x7f40bc077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.464+0000 7f40da000640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40bc0759f0 0x7f40bc077f00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f40d40af000 tx=0x7f40d401b5c0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:14.996+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99875", "target": ["mon-mgr", ""]}) -- 0x7f40b4000bd0 con 0x7f40bc0759f0 2024-09-17T03:38:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.308+0000 7f40d3400640 1 -- 172.21.15.46:0/1235733697 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f40b4000bd0 con 0x7f40bc0759f0 2024-09-17T03:38:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40bc0759f0 msgr2=0x7f40bc077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40bc0759f0 0x7f40bc077f00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f40d40af000 tx=0x7f40d401b5c0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 msgr2=0x7f40dc1f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 0x7f40dc1f6e30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0058d0 tx=0x7f40cc00dde0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 shutdown_connections 2024-09-17T03:38:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40bc0759f0 0x7f40bc077f00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f40dc15eeb0 0x7f40dc165e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40dc15b810 0x7f40dc15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 --2- 172.21.15.46:0/1235733697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40dc0cf620 0x7f40dc1f6e30 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.312+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 >> 172.21.15.46:0/1235733697 conn(0x7f40dc0b2040 msgr2=0x7f40dc073600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.316+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 shutdown_connections 2024-09-17T03:38:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.316+0000 7f40e1e00640 1 -- 172.21.15.46:0/1235733697 wait complete. 2024-09-17T03:38:15.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone17366 2024-09-17T03:38:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.708+0000 7f9e3da00640 1 -- 172.21.15.46:0/632791788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c003c20 msgr2=0x7f9e1c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.708+0000 7f9e3da00640 1 --2- 172.21.15.46:0/632791788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1c003c20 0x7f9e1c004070 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f9e28001860 tx=0x7f9e2801c8e0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.712+0000 7f9e37400640 1 -- 172.21.15.46:0/632791788 reap_dead start 2024-09-17T03:38:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.712+0000 7f9e3da00640 1 -- 172.21.15.46:0/632791788 shutdown_connections 2024-09-17T03:38:15.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.712+0000 7f9e3da00640 1 -- 172.21.15.46:0/632791788 >> 172.21.15.46:0/632791788 conn(0x7f9e380b2040 msgr2=0x7f9e380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.712+0000 7f9e3da00640 1 -- 172.21.15.46:0/632791788 shutdown_connections 2024-09-17T03:38:15.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.712+0000 7f9e3da00640 1 -- 172.21.15.46:0/632791788 wait complete. 2024-09-17T03:38:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 Processor -- start 2024-09-17T03:38:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 -- start start 2024-09-17T03:38:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e1c003c20 0x7f9e381f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e38160730 0x7f9e381f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 0x7f9e3815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38167090 con 0x7f9e38160b40 2024-09-17T03:38:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e380b4580 con 0x7f9e38160730 2024-09-17T03:38:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.716+0000 7f9e3da00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9e38167210 con 0x7f9e1c003c20 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 0x7f9e3815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 0x7f9e3815f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35182/0 (socket says 172.21.15.46:35182) 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 -- 172.21.15.46:0/3360541132 learned_addr learned my addr 172.21.15.46:0/3360541132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e36000640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e38160730 0x7f9e381f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 -- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e1c003c20 msgr2=0x7f9e381f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e1c003c20 0x7f9e381f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 -- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e38160730 msgr2=0x7f9e381f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e38160730 0x7f9e381f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e37400640 1 -- 172.21.15.46:0/3360541132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e381f4fa0 con 0x7f9e38160b40 2024-09-17T03:38:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.720+0000 7f9e36000640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e38160730 0x7f9e381f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.728+0000 7f9e37400640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 0x7f9e3815f480 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f9e300017d0 tx=0x7f9e30006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.732+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000fa80 con 0x7f9e38160b40 2024-09-17T03:38:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.732+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3815e070 con 0x7f9e38160b40 2024-09-17T03:38:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.732+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e380acfe0 con 0x7f9e38160b40 2024-09-17T03:38:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.732+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e3000fc20 con 0x7f9e38160b40 2024-09-17T03:38:15.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.732+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3001d470 con 0x7f9e38160b40 2024-09-17T03:38:15.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.736+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e100052a0 con 0x7f9e38160b40 2024-09-17T03:38:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.740+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f9e3001d610 con 0x7f9e38160b40 2024-09-17T03:38:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.740+0000 7f9e2ea00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e180759f0 0x7f9e18077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.740+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f9e30094030 con 0x7f9e38160b40 2024-09-17T03:38:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.740+0000 7f9e36a00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e180759f0 0x7f9e18077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.744+0000 7f9e36a00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e180759f0 0x7f9e18077f00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9e28001830 tx=0x7f9e1c005840 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:15.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:15.748+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9e30060a30 con 0x7f9e38160b40 2024-09-17T03:38:16.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.132+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone17366", "target": ["mon-mgr", ""]}) -- 0x7f9e10000bd0 con 0x7f9e180759f0 2024-09-17T03:38:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.284+0000 7f9e2ea00640 1 -- 172.21.15.46:0/3360541132 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9e10000bd0 con 0x7f9e180759f0 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.284+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e180759f0 msgr2=0x7f9e18077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e180759f0 0x7f9e18077f00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9e28001830 tx=0x7f9e1c005840 comp rx=0 tx=0).stop 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 msgr2=0x7f9e3815f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 0x7f9e3815f480 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f9e300017d0 tx=0x7f9e30006660 comp rx=0 tx=0).stop 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 shutdown_connections 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9e180759f0 0x7f9e18077f00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e38160b40 0x7f9e3815f480 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e38160730 0x7f9e381f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 --2- 172.21.15.46:0/3360541132 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9e1c003c20 0x7f9e381f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 >> 172.21.15.46:0/3360541132 conn(0x7f9e380b2040 msgr2=0x7f9e380cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 shutdown_connections 2024-09-17T03:38:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.288+0000 7f9e3da00640 1 -- 172.21.15.46:0/3360541132 wait complete. 2024-09-17T03:38:16.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone63430 2024-09-17T03:38:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 -- 172.21.15.46:0/2884621889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 msgr2=0x7f571c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 --2- 172.21.15.46:0/2884621889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c15bb60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f571c074b20 tx=0x7f570c01cae0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 -- 172.21.15.46:0/2884621889 shutdown_connections 2024-09-17T03:38:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 --2- 172.21.15.46:0/2884621889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c15bb60 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 -- 172.21.15.46:0/2884621889 >> 172.21.15.46:0/2884621889 conn(0x7f571c0b2040 msgr2=0x7f571c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 -- 172.21.15.46:0/2884621889 shutdown_connections 2024-09-17T03:38:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.468+0000 7f5721e00640 1 -- 172.21.15.46:0/2884621889 wait complete. 2024-09-17T03:38:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 Processor -- start 2024-09-17T03:38:16.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 -- start start 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f571c0b4de0 0x7f571c14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f571c1527b0 0x7f571c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f571c169b30 con 0x7f571c0b5c70 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f571c1699b0 con 0x7f571c0b4de0 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5721e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f571c169cb0 con 0x7f571c1527b0 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5712000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5712000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35198/0 (socket says 172.21.15.46:35198) 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f5712000640 1 -- 172.21.15.46:0/973949717 learned_addr learned my addr 172.21.15.46:0/973949717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f571aa00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f571c0b4de0 0x7f571c14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.472+0000 7f571b400640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f571c1527b0 0x7f571c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5712000640 1 -- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f571c1527b0 msgr2=0x7f571c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5712000640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f571c1527b0 0x7f571c0ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5712000640 1 -- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f571c0b4de0 msgr2=0x7f571c14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5712000640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f571c0b4de0 0x7f571c14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5712000640 1 -- 172.21.15.46:0/973949717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c154c50 con 0x7f571c0b5c70 2024-09-17T03:38:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5712000640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c154710 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f570400c420 tx=0x7f570400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704011790 con 0x7f571c0b5c70 2024-09-17T03:38:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5704017070 con 0x7f571c0b5c70 2024-09-17T03:38:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704011ce0 con 0x7f571c0b5c70 2024-09-17T03:38:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c157b30 con 0x7f571c0b5c70 2024-09-17T03:38:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c14e440 con 0x7f571c0b5c70 2024-09-17T03:38:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571c15bb60 con 0x7f571c0b5c70 2024-09-17T03:38:16.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.476+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f570401c460 con 0x7f571c0b5c70 2024-09-17T03:38:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.480+0000 7f5713e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56f80759f0 0x7f56f8077f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.480+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f5704093e10 con 0x7f571c0b5c70 2024-09-17T03:38:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.480+0000 7f571aa00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56f80759f0 0x7f56f8077f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.480+0000 7f571aa00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56f80759f0 0x7f56f8077f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f570c0af000 tx=0x7f570c01b320 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:16.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.480+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f571c154c50 con 0x7f571c0b5c70 2024-09-17T03:38:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.692+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone63430", "target": ["mon-mgr", ""]}) -- 0x7f571c06b540 con 0x7f56f80759f0 2024-09-17T03:38:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.700+0000 7f5713e00640 1 -- 172.21.15.46:0/973949717 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f570c0a5e50 con 0x7f56f80759f0 2024-09-17T03:38:16.703 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:38:16.703 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:16.704 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56f80759f0 msgr2=0x7f56f8077f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56f80759f0 0x7f56f8077f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f570c0af000 tx=0x7f570c01b320 comp rx=0 tx=0).stop 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 msgr2=0x7f571c154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c154710 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f570400c420 tx=0x7f570400c8e0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 shutdown_connections 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f56f80759f0 0x7f56f8077f00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f571c1527b0 0x7f571c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f571c0b5c70 0x7f571c154710 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 --2- 172.21.15.46:0/973949717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f571c0b4de0 0x7f571c14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 >> 172.21.15.46:0/973949717 conn(0x7f571c0b2040 msgr2=0x7f571c15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 shutdown_connections 2024-09-17T03:38:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.704+0000 7f5721e00640 1 -- 172.21.15.46:0/973949717 wait complete. 2024-09-17T03:38:16.727 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:38:16.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6331 2024-09-17T03:38:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 -- 172.21.15.46:0/2297149102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b41527b0 msgr2=0x7f23b4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 --2- 172.21.15.46:0/2297149102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b41527b0 0x7f23b4074e30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f23b4071cb0 tx=0x7f23a001c890 comp rx=0 tx=0).stop 2024-09-17T03:38:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 -- 172.21.15.46:0/2297149102 shutdown_connections 2024-09-17T03:38:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 --2- 172.21.15.46:0/2297149102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b41527b0 0x7f23b4074e30 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 --2- 172.21.15.46:0/2297149102 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23b40b4de0 0x7f23b40b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 --2- 172.21.15.46:0/2297149102 >> v2:172.21.15.99:3300/0 conn(0x7f23b40b5c70 0x7f23b415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 -- 172.21.15.46:0/2297149102 >> 172.21.15.46:0/2297149102 conn(0x7f23b40b2040 msgr2=0x7f23b40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 -- 172.21.15.46:0/2297149102 shutdown_connections 2024-09-17T03:38:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 -- 172.21.15.46:0/2297149102 wait complete. 2024-09-17T03:38:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 Processor -- start 2024-09-17T03:38:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.888+0000 7f23bc000640 1 -- start start 2024-09-17T03:38:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b40b4de0 0x7f23b4154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 0x7f23b4148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23b41527b0 0x7f23b40ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23b415f6b0 con 0x7f23b40b5c70 2024-09-17T03:38:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23b415f530 con 0x7f23b40b4de0 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f23b415f830 con 0x7f23b41527b0 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 0x7f23b4148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 0x7f23b4148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35216/0 (socket says 172.21.15.46:35216) 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 -- 172.21.15.46:0/2174673833 learned_addr learned my addr 172.21.15.46:0/2174673833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23ba000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23b41527b0 0x7f23b40ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b9600640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b40b4de0 0x7f23b4154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 -- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23b41527b0 msgr2=0x7f23b40ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23b41527b0 0x7f23b40ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 -- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b40b4de0 msgr2=0x7f23b4154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b40b4de0 0x7f23b4154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 -- 172.21.15.46:0/2174673833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b4148770 con 0x7f23b40b5c70 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b8c00640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 0x7f23b4148230 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f23a001af40 tx=0x7f23a00a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a00b2040 con 0x7f23b40b5c70 2024-09-17T03:38:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23a0004450 con 0x7f23b40b5c70 2024-09-17T03:38:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b4149160 con 0x7f23b40b5c70 2024-09-17T03:38:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0005290 con 0x7f23b40b5c70 2024-09-17T03:38:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b41544a0 con 0x7f23b40b5c70 2024-09-17T03:38:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.892+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b40b0dc0 con 0x7f23b40b5c70 2024-09-17T03:38:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.896+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f23a00be060 con 0x7f23b40b5c70 2024-09-17T03:38:16.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.896+0000 7f23b1600640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2394075a20 0x7f2394077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:16.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.896+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f23a012cab0 con 0x7f23b40b5c70 2024-09-17T03:38:16.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.896+0000 7f23b9600640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2394075a20 0x7f2394077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:16.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.896+0000 7f23b9600640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2394075a20 0x7f2394077f30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f23a800bb50 tx=0x7f23a800b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:16.900+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23a00164b0 con 0x7f23b40b5c70 2024-09-17T03:38:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.108+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6331", "target": ["mon-mgr", ""]}) -- 0x7f23b406b540 con 0x7f2394075a20 2024-09-17T03:38:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.116+0000 7f23b1600640 1 -- 172.21.15.46:0/2174673833 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f23a8015070 con 0x7f2394075a20 2024-09-17T03:38:17.118 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "complete" 2024-09-17T03:38:17.118 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:17.119 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2394075a20 msgr2=0x7f2394077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2394075a20 0x7f2394077f30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f23a800bb50 tx=0x7f23a800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 msgr2=0x7f23b4148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 0x7f23b4148230 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f23a001af40 tx=0x7f23a00a5820 comp rx=0 tx=0).stop 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 shutdown_connections 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2394075a20 0x7f2394077f30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f23b41527b0 0x7f23b40ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b40b5c70 0x7f23b4148230 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 --2- 172.21.15.46:0/2174673833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23b40b4de0 0x7f23b4154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 >> 172.21.15.46:0/2174673833 conn(0x7f23b40b2040 msgr2=0x7f23b415d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 shutdown_connections 2024-09-17T03:38:17.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.120+0000 7f23bc000640 1 -- 172.21.15.46:0/2174673833 wait complete. 2024-09-17T03:38:17.141 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-17T03:38:17.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone59742 2024-09-17T03:38:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 -- 172.21.15.46:0/4270550193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 msgr2=0x7f1e2c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/4270550193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e2c004c20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009b30 tx=0x7f1e3001d270 comp rx=0 tx=0).stop 2024-09-17T03:38:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e47400640 1 -- 172.21.15.46:0/4270550193 reap_dead start 2024-09-17T03:38:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 -- 172.21.15.46:0/4270550193 shutdown_connections 2024-09-17T03:38:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 -- 172.21.15.46:0/4270550193 >> 172.21.15.46:0/4270550193 conn(0x7f1e480b2c40 msgr2=0x7f1e480b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 -- 172.21.15.46:0/4270550193 shutdown_connections 2024-09-17T03:38:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 -- 172.21.15.46:0/4270550193 wait complete. 2024-09-17T03:38:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 Processor -- start 2024-09-17T03:38:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.308+0000 7f1e4dc00640 1 -- start start 2024-09-17T03:38:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e48159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e481394f0 0x7f1e4815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e4815bda0 0x7f1e4813bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4815d840 con 0x7f1e2c004810 2024-09-17T03:38:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480d03d0 con 0x7f1e481394f0 2024-09-17T03:38:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4815d9c0 con 0x7f1e4815bda0 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e48159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e48159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35232/0 (socket says 172.21.15.46:35232) 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 -- 172.21.15.46:0/472806738 learned_addr learned my addr 172.21.15.46:0/472806738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e47400640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e4815bda0 0x7f1e4813bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e3fe00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e481394f0 0x7f1e4815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 -- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e4815bda0 msgr2=0x7f1e4813bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e4815bda0 0x7f1e4813bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 -- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e481394f0 msgr2=0x7f1e4815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e481394f0 0x7f1e4815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 -- 172.21.15.46:0/472806738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4815c1b0 con 0x7f1e2c004810 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e46a00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e48159fe0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1e38011ab0 tx=0x7f1e38011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3801e070 con 0x7f1e2c004810 2024-09-17T03:38:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3800fe70 con 0x7f1e2c004810 2024-09-17T03:38:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4815a520 con 0x7f1e2c004810 2024-09-17T03:38:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380042b0 con 0x7f1e2c004810 2024-09-17T03:38:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481374c0 con 0x7f1e2c004810 2024-09-17T03:38:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.312+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48070ff0 con 0x7f1e2c004810 2024-09-17T03:38:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.316+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 58) ==== 100855+0+0 (secure 0 0 0) 0x7f1e38021070 con 0x7f1e2c004810 2024-09-17T03:38:17.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.316+0000 7f1e3ea00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e24075a20 0x7f1e24077f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:17.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.316+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f1e38093f40 con 0x7f1e2c004810 2024-09-17T03:38:17.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.316+0000 7f1e3fe00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e24075a20 0x7f1e24077f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:17.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.316+0000 7f1e3fe00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e24075a20 0x7f1e24077f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ae000 tx=0x7f1e3001a010 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.320+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e4815c1b0 con 0x7f1e2c004810 2024-09-17T03:38:17.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:17.528+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone59742", "target": ["mon-mgr", ""]}) -- 0x7f1e4815f860 con 0x7f1e24075a20 2024-09-17T03:38:18.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:18.424+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f1e3805c670 con 0x7f1e2c004810 2024-09-17T03:38:22.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e3ea00640 1 -- 172.21.15.46:0/472806738 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f1e4815f860 con 0x7f1e24075a20 2024-09-17T03:38:22.160 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "complete" 2024-09-17T03:38:22.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:22.161 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e24075a20 msgr2=0x7f1e24077f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e24075a20 0x7f1e24077f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1e300ae000 tx=0x7f1e3001a010 comp rx=0 tx=0).stop 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 msgr2=0x7f1e48159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e48159fe0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1e38011ab0 tx=0x7f1e38011f70 comp rx=0 tx=0).stop 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 shutdown_connections 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1e24075a20 0x7f1e24077f30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1e4815bda0 0x7f1e4813bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e481394f0 0x7f1e4815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 --2- 172.21.15.46:0/472806738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e2c004810 0x7f1e48159fe0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 >> 172.21.15.46:0/472806738 conn(0x7f1e480b2c40 msgr2=0x7f1e480b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 shutdown_connections 2024-09-17T03:38:22.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.160+0000 7f1e4dc00640 1 -- 172.21.15.46:0/472806738 wait complete. 2024-09-17T03:38:22.188 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-17T03:38:22.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone1673 2024-09-17T03:38:22.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 -- 172.21.15.46:0/1885767597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8368004810 msgr2=0x7f8368004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 --2- 172.21.15.46:0/1885767597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8368004810 0x7f8368004c20 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f836c009b40 tx=0x7f836c01d220 comp rx=0 tx=0).stop 2024-09-17T03:38:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 -- 172.21.15.46:0/1885767597 shutdown_connections 2024-09-17T03:38:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 --2- 172.21.15.46:0/1885767597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8368004810 0x7f8368004c20 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 -- 172.21.15.46:0/1885767597 >> 172.21.15.46:0/1885767597 conn(0x7f83800b2040 msgr2=0x7f83800b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 -- 172.21.15.46:0/1885767597 shutdown_connections 2024-09-17T03:38:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 -- 172.21.15.46:0/1885767597 wait complete. 2024-09-17T03:38:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 Processor -- start 2024-09-17T03:38:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.356+0000 7f8387a00640 1 -- start start 2024-09-17T03:38:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8368004810 0x7f83800cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83801566c0 0x7f83801f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 0x7f83800b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83800b55d0 con 0x7f8380156b80 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83800b5450 con 0x7f8368004810 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f83800b5750 con 0x7f83801566c0 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 0x7f83800b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 0x7f83800b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35252/0 (socket says 172.21.15.46:35252) 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 -- 172.21.15.46:0/2443114978 learned_addr learned my addr 172.21.15.46:0/2443114978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385000640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8368004810 0x7f83800cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f837fe00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83801566c0 0x7f83801f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 -- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83801566c0 msgr2=0x7f83801f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83801566c0 0x7f83801f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 -- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8368004810 msgr2=0x7f83800cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8368004810 0x7f83800cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 -- 172.21.15.46:0/2443114978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83801fc9d0 con 0x7f8380156b80 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8385a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 0x7f83800b02e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f8378000f90 tx=0x7f837800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837800c8f0 con 0x7f8380156b80 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8378018070 con 0x7f8380156b80 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837800ce00 con 0x7f8380156b80 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83801fccb0 con 0x7f8380156b80 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83801fd270 con 0x7f8380156b80 2024-09-17T03:38:22.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.360+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838015f560 con 0x7f8380156b80 2024-09-17T03:38:22.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.364+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f837801d460 con 0x7f8380156b80 2024-09-17T03:38:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.364+0000 7f837cc00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8360075960 0x7f8360077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.364+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f8378093870 con 0x7f8380156b80 2024-09-17T03:38:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.364+0000 7f8385000640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8360075960 0x7f8360077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.364+0000 7f8385000640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8360075960 0x7f8360077e70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8374000c10 tx=0x7f8374006da0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.368+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8378060390 con 0x7f8380156b80 2024-09-17T03:38:22.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.576+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone1673", "target": ["mon-mgr", ""]}) -- 0x7f838006b540 con 0x7f8360075960 2024-09-17T03:38:22.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f837cc00640 1 -- 172.21.15.46:0/2443114978 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f8374012560 con 0x7f8360075960 2024-09-17T03:38:22.588 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:22.589 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8360075960 msgr2=0x7f8360077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8360075960 0x7f8360077e70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8374000c10 tx=0x7f8374006da0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 msgr2=0x7f83800b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 0x7f83800b02e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f8378000f90 tx=0x7f837800eb00 comp rx=0 tx=0).stop 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 shutdown_connections 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8360075960 0x7f8360077e70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8380156b80 0x7f83800b02e0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83801566c0 0x7f83801f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 --2- 172.21.15.46:0/2443114978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8368004810 0x7f83800cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 >> 172.21.15.46:0/2443114978 conn(0x7f83800b2040 msgr2=0x7f8380159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 shutdown_connections 2024-09-17T03:38:22.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.588+0000 7f8387a00640 1 -- 172.21.15.46:0/2443114978 wait complete. 2024-09-17T03:38:22.611 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:38:22.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone88214 2024-09-17T03:38:22.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 -- 172.21.15.46:0/598734066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 msgr2=0x7f3e780b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 --2- 172.21.15.46:0/598734066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780b51f0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3e78071cb0 tx=0x7f3e7001ccc0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 -- 172.21.15.46:0/598734066 shutdown_connections 2024-09-17T03:38:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 --2- 172.21.15.46:0/598734066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780b51f0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 --2- 172.21.15.46:0/598734066 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e781527b0 0x7f3e78074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 --2- 172.21.15.46:0/598734066 >> v2:172.21.15.99:3300/0 conn(0x7f3e780b5c70 0x7f3e7815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 -- 172.21.15.46:0/598734066 >> 172.21.15.46:0/598734066 conn(0x7f3e780b2040 msgr2=0x7f3e780b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 -- 172.21.15.46:0/598734066 shutdown_connections 2024-09-17T03:38:22.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.772+0000 7f3e7e800640 1 -- 172.21.15.46:0/598734066 wait complete. 2024-09-17T03:38:22.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 Processor -- start 2024-09-17T03:38:22.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 -- start start 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e780b5c70 0x7f3e7814ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e781527b0 0x7f3e781f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7815f730 con 0x7f3e780b4de0 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7815f5b0 con 0x7f3e781527b0 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7815f8b0 con 0x7f3e780b5c70 2024-09-17T03:38:22.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42110/0 (socket says 172.21.15.46:42110) 2024-09-17T03:38:22.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 -- 172.21.15.46:0/3349720234 learned_addr learned my addr 172.21.15.46:0/3349720234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:22.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e6ea00640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e780b5c70 0x7f3e7814ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77e00640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e781527b0 0x7f3e781f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 -- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e780b5c70 msgr2=0x7f3e7814ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e780b5c70 0x7f3e7814ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 -- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e781527b0 msgr2=0x7f3e781f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:22.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e781527b0 0x7f3e781f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:22.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 -- 172.21.15.46:0/3349720234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7814fc90 con 0x7f3e780b4de0 2024-09-17T03:38:22.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e77400640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780af950 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800e7c0 tx=0x7f3e6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800ccc0 con 0x7f3e780b4de0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e6800ee70 con 0x7f3e780b4de0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e78150100 con 0x7f3e780b4de0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68010690 con 0x7f3e780b4de0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.776+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7814f830 con 0x7f3e780b4de0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.780+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7815fdd0 con 0x7f3e780b4de0 2024-09-17T03:38:22.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.780+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f3e68010830 con 0x7f3e780b4de0 2024-09-17T03:38:22.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.780+0000 7f3e6fe00640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e54075960 0x7f3e54077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.780+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f3e68093de0 con 0x7f3e780b4de0 2024-09-17T03:38:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.780+0000 7f3e6ea00640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e54075960 0x7f3e54077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.784+0000 7f3e6ea00640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e54075960 0x7f3e54077e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3e781f9b60 tx=0x7f3e600091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:22.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.784+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e68060340 con 0x7f3e780b4de0 2024-09-17T03:38:22.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:22.996+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone88214", "target": ["mon-mgr", ""]}) -- 0x7f3e7814b250 con 0x7f3e54075960 2024-09-17T03:38:23.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e6fe00640 1 -- 172.21.15.46:0/3349720234 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f3e7814b250 con 0x7f3e54075960 2024-09-17T03:38:23.005 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:23.005 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:23.005 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:38:23.005 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:38:23.005 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:23.006 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e54075960 msgr2=0x7f3e54077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e54075960 0x7f3e54077e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3e781f9b60 tx=0x7f3e600091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 msgr2=0x7f3e780af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780af950 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800e7c0 tx=0x7f3e6800ec80 comp rx=0 tx=0).stop 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 shutdown_connections 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3e54075960 0x7f3e54077e70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e781527b0 0x7f3e781f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3e780b5c70 0x7f3e7814ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 --2- 172.21.15.46:0/3349720234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e780b4de0 0x7f3e780af950 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 >> 172.21.15.46:0/3349720234 conn(0x7f3e780b2040 msgr2=0x7f3e7806e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:23.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 shutdown_connections 2024-09-17T03:38:23.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.004+0000 7f3e7e800640 1 -- 172.21.15.46:0/3349720234 wait complete. 2024-09-17T03:38:23.028 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:38:23.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99875 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 -- 172.21.15.46:0/596805714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840af520 msgr2=0x7ffa840af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 --2- 172.21.15.46:0/596805714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840af520 0x7ffa840af930 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009840 tx=0x7ffa7c01c790 comp rx=0 tx=0).stop 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 -- 172.21.15.46:0/596805714 shutdown_connections 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 --2- 172.21.15.46:0/596805714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840af520 0x7ffa840af930 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 --2- 172.21.15.46:0/596805714 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 0x7ffa84070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 -- 172.21.15.46:0/596805714 >> 172.21.15.46:0/596805714 conn(0x7ffa8406d3f0 msgr2=0x7ffa840b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 -- 172.21.15.46:0/596805714 shutdown_connections 2024-09-17T03:38:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.188+0000 7ffa88c00640 1 -- 172.21.15.46:0/596805714 wait complete. 2024-09-17T03:38:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 Processor -- start 2024-09-17T03:38:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 -- start start 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 0x7ffa84152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840b4fd0 0x7ffa8415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 0x7ffa8415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840cfef0 con 0x7ffa840af520 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840cfd70 con 0x7ffa840b4fd0 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa88c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840d0070 con 0x7ffa8415cb90 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 0x7ffa84152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 0x7ffa84152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42118/0 (socket says 172.21.15.46:42118) 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 -- 172.21.15.46:0/3276135765 learned_addr learned my addr 172.21.15.46:0/3276135765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa7aa00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840b4fd0 0x7ffa8415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83e00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 0x7ffa8415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 -- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 msgr2=0x7ffa8415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 0x7ffa8415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 -- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840b4fd0 msgr2=0x7ffa8415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840b4fd0 0x7ffa8415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 -- 172.21.15.46:0/3276135765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa84153550 con 0x7ffa840af520 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83e00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 0x7ffa8415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.192+0000 7ffa83400640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 0x7ffa84152660 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400ed30 tx=0x7ffa7400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74003b80 con 0x7ffa840af520 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa74003d20 con 0x7ffa840af520 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74004280 con 0x7ffa840af520 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8415b5c0 con 0x7ffa840af520 2024-09-17T03:38:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa840c6660 con 0x7ffa840af520 2024-09-17T03:38:23.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa840af930 con 0x7ffa840af520 2024-09-17T03:38:23.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7ffa74004420 con 0x7ffa840af520 2024-09-17T03:38:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7be00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa60075990 0x7ffa60077ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7ffa740942a0 con 0x7ffa840af520 2024-09-17T03:38:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.196+0000 7ffa7aa00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa60075990 0x7ffa60077ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.200+0000 7ffa7aa00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa60075990 0x7ffa60077ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ffa840c6200 tx=0x7ffa6c0013a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.200+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffa74060d20 con 0x7ffa840af520 2024-09-17T03:38:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.412+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99875", "target": ["mon-mgr", ""]}) -- 0x7ffa84152ba0 con 0x7ffa60075990 2024-09-17T03:38:23.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa7be00640 1 -- 172.21.15.46:0/3276135765 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7ffa84152ba0 con 0x7ffa60075990 2024-09-17T03:38:23.420 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:23.420 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:23.421 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:23.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa60075990 msgr2=0x7ffa60077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa60075990 0x7ffa60077ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ffa840c6200 tx=0x7ffa6c0013a0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 msgr2=0x7ffa84152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 0x7ffa84152660 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400ed30 tx=0x7ffa7400c6a0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 shutdown_connections 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ffa60075990 0x7ffa60077ea0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ffa8415cb90 0x7ffa8415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa840b4fd0 0x7ffa8415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 --2- 172.21.15.46:0/3276135765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa840af520 0x7ffa84152660 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 >> 172.21.15.46:0/3276135765 conn(0x7ffa8406d3f0 msgr2=0x7ffa8406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 shutdown_connections 2024-09-17T03:38:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.420+0000 7ffa88c00640 1 -- 172.21.15.46:0/3276135765 wait complete. 2024-09-17T03:38:23.443 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:38:23.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone17366 2024-09-17T03:38:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 -- 172.21.15.46:0/3646670332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 msgr2=0x7f8474074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 --2- 172.21.15.46:0/3646670332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f8474074240 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f84740cf5e0 tx=0x7f846001c830 comp rx=0 tx=0).stop 2024-09-17T03:38:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 -- 172.21.15.46:0/3646670332 shutdown_connections 2024-09-17T03:38:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 --2- 172.21.15.46:0/3646670332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f8474074240 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 --2- 172.21.15.46:0/3646670332 >> v2:172.21.15.99:3300/0 conn(0x7f8474160470 0x7f8474074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 --2- 172.21.15.46:0/3646670332 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84741536f0 0x7f847415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 -- 172.21.15.46:0/3646670332 >> 172.21.15.46:0/3646670332 conn(0x7f84740b4560 msgr2=0x7f84740d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 -- 172.21.15.46:0/3646670332 shutdown_connections 2024-09-17T03:38:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.604+0000 7f847a200640 1 -- 172.21.15.46:0/3646670332 wait complete. 2024-09-17T03:38:23.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 Processor -- start 2024-09-17T03:38:23.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 -- start start 2024-09-17T03:38:23.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84741536f0 0x7f847413ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f847415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8474160470 0x7f847415f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8474071dd0 con 0x7f8474160060 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8474071c50 con 0x7f8474160470 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8474071f50 con 0x7f84741536f0 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f847415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f847415ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42144/0 (socket says 172.21.15.46:42144) 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 -- 172.21.15.46:0/3947655289 learned_addr learned my addr 172.21.15.46:0/3947655289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8478e00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84741536f0 0x7f847413ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8479800640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8474160470 0x7f847415f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 -- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84741536f0 msgr2=0x7f847413ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84741536f0 0x7f847413ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 -- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8474160470 msgr2=0x7f847415f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8474160470 0x7f847415f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 -- 172.21.15.46:0/3947655289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8474155da0 con 0x7f8474160060 2024-09-17T03:38:23.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8473e00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f847415ed20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f84600b0040 tx=0x7f8460004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:23.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460005020 con 0x7f8474160060 2024-09-17T03:38:23.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84600052d0 con 0x7f8474160060 2024-09-17T03:38:23.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8474156080 con 0x7f8474160060 2024-09-17T03:38:23.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84600046d0 con 0x7f8474160060 2024-09-17T03:38:23.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.608+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8474156640 con 0x7f8474160060 2024-09-17T03:38:23.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.612+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84740724f0 con 0x7f8474160060 2024-09-17T03:38:23.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.612+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f84600be060 con 0x7f8474160060 2024-09-17T03:38:23.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.612+0000 7f8470c00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8454075960 0x7f8454077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:23.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.612+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f846012d9b0 con 0x7f8474160060 2024-09-17T03:38:23.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.612+0000 7f8478e00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8454075960 0x7f8454077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:23.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.616+0000 7f8478e00640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8454075960 0x7f8454077e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8468006fd0 tx=0x7f8468008040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:23.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.620+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f84600164b0 con 0x7f8474160060 2024-09-17T03:38:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.828+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone17366", "target": ["mon-mgr", ""]}) -- 0x7f8474155b50 con 0x7f8454075960 2024-09-17T03:38:23.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.836+0000 7f8470c00640 1 -- 172.21.15.46:0/3947655289 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f8474155b50 con 0x7f8454075960 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "status": { 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "canceled", 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "source": { 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "volume": "cephfs", 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "subvolume": "sv1", 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot": "ss1" 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "failure": { 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "errno": "4", 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T03:38:23.840 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:23.841 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:23.841 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8454075960 msgr2=0x7f8454077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8454075960 0x7f8454077e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8468006fd0 tx=0x7f8468008040 comp rx=0 tx=0).stop 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 msgr2=0x7f847415ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f847415ed20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f84600b0040 tx=0x7f8460004040 comp rx=0 tx=0).stop 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 shutdown_connections 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8454075960 0x7f8454077e70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8474160470 0x7f847415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8474160060 0x7f847415ed20 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 --2- 172.21.15.46:0/3947655289 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84741536f0 0x7f847413ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:23.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 >> 172.21.15.46:0/3947655289 conn(0x7f84740b4560 msgr2=0x7f84741529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:23.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 shutdown_connections 2024-09-17T03:38:23.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:23.840+0000 7f847a200640 1 -- 172.21.15.46:0/3947655289 wait complete. 2024-09-17T03:38:23.863 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T03:38:23.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T03:38:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 -- 172.21.15.46:0/880334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406fc90 msgr2=0x7fe9e40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 --2- 172.21.15.46:0/880334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406fc90 0x7fe9e40700a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40749e0 tx=0x7fe9dc01c830 comp rx=0 tx=0).stop 2024-09-17T03:38:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 -- 172.21.15.46:0/880334651 shutdown_connections 2024-09-17T03:38:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 --2- 172.21.15.46:0/880334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406fc90 0x7fe9e40700a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 --2- 172.21.15.46:0/880334651 >> v2:172.21.15.99:3300/0 conn(0x7fe9e406cdf0 0x7fe9e406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 --2- 172.21.15.46:0/880334651 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe9e4159130 0x7fe9e415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 -- 172.21.15.46:0/880334651 >> 172.21.15.46:0/880334651 conn(0x7fe9e40b35f0 msgr2=0x7fe9e40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 -- 172.21.15.46:0/880334651 shutdown_connections 2024-09-17T03:38:24.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 -- 172.21.15.46:0/880334651 wait complete. 2024-09-17T03:38:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 Processor -- start 2024-09-17T03:38:24.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.020+0000 7fe9eb000640 1 -- start start 2024-09-17T03:38:24.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 0x7fe9e41663d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e406fc90 0x7fe9e41646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe9e4159130 0x7fe9e4155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e415fa60 con 0x7fe9e406cdf0 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e415f8e0 con 0x7fe9e406fc90 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e415fbe0 con 0x7fe9e4159130 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 0x7fe9e41663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 0x7fe9e41663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42168/0 (socket says 172.21.15.46:42168) 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 -- 172.21.15.46:0/2307847425 learned_addr learned my addr 172.21.15.46:0/2307847425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9db400640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e406fc90 0x7fe9e41646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e9000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe9e4159130 0x7fe9e4155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 -- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe9e4159130 msgr2=0x7fe9e4155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe9e4159130 0x7fe9e4155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 -- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e406fc90 msgr2=0x7fe9e41646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e406fc90 0x7fe9e41646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 -- 172.21.15.46:0/2307847425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e4166cd0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e3e00640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 0x7fe9e41663d0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400cce0 tx=0x7fe9d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:24.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400f450 con 0x7fe9e406cdf0 2024-09-17T03:38:24.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9d400f6e0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400fb10 con 0x7fe9e406cdf0 2024-09-17T03:38:24.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e4166fb0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e41599c0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.024+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e40700a0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.028+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7fe9d400fcb0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.028+0000 7fe9e0c00640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe9c0075960 0x7fe9c0077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.028+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7fe9d4093f00 con 0x7fe9e406cdf0 2024-09-17T03:38:24.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.028+0000 7fe9db400640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe9c0075960 0x7fe9c0077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.028+0000 7fe9db400640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe9c0075960 0x7fe9c0077e70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe9e41f9f00 tx=0x7fe9cc00a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:24.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.032+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe9d4014030 con 0x7fe9e406cdf0 2024-09-17T03:38:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe9e4167490 con 0x7fe9e406cdf0 2024-09-17T03:38:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9e0c00640 1 -- 172.21.15.46:0/2307847425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1929 (secure 0 0 0) 0x7fe9d40603c0 con 0x7fe9e406cdf0 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: "fsid": "6c25af6b-3689-4426-acb2-6ebe6de6efd5", 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: "health": { 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "HEALTH_OK", 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: "checks": {}, 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: "mutes": [] 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.273 INFO:teuthology.orchestra.run.smithi046.stdout: "election_epoch": 8, 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum": [ 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: 0, 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: 1, 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: 2 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_names": [ 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "a", 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "b", 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "c" 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "quorum_age": 1062, 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "monmap": { 2024-09-17T03:38:24.274 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 1, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min_mon_release_name": "squid", 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "num_mons": 3 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap": { 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 145, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "num_osds": 8, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "num_up_osds": 8, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_up_since": 1726543255, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "num_in_osds": 8, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_in_since": 1726543245, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "num_remapped_pgs": 0 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap": { 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: "pgs_by_state": [ 2024-09-17T03:38:24.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "state_name": "active+clean", 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "count": 97 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pgs": 97, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "num_pools": 3, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "num_objects": 5187, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "data_bytes": 21594487873, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_used": 44413808640, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_avail": 728680304640, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "bytes_total": 773094113280, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "read_bytes_sec": 531518763, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "write_bytes_sec": 532959900, 2024-09-17T03:38:24.276 INFO:teuthology.orchestra.run.smithi046.stdout: "read_op_per_sec": 129, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "write_op_per_sec": 142 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "fsmap": { 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 95, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "btime": "2024-09-17T03:37:14:564527+0000", 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 10, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "up": 1, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "in": 1, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "by_rank": [ 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "filesystem_id": 10, 2024-09-17T03:38:24.277 INFO:teuthology.orchestra.run.smithi046.stdout: "rank": 0, 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "a", 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "up:active", 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 248758 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "up:standby": 3 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "mgrmap": { 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "num_standbys": 1, 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-09-17T03:38:24.278 INFO:teuthology.orchestra.run.smithi046.stdout: "restful" 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: "servicemap": { 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 28, 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: "modified": "2024-09-17T03:38:09.622024+0000", 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: "services": {} 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout: "progress_events": {} 2024-09-17T03:38:24.279 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe9c0075960 msgr2=0x7fe9c0077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe9c0075960 0x7fe9c0077e70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe9e41f9f00 tx=0x7fe9cc00a250 comp rx=0 tx=0).stop 2024-09-17T03:38:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 msgr2=0x7fe9e41663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 0x7fe9e41663d0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400cce0 tx=0x7fe9d400a620 comp rx=0 tx=0).stop 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 shutdown_connections 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe9c0075960 0x7fe9c0077e70 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe9e4159130 0x7fe9e4155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e406fc90 0x7fe9e41646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 --2- 172.21.15.46:0/2307847425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9e406cdf0 0x7fe9e41663d0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 >> 172.21.15.46:0/2307847425 conn(0x7fe9e40b35f0 msgr2=0x7fe9e40b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 shutdown_connections 2024-09-17T03:38:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.272+0000 7fe9eb000640 1 -- 172.21.15.46:0/2307847425 wait complete. 2024-09-17T03:38:24.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:24.295 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone63430/e7b6ca95-3b05-468d-ae4b-d174c4830b3d 2024-09-17T03:38:24.295 DEBUG:teuthology.orchestra.run.smithi046:> ) 2024-09-17T03:38:24.306 INFO:teuthology.orchestra.run.smithi046.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone63430/e7b6ca95-3b05-468d-ae4b-d174c4830b3d': No such file or directory 2024-09-17T03:38:24.307 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T03:38:24.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T03:38:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 -- 172.21.15.46:0/2141855713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b4de0 msgr2=0x7fbbd80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 --2- 172.21.15.46:0/2141855713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b4de0 0x7fbbd80b51f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009c80 tx=0x7fbbd001cce0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 -- 172.21.15.46:0/2141855713 shutdown_connections 2024-09-17T03:38:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 --2- 172.21.15.46:0/2141855713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b4de0 0x7fbbd80b51f0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 --2- 172.21.15.46:0/2141855713 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbd80b5c70 0x7fbbd815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 -- 172.21.15.46:0/2141855713 >> 172.21.15.46:0/2141855713 conn(0x7fbbd80b2040 msgr2=0x7fbbd80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 -- 172.21.15.46:0/2141855713 shutdown_connections 2024-09-17T03:38:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.508+0000 7fbbdee00640 1 -- 172.21.15.46:0/2141855713 wait complete. 2024-09-17T03:38:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 Processor -- start 2024-09-17T03:38:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 -- start start 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbd80b4de0 0x7fbbd80cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 0x7fbbd814ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbd81527b0 0x7fbbd81fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd8168ab0 con 0x7fbbd80b5c70 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd8168930 con 0x7fbbd80b4de0 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd8168c30 con 0x7fbbd81527b0 2024-09-17T03:38:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 0x7fbbd814ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 0x7fbbd814ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42180/0 (socket says 172.21.15.46:42180) 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 -- 172.21.15.46:0/3351329474 learned_addr learned my addr 172.21.15.46:0/3351329474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbd7e00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbd80b4de0 0x7fbbd80cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdce00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbd81527b0 0x7fbbd81fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 -- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbd81527b0 msgr2=0x7fbbd81fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbd81527b0 0x7fbbd81fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 -- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbd80b4de0 msgr2=0x7fbbd80cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbd80b4de0 0x7fbbd80cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 -- 172.21.15.46:0/3351329474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd80cefc0 con 0x7fbbd80b5c70 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbcf400640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 0x7fbbd814ee50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c3c0 tx=0x7fbbc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0017070 con 0x7fbbd80b5c70 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbc00053c0 con 0x7fbbd80b5c70 2024-09-17T03:38:24.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd80ce720 con 0x7fbbd80b5c70 2024-09-17T03:38:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0003840 con 0x7fbbd80b5c70 2024-09-17T03:38:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.512+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd80cece0 con 0x7fbbd80b5c70 2024-09-17T03:38:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.516+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd80b51f0 con 0x7fbbd80b5c70 2024-09-17T03:38:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.516+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7fbbc001a070 con 0x7fbbd80b5c70 2024-09-17T03:38:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.516+0000 7fbbd4c00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbb4075960 0x7fbbb4077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.516+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7fbbc00927d0 con 0x7fbbd80b5c70 2024-09-17T03:38:24.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.516+0000 7fbbd7e00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbb4075960 0x7fbbb4077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.516+0000 7fbbd7e00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbb4075960 0x7fbbb4077e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009b40 tx=0x7fbbc80091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:24.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.520+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbbd80b5730 con 0x7fbbd80b5c70 2024-09-17T03:38:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbbd806b540 con 0x7fbbb4075960 2024-09-17T03:38:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbd4c00640 1 -- 172.21.15.46:0/3351329474 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbbc8011be0 con 0x7fbbb4075960 2024-09-17T03:38:24.721 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:38:24.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:24.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2024-09-17T03:38:24.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:24.722 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:38:24.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbb4075960 msgr2=0x7fbbb4077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbb4075960 0x7fbbb4077e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009b40 tx=0x7fbbc80091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 msgr2=0x7fbbd814ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 0x7fbbd814ee50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c3c0 tx=0x7fbbc000c880 comp rx=0 tx=0).stop 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 shutdown_connections 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fbbb4075960 0x7fbbb4077e70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fbbd81527b0 0x7fbbd81fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbd80b5c70 0x7fbbd814ee50 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 --2- 172.21.15.46:0/3351329474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbd80b4de0 0x7fbbd80cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.720+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 >> 172.21.15.46:0/3351329474 conn(0x7fbbd80b2040 msgr2=0x7fbbd80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.724+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 shutdown_connections 2024-09-17T03:38:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.724+0000 7fbbdee00640 1 -- 172.21.15.46:0/3351329474 wait complete. 2024-09-17T03:38:24.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T03:38:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 -- 172.21.15.46:0/1055747358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fdc004810 msgr2=0x7f0fdc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 --2- 172.21.15.46:0/1055747358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fdc004810 0x7f0fdc004c20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009b40 tx=0x7f0fe001d220 comp rx=0 tx=0).stop 2024-09-17T03:38:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 -- 172.21.15.46:0/1055747358 shutdown_connections 2024-09-17T03:38:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 --2- 172.21.15.46:0/1055747358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fdc004810 0x7f0fdc004c20 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 -- 172.21.15.46:0/1055747358 >> 172.21.15.46:0/1055747358 conn(0x7f0ff80b2040 msgr2=0x7f0ff80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 -- 172.21.15.46:0/1055747358 shutdown_connections 2024-09-17T03:38:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.904+0000 7f0ffd800640 1 -- 172.21.15.46:0/1055747358 wait complete. 2024-09-17T03:38:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 Processor -- start 2024-09-17T03:38:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 -- start start 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc004810 0x7f0ff81f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 0x7f0ff813f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff81f8060 0x7f0ff81f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80b55d0 con 0x7f0ff813f280 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80b5450 con 0x7f0ff81f8060 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ffd800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff80b5750 con 0x7f0fdc004810 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0fee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 0x7f0ff813f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0fee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 0x7f0ff813f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42204/0 (socket says 172.21.15.46:42204) 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc004810 0x7f0ff81f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0fee000640 1 -- 172.21.15.46:0/3956824024 learned_addr learned my addr 172.21.15.46:0/3956824024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.908+0000 7f0ff7400640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff81f8060 0x7f0ff81f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fee000640 1 -- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc004810 msgr2=0x7f0ff81f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fee000640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc004810 0x7f0ff81f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fee000640 1 -- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff81f8060 msgr2=0x7f0ff81f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fee000640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff81f8060 0x7f0ff81f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fee000640 1 -- 172.21.15.46:0/3956824024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff81576a0 con 0x7f0ff813f280 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0ff6a00640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc004810 0x7f0ff81f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fee000640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 0x7f0ff813f690 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001d050 tx=0x7f0fe0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00bb030 con 0x7f0ff813f280 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fe00046d0 con 0x7f0ff813f280 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff81f8ed0 con 0x7f0ff813f280 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0004b20 con 0x7f0ff813f280 2024-09-17T03:38:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff813fe20 con 0x7f0ff813f280 2024-09-17T03:38:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff81e7220 con 0x7f0ff813f280 2024-09-17T03:38:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.912+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f0fe00be030 con 0x7f0ff813f280 2024-09-17T03:38:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.916+0000 7f0fef400640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd4075960 0x7f0fd4077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:24.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.916+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f0fe012e990 con 0x7f0ff813f280 2024-09-17T03:38:24.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.916+0000 7f0ff6a00640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd4075960 0x7f0fd4077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:24.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.916+0000 7f0ff6a00640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd4075960 0x7f0fd4077e70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80066c0 tx=0x7f0fe8006b80 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:24.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:24.920+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0fe0016f60 con 0x7f0ff813f280 2024-09-17T03:38:25.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.120+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ff806b540 con 0x7f0fd4075960 2024-09-17T03:38:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0fef400640 1 -- 172.21.15.46:0/3956824024 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+379 (secure 0 0 0) 0x7f0fe8012a30 con 0x7f0fd4075960 2024-09-17T03:38:25.129 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:25.129 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:38:25.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone88214" 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sv1" 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone59742" 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone63430" 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone17366" 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.130 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone1673" 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone99875" 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subvol_clone6331" 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:25.131 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:38:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd4075960 msgr2=0x7f0fd4077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd4075960 0x7f0fd4077e70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80066c0 tx=0x7f0fe8006b80 comp rx=0 tx=0).stop 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 msgr2=0x7f0ff813f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 0x7f0ff813f690 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001d050 tx=0x7f0fe0005850 comp rx=0 tx=0).stop 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 shutdown_connections 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fd4075960 0x7f0fd4077e70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ff81f8060 0x7f0ff81f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff813f280 0x7f0ff813f690 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 --2- 172.21.15.46:0/3956824024 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fdc004810 0x7f0ff81f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.128+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 >> 172.21.15.46:0/3956824024 conn(0x7f0ff80b2040 msgr2=0x7f0ff81599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.132+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 shutdown_connections 2024-09-17T03:38:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.132+0000 7f0ffd800640 1 -- 172.21.15.46:0/3956824024 wait complete. 2024-09-17T03:38:25.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone88214 2024-09-17T03:38:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f835d200640 1 -- 172.21.15.46:0/1191575711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 msgr2=0x7f833c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f835d200640 1 --2- 172.21.15.46:0/1191575711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f833c004c20 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f834000c470 tx=0x7f834001f700 comp rx=0 tx=0).stop 2024-09-17T03:38:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f8356800640 1 -- 172.21.15.46:0/1191575711 reap_dead start 2024-09-17T03:38:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f835d200640 1 -- 172.21.15.46:0/1191575711 shutdown_connections 2024-09-17T03:38:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f835d200640 1 -- 172.21.15.46:0/1191575711 >> 172.21.15.46:0/1191575711 conn(0x7f83580b2040 msgr2=0x7f83580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f835d200640 1 -- 172.21.15.46:0/1191575711 shutdown_connections 2024-09-17T03:38:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.315+0000 7f835d200640 1 -- 172.21.15.46:0/1191575711 wait complete. 2024-09-17T03:38:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 Processor -- start 2024-09-17T03:38:25.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 -- start start 2024-09-17T03:38:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f8358149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83581e4d30 0x7f83581e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580ae8e0 0x7f83580aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f835815cf30 con 0x7f833c004810 2024-09-17T03:38:25.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f835815cdb0 con 0x7f83580ae8e0 2024-09-17T03:38:25.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f835815d0b0 con 0x7f83581e4d30 2024-09-17T03:38:25.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f8358149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f8358149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42228/0 (socket says 172.21.15.46:42228) 2024-09-17T03:38:25.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 -- 172.21.15.46:0/1590627273 learned_addr learned my addr 172.21.15.46:0/1590627273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:25.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f834fe00640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83581e4d30 0x7f83581e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8356800640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580ae8e0 0x7f83580aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 -- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83581e4d30 msgr2=0x7f83581e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83581e4d30 0x7f83581e5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 -- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580ae8e0 msgr2=0x7f83580aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580ae8e0 0x7f83580aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 -- 172.21.15.46:0/1590627273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83580af230 con 0x7f833c004810 2024-09-17T03:38:25.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f8355e00640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f8358149b70 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8348011ae0 tx=0x7f8348011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:25.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834801e070 con 0x7f833c004810 2024-09-17T03:38:25.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f834800fdf0 con 0x7f833c004810 2024-09-17T03:38:25.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348019690 con 0x7f833c004810 2024-09-17T03:38:25.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83581f9900 con 0x7f833c004810 2024-09-17T03:38:25.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.319+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581f9ec0 con 0x7f833c004810 2024-09-17T03:38:25.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.323+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83580768c0 con 0x7f833c004810 2024-09-17T03:38:25.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.323+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f8348021070 con 0x7f833c004810 2024-09-17T03:38:25.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.323+0000 7f834e000640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8334075990 0x7f8334077ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.323+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f8348094040 con 0x7f833c004810 2024-09-17T03:38:25.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.323+0000 7f834fe00640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8334075990 0x7f8334077ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.327+0000 7f834fe00640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8334075990 0x7f8334077ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f834000c470 tx=0x7f834001c540 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:25.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.327+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f83580af230 con 0x7f833c004810 2024-09-17T03:38:25.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.531+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone88214", "target": ["mon-mgr", ""]}) -- 0x7f835806b540 con 0x7f8334075990 2024-09-17T03:38:25.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f834e000640 1 -- 172.21.15.46:0/1590627273 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83400b4070 con 0x7f8334075990 2024-09-17T03:38:25.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8334075990 msgr2=0x7f8334077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8334075990 0x7f8334077ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f834000c470 tx=0x7f834001c540 comp rx=0 tx=0).stop 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 msgr2=0x7f8358149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f8358149b70 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8348011ae0 tx=0x7f8348011ee0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 shutdown_connections 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8334075990 0x7f8334077ea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83580ae8e0 0x7f83580aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f83581e4d30 0x7f83581e5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 --2- 172.21.15.46:0/1590627273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f833c004810 0x7f8358149b70 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.547+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 >> 172.21.15.46:0/1590627273 conn(0x7f83580b2040 msgr2=0x7f835815ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.551+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 shutdown_connections 2024-09-17T03:38:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.551+0000 7f835d200640 1 -- 172.21.15.46:0/1590627273 wait complete. 2024-09-17T03:38:25.553 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.551+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.553 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.551+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.553 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.551+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.553 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.551+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.554 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.551+0000 7fbe50e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.571 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.567+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.571 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.567+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.571 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.567+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.571 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.567+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.571 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-09-17T03:38:25.567+0000 7fbe50400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T03:38:25.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T03:38:25.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 -- 172.21.15.46:0/1444818604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b5c70 msgr2=0x7f586c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 --2- 172.21.15.46:0/1444818604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b5c70 0x7f586c15bb60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f586001c6c0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 -- 172.21.15.46:0/1444818604 shutdown_connections 2024-09-17T03:38:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 --2- 172.21.15.46:0/1444818604 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b5c70 0x7f586c15bb60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 -- 172.21.15.46:0/1444818604 >> 172.21.15.46:0/1444818604 conn(0x7f586c0b2040 msgr2=0x7f586c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 -- 172.21.15.46:0/1444818604 shutdown_connections 2024-09-17T03:38:25.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.743+0000 7f5873400640 1 -- 172.21.15.46:0/1444818604 wait complete. 2024-09-17T03:38:25.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 Processor -- start 2024-09-17T03:38:25.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 -- start start 2024-09-17T03:38:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b4de0 0x7f586c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c0b5c70 0x7f586c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 0x7f586c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f586c169d00 con 0x7f586c1527b0 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f586c169b80 con 0x7f586c0b5c70 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5873400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f586c169e80 con 0x7f586c0b4de0 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5871400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 0x7f586c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5871400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 0x7f586c154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42262/0 (socket says 172.21.15.46:42262) 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5871400640 1 -- 172.21.15.46:0/547965221 learned_addr learned my addr 172.21.15.46:0/547965221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f5870a00640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b4de0 0x7f586c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.747+0000 7f586be00640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c0b5c70 0x7f586c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5871400640 1 -- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b4de0 msgr2=0x7f586c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5871400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b4de0 0x7f586c157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5871400640 1 -- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c0b5c70 msgr2=0x7f586c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5871400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c0b5c70 0x7f586c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5871400640 1 -- 172.21.15.46:0/547965221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c153540 con 0x7f586c1527b0 2024-09-17T03:38:25.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5871400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 0x7f586c154800 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f586400cbc0 tx=0x7f586400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864016020 con 0x7f586c1527b0 2024-09-17T03:38:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f586400ab00 con 0x7f586c1527b0 2024-09-17T03:38:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864012420 con 0x7f586c1527b0 2024-09-17T03:38:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c1537e0 con 0x7f586c1527b0 2024-09-17T03:38:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c153da0 con 0x7f586c1527b0 2024-09-17T03:38:25.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c157610 con 0x7f586c1527b0 2024-09-17T03:38:25.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.751+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7f5864019020 con 0x7f586c1527b0 2024-09-17T03:38:25.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.755+0000 7f5868c00640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f584c075990 0x7f584c077ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.755+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f5864093740 con 0x7f586c1527b0 2024-09-17T03:38:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.755+0000 7f5870a00640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f584c075990 0x7f584c077ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:25.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.755+0000 7f5870a00640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f584c075990 0x7f584c077ea0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f58600af000 tx=0x7f586001af00 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.759+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5864020020 con 0x7f586c1527b0 2024-09-17T03:38:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.963+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f586c06b540 con 0x7f584c075990 2024-09-17T03:38:25.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5868c00640 1 -- 172.21.15.46:0/547965221 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f58600040c0 con 0x7f584c075990 2024-09-17T03:38:25.976 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:25.976 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-09-17T03:38:25.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-09-17T03:38:25.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ss1" 2024-09-17T03:38:25.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-17T03:38:25.976 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-09-17T03:38:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f584c075990 msgr2=0x7f584c077ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f584c075990 0x7f584c077ea0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f58600af000 tx=0x7f586001af00 comp rx=0 tx=0).stop 2024-09-17T03:38:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 msgr2=0x7f586c154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 0x7f586c154800 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f586400cbc0 tx=0x7f586400ccb0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 shutdown_connections 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f584c075990 0x7f584c077ea0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f586c1527b0 0x7f586c154800 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f586c0b5c70 0x7f586c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 --2- 172.21.15.46:0/547965221 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f586c0b4de0 0x7f586c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 >> 172.21.15.46:0/547965221 conn(0x7f586c0b2040 msgr2=0x7f586c154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 shutdown_connections 2024-09-17T03:38:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:25.975+0000 7f5873400640 1 -- 172.21.15.46:0/547965221 wait complete. 2024-09-17T03:38:26.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T03:38:26.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 -- 172.21.15.46:0/2033230454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908074950 msgr2=0x7fa9080b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2033230454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908074950 0x7fa9080b5260 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa908073cf0 tx=0x7fa90001ccf0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 -- 172.21.15.46:0/2033230454 shutdown_connections 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2033230454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908074950 0x7fa9080b5260 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2033230454 >> v2:172.21.15.99:3300/0 conn(0x7fa9081536f0 0x7fa90815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 -- 172.21.15.46:0/2033230454 >> 172.21.15.46:0/2033230454 conn(0x7fa9080b4560 msgr2=0x7fa9080d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 -- 172.21.15.46:0/2033230454 shutdown_connections 2024-09-17T03:38:26.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 -- 172.21.15.46:0/2033230454 wait complete. 2024-09-17T03:38:26.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 Processor -- start 2024-09-17T03:38:26.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.163+0000 7fa90ea00640 1 -- start start 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9081536f0 0x7fa90814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 0x7fa9080d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9080d1870 0x7fa908153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa908169e10 con 0x7fa908160060 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa908169c90 con 0x7fa9081536f0 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fa908169f90 con 0x7fa9080d1870 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 0x7fa9080d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 0x7fa9080d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42276/0 (socket says 172.21.15.46:42276) 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 -- 172.21.15.46:0/2165441016 learned_addr learned my addr 172.21.15.46:0/2165441016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90d600640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9081536f0 0x7fa90814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90e000640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9080d1870 0x7fa908153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 -- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9080d1870 msgr2=0x7fa908153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9080d1870 0x7fa908153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 -- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9081536f0 msgr2=0x7fa90814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9081536f0 0x7fa90814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 -- 172.21.15.46:0/2165441016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90814fdc0 con 0x7fa908160060 2024-09-17T03:38:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90cc00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 0x7fa9080d1330 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c3a0 tx=0x7fa8f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4017070 con 0x7fa908160060 2024-09-17T03:38:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8f4003600 con 0x7fa908160060 2024-09-17T03:38:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4003a50 con 0x7fa908160060 2024-09-17T03:38:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9081500a0 con 0x7fa908160060 2024-09-17T03:38:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa908154d50 con 0x7fa908160060 2024-09-17T03:38:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.167+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9080b5260 con 0x7fa908160060 2024-09-17T03:38:26.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.171+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) ==== 100804+0+0 (secure 0 0 0) 0x7fa8f401a070 con 0x7fa908160060 2024-09-17T03:38:26.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.171+0000 7fa905600640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8e8075960 0x7fa8e8077e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.171+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7fa8f4092c60 con 0x7fa908160060 2024-09-17T03:38:26.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.171+0000 7fa90d600640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8e8075960 0x7fa8e8077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.171+0000 7fa90d600640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8e8075960 0x7fa8e8077e70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc004630 tx=0x7fa8fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:26.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.175+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8f4021020 con 0x7fa908160060 2024-09-17T03:38:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.379+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa90806b540 con 0x7fa8e8075960 2024-09-17T03:38:26.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.411+0000 7fa905600640 1 -- 172.21.15.46:0/2165441016 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8fc004280 con 0x7fa8e8075960 2024-09-17T03:38:26.416 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8e8075960 msgr2=0x7fa8e8077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8e8075960 0x7fa8e8077e70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc004630 tx=0x7fa8fc00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 msgr2=0x7fa9080d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 0x7fa9080d1330 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c3a0 tx=0x7fa8f400c860 comp rx=0 tx=0).stop 2024-09-17T03:38:26.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 shutdown_connections 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fa8e8075960 0x7fa8e8077e70 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fa9080d1870 0x7fa908153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa908160060 0x7fa9080d1330 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 --2- 172.21.15.46:0/2165441016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9081536f0 0x7fa90814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 >> 172.21.15.46:0/2165441016 conn(0x7fa9080b4560 msgr2=0x7fa908071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 shutdown_connections 2024-09-17T03:38:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.415+0000 7fa90ea00640 1 -- 172.21.15.46:0/2165441016 wait complete. 2024-09-17T03:38:26.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T03:38:26.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 -- 172.21.15.46:0/2332274237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74600738b0 msgr2=0x7f7460075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 --2- 172.21.15.46:0/2332274237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74600738b0 0x7f7460075010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7450009870 tx=0x7f745001c830 comp rx=0 tx=0).stop 2024-09-17T03:38:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 -- 172.21.15.46:0/2332274237 shutdown_connections 2024-09-17T03:38:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 --2- 172.21.15.46:0/2332274237 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74600738b0 0x7f7460075010 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 -- 172.21.15.46:0/2332274237 >> 172.21.15.46:0/2332274237 conn(0x7f74600b2040 msgr2=0x7f74600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 -- 172.21.15.46:0/2332274237 shutdown_connections 2024-09-17T03:38:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.603+0000 7f7467000640 1 -- 172.21.15.46:0/2332274237 wait complete. 2024-09-17T03:38:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 Processor -- start 2024-09-17T03:38:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 -- start start 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 0x7f7460145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74600b4de0 0x7f746014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f746015cc60 0x7f7460151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7460169d40 con 0x7f74600738b0 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7460169ec0 con 0x7f74600b4de0 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7467000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f746016a040 con 0x7f746015cc60 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f745fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 0x7f7460145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f745fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 0x7f7460145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42286/0 (socket says 172.21.15.46:42286) 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f745fe00640 1 -- 172.21.15.46:0/3609768002 learned_addr learned my addr 172.21.15.46:0/3609768002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7465000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f746015cc60 0x7f7460151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.607+0000 7f7457e00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74600b4de0 0x7f746014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745fe00640 1 -- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f746015cc60 msgr2=0x7f7460151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745fe00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f746015cc60 0x7f7460151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745fe00640 1 -- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74600b4de0 msgr2=0x7f746014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745fe00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74600b4de0 0x7f746014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745fe00640 1 -- 172.21.15.46:0/3609768002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7460145fb0 con 0x7f74600738b0 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f7465000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f746015cc60 0x7f7460151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745fe00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 0x7f7460145800 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f7450019440 tx=0x7f7450005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500b2040 con 0x7f74600738b0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7450004130 con 0x7f74600738b0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500044e0 con 0x7f74600738b0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7460146290 con 0x7f74600738b0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7460146850 con 0x7f74600738b0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7460075010 con 0x7f74600738b0 2024-09-17T03:38:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.611+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f74500bd030 con 0x7f74600738b0 2024-09-17T03:38:26.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.615+0000 7f745cc00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f743c075c20 0x7f743c078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:26.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.615+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7f745012dc40 con 0x7f74600738b0 2024-09-17T03:38:26.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.615+0000 7f7457e00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f743c075c20 0x7f743c078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:26.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.615+0000 7f7457e00640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f743c075c20 0x7f743c078130 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f746014c6d0 tx=0x7f7448009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:26.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.615+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74500c4020 con 0x7f74600738b0 2024-09-17T03:38:26.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.815+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f746006b540 con 0x7f743c075c20 2024-09-17T03:38:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f745cc00640 1 -- 172.21.15.46:0/3609768002 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7448001440 con 0x7f743c075c20 2024-09-17T03:38:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f743c075c20 msgr2=0x7f743c078130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f743c075c20 0x7f743c078130 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f746014c6d0 tx=0x7f7448009040 comp rx=0 tx=0).stop 2024-09-17T03:38:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 msgr2=0x7f7460145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 0x7f7460145800 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f7450019440 tx=0x7f7450005d20 comp rx=0 tx=0).stop 2024-09-17T03:38:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 shutdown_connections 2024-09-17T03:38:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f743c075c20 0x7f743c078130 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f746015cc60 0x7f7460151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74600b4de0 0x7f746014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 --2- 172.21.15.46:0/3609768002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74600738b0 0x7f7460145800 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 >> 172.21.15.46:0/3609768002 conn(0x7f74600b2040 msgr2=0x7f74600b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 shutdown_connections 2024-09-17T03:38:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:26.835+0000 7f7467000640 1 -- 172.21.15.46:0/3609768002 wait complete. 2024-09-17T03:38:26.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone59742 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 -- 172.21.15.46:0/2534839941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900cefa0 msgr2=0x7faf900cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 --2- 172.21.15.46:0/2534839941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900cefa0 0x7faf900cf3b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7faf900b3040 tx=0x7faf7c01c8c0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 -- 172.21.15.46:0/2534839941 shutdown_connections 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 --2- 172.21.15.46:0/2534839941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900cefa0 0x7faf900cf3b0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 --2- 172.21.15.46:0/2534839941 >> v2:172.21.15.99:3300/0 conn(0x7faf900749a0 0x7faf90074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 -- 172.21.15.46:0/2534839941 >> 172.21.15.46:0/2534839941 conn(0x7faf900b2040 msgr2=0x7faf900b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 -- 172.21.15.46:0/2534839941 shutdown_connections 2024-09-17T03:38:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.027+0000 7faf97400640 1 -- 172.21.15.46:0/2534839941 wait complete. 2024-09-17T03:38:27.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 Processor -- start 2024-09-17T03:38:27.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 -- start start 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf900749a0 0x7faf9012d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf900cefa0 0x7faf901f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 0x7faf9013bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf9016a060 con 0x7faf90160330 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf90169ee0 con 0x7faf900749a0 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf97400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faf9016a1e0 con 0x7faf900cefa0 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 0x7faf9013bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 0x7faf9013bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42306/0 (socket says 172.21.15.46:42306) 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 -- 172.21.15.46:0/1443119498 learned_addr learned my addr 172.21.15.46:0/1443119498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:27.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf8fe00640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf900cefa0 0x7faf901f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf94a00640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf900749a0 0x7faf9012d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 -- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf900cefa0 msgr2=0x7faf901f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf900cefa0 0x7faf901f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 -- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf900749a0 msgr2=0x7faf9012d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf900749a0 0x7faf9012d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 -- 172.21.15.46:0/1443119498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9012daa0 con 0x7faf90160330 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.031+0000 7faf95400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 0x7faf9013bd60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c760 tx=0x7faf8800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88003b80 con 0x7faf90160330 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf88003d20 con 0x7faf90160330 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8800aab0 con 0x7faf90160330 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf901f8b30 con 0x7faf90160330 2024-09-17T03:38:27.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf901f90f0 con 0x7faf90160330 2024-09-17T03:38:27.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf90074db0 con 0x7faf90160330 2024-09-17T03:38:27.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7faf8800ac50 con 0x7faf90160330 2024-09-17T03:38:27.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf8cc00640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf70075a80 0x7faf70077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5766+0+0 (secure 0 0 0) 0x7faf88093230 con 0x7faf90160330 2024-09-17T03:38:27.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.035+0000 7faf94a00640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf70075a80 0x7faf70077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.039+0000 7faf94a00640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf70075a80 0x7faf70077f90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7faf84009b40 tx=0x7faf840091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:27.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.039+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf8805f660 con 0x7faf90160330 2024-09-17T03:38:27.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.243+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone59742", "target": ["mon-mgr", ""]}) -- 0x7faf901461f0 con 0x7faf70075a80 2024-09-17T03:38:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf8cc00640 1 -- 172.21.15.46:0/1443119498 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf901461f0 con 0x7faf70075a80 2024-09-17T03:38:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf70075a80 msgr2=0x7faf70077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf70075a80 0x7faf70077f90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7faf84009b40 tx=0x7faf840091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 msgr2=0x7faf9013bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 0x7faf9013bd60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c760 tx=0x7faf8800cc20 comp rx=0 tx=0).stop 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 shutdown_connections 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf70075a80 0x7faf70077f90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90160330 0x7faf9013bd60 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf900cefa0 0x7faf901f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 --2- 172.21.15.46:0/1443119498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf900749a0 0x7faf9012d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.259+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 >> 172.21.15.46:0/1443119498 conn(0x7faf900b2040 msgr2=0x7faf9015b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.263+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 shutdown_connections 2024-09-17T03:38:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.263+0000 7faf97400640 1 -- 172.21.15.46:0/1443119498 wait complete. 2024-09-17T03:38:27.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone63430 2024-09-17T03:38:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 -- 172.21.15.46:0/3780914029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 msgr2=0x7faf3c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 --2- 172.21.15.46:0/3780914029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf3c00b9d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e90 tx=0x7faf5001d5f0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 -- 172.21.15.46:0/3780914029 shutdown_connections 2024-09-17T03:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 --2- 172.21.15.46:0/3780914029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf3c00b9d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 -- 172.21.15.46:0/3780914029 >> 172.21.15.46:0/3780914029 conn(0x7faf580b2040 msgr2=0x7faf580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 -- 172.21.15.46:0/3780914029 shutdown_connections 2024-09-17T03:38:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.455+0000 7faf5f000640 1 -- 172.21.15.46:0/3780914029 wait complete. 2024-09-17T03:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.459+0000 7faf5f000640 1 Processor -- start 2024-09-17T03:38:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.459+0000 7faf5f000640 1 -- start start 2024-09-17T03:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf5815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf58159d30 0x7faf58159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf5815deb0 0x7faf5813e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf58075190 con 0x7faf3c005260 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf58075010 con 0x7faf5815deb0 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faf58075310 con 0x7faf58159d30 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf5815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf5815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42332/0 (socket says 172.21.15.46:42332) 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 -- 172.21.15.46:0/4278163911 learned_addr learned my addr 172.21.15.46:0/4278163911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5d000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf5815deb0 0x7faf5813e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf4fe00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf58159d30 0x7faf58159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 -- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf58159d30 msgr2=0x7faf58159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf58159d30 0x7faf58159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 -- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf5815deb0 msgr2=0x7faf5813e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf5815deb0 0x7faf5813e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 -- 172.21.15.46:0/4278163911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf580cd510 con 0x7faf3c005260 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf57e00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf5815f0f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7faf48004da0 tx=0x7faf48001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48001d50 con 0x7faf3c005260 2024-09-17T03:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf48016070 con 0x7faf3c005260 2024-09-17T03:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf58159880 con 0x7faf3c005260 2024-09-17T03:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48010b80 con 0x7faf3c005260 2024-09-17T03:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5815fda0 con 0x7faf3c005260 2024-09-17T03:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.463+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf580cf650 con 0x7faf3c005260 2024-09-17T03:38:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.467+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7faf4801b460 con 0x7faf3c005260 2024-09-17T03:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.467+0000 7faf54c00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf34075a80 0x7faf34077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.467+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7faf48094690 con 0x7faf3c005260 2024-09-17T03:38:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.467+0000 7faf4fe00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf34075a80 0x7faf34077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.467+0000 7faf4fe00640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf34075a80 0x7faf34077f90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faf40001870 tx=0x7faf40006040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.471+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf48060bd0 con 0x7faf3c005260 2024-09-17T03:38:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.691+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone63430", "target": ["mon-mgr", ""]}) -- 0x7faf5806b540 con 0x7faf34075a80 2024-09-17T03:38:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.727+0000 7faf54c00640 1 -- 172.21.15.46:0/4278163911 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf40016070 con 0x7faf34075a80 2024-09-17T03:38:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf34075a80 msgr2=0x7faf34077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf34075a80 0x7faf34077f90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7faf40001870 tx=0x7faf40006040 comp rx=0 tx=0).stop 2024-09-17T03:38:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 msgr2=0x7faf5815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf5815f0f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7faf48004da0 tx=0x7faf48001640 comp rx=0 tx=0).stop 2024-09-17T03:38:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 shutdown_connections 2024-09-17T03:38:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faf34075a80 0x7faf34077f90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf5815deb0 0x7faf5813e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faf58159d30 0x7faf58159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 --2- 172.21.15.46:0/4278163911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c005260 0x7faf5815f0f0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 >> 172.21.15.46:0/4278163911 conn(0x7faf580b2040 msgr2=0x7faf58140c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 shutdown_connections 2024-09-17T03:38:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.731+0000 7faf5f000640 1 -- 172.21.15.46:0/4278163911 wait complete. 2024-09-17T03:38:27.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone17366 2024-09-17T03:38:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 -- 172.21.15.46:0/2470019367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc54160470 msgr2=0x7fcc54074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 --2- 172.21.15.46:0/2470019367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc54160470 0x7fcc54074780 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fcc54073cf0 tx=0x7fcc5001ccf0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 -- 172.21.15.46:0/2470019367 shutdown_connections 2024-09-17T03:38:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 --2- 172.21.15.46:0/2470019367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc54160470 0x7fcc54074780 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 --2- 172.21.15.46:0/2470019367 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc54160060 0x7fcc54074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 --2- 172.21.15.46:0/2470019367 >> v2:172.21.15.99:3300/0 conn(0x7fcc541536f0 0x7fcc5415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 -- 172.21.15.46:0/2470019367 >> 172.21.15.46:0/2470019367 conn(0x7fcc540b4560 msgr2=0x7fcc540d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 -- 172.21.15.46:0/2470019367 shutdown_connections 2024-09-17T03:38:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.983+0000 7fcc5b200640 1 -- 172.21.15.46:0/2470019367 wait complete. 2024-09-17T03:38:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 Processor -- start 2024-09-17T03:38:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 -- start start 2024-09-17T03:38:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 0x7fcc54154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc54160060 0x7fcc5413ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc54160470 0x7fcc5413f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54071e00 con 0x7fcc541536f0 2024-09-17T03:38:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54071c80 con 0x7fcc54160470 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fcc54071f80 con 0x7fcc54160060 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 0x7fcc54154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 0x7fcc54154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42344/0 (socket says 172.21.15.46:42344) 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 -- 172.21.15.46:0/255113280 learned_addr learned my addr 172.21.15.46:0/255113280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5a800640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc54160470 0x7fcc5413f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59400640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc54160060 0x7fcc5413ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 -- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc54160060 msgr2=0x7fcc5413ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc54160060 0x7fcc5413ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 -- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc54160470 msgr2=0x7fcc5413f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc54160470 0x7fcc5413f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:27.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 -- 172.21.15.46:0/255113280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc54154fc0 con 0x7fcc541536f0 2024-09-17T03:38:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc59e00640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 0x7fcc54154a80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e7c0 tx=0x7fcc4800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4800ccc0 con 0x7fcc541536f0 2024-09-17T03:38:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc4800ee70 con 0x7fcc541536f0 2024-09-17T03:38:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc54151f80 con 0x7fcc541536f0 2024-09-17T03:38:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48010690 con 0x7fcc541536f0 2024-09-17T03:38:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.987+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc54152540 con 0x7fcc541536f0 2024-09-17T03:38:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.991+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc540724c0 con 0x7fcc541536f0 2024-09-17T03:38:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.991+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fcc48010830 con 0x7fcc541536f0 2024-09-17T03:38:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.991+0000 7fcc4e000640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcc38075a80 0x7fcc38077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.991+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7fcc48093f60 con 0x7fcc541536f0 2024-09-17T03:38:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.991+0000 7fcc59400640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcc38075a80 0x7fcc38077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.991+0000 7fcc59400640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcc38075a80 0x7fcc38077f90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc54157050 tx=0x7fcc440091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:27.995+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcc48060310 con 0x7fcc541536f0 2024-09-17T03:38:28.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.195+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone17366", "target": ["mon-mgr", ""]}) -- 0x7fcc5406b540 con 0x7fcc38075a80 2024-09-17T03:38:28.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.223+0000 7fcc4e000640 1 -- 172.21.15.46:0/255113280 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcc44011ca0 con 0x7fcc38075a80 2024-09-17T03:38:28.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.223+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcc38075a80 msgr2=0x7fcc38077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.223+0000 7fcc5b200640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcc38075a80 0x7fcc38077f90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc54157050 tx=0x7fcc440091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.223+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 msgr2=0x7fcc54154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.223+0000 7fcc5b200640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 0x7fcc54154a80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e7c0 tx=0x7fcc4800ec80 comp rx=0 tx=0).stop 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 shutdown_connections 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fcc38075a80 0x7fcc38077f90 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc54160470 0x7fcc5413f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fcc54160060 0x7fcc5413ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 --2- 172.21.15.46:0/255113280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc541536f0 0x7fcc54154a80 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 >> 172.21.15.46:0/255113280 conn(0x7fcc540b4560 msgr2=0x7fcc540b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 shutdown_connections 2024-09-17T03:38:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.227+0000 7fcc5b200640 1 -- 172.21.15.46:0/255113280 wait complete. 2024-09-17T03:38:28.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1673 2024-09-17T03:38:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 -- 172.21.15.46:0/1602457305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8160d50 msgr2=0x7f74d8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 --2- 172.21.15.46:0/1602457305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8160d50 0x7f74d8074410 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f74d80713e0 tx=0x7f74c001c830 comp rx=0 tx=0).stop 2024-09-17T03:38:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 -- 172.21.15.46:0/1602457305 shutdown_connections 2024-09-17T03:38:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 --2- 172.21.15.46:0/1602457305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8160d50 0x7f74d8074410 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 --2- 172.21.15.46:0/1602457305 >> v2:172.21.15.99:3300/0 conn(0x7f74d8157030 0x7f74d80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 -- 172.21.15.46:0/1602457305 >> 172.21.15.46:0/1602457305 conn(0x7f74d80708c0 msgr2=0x7f74d8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 -- 172.21.15.46:0/1602457305 shutdown_connections 2024-09-17T03:38:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.415+0000 7f74dde00640 1 -- 172.21.15.46:0/1602457305 wait complete. 2024-09-17T03:38:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 Processor -- start 2024-09-17T03:38:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 -- start start 2024-09-17T03:38:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 0x7f74d8148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d8160d50 0x7f74d81f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d8161160 0x7f74d806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74d816a740 con 0x7f74d8157030 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f74d816a5c0 con 0x7f74d8160d50 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f74d816a8c0 con 0x7f74d8161160 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 0x7f74d8148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 0x7f74d8148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42354/0 (socket says 172.21.15.46:42354) 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 -- 172.21.15.46:0/4207075891 learned_addr learned my addr 172.21.15.46:0/4207075891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 -- 172.21.15.46:0/4207075891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d8161160 msgr2=0x7f74d806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d8161160 0x7f74d806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 -- 172.21.15.46:0/4207075891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d8160d50 msgr2=0x7f74d81f9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d8160d50 0x7f74d81f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 -- 172.21.15.46:0/4207075891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8151260 con 0x7f74d8157030 2024-09-17T03:38:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74d6a00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 0x7f74d8148570 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e7f0 tx=0x7f74c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800ccc0 con 0x7f74d8157030 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74c800ce80 con 0x7f74d8157030 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d8151540 con 0x7f74d8157030 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8005280 con 0x7f74d8157030 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.419+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d806d310 con 0x7f74d8157030 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.423+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d814e290 con 0x7f74d8157030 2024-09-17T03:38:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.423+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f74c8005420 con 0x7f74d8157030 2024-09-17T03:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.423+0000 7f74cf400640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74b4075a80 0x7f74b4077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.423+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f74c80944b0 con 0x7f74d8157030 2024-09-17T03:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.423+0000 7f74cfe00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74b4075a80 0x7f74b4077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:28.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.427+0000 7f74cfe00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74b4075a80 0x7f74b4077f90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f74c00af000 tx=0x7f74c00195d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:28.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.427+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74c8060fb0 con 0x7f74d8157030 2024-09-17T03:38:28.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.631+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1673", "target": ["mon-mgr", ""]}) -- 0x7f74d806ee20 con 0x7f74b4075a80 2024-09-17T03:38:28.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.643+0000 7f74cf400640 1 -- 172.21.15.46:0/4207075891 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f74d806ee20 con 0x7f74b4075a80 2024-09-17T03:38:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74b4075a80 msgr2=0x7f74b4077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74b4075a80 0x7f74b4077f90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f74c00af000 tx=0x7f74c00195d0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 msgr2=0x7f74d8148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 0x7f74d8148570 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e7f0 tx=0x7f74c800ecb0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 shutdown_connections 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74b4075a80 0x7f74b4077f90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f74d8161160 0x7f74d806e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d8160d50 0x7f74d81f9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 --2- 172.21.15.46:0/4207075891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74d8157030 0x7f74d8148570 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 >> 172.21.15.46:0/4207075891 conn(0x7f74d80708c0 msgr2=0x7f74d8152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 shutdown_connections 2024-09-17T03:38:28.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.647+0000 7f74dde00640 1 -- 172.21.15.46:0/4207075891 wait complete. 2024-09-17T03:38:28.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99875 2024-09-17T03:38:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 -- 172.21.15.46:0/3691989736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 msgr2=0x7f9ad00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3691989736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad00b4de0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009840 tx=0x7f9ab801c760 comp rx=0 tx=0).stop 2024-09-17T03:38:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 -- 172.21.15.46:0/3691989736 shutdown_connections 2024-09-17T03:38:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3691989736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad00b4de0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3691989736 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ad00738b0 0x7f9ad0152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 -- 172.21.15.46:0/3691989736 >> 172.21.15.46:0/3691989736 conn(0x7f9ad006e1d0 msgr2=0x7f9ad006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 -- 172.21.15.46:0/3691989736 shutdown_connections 2024-09-17T03:38:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.955+0000 7f9ad6800640 1 -- 172.21.15.46:0/3691989736 wait complete. 2024-09-17T03:38:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.959+0000 7f9ad6800640 1 Processor -- start 2024-09-17T03:38:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.959+0000 7f9ad6800640 1 -- start start 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ad6800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ad00738b0 0x7f9ad0157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ad6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ad6800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ad00b5320 0x7f9ad014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ad6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b4060 con 0x7f9ad0075010 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ad6800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b3ee0 con 0x7f9ad00738b0 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ad6800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad00b41e0 con 0x7f9ad00b5320 2024-09-17T03:38:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9acf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ad00738b0 0x7f9ad0157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9acf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ad00738b0 0x7f9ad0157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43260/0 (socket says 172.21.15.46:43260) 2024-09-17T03:38:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9acf400640 1 -- 172.21.15.46:0/3604611801 learned_addr learned my addr 172.21.15.46:0/3604611801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9acfe00640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ad00b5320 0x7f9ad014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 -- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ad00b5320 msgr2=0x7f9ad014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ad00b5320 0x7f9ad014d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 -- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ad00738b0 msgr2=0x7f9ad0157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ad00738b0 0x7f9ad0157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 -- 172.21.15.46:0/3604611801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad0155f00 con 0x7f9ad0075010 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.963+0000 7f9ac7e00640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad014cad0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80ac000 tx=0x7f9ab80a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.967+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80ba020 con 0x7f9ad0075010 2024-09-17T03:38:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.967+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ab80043e0 con 0x7f9ad0075010 2024-09-17T03:38:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.967+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80b2040 con 0x7f9ad0075010 2024-09-17T03:38:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.967+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad00cd5e0 con 0x7f9ad0075010 2024-09-17T03:38:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.967+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad0156a60 con 0x7f9ad0075010 2024-09-17T03:38:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.967+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad0152580 con 0x7f9ad0075010 2024-09-17T03:38:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.975+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f9ab80053a0 con 0x7f9ad0075010 2024-09-17T03:38:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.975+0000 7f9ac7400640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9aac075a80 0x7f9aac077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.975+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f9ab812d350 con 0x7f9ad0075010 2024-09-17T03:38:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.975+0000 7f9acf400640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9aac075a80 0x7f9aac077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.979+0000 7f9acf400640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9aac075a80 0x7f9aac077f90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009c00 tx=0x7f9ac00091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:28.979+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ab8016410 con 0x7f9ad0075010 2024-09-17T03:38:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.395+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99875", "target": ["mon-mgr", ""]}) -- 0x7f9ad0155c20 con 0x7f9aac075a80 2024-09-17T03:38:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.435+0000 7f9ac7400640 1 -- 172.21.15.46:0/3604611801 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ad0155c20 con 0x7f9aac075a80 2024-09-17T03:38:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9aac075a80 msgr2=0x7f9aac077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9aac075a80 0x7f9aac077f90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009c00 tx=0x7f9ac00091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 msgr2=0x7f9ad014cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad014cad0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80ac000 tx=0x7f9ab80a5780 comp rx=0 tx=0).stop 2024-09-17T03:38:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 shutdown_connections 2024-09-17T03:38:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9aac075a80 0x7f9aac077f90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9ad00b5320 0x7f9ad014d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ad0075010 0x7f9ad014cad0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 --2- 172.21.15.46:0/3604611801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ad00738b0 0x7f9ad0157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 >> 172.21.15.46:0/3604611801 conn(0x7f9ad006e1d0 msgr2=0x7f9ad015ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:29.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 shutdown_connections 2024-09-17T03:38:29.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.439+0000 7f9ad6800640 1 -- 172.21.15.46:0/3604611801 wait complete. 2024-09-17T03:38:29.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6331 2024-09-17T03:38:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 -- 172.21.15.46:0/321889980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 msgr2=0x7faa6c0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 --2- 172.21.15.46:0/321889980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c0700a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0d14d0 tx=0x7faa6801caf0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 -- 172.21.15.46:0/321889980 shutdown_connections 2024-09-17T03:38:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 --2- 172.21.15.46:0/321889980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c0700a0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 --2- 172.21.15.46:0/321889980 >> v2:172.21.15.99:3300/0 conn(0x7faa6c06cdf0 0x7faa6c06d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 -- 172.21.15.46:0/321889980 >> 172.21.15.46:0/321889980 conn(0x7faa6c0b35f0 msgr2=0x7faa6c0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 -- 172.21.15.46:0/321889980 shutdown_connections 2024-09-17T03:38:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 -- 172.21.15.46:0/321889980 wait complete. 2024-09-17T03:38:29.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 Processor -- start 2024-09-17T03:38:29.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.639+0000 7faa74800640 1 -- start start 2024-09-17T03:38:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa6c06cdf0 0x7faa6c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6c159130 0x7faa6c1fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c168da0 con 0x7faa6c06fc90 2024-09-17T03:38:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c168c20 con 0x7faa6c159130 2024-09-17T03:38:29.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c168f20 con 0x7faa6c06cdf0 2024-09-17T03:38:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42394/0 (socket says 172.21.15.46:42394) 2024-09-17T03:38:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 -- 172.21.15.46:0/1010988690 learned_addr learned my addr 172.21.15.46:0/1010988690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71e00640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa6c06cdf0 0x7faa6c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:29.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa72800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6c159130 0x7faa6c1fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 -- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa6c06cdf0 msgr2=0x7faa6c156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa6c06cdf0 0x7faa6c156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 -- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6c159130 msgr2=0x7faa6c1fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6c159130 0x7faa6c1fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 -- 172.21.15.46:0/1010988690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c1603e0 con 0x7faa6c06fc90 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa71400640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c153a60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00e7d0 tx=0x7faa5c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:29.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00ccc0 con 0x7faa6c06fc90 2024-09-17T03:38:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5c010430 con 0x7faa6c06fc90 2024-09-17T03:38:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c010860 con 0x7faa6c06fc90 2024-09-17T03:38:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c1606c0 con 0x7faa6c06fc90 2024-09-17T03:38:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c160c80 con 0x7faa6c06fc90 2024-09-17T03:38:29.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.643+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c06d200 con 0x7faa6c06fc90 2024-09-17T03:38:29.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.647+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7faa5c010a00 con 0x7faa6c06fc90 2024-09-17T03:38:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.647+0000 7faa66000640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa50075a80 0x7faa50077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:29.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.647+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7faa5c0946d0 con 0x7faa6c06fc90 2024-09-17T03:38:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.647+0000 7faa71e00640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa50075a80 0x7faa50077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.647+0000 7faa71e00640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa50075a80 0x7faa50077f90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faa60000f70 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:29.651+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faa5c0612d0 con 0x7faa6c06fc90 2024-09-17T03:38:30.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.015+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6331", "target": ["mon-mgr", ""]}) -- 0x7faa6c06b540 con 0x7faa50075a80 2024-09-17T03:38:30.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.043+0000 7faa66000640 1 -- 172.21.15.46:0/1010988690 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa600050d0 con 0x7faa50075a80 2024-09-17T03:38:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa50075a80 msgr2=0x7faa50077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa50075a80 0x7faa50077f90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faa60000f70 tx=0x7faa60009040 comp rx=0 tx=0).stop 2024-09-17T03:38:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 msgr2=0x7faa6c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c153a60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00e7d0 tx=0x7faa5c00ec90 comp rx=0 tx=0).stop 2024-09-17T03:38:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 shutdown_connections 2024-09-17T03:38:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7faa50075a80 0x7faa50077f90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa6c159130 0x7faa6c1fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa6c06fc90 0x7faa6c153a60 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 --2- 172.21.15.46:0/1010988690 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7faa6c06cdf0 0x7faa6c156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 >> 172.21.15.46:0/1010988690 conn(0x7faa6c0b35f0 msgr2=0x7faa6c073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 shutdown_connections 2024-09-17T03:38:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.047+0000 7faa74800640 1 -- 172.21.15.46:0/1010988690 wait complete. 2024-09-17T03:38:30.092 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T03:38:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T03:38:30.193 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-09-17T03:38:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:38:30.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.403+0000 7fe04e200640 1 -- 172.21.15.46:0/2798017116 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 msgr2=0x7fe048075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.403+0000 7fe04e200640 1 --2- 172.21.15.46:0/2798017116 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe048075010 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe038009840 tx=0x7fe03801c7f0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.403+0000 7fe04e200640 1 -- 172.21.15.46:0/2798017116 shutdown_connections 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.403+0000 7fe04e200640 1 --2- 172.21.15.46:0/2798017116 >> v2:172.21.15.99:3300/0 conn(0x7fe0480b4de0 0x7fe0480b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.403+0000 7fe04e200640 1 --2- 172.21.15.46:0/2798017116 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe048075010 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.403+0000 7fe04e200640 1 -- 172.21.15.46:0/2798017116 >> 172.21.15.46:0/2798017116 conn(0x7fe0480b2040 msgr2=0x7fe0480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 -- 172.21.15.46:0/2798017116 shutdown_connections 2024-09-17T03:38:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 -- 172.21.15.46:0/2798017116 wait complete. 2024-09-17T03:38:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 Processor -- start 2024-09-17T03:38:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 -- start start 2024-09-17T03:38:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe0480cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 0x7fe048143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815cc60 0x7fe048158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe048168b20 con 0x7fe0480b4de0 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe048152a50 con 0x7fe04815cc60 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe04e200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fe048168ca0 con 0x7fe0480738b0 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.407+0000 7fe03fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 0x7fe048143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 0x7fe048143ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42404/0 (socket says 172.21.15.46:42404) 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03fe00640 1 -- 172.21.15.46:0/232610775 learned_addr learned my addr 172.21.15.46:0/232610775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe0480cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047e00640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815cc60 0x7fe048158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 -- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815cc60 msgr2=0x7fe048158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815cc60 0x7fe048158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 -- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 msgr2=0x7fe048143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 0x7fe048143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 -- 172.21.15.46:0/232610775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0480cdd40 con 0x7fe0480738b0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047e00640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815cc60 0x7fe048158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03fe00640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 0x7fe048143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe047400640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe0480cd4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0480706f0 tx=0x7fe0380a57c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0380b2040 con 0x7fe0480738b0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe038004430 con 0x7fe0480738b0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038005290 con 0x7fe0480738b0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0480ce020 con 0x7fe0480738b0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0480ce5e0 con 0x7fe0480738b0 2024-09-17T03:38:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe048075010 con 0x7fe0480738b0 2024-09-17T03:38:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.411+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fe038005430 con 0x7fe0480738b0 2024-09-17T03:38:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.415+0000 7fe03f400640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe024075a80 0x7fe024077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.415+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7fe03812cd10 con 0x7fe0480738b0 2024-09-17T03:38:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.415+0000 7fe03fe00640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe024075a80 0x7fe024077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:30.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.415+0000 7fe03fe00640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe024075a80 0x7fe024077f90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe0300096c0 tx=0x7fe030009260 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.415+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe038016410 con 0x7fe0480738b0 2024-09-17T03:38:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.591+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe0480b4bc0 con 0x7fe0480738b0 2024-09-17T03:38:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe03f400640 1 -- 172.21.15.46:0/232610775 <== mon.2 v2:172.21.15.99:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v60)=0 v60) ==== 131+0+0 (secure 0 0 0) 0x7fe0380f91c0 con 0x7fe0480738b0 2024-09-17T03:38:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe024075a80 msgr2=0x7fe024077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe024075a80 0x7fe024077f90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe0300096c0 tx=0x7fe030009260 comp rx=0 tx=0).stop 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 msgr2=0x7fe0480cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe0480cd4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0480706f0 tx=0x7fe0380a57c0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 shutdown_connections 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fe024075a80 0x7fe024077f90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815cc60 0x7fe048158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0480b4de0 0x7fe048143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 --2- 172.21.15.46:0/232610775 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fe0480738b0 0x7fe0480cd4f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 >> 172.21.15.46:0/232610775 conn(0x7fe0480b2040 msgr2=0x7fe048151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 shutdown_connections 2024-09-17T03:38:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:30.595+0000 7fe04e200640 1 -- 172.21.15.46:0/232610775 wait complete. 2024-09-17T03:38:30.618 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:38:30.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:38:30.707 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:38:30.834 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:38:30.834 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:38:30.834 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:38:30.834 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:30.849 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:38:30.849 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:30.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 -- 172.21.15.46:0/2449774636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb61815c5d0 msgr2=0x7fb618073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 --2- 172.21.15.46:0/2449774636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb61815c5d0 0x7fb618073dc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb608009840 tx=0x7fb60801c790 comp rx=0 tx=0).stop 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 -- 172.21.15.46:0/2449774636 shutdown_connections 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 --2- 172.21.15.46:0/2449774636 >> v2:172.21.15.99:3300/0 conn(0x7fb61806fa70 0x7fb61815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 --2- 172.21.15.46:0/2449774636 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb61815c5d0 0x7fb618073dc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 -- 172.21.15.46:0/2449774636 >> 172.21.15.46:0/2449774636 conn(0x7fb6180b2c40 msgr2=0x7fb6180b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 -- 172.21.15.46:0/2449774636 shutdown_connections 2024-09-17T03:38:31.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 -- 172.21.15.46:0/2449774636 wait complete. 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 Processor -- start 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.135+0000 7fb61da00640 1 -- start start 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 0x7fb6181f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb618074300 0x7fb61814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb61815c5d0 0x7fb618154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb618168b10 con 0x7fb61806fa70 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb61815ffd0 con 0x7fb61815c5d0 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb618168c90 con 0x7fb618074300 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 0x7fb6181f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 0x7fb6181f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42420/0 (socket says 172.21.15.46:42420) 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 -- 172.21.15.46:0/1254016696 learned_addr learned my addr 172.21.15.46:0/1254016696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 -- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb618074300 msgr2=0x7fb61814cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb618074300 0x7fb61814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb617400640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb61815c5d0 0x7fb618154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 -- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb61815c5d0 msgr2=0x7fb618154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb61815c5d0 0x7fb618154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 -- 172.21.15.46:0/1254016696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6181f9440 con 0x7fb61806fa70 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb617400640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb61815c5d0 0x7fb618154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb616a00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 0x7fb6181f8f00 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fb608019400 tx=0x7fb6080a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6080bb070 con 0x7fb61806fa70 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb608004470 con 0x7fb61806fa70 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb618153f40 con 0x7fb61806fa70 2024-09-17T03:38:31.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6080b2040 con 0x7fb61806fa70 2024-09-17T03:38:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb618154500 con 0x7fb61806fa70 2024-09-17T03:38:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.139+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb618073dc0 con 0x7fb61806fa70 2024-09-17T03:38:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.143+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fb6080be070 con 0x7fb61806fa70 2024-09-17T03:38:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.143+0000 7fb60ea00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb5f4075ab0 0x7fb5f4077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.143+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7fb60812cd30 con 0x7fb61806fa70 2024-09-17T03:38:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.143+0000 7fb60fe00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb5f4075ab0 0x7fb5f4077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:31.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.143+0000 7fb60fe00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb5f4075ab0 0x7fb5f4077fc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb6000046c0 tx=0x7fb600007910 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:31.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.147+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb608016410 con 0x7fb61806fa70 2024-09-17T03:38:31.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:38:31.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:38:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.331+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb618152a40 con 0x7fb61806fa70 2024-09-17T03:38:31.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb60ea00640 1 -- 172.21.15.46:0/1254016696 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v146) ==== 80+0+0 (secure 0 0 0) 0x7fb6080f9840 con 0x7fb61806fa70 2024-09-17T03:38:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-09-17T03:38:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb5f4075ab0 msgr2=0x7fb5f4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb5f4075ab0 0x7fb5f4077fc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb6000046c0 tx=0x7fb600007910 comp rx=0 tx=0).stop 2024-09-17T03:38:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 msgr2=0x7fb6181f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 0x7fb6181f8f00 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fb608019400 tx=0x7fb6080a5650 comp rx=0 tx=0).stop 2024-09-17T03:38:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 shutdown_connections 2024-09-17T03:38:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb5f4075ab0 0x7fb5f4077fc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb61815c5d0 0x7fb618154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb618074300 0x7fb61814cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 --2- 172.21.15.46:0/1254016696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb61806fa70 0x7fb6181f8f00 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 >> 172.21.15.46:0/1254016696 conn(0x7fb6180b2c40 msgr2=0x7fb618152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 shutdown_connections 2024-09-17T03:38:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.335+0000 7fb61da00640 1 -- 172.21.15.46:0/1254016696 wait complete. 2024-09-17T03:38:31.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T03:38:31.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:31.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T03:38:31.454 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:38:31.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:31.455 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:38:31.474 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:38:31.474 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:38:31.474 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:31.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:38:31.485 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:38:31.485 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:38:31.485 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:31.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:38:31.528 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:38:31.528 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:38:31.528 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:31.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:38:31.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:38:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 -- 172.21.15.46:0/608770632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 msgr2=0x7f0328004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 --2- 172.21.15.46:0/608770632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 0x7f0328004c20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0330009b60 tx=0x7f033001d270 comp rx=0 tx=0).stop 2024-09-17T03:38:31.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 -- 172.21.15.46:0/608770632 shutdown_connections 2024-09-17T03:38:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 --2- 172.21.15.46:0/608770632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 0x7f0328004c20 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 -- 172.21.15.46:0/608770632 >> 172.21.15.46:0/608770632 conn(0x7f0344073a10 msgr2=0x7f034406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 -- 172.21.15.46:0/608770632 shutdown_connections 2024-09-17T03:38:31.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.779+0000 7f0349c00640 1 -- 172.21.15.46:0/608770632 wait complete. 2024-09-17T03:38:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 Processor -- start 2024-09-17T03:38:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 -- start start 2024-09-17T03:38:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 0x7f03440cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 0x7f03440b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03440b68d0 0x7f03440b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03440d1690 con 0x7f0328004810 2024-09-17T03:38:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03440d1510 con 0x7f03441f63c0 2024-09-17T03:38:31.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0349c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0344162a00 con 0x7f03440b68d0 2024-09-17T03:38:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0343400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 0x7f03440b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0343400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 0x7f03440b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.46:43354/0 (socket says 172.21.15.46:43354) 2024-09-17T03:38:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.783+0000 7f0343400640 1 -- 172.21.15.46:0/1626090497 learned_addr learned my addr 172.21.15.46:0/1626090497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343e00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 0x7f03440cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343400640 1 -- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03440b68d0 msgr2=0x7f03440b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343400640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03440b68d0 0x7f03440b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343400640 1 -- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 msgr2=0x7f03440cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343400640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 0x7f03440cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343400640 1 -- 172.21.15.46:0/1626090497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03440cb680 con 0x7f03441f63c0 2024-09-17T03:38:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0343400640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 0x7f03440b6390 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f033001d0a0 tx=0x7f03300056c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300b1040 con 0x7f03441f63c0 2024-09-17T03:38:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03300bc070 con 0x7f03441f63c0 2024-09-17T03:38:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300acb00 con 0x7f03441f63c0 2024-09-17T03:38:31.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03440cb960 con 0x7f03441f63c0 2024-09-17T03:38:31.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.787+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03440b4ca0 con 0x7f03441f63c0 2024-09-17T03:38:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.791+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03440cd780 con 0x7f03441f63c0 2024-09-17T03:38:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.795+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f03300acca0 con 0x7f03441f63c0 2024-09-17T03:38:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.795+0000 7f033be00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0324075ad0 0x7f0324077fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.795+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f033012e570 con 0x7f03441f63c0 2024-09-17T03:38:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.795+0000 7f0343e00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0324075ad0 0x7f0324077fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:31.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.795+0000 7f0343e00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0324075ad0 0x7f0324077fe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0334000fe0 tx=0x7f0334001e60 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:31.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:31.811+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0330016f80 con 0x7f03441f63c0 2024-09-17T03:38:32.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.207+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0344162f60 con 0x7f03441f63c0 2024-09-17T03:38:32.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.207+0000 7f033be00640 1 -- 172.21.15.46:0/1626090497 <== mon.1 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5243 (secure 0 0 0) 0x7f03300faad0 con 0x7f03441f63c0 2024-09-17T03:38:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:32.212 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"btime":"2024-09-17T03:37:14:564527+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:37:00.912074+0000","modified":"2024-09-17T03:37:03.946655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248758":{"gid":248758,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/2038056563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2038056563},{"type":"v1","addr":"172.21.15.46:6837","nonce":2038056563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248758,"qdb_cluster":[248758]},"id":10}]} 2024-09-17T03:38:32.212 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 95 2024-09-17T03:38:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0324075ad0 msgr2=0x7f0324077fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0324075ad0 0x7f0324077fe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0334000fe0 tx=0x7f0334001e60 comp rx=0 tx=0).stop 2024-09-17T03:38:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 msgr2=0x7f03440b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 0x7f03440b6390 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f033001d0a0 tx=0x7f03300056c0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 shutdown_connections 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0324075ad0 0x7f0324077fe0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f03440b68d0 0x7f03440b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03441f63c0 0x7f03440b6390 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 --2- 172.21.15.46:0/1626090497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0328004810 0x7f03440cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 >> 172.21.15.46:0/1626090497 conn(0x7f0344073a10 msgr2=0x7f0344160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 shutdown_connections 2024-09-17T03:38:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.215+0000 7f0349c00640 1 -- 172.21.15.46:0/1626090497 wait complete. 2024-09-17T03:38:32.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:38:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 -- 172.21.15.46:0/3043550644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 msgr2=0x7f750415f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 --2- 172.21.15.46:0/3043550644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f750415f070 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009930 tx=0x7f74f401c820 comp rx=0 tx=0).stop 2024-09-17T03:38:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 -- 172.21.15.46:0/3043550644 shutdown_connections 2024-09-17T03:38:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 --2- 172.21.15.46:0/3043550644 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f75040b24d0 0x7f75040b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 --2- 172.21.15.46:0/3043550644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f750415f070 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 -- 172.21.15.46:0/3043550644 >> 172.21.15.46:0/3043550644 conn(0x7f750406cdf0 msgr2=0x7f750406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 -- 172.21.15.46:0/3043550644 shutdown_connections 2024-09-17T03:38:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 -- 172.21.15.46:0/3043550644 wait complete. 2024-09-17T03:38:32.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 Processor -- start 2024-09-17T03:38:32.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.431+0000 7f7509200640 1 -- start start 2024-09-17T03:38:32.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f75041523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f750415f5b0 0x7f750414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750414f850 0x7f75040ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7504168a40 con 0x7f7504155a00 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75041688c0 con 0x7f750414f850 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7504168bc0 con 0x7f750415f5b0 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f75041523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f75041523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45176/0 (socket says 172.21.15.46:45176) 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 -- 172.21.15.46:0/902947418 learned_addr learned my addr 172.21.15.46:0/902947418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503e00640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750414f850 0x7f75040ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 -- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f750415f5b0 msgr2=0x7f750414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f74fbe00640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f750415f5b0 0x7f750414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f750415f5b0 0x7f750414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 -- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750414f850 msgr2=0x7f75040ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750414f850 0x7f75040ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 -- 172.21.15.46:0/902947418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750414dc50 con 0x7f7504155a00 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503e00640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750414f850 0x7f75040ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f74fbe00640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f750415f5b0 0x7f750414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7503400640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f75041523d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009850 tx=0x7f74f40a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f40b2040 con 0x7f7504155a00 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74f4004430 con 0x7f7504155a00 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4005280 con 0x7f7504155a00 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75040aa600 con 0x7f7504155a00 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75040aabc0 con 0x7f7504155a00 2024-09-17T03:38:32.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.435+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75040b3130 con 0x7f7504155a00 2024-09-17T03:38:32.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.439+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f74f4005420 con 0x7f7504155a00 2024-09-17T03:38:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.439+0000 7f74fb400640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74e0075a80 0x7f74e0077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.439+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f74f412d590 con 0x7f7504155a00 2024-09-17T03:38:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.439+0000 7f74fbe00640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74e0075a80 0x7f74e0077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.439+0000 7f74fbe00640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74e0075a80 0x7f74e0077f90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f75040ad0a0 tx=0x7f74ec009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.443+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74f40164d0 con 0x7f7504155a00 2024-09-17T03:38:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75040cc440 con 0x7f7504155a00 2024-09-17T03:38:32.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f74fb400640 1 -- 172.21.15.46:0/902947418 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5243 (secure 0 0 0) 0x7f74f40fa220 con 0x7f7504155a00 2024-09-17T03:38:32.660 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:32.661 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"btime":"2024-09-17T03:37:14:564527+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:37:00.912074+0000","modified":"2024-09-17T03:37:03.946655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248758":{"gid":248758,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/2038056563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2038056563},{"type":"v1","addr":"172.21.15.46:6837","nonce":2038056563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248758,"qdb_cluster":[248758]},"id":10}]} 2024-09-17T03:38:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 95 2024-09-17T03:38:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74e0075a80 msgr2=0x7f74e0077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74e0075a80 0x7f74e0077f90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f75040ad0a0 tx=0x7f74ec009040 comp rx=0 tx=0).stop 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 msgr2=0x7f75041523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f75041523d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009850 tx=0x7f74f40a57b0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 shutdown_connections 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f74e0075a80 0x7f74e0077f90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750414f850 0x7f75040ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f750415f5b0 0x7f750414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 --2- 172.21.15.46:0/902947418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7504155a00 0x7f75041523d0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 >> 172.21.15.46:0/902947418 conn(0x7f750406cdf0 msgr2=0x7f7504075d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 shutdown_connections 2024-09-17T03:38:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.659+0000 7f7509200640 1 -- 172.21.15.46:0/902947418 wait complete. 2024-09-17T03:38:32.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:38:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 -- 172.21.15.46:0/1343566039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580cefa0 msgr2=0x7f31580cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 --2- 172.21.15.46:0/1343566039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580cefa0 0x7f31580cf3b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f31580b3040 tx=0x7f314001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 -- 172.21.15.46:0/1343566039 shutdown_connections 2024-09-17T03:38:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 --2- 172.21.15.46:0/1343566039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580cefa0 0x7f31580cf3b0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 --2- 172.21.15.46:0/1343566039 >> v2:172.21.15.99:3300/0 conn(0x7f31580749a0 0x7f3158074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 -- 172.21.15.46:0/1343566039 >> 172.21.15.46:0/1343566039 conn(0x7f31580b2040 msgr2=0x7f31580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 -- 172.21.15.46:0/1343566039 shutdown_connections 2024-09-17T03:38:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 -- 172.21.15.46:0/1343566039 wait complete. 2024-09-17T03:38:32.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 Processor -- start 2024-09-17T03:38:32.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.843+0000 7f315d200640 1 -- start start 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 0x7f3158166b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580cefa0 0x7f3158145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3158160330 0x7f31580af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f315816a060 con 0x7f31580749a0 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3158169ee0 con 0x7f31580cefa0 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f315816a1e0 con 0x7f3158160330 2024-09-17T03:38:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 0x7f3158166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 0x7f3158166b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45196/0 (socket says 172.21.15.46:45196) 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 -- 172.21.15.46:0/2647517404 learned_addr learned my addr 172.21.15.46:0/2647517404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f314d400640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580cefa0 0x7f3158145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3156800640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3158160330 0x7f31580af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 -- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3158160330 msgr2=0x7f31580af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3158160330 0x7f31580af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 -- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580cefa0 msgr2=0x7f3158145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580cefa0 0x7f3158145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 -- 172.21.15.46:0/2647517404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31581f9660 con 0x7f31580749a0 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f314d400640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580cefa0 0x7f3158145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f3155e00640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 0x7f3158166b10 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f314800eb00 tx=0x7f314800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148004b30 con 0x7f31580749a0 2024-09-17T03:38:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3148004cf0 con 0x7f31580749a0 2024-09-17T03:38:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31581f9940 con 0x7f31580749a0 2024-09-17T03:38:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148005280 con 0x7f31580749a0 2024-09-17T03:38:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31581456b0 con 0x7f31580749a0 2024-09-17T03:38:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.847+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31580ae8e0 con 0x7f31580749a0 2024-09-17T03:38:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.851+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f3148005420 con 0x7f31580749a0 2024-09-17T03:38:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.851+0000 7f314ea00640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3134075ab0 0x7f3134077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.851+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f3148094300 con 0x7f31580749a0 2024-09-17T03:38:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.851+0000 7f314d400640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3134075ab0 0x7f3134077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.851+0000 7f314d400640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3134075ab0 0x7f3134077fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f31400af000 tx=0x7f314001b100 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:32.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:32.855+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31480606b0 con 0x7f31580749a0 2024-09-17T03:38:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3158074db0 con 0x7f31580749a0 2024-09-17T03:38:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f314ea00640 1 -- 172.21.15.46:0/2647517404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16909 (secure 0 0 0) 0x7f31480650e0 con 0x7f31580749a0 2024-09-17T03:38:33.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:33.037 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":146,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:38:27.448156+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:36:58.659502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T03:36:59.882404+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":146,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:38:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3134075ab0 msgr2=0x7f3134077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3134075ab0 0x7f3134077fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f31400af000 tx=0x7f314001b100 comp rx=0 tx=0).stop 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 msgr2=0x7f3158166b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 0x7f3158166b10 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f314800eb00 tx=0x7f314800c2d0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 shutdown_connections 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f3134075ab0 0x7f3134077fc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f3158160330 0x7f31580af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580cefa0 0x7f3158145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 --2- 172.21.15.46:0/2647517404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31580749a0 0x7f3158166b10 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 >> 172.21.15.46:0/2647517404 conn(0x7f31580b2040 msgr2=0x7f31580afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 shutdown_connections 2024-09-17T03:38:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.035+0000 7f315d200640 1 -- 172.21.15.46:0/2647517404 wait complete. 2024-09-17T03:38:33.061 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:38:33.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:38:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 -- 172.21.15.46:0/289404559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 msgr2=0x7fd04815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 --2- 172.21.15.46:0/289404559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 0x7fd04815bb60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009870 tx=0x7fd03c01c7a0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 -- 172.21.15.46:0/289404559 shutdown_connections 2024-09-17T03:38:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 --2- 172.21.15.46:0/289404559 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 0x7fd04815bb60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 -- 172.21.15.46:0/289404559 >> 172.21.15.46:0/289404559 conn(0x7fd0480b2040 msgr2=0x7fd0480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 -- 172.21.15.46:0/289404559 shutdown_connections 2024-09-17T03:38:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 -- 172.21.15.46:0/289404559 wait complete. 2024-09-17T03:38:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 Processor -- start 2024-09-17T03:38:33.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.219+0000 7fd04f600640 1 -- start start 2024-09-17T03:38:33.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 0x7fd04814c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 0x7fd0480ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0481527b0 0x7fd0480aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd048169d00 con 0x7fd0480b4de0 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd048169b80 con 0x7fd0481527b0 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd048169e80 con 0x7fd0480b5c70 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 0x7fd04814c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 0x7fd04814c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45198/0 (socket says 172.21.15.46:45198) 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 -- 172.21.15.46:0/1002752567 learned_addr learned my addr 172.21.15.46:0/1002752567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd047e00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 0x7fd0480ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04d600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0481527b0 0x7fd0480aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 -- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 msgr2=0x7fd0480ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 0x7fd0480ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 -- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0481527b0 msgr2=0x7fd0480aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0481527b0 0x7fd0480aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 -- 172.21.15.46:0/1002752567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd048155460 con 0x7fd0480b4de0 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04cc00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 0x7fd04814c270 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fd03c01cc80 tx=0x7fd03c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0b2040 con 0x7fd0480b4de0 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd03c0bb070 con 0x7fd0480b4de0 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c003600 con 0x7fd0480b4de0 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0480cf780 con 0x7fd0480b4de0 2024-09-17T03:38:33.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd048153640 con 0x7fd0480b4de0 2024-09-17T03:38:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.223+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04815bb60 con 0x7fd0480b4de0 2024-09-17T03:38:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.227+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fd03c0037a0 con 0x7fd0480b4de0 2024-09-17T03:38:33.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.227+0000 7fd044c00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd028075a80 0x7fd028077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.227+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7fd03c12d980 con 0x7fd0480b4de0 2024-09-17T03:38:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.227+0000 7fd047e00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd028075a80 0x7fd028077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.227+0000 7fd047e00640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd028075a80 0x7fd028077f90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd0480cfcf0 tx=0x7fd034009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.231+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd03c016450 con 0x7fd0480b4de0 2024-09-17T03:38:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd048155850 con 0x7fd0480b4de0 2024-09-17T03:38:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd044c00640 1 -- 172.21.15.46:0/1002752567 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5243 (secure 0 0 0) 0x7fd03c0f9e50 con 0x7fd0480b4de0 2024-09-17T03:38:33.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:33.445 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"btime":"2024-09-17T03:37:14:564527+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:37:00.912074+0000","modified":"2024-09-17T03:37:03.946655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248758":{"gid":248758,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/2038056563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2038056563},{"type":"v1","addr":"172.21.15.46:6837","nonce":2038056563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248758,"qdb_cluster":[248758]},"id":10}]} 2024-09-17T03:38:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 95 2024-09-17T03:38:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd028075a80 msgr2=0x7fd028077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd028075a80 0x7fd028077f90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd0480cfcf0 tx=0x7fd034009040 comp rx=0 tx=0).stop 2024-09-17T03:38:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 msgr2=0x7fd04814c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 0x7fd04814c270 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fd03c01cc80 tx=0x7fd03c005db0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 shutdown_connections 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd028075a80 0x7fd028077f90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0481527b0 0x7fd0480aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0480b5c70 0x7fd0480ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 --2- 172.21.15.46:0/1002752567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0480b4de0 0x7fd04814c270 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 >> 172.21.15.46:0/1002752567 conn(0x7fd0480b2040 msgr2=0x7fd048159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 shutdown_connections 2024-09-17T03:38:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.443+0000 7fd04f600640 1 -- 172.21.15.46:0/1002752567 wait complete. 2024-09-17T03:38:33.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:38:33.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 -- 172.21.15.46:0/1900537607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b4de0 msgr2=0x7f72280b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 --2- 172.21.15.46:0/1900537607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b4de0 0x7f72280b51f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f7228070710 tx=0x7f722001cac0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 -- 172.21.15.46:0/1900537607 shutdown_connections 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 --2- 172.21.15.46:0/1900537607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b4de0 0x7f72280b51f0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 --2- 172.21.15.46:0/1900537607 >> v2:172.21.15.99:3300/0 conn(0x7f72281527b0 0x7f722815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 -- 172.21.15.46:0/1900537607 >> 172.21.15.46:0/1900537607 conn(0x7f72280b2040 msgr2=0x7f72280b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 -- 172.21.15.46:0/1900537607 shutdown_connections 2024-09-17T03:38:33.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 -- 172.21.15.46:0/1900537607 wait complete. 2024-09-17T03:38:33.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 Processor -- start 2024-09-17T03:38:33.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.627+0000 7f722d000640 1 -- start start 2024-09-17T03:38:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72280b4de0 0x7f72280cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 0x7f722814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72281527b0 0x7f72280b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72281688c0 con 0x7f72280b5c70 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7228168740 con 0x7f72281527b0 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7228168a40 con 0x7f72280b4de0 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 0x7f722814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 0x7f722814ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45220/0 (socket says 172.21.15.46:45220) 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 -- 172.21.15.46:0/745993958 learned_addr learned my addr 172.21.15.46:0/745993958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f7225e00640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72280b4de0 0x7f72280cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f7226800640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72281527b0 0x7f72280b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 -- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72280b4de0 msgr2=0x7f72280cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72280b4de0 0x7f72280cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 -- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72281527b0 msgr2=0x7f72280b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72281527b0 0x7f72280b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 -- 172.21.15.46:0/745993958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72280cf850 con 0x7f72280b5c70 2024-09-17T03:38:33.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721d400640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 0x7f722814ab80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f721000e7f0 tx=0x7f721000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721000ccc0 con 0x7f72280b5c70 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f721000ce80 con 0x7f72280b5c70 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210005280 con 0x7f72280b5c70 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72280cf290 con 0x7f72280b5c70 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72280cd400 con 0x7f72280b5c70 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.631+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72280b51f0 con 0x7f72280b5c70 2024-09-17T03:38:33.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.635+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f7210005420 con 0x7f72280b5c70 2024-09-17T03:38:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.635+0000 7f721ea00640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7204075ab0 0x7f7204077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.635+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f72100943a0 con 0x7f72280b5c70 2024-09-17T03:38:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.635+0000 7f7225e00640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7204075ab0 0x7f7204077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.635+0000 7f7225e00640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7204075ab0 0x7f7204077fc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7218001400 tx=0x7f7218009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.639+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7210095e20 con 0x7f72280b5c70 2024-09-17T03:38:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f722814b500 con 0x7f72280b5c70 2024-09-17T03:38:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f721ea00640 1 -- 172.21.15.46:0/745993958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5243 (secure 0 0 0) 0x7f7210060ea0 con 0x7f72280b5c70 2024-09-17T03:38:33.853 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"btime":"2024-09-17T03:37:14:564527+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:37:00.912074+0000","modified":"2024-09-17T03:37:03.946655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":248758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_248758":{"gid":248758,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/2038056563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2038056563},{"type":"v1","addr":"172.21.15.46:6837","nonce":2038056563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":248758,"qdb_cluster":[248758]},"id":10}]} 2024-09-17T03:38:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 95 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7204075ab0 msgr2=0x7f7204077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7204075ab0 0x7f7204077fc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7218001400 tx=0x7f7218009040 comp rx=0 tx=0).stop 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 msgr2=0x7f722814ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 0x7f722814ab80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f721000e7f0 tx=0x7f721000ecb0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 shutdown_connections 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7204075ab0 0x7f7204077fc0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72281527b0 0x7f72280b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72280b5c70 0x7f722814ab80 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 --2- 172.21.15.46:0/745993958 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f72280b4de0 0x7f72280cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 >> 172.21.15.46:0/745993958 conn(0x7f72280b2040 msgr2=0x7f72281511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 shutdown_connections 2024-09-17T03:38:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:33.851+0000 7f722d000640 1 -- 172.21.15.46:0/745993958 wait complete. 2024-09-17T03:38:33.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:38:34.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 -- 172.21.15.46:0/3297002032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b4de0 msgr2=0x7f5e840b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 --2- 172.21.15.46:0/3297002032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b4de0 0x7f5e840b51f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009c80 tx=0x7f5e7c01cce0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 -- 172.21.15.46:0/3297002032 shutdown_connections 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 --2- 172.21.15.46:0/3297002032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b4de0 0x7f5e840b51f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 --2- 172.21.15.46:0/3297002032 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e841527b0 0x7f5e8415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 -- 172.21.15.46:0/3297002032 >> 172.21.15.46:0/3297002032 conn(0x7f5e840b2040 msgr2=0x7f5e840b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 -- 172.21.15.46:0/3297002032 shutdown_connections 2024-09-17T03:38:34.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.031+0000 7f5e8a600640 1 -- 172.21.15.46:0/3297002032 wait complete. 2024-09-17T03:38:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 Processor -- start 2024-09-17T03:38:34.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 -- start start 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e840b4de0 0x7f5e841415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 0x7f5e84140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e841527b0 0x7f5e84156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84168ab0 con 0x7f5e840b5c70 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84168930 con 0x7f5e840b4de0 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5e84168c30 con 0x7f5e841527b0 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 0x7f5e84140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 0x7f5e84140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45234/0 (socket says 172.21.15.46:45234) 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 -- 172.21.15.46:0/1239858489 learned_addr learned my addr 172.21.15.46:0/1239858489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:34.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e83400640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e840b4de0 0x7f5e841415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 -- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e841527b0 msgr2=0x7f5e84156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e841527b0 0x7f5e84156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 -- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e840b4de0 msgr2=0x7f5e841415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e840b4de0 0x7f5e841415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 -- 172.21.15.46:0/1239858489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e841410b0 con 0x7f5e840b5c70 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e83400640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e840b4de0 0x7f5e841415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:34.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7be00640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 0x7f5e84140b70 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00e7c0 tx=0x7f5e6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00cc60 con 0x7f5e840b5c70 2024-09-17T03:38:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e6c00ce00 con 0x7f5e840b5c70 2024-09-17T03:38:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c005280 con 0x7f5e840b5c70 2024-09-17T03:38:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.035+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e84157920 con 0x7f5e840b5c70 2024-09-17T03:38:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.039+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e84154720 con 0x7f5e840b5c70 2024-09-17T03:38:34.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.039+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e840b51f0 con 0x7f5e840b5c70 2024-09-17T03:38:34.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.039+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f5e6c005420 con 0x7f5e840b5c70 2024-09-17T03:38:34.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.039+0000 7f5e7b400640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e60075ab0 0x7f5e60077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.039+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f5e6c093eb0 con 0x7f5e840b5c70 2024-09-17T03:38:34.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.039+0000 7f5e83400640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e60075ab0 0x7f5e60077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.043+0000 7f5e83400640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e60075ab0 0x7f5e60077fc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5e74000f70 tx=0x7f5e74009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:34.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.047+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e6c0609d0 con 0x7f5e840b5c70 2024-09-17T03:38:34.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.219+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e8415a7d0 con 0x7f5e840b5c70 2024-09-17T03:38:34.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.219+0000 7f5e7b400640 1 -- 172.21.15.46:0/1239858489 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16909 (secure 0 0 0) 0x7f5e6c0602d0 con 0x7f5e840b5c70 2024-09-17T03:38:34.223 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:34.224 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":146,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:38:27.448156+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:36:58.659502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T03:36:59.882404+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":146,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e60075ab0 msgr2=0x7f5e60077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e60075ab0 0x7f5e60077fc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5e74000f70 tx=0x7f5e74009040 comp rx=0 tx=0).stop 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 msgr2=0x7f5e84140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 0x7f5e84140b70 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00e7c0 tx=0x7f5e6c00ec80 comp rx=0 tx=0).stop 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 shutdown_connections 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5e60075ab0 0x7f5e60077fc0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f5e841527b0 0x7f5e84156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e840b5c70 0x7f5e84140b70 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 --2- 172.21.15.46:0/1239858489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e840b4de0 0x7f5e841415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 >> 172.21.15.46:0/1239858489 conn(0x7f5e840b2040 msgr2=0x7f5e84151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 shutdown_connections 2024-09-17T03:38:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.223+0000 7f5e8a600640 1 -- 172.21.15.46:0/1239858489 wait complete. 2024-09-17T03:38:34.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 -- 172.21.15.46:0/535023094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 msgr2=0x7f7bdc0cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 --2- 172.21.15.46:0/535023094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc0cc320 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009840 tx=0x7f7bd001c730 comp rx=0 tx=0).stop 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 -- 172.21.15.46:0/535023094 shutdown_connections 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 --2- 172.21.15.46:0/535023094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc0cc320 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 -- 172.21.15.46:0/535023094 >> 172.21.15.46:0/535023094 conn(0x7f7bdc0b2040 msgr2=0x7f7bdc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 -- 172.21.15.46:0/535023094 shutdown_connections 2024-09-17T03:38:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.403+0000 7f7be4200640 1 -- 172.21.15.46:0/535023094 wait complete. 2024-09-17T03:38:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 Processor -- start 2024-09-17T03:38:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 -- start start 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc0cc860 0x7f7bdc13a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc1fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bdc13a7d0 0x7f7bdc1fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc140db0 con 0x7f7bdc13e790 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc140c30 con 0x7f7bdc0cc860 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be4200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc140f30 con 0x7f7bdc13a7d0 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc1fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc1fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45244/0 (socket says 172.21.15.46:45244) 2024-09-17T03:38:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 -- 172.21.15.46:0/1648409160 learned_addr learned my addr 172.21.15.46:0/1648409160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be1800640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc0cc860 0x7f7bdc13a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be2200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bdc13a7d0 0x7f7bdc1fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 -- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bdc13a7d0 msgr2=0x7f7bdc1fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bdc13a7d0 0x7f7bdc1fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 -- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc0cc860 msgr2=0x7f7bdc13a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc0cc860 0x7f7bdc13a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 -- 172.21.15.46:0/1648409160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc1418a0 con 0x7f7bdc13e790 2024-09-17T03:38:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7be0e00640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc1fc1a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c420 tx=0x7f7bc800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.407+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8017070 con 0x7f7bdc13e790 2024-09-17T03:38:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bc80053c0 con 0x7f7bdc13e790 2024-09-17T03:38:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8003860 con 0x7f7bdc13e790 2024-09-17T03:38:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc152040 con 0x7f7bdc13e790 2024-09-17T03:38:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc153cf0 con 0x7f7bdc13e790 2024-09-17T03:38:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc0cc320 con 0x7f7bdc13e790 2024-09-17T03:38:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f7bc801a070 con 0x7f7bdc13e790 2024-09-17T03:38:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.411+0000 7f7bd9600640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7bbc075a80 0x7f7bbc077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.415+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5822+0+0 (secure 0 0 0) 0x7f7bc80929f0 con 0x7f7bdc13e790 2024-09-17T03:38:34.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.415+0000 7f7be1800640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7bbc075a80 0x7f7bbc077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.415+0000 7f7be1800640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7bbc075a80 0x7f7bbc077f90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00af000 tx=0x7f7bd001afd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:34.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.415+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7bc8021080 con 0x7f7bdc13e790 2024-09-17T03:38:34.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.627+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7bdc140300 con 0x7f7bdc13e790 2024-09-17T03:38:34.631 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-09-17T03:38:34.631+0000 7f3b57600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:38:34.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.631+0000 7f7bd9600640 1 -- 172.21.15.46:0/1648409160 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) ==== 161+0+0 (secure 0 0 0) 0x7f7bc805f680 con 0x7f7bdc13e790 2024-09-17T03:38:34.633 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:38:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7bbc075a80 msgr2=0x7f7bbc077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7bbc075a80 0x7f7bbc077f90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00af000 tx=0x7f7bd001afd0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 msgr2=0x7f7bdc1fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc1fc1a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c420 tx=0x7f7bc800c8e0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 shutdown_connections 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7bbc075a80 0x7f7bbc077f90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7bdc13a7d0 0x7f7bdc1fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bdc13e790 0x7f7bdc1fc1a0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 --2- 172.21.15.46:0/1648409160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc0cc860 0x7f7bdc13a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 >> 172.21.15.46:0/1648409160 conn(0x7f7bdc0b2040 msgr2=0x7f7bdc0aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.635+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 shutdown_connections 2024-09-17T03:38:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.639+0000 7f7be4200640 1 -- 172.21.15.46:0/1648409160 wait complete. 2024-09-17T03:38:34.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:38:34.791 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-09-17T03:38:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 -- 172.21.15.46:0/2138714926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0b4de0 msgr2=0x7fb44c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 --2- 172.21.15.46:0/2138714926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0b4de0 0x7fb44c0b51f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb444009bc0 tx=0x7fb44401cae0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 -- 172.21.15.46:0/2138714926 shutdown_connections 2024-09-17T03:38:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 --2- 172.21.15.46:0/2138714926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0b4de0 0x7fb44c0b51f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 --2- 172.21.15.46:0/2138714926 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb44c1527b0 0x7fb44c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 -- 172.21.15.46:0/2138714926 >> 172.21.15.46:0/2138714926 conn(0x7fb44c0b2040 msgr2=0x7fb44c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 -- 172.21.15.46:0/2138714926 shutdown_connections 2024-09-17T03:38:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.835+0000 7fb451e00640 1 -- 172.21.15.46:0/2138714926 wait complete. 2024-09-17T03:38:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 Processor -- start 2024-09-17T03:38:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 -- start start 2024-09-17T03:38:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb44c0b4de0 0x7fb44c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 0x7fb44c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c1527b0 0x7fb44c150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c168a10 con 0x7fb44c0b5c70 2024-09-17T03:38:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c168890 con 0x7fb44c1527b0 2024-09-17T03:38:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb451e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c168b90 con 0x7fb44c0b4de0 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 0x7fb44c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 0x7fb44c0cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45260/0 (socket says 172.21.15.46:45260) 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 -- 172.21.15.46:0/1023792823 learned_addr learned my addr 172.21.15.46:0/1023792823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44b400640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c1527b0 0x7fb44c150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44aa00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb44c0b4de0 0x7fb44c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 -- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb44c0b4de0 msgr2=0x7fb44c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb44c0b4de0 0x7fb44c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 -- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c1527b0 msgr2=0x7fb44c150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c1527b0 0x7fb44c150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 -- 172.21.15.46:0/1023792823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c1fce00 con 0x7fb44c0b5c70 2024-09-17T03:38:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.839+0000 7fb44a000640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 0x7fb44c0cf530 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e790 tx=0x7fb43800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43800eea0 con 0x7fb44c0b5c70 2024-09-17T03:38:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb43800cc60 con 0x7fb44c0b5c70 2024-09-17T03:38:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438005280 con 0x7fb44c0b5c70 2024-09-17T03:38:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c1fc280 con 0x7fb44c0b5c70 2024-09-17T03:38:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c1fc840 con 0x7fb44c0b5c70 2024-09-17T03:38:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c0b51f0 con 0x7fb44c0b5c70 2024-09-17T03:38:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.843+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fb438005420 con 0x7fb44c0b5c70 2024-09-17T03:38:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.847+0000 7fb443400640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb42c075ab0 0x7fb42c077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.847+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5880+0+0 (secure 0 0 0) 0x7fb438093ec0 con 0x7fb44c0b5c70 2024-09-17T03:38:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.847+0000 7fb44aa00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb42c075ab0 0x7fb42c077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.847+0000 7fb44aa00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb42c075ab0 0x7fb42c077fc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb43c001400 tx=0x7fb43c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:34.851+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb4380609e0 con 0x7fb44c0b5c70 2024-09-17T03:38:35.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.063+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb44c15bf20 con 0x7fb44c0b5c70 2024-09-17T03:38:35.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.631+0000 7fb443400640 1 -- 172.21.15.46:0/1023792823 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v97) ==== 104+0+0 (secure 0 0 0) 0x7fb4380602e0 con 0x7fb44c0b5c70 2024-09-17T03:38:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb42c075ab0 msgr2=0x7fb42c077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:35.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb42c075ab0 0x7fb42c077fc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb43c001400 tx=0x7fb43c009040 comp rx=0 tx=0).stop 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 msgr2=0x7fb44c0cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 0x7fb44c0cf530 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e790 tx=0x7fb43800ec50 comp rx=0 tx=0).stop 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 shutdown_connections 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb42c075ab0 0x7fb42c077fc0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c1527b0 0x7fb44c150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb44c0b5c70 0x7fb44c0cf530 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 --2- 172.21.15.46:0/1023792823 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb44c0b4de0 0x7fb44c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 >> 172.21.15.46:0/1023792823 conn(0x7fb44c0b2040 msgr2=0x7fb44c0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 shutdown_connections 2024-09-17T03:38:35.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.635+0000 7fb451e00640 1 -- 172.21.15.46:0/1023792823 wait complete. 2024-09-17T03:38:35.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4007507631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d40b54f0 msgr2=0x7ff6d415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4007507631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d40b54f0 0x7ff6d415bb60 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009bc0 tx=0x7ff6cc01cae0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4007507631 shutdown_connections 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4007507631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d40b54f0 0x7ff6d415bb60 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4007507631 >> 172.21.15.46:0/4007507631 conn(0x7ff6d40b2040 msgr2=0x7ff6d40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4007507631 shutdown_connections 2024-09-17T03:38:35.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.831+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4007507631 wait complete. 2024-09-17T03:38:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 Processor -- start 2024-09-17T03:38:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 -- start start 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6d40b5c70 0x7ff6d414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d41527b0 0x7ff6d4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 0x7ff6d4156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4169b30 con 0x7ff6d414ba20 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41699b0 con 0x7ff6d41527b0 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d4169cb0 con 0x7ff6d40b5c70 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 0x7ff6d4156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 0x7ff6d4156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45264/0 (socket says 172.21.15.46:45264) 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 -- 172.21.15.46:0/4191417885 learned_addr learned my addr 172.21.15.46:0/4191417885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9400640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6d40b5c70 0x7ff6d414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d8a00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d41527b0 0x7ff6d4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 -- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6d40b5c70 msgr2=0x7ff6d414b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6d40b5c70 0x7ff6d414b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 -- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d41527b0 msgr2=0x7ff6d4156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d41527b0 0x7ff6d4156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 -- 172.21.15.46:0/4191417885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d41543e0 con 0x7ff6d414ba20 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d8a00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d41527b0 0x7ff6d4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d9e00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 0x7ff6d4156ff0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0b0040 tx=0x7ff6cc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0b2040 con 0x7ff6d414ba20 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6cc0a5e50 con 0x7ff6d414ba20 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc004450 con 0x7ff6d414ba20 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d4158c70 con 0x7ff6d414ba20 2024-09-17T03:38:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.835+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d4159230 con 0x7ff6d414ba20 2024-09-17T03:38:35.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.839+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d4154150 con 0x7ff6d414ba20 2024-09-17T03:38:35.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.839+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7ff6cc0bd030 con 0x7ff6d414ba20 2024-09-17T03:38:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.839+0000 7ff6d1600640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff6b4075ab0 0x7ff6b4077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.839+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5880+0+0 (secure 0 0 0) 0x7ff6cc12d5f0 con 0x7ff6d414ba20 2024-09-17T03:38:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.839+0000 7ff6d9400640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff6b4075ab0 0x7ff6b4077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.839+0000 7ff6d9400640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff6b4075ab0 0x7ff6b4077fc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8007fd0 tx=0x7ff6c8009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:35.843+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6cc0c4060 con 0x7ff6d414ba20 2024-09-17T03:38:36.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.019+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6d415bb60 con 0x7ff6d414ba20 2024-09-17T03:38:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.207+0000 7ff6d1600640 1 -- 172.21.15.46:0/4191417885 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7ff6cc016790 con 0x7ff6d414ba20 2024-09-17T03:38:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff6b4075ab0 msgr2=0x7ff6b4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff6b4075ab0 0x7ff6b4077fc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8007fd0 tx=0x7ff6c8009040 comp rx=0 tx=0).stop 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 msgr2=0x7ff6d4156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 0x7ff6d4156ff0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0b0040 tx=0x7ff6cc005ae0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 shutdown_connections 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff6b4075ab0 0x7ff6b4077fc0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6d414ba20 0x7ff6d4156ff0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d41527b0 0x7ff6d4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 --2- 172.21.15.46:0/4191417885 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff6d40b5c70 0x7ff6d414b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 >> 172.21.15.46:0/4191417885 conn(0x7ff6d40b2040 msgr2=0x7ff6d4159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 shutdown_connections 2024-09-17T03:38:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.211+0000 7ff6dbe00640 1 -- 172.21.15.46:0/4191417885 wait complete. 2024-09-17T03:38:36.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:38:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 -- 172.21.15.46:0/829117264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40072100 msgr2=0x7f8d40155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 --2- 172.21.15.46:0/829117264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40072100 0x7f8d40155b50 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8d40073920 tx=0x7f8d3401c770 comp rx=0 tx=0).stop 2024-09-17T03:38:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 -- 172.21.15.46:0/829117264 shutdown_connections 2024-09-17T03:38:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 --2- 172.21.15.46:0/829117264 >> v2:172.21.15.99:3300/0 conn(0x7f8d40073a90 0x7f8d40073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 --2- 172.21.15.46:0/829117264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40072100 0x7f8d40155b50 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 -- 172.21.15.46:0/829117264 >> 172.21.15.46:0/829117264 conn(0x7f8d400b2040 msgr2=0x7f8d400b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 -- 172.21.15.46:0/829117264 shutdown_connections 2024-09-17T03:38:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 -- 172.21.15.46:0/829117264 wait complete. 2024-09-17T03:38:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 Processor -- start 2024-09-17T03:38:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.403+0000 7f8d47600640 1 -- start start 2024-09-17T03:38:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40072100 0x7f8d4014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 0x7f8d401f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d400b4de0 0x7f8d400b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8d40168e20 con 0x7f8d40073a90 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d40168ca0 con 0x7f8d40072100 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f8d40168fa0 con 0x7f8d400b4de0 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 0x7f8d401f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 0x7f8d401f9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45286/0 (socket says 172.21.15.46:45286) 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 -- 172.21.15.46:0/3487852346 learned_addr learned my addr 172.21.15.46:0/3487852346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d44c00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40072100 0x7f8d4014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d45600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d400b4de0 0x7f8d400b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 -- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d400b4de0 msgr2=0x7f8d400b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d400b4de0 0x7f8d400b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 -- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40072100 msgr2=0x7f8d4014b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40072100 0x7f8d4014b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 -- 172.21.15.46:0/3487852346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4014a5c0 con 0x7f8d40073a90 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d44c00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40072100 0x7f8d4014b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d45600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d400b4de0 0x7f8d400b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3fe00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 0x7f8d401f9960 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00c3c0 tx=0x7f8d2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c011790 con 0x7f8d40073a90 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d2c017070 con 0x7f8d40073a90 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c011d00 con 0x7f8d40073a90 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4014c280 con 0x7f8d40073a90 2024-09-17T03:38:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4014c840 con 0x7f8d40073a90 2024-09-17T03:38:36.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.407+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d40155b50 con 0x7f8d40073a90 2024-09-17T03:38:36.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.411+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f8d2c01c460 con 0x7f8d40073a90 2024-09-17T03:38:36.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.411+0000 7f8d3cc00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d20075ab0 0x7f8d20077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:36.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.411+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5442+0+0 (secure 0 0 0) 0x7f8d2c0931e0 con 0x7f8d40073a90 2024-09-17T03:38:36.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.411+0000 7f8d44c00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d20075ab0 0x7f8d20077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:36.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.411+0000 7f8d44c00640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d20075ab0 0x7f8d20077fc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8d340af000 tx=0x7f8d3401afb0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.415+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d2c05fcf0 con 0x7f8d40073a90 2024-09-17T03:38:36.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:36.599+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d4014a380 con 0x7f8d40073a90 2024-09-17T03:38:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.499+0000 7f8d3cc00640 1 -- 172.21.15.46:0/3487852346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7f8d2c05f5f0 con 0x7f8d40073a90 2024-09-17T03:38:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-09-17T03:38:37.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d20075ab0 msgr2=0x7f8d20077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d20075ab0 0x7f8d20077fc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8d340af000 tx=0x7f8d3401afb0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 msgr2=0x7f8d401f9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 0x7f8d401f9960 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00c3c0 tx=0x7f8d2c00c880 comp rx=0 tx=0).stop 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 shutdown_connections 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8d20075ab0 0x7f8d20077fc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f8d400b4de0 0x7f8d400b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d40073a90 0x7f8d401f9960 secure :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00c3c0 tx=0x7f8d2c00c880 comp rx=0 tx=0).stop 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 --2- 172.21.15.46:0/3487852346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40072100 0x7f8d4014b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.503+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 >> 172.21.15.46:0/3487852346 conn(0x7f8d400b2040 msgr2=0x7f8d40154b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.507+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 shutdown_connections 2024-09-17T03:38:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.507+0000 7f8d47600640 1 -- 172.21.15.46:0/3487852346 wait complete. 2024-09-17T03:38:37.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:38:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 -- 172.21.15.46:0/4272199836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f561c003c40 msgr2=0x7f561c0040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 --2- 172.21.15.46:0/4272199836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f561c003c40 0x7f561c0040b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5628009b40 tx=0x7f562801d220 comp rx=0 tx=0).stop 2024-09-17T03:38:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 -- 172.21.15.46:0/4272199836 shutdown_connections 2024-09-17T03:38:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 --2- 172.21.15.46:0/4272199836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f561c003c40 0x7f561c0040b0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 -- 172.21.15.46:0/4272199836 >> 172.21.15.46:0/4272199836 conn(0x7f56380b2040 msgr2=0x7f56380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 -- 172.21.15.46:0/4272199836 shutdown_connections 2024-09-17T03:38:37.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.707+0000 7f563f000640 1 -- 172.21.15.46:0/4272199836 wait complete. 2024-09-17T03:38:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 Processor -- start 2024-09-17T03:38:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 -- start start 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f561c003c40 0x7f5638151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5638157210 0x7f56381f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 0x7f56381fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56380b5580 con 0x7f5638151a00 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56380b5400 con 0x7f5638157210 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563f000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f56380b5700 con 0x7f561c003c40 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 0x7f56381fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 0x7f56381fa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45318/0 (socket says 172.21.15.46:45318) 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f563d000640 1 -- 172.21.15.46:0/1015035987 learned_addr learned my addr 172.21.15.46:0/1015035987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f5637e00640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f561c003c40 0x7f5638151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.711+0000 7f562f400640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5638157210 0x7f56381f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563d000640 1 -- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f561c003c40 msgr2=0x7f5638151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563d000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f561c003c40 0x7f5638151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563d000640 1 -- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5638157210 msgr2=0x7f56381f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563d000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5638157210 0x7f56381f9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563d000640 1 -- 172.21.15.46:0/1015035987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5638156190 con 0x7f5638151a00 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563d000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 0x7f56381fa3f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5638152bf0 tx=0x7f5630001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630011840 con 0x7f5638151a00 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56300119e0 con 0x7f5638151a00 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5638156450 con 0x7f5638151a00 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630011e30 con 0x7f5638151a00 2024-09-17T03:38:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5638156a10 con 0x7f5638151a00 2024-09-17T03:38:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56381fd160 con 0x7f5638151a00 2024-09-17T03:38:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.715+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f563001d470 con 0x7f5638151a00 2024-09-17T03:38:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.719+0000 7f5634c00640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5614075a80 0x7f5614077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.719+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f5630093c90 con 0x7f5638151a00 2024-09-17T03:38:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.719+0000 7f5637e00640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5614075a80 0x7f5614077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.719+0000 7f5637e00640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5614075a80 0x7f5614077f90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f56280ad000 tx=0x7f5628019fc0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.723+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5630060150 con 0x7f5638151a00 2024-09-17T03:38:37.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.899+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f563815f500 con 0x7f5638151a00 2024-09-17T03:38:37.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.903+0000 7f5634c00640 1 -- 172.21.15.46:0/1015035987 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v61) ==== 71+0+0 (secure 0 0 0) 0x7f5630064b80 con 0x7f5638151a00 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5614075a80 msgr2=0x7f5614077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5614075a80 0x7f5614077f90 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f56280ad000 tx=0x7f5628019fc0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 msgr2=0x7f56381fa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 0x7f56381fa3f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5638152bf0 tx=0x7f5630001810 comp rx=0 tx=0).stop 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 shutdown_connections 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5614075a80 0x7f5614077f90 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5638151a00 0x7f56381fa3f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5638157210 0x7f56381f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 --2- 172.21.15.46:0/1015035987 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f561c003c40 0x7f5638151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 >> 172.21.15.46:0/1015035987 conn(0x7f56380b2040 msgr2=0x7f56380cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 shutdown_connections 2024-09-17T03:38:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:37.907+0000 7f563f000640 1 -- 172.21.15.46:0/1015035987 wait complete. 2024-09-17T03:38:37.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 -- 172.21.15.46:0/3072323963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c0b5c70 msgr2=0x7f938c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 --2- 172.21.15.46:0/3072323963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c0b5c70 0x7f938c15bb60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f938c074b20 tx=0x7f938001c820 comp rx=0 tx=0).stop 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 -- 172.21.15.46:0/3072323963 shutdown_connections 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 --2- 172.21.15.46:0/3072323963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c0b5c70 0x7f938c15bb60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 -- 172.21.15.46:0/3072323963 >> 172.21.15.46:0/3072323963 conn(0x7f938c0b2040 msgr2=0x7f938c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 -- 172.21.15.46:0/3072323963 shutdown_connections 2024-09-17T03:38:38.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.099+0000 7f9393800640 1 -- 172.21.15.46:0/3072323963 wait complete. 2024-09-17T03:38:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 Processor -- start 2024-09-17T03:38:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 -- start start 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c0b4de0 0x7f938c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f938c0b5c70 0x7f938c154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 0x7f938c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f938c169bd0 con 0x7f938c1527b0 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f938c169a50 con 0x7f938c0b4de0 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f938c169d50 con 0x7f938c0b5c70 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 0x7f938c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 0x7f938c1547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45332/0 (socket says 172.21.15.46:45332) 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 -- 172.21.15.46:0/1847804869 learned_addr learned my addr 172.21.15.46:0/1847804869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9390e00640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c0b4de0 0x7f938c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f938be00640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f938c0b5c70 0x7f938c154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 -- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f938c0b5c70 msgr2=0x7f938c154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f938c0b5c70 0x7f938c154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 -- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c0b4de0 msgr2=0x7f938c150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c0b4de0 0x7f938c150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 -- 172.21.15.46:0/1847804869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c154ce0 con 0x7f938c1527b0 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9391800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 0x7f938c1547a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f938400c6b0 tx=0x7f938400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384003b80 con 0x7f938c1527b0 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9384003d20 con 0x7f938c1527b0 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c0b0880 con 0x7f938c1527b0 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384005330 con 0x7f938c1527b0 2024-09-17T03:38:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.103+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c0b0e40 con 0x7f938c1527b0 2024-09-17T03:38:38.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.107+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c15bb60 con 0x7f938c1527b0 2024-09-17T03:38:38.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.107+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f93840054d0 con 0x7f938c1527b0 2024-09-17T03:38:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.107+0000 7f9388c00640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f936c075a80 0x7f936c077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.107+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f9384093230 con 0x7f938c1527b0 2024-09-17T03:38:38.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.107+0000 7f9390e00640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f936c075a80 0x7f936c077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:38.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.107+0000 7f9390e00640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f936c075a80 0x7f936c077f90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f93800af000 tx=0x7f938001b060 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:38.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.111+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f938405f6f0 con 0x7f938c1527b0 2024-09-17T03:38:38.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.347+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f938c152070 con 0x7f938c1527b0 2024-09-17T03:38:38.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9388c00640 1 -- 172.21.15.46:0/1847804869 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v812) ==== 186+0+0 (secure 0 0 0) 0x7f9384064120 con 0x7f938c1527b0 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f936c075a80 msgr2=0x7f936c077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f936c075a80 0x7f936c077f90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f93800af000 tx=0x7f938001b060 comp rx=0 tx=0).stop 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 msgr2=0x7f938c1547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 0x7f938c1547a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f938400c6b0 tx=0x7f938400cb70 comp rx=0 tx=0).stop 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 shutdown_connections 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f936c075a80 0x7f936c077f90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f938c1527b0 0x7f938c1547a0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f938c0b5c70 0x7f938c154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 --2- 172.21.15.46:0/1847804869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c0b4de0 0x7f938c150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 >> 172.21.15.46:0/1847804869 conn(0x7f938c0b2040 msgr2=0x7f938c075f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:38.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 shutdown_connections 2024-09-17T03:38:38.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:38.515+0000 7f9393800640 1 -- 172.21.15.46:0/1847804869 wait complete. 2024-09-17T03:38:38.540 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-17T03:38:38.540 INFO:tasks.cephfs_test_runner: 2024-09-17T03:38:38.540 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T03:38:38.540 INFO:tasks.cephfs_test_runner:Ran 9 tests in 1023.783s 2024-09-17T03:38:38.540 INFO:tasks.cephfs_test_runner: 2024-09-17T03:38:38.541 INFO:tasks.cephfs_test_runner:OK 2024-09-17T03:38:38.541 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T03:38:38.553 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-17T03:38:38.561 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-17T03:38:38.561 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:38:38.561 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:38.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:38:38.570 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-17T03:38:38.570 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:38.623 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353340k,nr_inodes=4088335,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7241 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:38:38.624 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:38:38.624 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:38.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:38:38.634 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-17T03:38:38.634 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:38:38.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=579 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:38:38.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:38.684 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:38:38.694 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:38:38.694 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:38:38.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:38.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:38:38.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:38.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:38:39.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:39.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-17T03:38:39.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:39.370 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-17T03:38:39.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:39.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-17T03:38:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-09-17T03:38:39.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:38:39.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:39.706 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-17T03:38:39.716 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T03:38:39.716 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T03:38:39.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:39.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:38:39.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:39.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:38:40.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:40.223 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-09-17T03:38:40.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:40.347 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-17T03:38:40.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:38:40.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-09-17T03:38:40.781 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-09-17T03:38:40.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:38:40.784 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T03:38:40.794 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:38:40.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:38:40.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fdc800640 1 -- 172.21.15.46:0/2508612829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 msgr2=0x7f0fbc003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fdc800640 1 --2- 172.21.15.46:0/2508612829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fbc003eb0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800c2e0 tx=0x7f0fc801cf60 comp rx=0 tx=0).stop 2024-09-17T03:38:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fda800640 1 -- 172.21.15.46:0/2508612829 reap_dead start 2024-09-17T03:38:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fdc800640 1 -- 172.21.15.46:0/2508612829 shutdown_connections 2024-09-17T03:38:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fdc800640 1 -- 172.21.15.46:0/2508612829 >> 172.21.15.46:0/2508612829 conn(0x7f0fd40b2040 msgr2=0x7f0fd40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fdc800640 1 -- 172.21.15.46:0/2508612829 shutdown_connections 2024-09-17T03:38:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.963+0000 7f0fdc800640 1 -- 172.21.15.46:0/2508612829 wait complete. 2024-09-17T03:38:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 Processor -- start 2024-09-17T03:38:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 -- start start 2024-09-17T03:38:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fd4160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fd4157f80 0x7f0fd4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd41fc040 0x7f0fd41fc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4167900 con 0x7f0fbc003a40 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4167780 con 0x7f0fd41fc040 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fdc800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd4167a80 con 0x7f0fd4157f80 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fd4160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fd4160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45350/0 (socket says 172.21.15.46:45350) 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fd9e00640 1 -- 172.21.15.46:0/731570983 learned_addr learned my addr 172.21.15.46:0/731570983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fd9400640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fd4157f80 0x7f0fd4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.967+0000 7f0fda800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd41fc040 0x7f0fd41fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fd9e00640 1 -- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fd4157f80 msgr2=0x7f0fd4158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fd9e00640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fd4157f80 0x7f0fd4158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fd9e00640 1 -- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd41fc040 msgr2=0x7f0fd41fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fd9e00640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd41fc040 0x7f0fd41fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fd9e00640 1 -- 172.21.15.46:0/731570983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd41588d0 con 0x7f0fbc003a40 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fd9e00640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fd4160e30 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f0fc801b830 tx=0x7f0fc8006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80014c0 con 0x7f0fbc003a40 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fc8001660 con 0x7f0fbc003a40 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8006330 con 0x7f0fbc003a40 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd41fc990 con 0x7f0fbc003a40 2024-09-17T03:38:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd4163900 con 0x7f0fbc003a40 2024-09-17T03:38:40.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd415ecd0 con 0x7f0fbc003a40 2024-09-17T03:38:40.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.971+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f0fc80b7070 con 0x7f0fbc003a40 2024-09-17T03:38:40.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.975+0000 7f0fce000640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fb8075a80 0x7f0fb8077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:40.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.975+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f0fc812c680 con 0x7f0fbc003a40 2024-09-17T03:38:40.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.975+0000 7f0fd9400640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fb8075a80 0x7f0fb8077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:40.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.975+0000 7f0fd9400640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fb8075a80 0x7f0fb8077f90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4074c10 tx=0x7f0fc4006d70 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:40.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:40.975+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0fd41fc990 con 0x7f0fbc003a40 2024-09-17T03:38:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.147+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fd415e5a0 con 0x7f0fb8075a80 2024-09-17T03:38:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.147+0000 7f0fce000640 1 -- 172.21.15.46:0/731570983 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13580 (secure 0 0 0) 0x7f0fd415e5a0 con 0x7f0fb8075a80 2024-09-17T03:38:41.151 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fb8075a80 msgr2=0x7f0fb8077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fb8075a80 0x7f0fb8077f90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4074c10 tx=0x7f0fc4006d70 comp rx=0 tx=0).stop 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 msgr2=0x7f0fd4160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fd4160e30 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f0fc801b830 tx=0x7f0fc8006d70 comp rx=0 tx=0).stop 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 shutdown_connections 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0fb8075a80 0x7f0fb8077f90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd41fc040 0x7f0fd41fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f0fd4157f80 0x7f0fd4158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 --2- 172.21.15.46:0/731570983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fbc003a40 0x7f0fd4160e30 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 >> 172.21.15.46:0/731570983 conn(0x7f0fd40b2040 msgr2=0x7f0fd415ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 shutdown_connections 2024-09-17T03:38:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.151+0000 7f0fdc800640 1 -- 172.21.15.46:0/731570983 wait complete. 2024-09-17T03:38:41.173 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":674,"stamp":"2024-09-17T03:38:39.636505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1211080,"kb_used_data":19560,"kb_used_omap":12,"kb_used_meta":1191347,"kb_avail":753763640,"statfs":{"total":773094113280,"available":771853967360,"internally_reserved":0,"allocated":20029440,"data_stored":11756600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1219939888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2753488889,"num_objects":-680,"num_object_clones":0,"num_object_copies":-1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-680,"num_whiteouts":0,"num_read":-4500,"num_read_kb":-18038880,"num_write":-10412,"num_write_kb":-21088505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-50331648,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001705"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":345,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-17T03:38:36.949220+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:38:36.949220+0000","last_peered":"2024-09-17T03:38:36.949220+0000","last_clean":"2024-09-17T03:38:36.949220+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:38:36.949220+0000","last_undegraded":"2024-09-17T03:38:36.949220+0000","last_fullsized":"2024-09-17T03:38:36.949220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150108,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221732,"statfs":{"total":96636764160,"available":96483053568,"internally_reserved":0,"allocated":2433024,"data_stored":1397867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":151255498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509655,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159204,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":157374,"kb_avail":94212636,"statfs":{"total":96636764160,"available":96473739264,"internally_reserved":0,"allocated":1851392,"data_stored":815081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161151430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509655,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155088,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216752,"statfs":{"total":96636764160,"available":96477954048,"internally_reserved":0,"allocated":1830912,"data_stored":806847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137056,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234784,"statfs":{"total":96636764160,"available":96496418816,"internally_reserved":0,"allocated":1847296,"data_stored":812963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509655,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161260,"kb_used_data":3800,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94210580,"statfs":{"total":96636764160,"available":96471633920,"internally_reserved":0,"allocated":3891200,"data_stored":2852795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509655,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143800,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94228040,"statfs":{"total":96636764160,"available":96489512960,"internally_reserved":0,"allocated":4493312,"data_stored":3451215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542359,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123156,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248684,"statfs":{"total":96636764160,"available":96510652416,"internally_reserved":0,"allocated":1835008,"data_stored":806869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542359,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181408,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":179582,"kb_avail":94190432,"statfs":{"total":96636764160,"available":96451002368,"internally_reserved":0,"allocated":1847296,"data_stored":812963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183892421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:38:41.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 -- 172.21.15.46:0/1339318605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 msgr2=0x7f9bc415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 --2- 172.21.15.46:0/1339318605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc415bb60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4074b20 tx=0x7f9bb401c5a0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 -- 172.21.15.46:0/1339318605 shutdown_connections 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 --2- 172.21.15.46:0/1339318605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc415bb60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 -- 172.21.15.46:0/1339318605 >> 172.21.15.46:0/1339318605 conn(0x7f9bc40b2040 msgr2=0x7f9bc40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 -- 172.21.15.46:0/1339318605 shutdown_connections 2024-09-17T03:38:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.331+0000 7f9bca200640 1 -- 172.21.15.46:0/1339318605 wait complete. 2024-09-17T03:38:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 Processor -- start 2024-09-17T03:38:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 -- start start 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 0x7f9bc414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc413fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bc41527b0 0x7f9bc40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4169bd0 con 0x7f9bc40b4de0 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4169a50 con 0x7f9bc40b5c70 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc4169d50 con 0x7f9bc41527b0 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 0x7f9bc414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 0x7f9bc414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45366/0 (socket says 172.21.15.46:45366) 2024-09-17T03:38:41.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 -- 172.21.15.46:0/2641315241 learned_addr learned my addr 172.21.15.46:0/2641315241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bbbe00640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc413fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3e00640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bc41527b0 0x7f9bc40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 -- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bc41527b0 msgr2=0x7f9bc40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bc41527b0 0x7f9bc40ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 -- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 msgr2=0x7f9bc413fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc413fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 -- 172.21.15.46:0/2641315241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc415e3a0 con 0x7f9bc40b4de0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bbbe00640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc413fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bc3400640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 0x7f9bc414ffa0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4000c10 tx=0x7f9bb4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4004030 con 0x7f9bc40b4de0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bb40041d0 con 0x7f9bc40b4de0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4004850 con 0x7f9bc40b4de0 2024-09-17T03:38:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc415e680 con 0x7f9bc40b4de0 2024-09-17T03:38:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.335+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc41408b0 con 0x7f9bc40b4de0 2024-09-17T03:38:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.339+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc415bb60 con 0x7f9bc40b4de0 2024-09-17T03:38:41.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.339+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f9bb40b8030 con 0x7f9bc40b4de0 2024-09-17T03:38:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.339+0000 7f9bbb400640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ba0075a80 0x7f9ba0077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.339+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f9bb412d280 con 0x7f9bc40b4de0 2024-09-17T03:38:41.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.339+0000 7f9bbbe00640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ba0075a80 0x7f9ba0077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.339+0000 7f9bbbe00640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ba0075a80 0x7f9ba0077f90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001ec0 tx=0x7f9bac009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:41.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.343+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9bb40bf030 con 0x7f9bc40b4de0 2024-09-17T03:38:41.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.519+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bc4140b50 con 0x7f9ba0075a80 2024-09-17T03:38:41.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.519+0000 7f9bbb400640 1 -- 172.21.15.46:0/2641315241 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13580 (secure 0 0 0) 0x7f9bc4140b50 con 0x7f9ba0075a80 2024-09-17T03:38:41.523 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:41.523 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:38:41.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ba0075a80 msgr2=0x7f9ba0077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ba0075a80 0x7f9ba0077f90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001ec0 tx=0x7f9bac009040 comp rx=0 tx=0).stop 2024-09-17T03:38:41.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 msgr2=0x7f9bc414ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 0x7f9bc414ffa0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4000c10 tx=0x7f9bb4005760 comp rx=0 tx=0).stop 2024-09-17T03:38:41.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 shutdown_connections 2024-09-17T03:38:41.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9ba0075a80 0x7f9ba0077f90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f9bc41527b0 0x7f9bc40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bc40b5c70 0x7f9bc413fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 --2- 172.21.15.46:0/2641315241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc40b4de0 0x7f9bc414ffa0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 >> 172.21.15.46:0/2641315241 conn(0x7f9bc40b2040 msgr2=0x7f9bc40b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 shutdown_connections 2024-09-17T03:38:41.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.523+0000 7f9bca200640 1 -- 172.21.15.46:0/2641315241 wait complete. 2024-09-17T03:38:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":674,"stamp":"2024-09-17T03:38:39.636505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1211080,"kb_used_data":19560,"kb_used_omap":12,"kb_used_meta":1191347,"kb_avail":753763640,"statfs":{"total":773094113280,"available":771853967360,"internally_reserved":0,"allocated":20029440,"data_stored":11756600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1219939888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2753488889,"num_objects":-680,"num_object_clones":0,"num_object_copies":-1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-680,"num_whiteouts":0,"num_read":-4500,"num_read_kb":-18038880,"num_write":-10412,"num_write_kb":-21088505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-50331648,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001705"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":345,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-17T03:38:36.949220+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:38:36.949220+0000","last_peered":"2024-09-17T03:38:36.949220+0000","last_clean":"2024-09-17T03:38:36.949220+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:38:36.949220+0000","last_undegraded":"2024-09-17T03:38:36.949220+0000","last_fullsized":"2024-09-17T03:38:36.949220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150108,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221732,"statfs":{"total":96636764160,"available":96483053568,"internally_reserved":0,"allocated":2433024,"data_stored":1397867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":151255498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509655,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159204,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":157374,"kb_avail":94212636,"statfs":{"total":96636764160,"available":96473739264,"internally_reserved":0,"allocated":1851392,"data_stored":815081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161151430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509655,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155088,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216752,"statfs":{"total":96636764160,"available":96477954048,"internally_reserved":0,"allocated":1830912,"data_stored":806847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137056,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234784,"statfs":{"total":96636764160,"available":96496418816,"internally_reserved":0,"allocated":1847296,"data_stored":812963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509655,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161260,"kb_used_data":3800,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94210580,"statfs":{"total":96636764160,"available":96471633920,"internally_reserved":0,"allocated":3891200,"data_stored":2852795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509655,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143800,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94228040,"statfs":{"total":96636764160,"available":96489512960,"internally_reserved":0,"allocated":4493312,"data_stored":3451215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542359,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123156,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248684,"statfs":{"total":96636764160,"available":96510652416,"internally_reserved":0,"allocated":1835008,"data_stored":806869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542359,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181408,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":179582,"kb_avail":94190432,"statfs":{"total":96636764160,"available":96451002368,"internally_reserved":0,"allocated":1847296,"data_stored":812963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183892421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:38:41.546 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:38:41.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:38:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f279b800640 1 -- 172.21.15.46:0/4094614025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 msgr2=0x7f277c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f279b800640 1 --2- 172.21.15.46:0/4094614025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f277c005160 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f278c009eb0 tx=0x7f278c01d5f0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f2799800640 1 -- 172.21.15.46:0/4094614025 reap_dead start 2024-09-17T03:38:41.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f279b800640 1 -- 172.21.15.46:0/4094614025 shutdown_connections 2024-09-17T03:38:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f279b800640 1 -- 172.21.15.46:0/4094614025 >> 172.21.15.46:0/4094614025 conn(0x7f27940b2040 msgr2=0x7f27940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f279b800640 1 -- 172.21.15.46:0/4094614025 shutdown_connections 2024-09-17T03:38:41.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.703+0000 7f279b800640 1 -- 172.21.15.46:0/4094614025 wait complete. 2024-09-17T03:38:41.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 Processor -- start 2024-09-17T03:38:41.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 -- start start 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f27941f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f279415ab10 0x7f27940ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279415af20 0x7f27940aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f279415cd20 con 0x7f277c004d50 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f279415cba0 con 0x7f279415af20 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f279b800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f279415cea0 con 0x7f279415ab10 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f2798e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f27941f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f2798e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f27941f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45384/0 (socket says 172.21.15.46:45384) 2024-09-17T03:38:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f2798e00640 1 -- 172.21.15.46:0/2915447150 learned_addr learned my addr 172.21.15.46:0/2915447150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f2793e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f279415ab10 0x7f27940ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.707+0000 7f2799800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279415af20 0x7f27940aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2798e00640 1 -- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f279415ab10 msgr2=0x7f27940ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2798e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f279415ab10 0x7f27940ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2798e00640 1 -- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279415af20 msgr2=0x7f27940aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2798e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279415af20 0x7f27940aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2798e00640 1 -- 172.21.15.46:0/2915447150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27941f5ca0 con 0x7f277c004d50 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2793e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f279415ab10 0x7f27940ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2798e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f27941f5760 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2788011ab0 tx=0x7f2788011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278801e070 con 0x7f277c004d50 2024-09-17T03:38:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f278800fc70 con 0x7f277c004d50 2024-09-17T03:38:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788019670 con 0x7f277c004d50 2024-09-17T03:38:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27941f61b0 con 0x7f277c004d50 2024-09-17T03:38:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27941e58a0 con 0x7f277c004d50 2024-09-17T03:38:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27941f66d0 con 0x7f277c004d50 2024-09-17T03:38:41.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.711+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f2788021070 con 0x7f277c004d50 2024-09-17T03:38:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.715+0000 7f2790c00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2774075a80 0x7f2774077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.715+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f2788093e90 con 0x7f277c004d50 2024-09-17T03:38:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.715+0000 7f2793e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2774075a80 0x7f2774077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:41.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.715+0000 7f2793e00640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2774075a80 0x7f2774077f90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2780001420 tx=0x7f2780006920 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.719+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27941f66d0 con 0x7f277c004d50 2024-09-17T03:38:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.887+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f279406b540 con 0x7f2774075a80 2024-09-17T03:38:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.887+0000 7f2790c00640 1 -- 172.21.15.46:0/2915447150 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13559 (secure 0 0 0) 0x7f2780004380 con 0x7f2774075a80 2024-09-17T03:38:41.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2774075a80 msgr2=0x7f2774077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2774075a80 0x7f2774077f90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2780001420 tx=0x7f2780006920 comp rx=0 tx=0).stop 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 msgr2=0x7f27941f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f27941f5760 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2788011ab0 tx=0x7f2788011f70 comp rx=0 tx=0).stop 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 shutdown_connections 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2774075a80 0x7f2774077f90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279415af20 0x7f27940aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f279415ab10 0x7f27940ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 --2- 172.21.15.46:0/2915447150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c004d50 0x7f27941f5760 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 >> 172.21.15.46:0/2915447150 conn(0x7f27940b2040 msgr2=0x7f2794151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 shutdown_connections 2024-09-17T03:38:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:41.891+0000 7f279b800640 1 -- 172.21.15.46:0/2915447150 wait complete. 2024-09-17T03:38:41.913 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":675,"stamp":"2024-09-17T03:38:41.636868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":1211112,"kb_used_data":19588,"kb_used_omap":12,"kb_used_meta":1191347,"kb_avail":753763608,"statfs":{"total":773094113280,"available":771853934592,"internally_reserved":0,"allocated":20058112,"data_stored":11768304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1219939888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2753488889,"num_objects":-680,"num_object_clones":0,"num_object_copies":-1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-680,"num_whiteouts":0,"num_read":-4500,"num_read_kb":-18038880,"num_write":-10412,"num_write_kb":-21088505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-50331648,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.999869"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":345,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-17T03:38:36.949220+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:38:36.949220+0000","last_peered":"2024-09-17T03:38:36.949220+0000","last_clean":"2024-09-17T03:38:36.949220+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:38:36.949220+0000","last_undegraded":"2024-09-17T03:38:36.949220+0000","last_fullsized":"2024-09-17T03:38:36.949220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150108,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221732,"statfs":{"total":96636764160,"available":96483053568,"internally_reserved":0,"allocated":2433024,"data_stored":1397867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":151255498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509655,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159204,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":157374,"kb_avail":94212636,"statfs":{"total":96636764160,"available":96473739264,"internally_reserved":0,"allocated":1851392,"data_stored":815081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161151430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509655,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155088,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216752,"statfs":{"total":96636764160,"available":96477954048,"internally_reserved":0,"allocated":1830912,"data_stored":806847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137056,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234784,"statfs":{"total":96636764160,"available":96496418816,"internally_reserved":0,"allocated":1847296,"data_stored":812963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509655,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161260,"kb_used_data":3800,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94210580,"statfs":{"total":96636764160,"available":96471633920,"internally_reserved":0,"allocated":3891200,"data_stored":2852795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509655,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143800,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94228040,"statfs":{"total":96636764160,"available":96489512960,"internally_reserved":0,"allocated":4493312,"data_stored":3451215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542360,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123188,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248652,"statfs":{"total":96636764160,"available":96510619648,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542359,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181408,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":179582,"kb_avail":94190432,"statfs":{"total":96636764160,"available":96451002368,"internally_reserved":0,"allocated":1847296,"data_stored":812963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183892421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:38:41.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:38:42.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 -- 172.21.15.46:0/1806341370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84641527b0 msgr2=0x7f8464074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 --2- 172.21.15.46:0/1806341370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84641527b0 0x7f8464074e30 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f8464071cb0 tx=0x7f844c01c890 comp rx=0 tx=0).stop 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 -- 172.21.15.46:0/1806341370 shutdown_connections 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 --2- 172.21.15.46:0/1806341370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84641527b0 0x7f8464074e30 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 --2- 172.21.15.46:0/1806341370 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84640b4de0 0x7f84640b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 --2- 172.21.15.46:0/1806341370 >> v2:172.21.15.99:3300/0 conn(0x7f84640b5c70 0x7f846415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 -- 172.21.15.46:0/1806341370 >> 172.21.15.46:0/1806341370 conn(0x7f84640b2040 msgr2=0x7f84640b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 -- 172.21.15.46:0/1806341370 shutdown_connections 2024-09-17T03:38:42.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 -- 172.21.15.46:0/1806341370 wait complete. 2024-09-17T03:38:42.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 Processor -- start 2024-09-17T03:38:42.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.071+0000 7f8469c00640 1 -- start start 2024-09-17T03:38:42.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 0x7f8464144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:42.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84640b5c70 0x7f84640cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:42.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84641527b0 0x7f84641fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:42.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f846415f6b0 con 0x7f84640b4de0 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f846415f530 con 0x7f84641527b0 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f846415f830 con 0x7f84640b5c70 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 0x7f8464144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 0x7f8464144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45410/0 (socket says 172.21.15.46:45410) 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 -- 172.21.15.46:0/127714463 learned_addr learned my addr 172.21.15.46:0/127714463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8463400640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84641527b0 0x7f84641fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f845a000640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84640b5c70 0x7f84640cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 -- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84640b5c70 msgr2=0x7f84640cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84640b5c70 0x7f84640cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 -- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84641527b0 msgr2=0x7f84641fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84641527b0 0x7f84641fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 -- 172.21.15.46:0/127714463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84641fcc50 con 0x7f84640b4de0 2024-09-17T03:38:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8462a00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 0x7f8464144c20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f845400e7a0 tx=0x7f845400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845400ccc0 con 0x7f84640b4de0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8454010430 con 0x7f84640b4de0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84640cfc50 con 0x7f84640b4de0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454010860 con 0x7f84640b4de0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8464154460 con 0x7f84640b4de0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.075+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846415fd50 con 0x7f84640b4de0 2024-09-17T03:38:42.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.079+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f8454010a00 con 0x7f84640b4de0 2024-09-17T03:38:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.079+0000 7f845b400640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8440075ab0 0x7f8440077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.079+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f8454093fb0 con 0x7f84640b4de0 2024-09-17T03:38:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.079+0000 7f845a000640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8440075ab0 0x7f8440077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:42.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.079+0000 7f845a000640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8440075ab0 0x7f8440077fc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f844c0af000 tx=0x7f844c01b0d0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:42.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.083+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8454060ac0 con 0x7f84640b4de0 2024-09-17T03:38:42.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.259+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84640cd810 con 0x7f84640b4de0 2024-09-17T03:38:42.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.259+0000 7f845b400640 1 -- 172.21.15.46:0/127714463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+13050 (secure 0 0 0) 0x7f84540603c0 con 0x7f84640b4de0 2024-09-17T03:38:42.263 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:42.263 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"fsid":"6c25af6b-3689-4426-acb2-6ebe6de6efd5","created":"2024-09-17T03:20:41.125148+0000","modified":"2024-09-17T03:38:37.500161+0000","last_up_change":"2024-09-17T03:20:55.951112+0000","last_in_change":"2024-09-17T03:20:45.817503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:20:55.230199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5991f809-e043-4b36-aa08-043a41133140","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6811","nonce":1962470687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6813","nonce":1962470687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6817","nonce":1962470687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1962470687},{"type":"v1","addr":"172.21.15.46:6815","nonce":1962470687}]},"public_addr":"172.21.15.46:6811/1962470687","cluster_addr":"172.21.15.46:6813/1962470687","heartbeat_back_addr":"172.21.15.46:6817/1962470687","heartbeat_front_addr":"172.21.15.46:6815/1962470687","state":["exists","up"]},{"osd":1,"uuid":"332bd3c4-c4cd-4788-9a2d-ae1d68f4686f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6803","nonce":2558692648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6805","nonce":2558692648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6809","nonce":2558692648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2558692648},{"type":"v1","addr":"172.21.15.46:6807","nonce":2558692648}]},"public_addr":"172.21.15.46:6803/2558692648","cluster_addr":"172.21.15.46:6805/2558692648","heartbeat_back_addr":"172.21.15.46:6809/2558692648","heartbeat_front_addr":"172.21.15.46:6807/2558692648","state":["exists","up"]},{"osd":2,"uuid":"a6821825-458d-44c4-b5e2-a72d7f7cb840","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6827","nonce":1373384937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6829","nonce":1373384937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6833","nonce":1373384937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1373384937},{"type":"v1","addr":"172.21.15.46:6831","nonce":1373384937}]},"public_addr":"172.21.15.46:6827/1373384937","cluster_addr":"172.21.15.46:6829/1373384937","heartbeat_back_addr":"172.21.15.46:6833/1373384937","heartbeat_front_addr":"172.21.15.46:6831/1373384937","state":["exists","up"]},{"osd":3,"uuid":"d9adfef4-8f8c-4aac-876e-4014eafeac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6819","nonce":2947041972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6821","nonce":2947041972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6825","nonce":2947041972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2947041972},{"type":"v1","addr":"172.21.15.46:6823","nonce":2947041972}]},"public_addr":"172.21.15.46:6819/2947041972","cluster_addr":"172.21.15.46:6821/2947041972","heartbeat_back_addr":"172.21.15.46:6825/2947041972","heartbeat_front_addr":"172.21.15.46:6823/2947041972","state":["exists","up"]},{"osd":4,"uuid":"72138866-38d1-45f3-a209-6f9a4523c997","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6809","nonce":2242927697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6811","nonce":2242927697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6815","nonce":2242927697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2242927697},{"type":"v1","addr":"172.21.15.99:6813","nonce":2242927697}]},"public_addr":"172.21.15.99:6809/2242927697","cluster_addr":"172.21.15.99:6811/2242927697","heartbeat_back_addr":"172.21.15.99:6815/2242927697","heartbeat_front_addr":"172.21.15.99:6813/2242927697","state":["exists","up"]},{"osd":5,"uuid":"77e04047-9271-4fdf-aac9-29db70985c96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6801","nonce":4253831876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6803","nonce":4253831876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6807","nonce":4253831876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4253831876},{"type":"v1","addr":"172.21.15.99:6805","nonce":4253831876}]},"public_addr":"172.21.15.99:6801/4253831876","cluster_addr":"172.21.15.99:6803/4253831876","heartbeat_back_addr":"172.21.15.99:6807/4253831876","heartbeat_front_addr":"172.21.15.99:6805/4253831876","state":["exists","up"]},{"osd":6,"uuid":"c3c7a1a2-d8bf-4068-9dd8-bee95f0ef49a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6817","nonce":788789999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6819","nonce":788789999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6823","nonce":788789999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":788789999},{"type":"v1","addr":"172.21.15.99:6821","nonce":788789999}]},"public_addr":"172.21.15.99:6817/788789999","cluster_addr":"172.21.15.99:6819/788789999","heartbeat_back_addr":"172.21.15.99:6823/788789999","heartbeat_front_addr":"172.21.15.99:6821/788789999","state":["exists","up"]},{"osd":7,"uuid":"e3a1d2f7-e367-45f8-a71c-3aafeb3bc6c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6825","nonce":3170570057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6827","nonce":3170570057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6831","nonce":3170570057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3170570057},{"type":"v1","addr":"172.21.15.99:6829","nonce":3170570057}]},"public_addr":"172.21.15.99:6825/3170570057","cluster_addr":"172.21.15.99:6827/3170570057","heartbeat_back_addr":"172.21.15.99:6831/3170570057","heartbeat_front_addr":"172.21.15.99:6829/3170570057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.783140+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.756265+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.942420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.895992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.923803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.700222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:52.893373+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:20:53.520921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6837/2038056563":"2024-09-18T03:38:34.633901+0000","172.21.15.46:6836/2038056563":"2024-09-18T03:38:34.633901+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:38:42.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8440075ab0 msgr2=0x7f8440077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:42.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8440075ab0 0x7f8440077fc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f844c0af000 tx=0x7f844c01b0d0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 msgr2=0x7f8464144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:42.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 0x7f8464144c20 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f845400e7a0 tx=0x7f845400ec60 comp rx=0 tx=0).stop 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 shutdown_connections 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8440075ab0 0x7f8440077fc0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84641527b0 0x7f84641fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f84640b5c70 0x7f84640cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 --2- 172.21.15.46:0/127714463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84640b4de0 0x7f8464144c20 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 >> 172.21.15.46:0/127714463 conn(0x7f84640b2040 msgr2=0x7f8464159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 shutdown_connections 2024-09-17T03:38:42.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:42.263+0000 7f8469c00640 1 -- 172.21.15.46:0/127714463 wait complete. 2024-09-17T03:38:43.286 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T03:38:43.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 -- 172.21.15.46:0/3803583110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 msgr2=0x7fef0415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 --2- 172.21.15.46:0/3803583110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef0415bb60 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fef040cca90 tx=0x7feef401c800 comp rx=0 tx=0).stop 2024-09-17T03:38:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 -- 172.21.15.46:0/3803583110 shutdown_connections 2024-09-17T03:38:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 --2- 172.21.15.46:0/3803583110 >> v2:172.21.15.99:3300/0 conn(0x7fef040b4de0 0x7fef040b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 --2- 172.21.15.46:0/3803583110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef0415bb60 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 -- 172.21.15.46:0/3803583110 >> 172.21.15.46:0/3803583110 conn(0x7fef040b2040 msgr2=0x7fef040b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 -- 172.21.15.46:0/3803583110 shutdown_connections 2024-09-17T03:38:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 -- 172.21.15.46:0/3803583110 wait complete. 2024-09-17T03:38:43.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 Processor -- start 2024-09-17T03:38:43.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.463+0000 7fef08a00640 1 -- start start 2024-09-17T03:38:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef040b4de0 0x7fef04154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef041f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef041527b0 0x7fef04142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef04168970 con 0x7fef040b5c70 2024-09-17T03:38:43.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef041687f0 con 0x7fef041527b0 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fef04168af0 con 0x7fef040b4de0 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef041f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef041f8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45290/0 (socket says 172.21.15.46:45290) 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 -- 172.21.15.46:0/2542588951 learned_addr learned my addr 172.21.15.46:0/2542588951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01e00640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef040b4de0 0x7fef04154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef02800640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef041527b0 0x7fef04142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 -- 172.21.15.46:0/2542588951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef040b4de0 msgr2=0x7fef04154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fef040b4de0 0x7fef04154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 -- 172.21.15.46:0/2542588951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef041527b0 msgr2=0x7fef04142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef041527b0 0x7fef04142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 -- 172.21.15.46:0/2542588951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef04154c00 con 0x7fef040b5c70 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef01400640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef041f8200 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7feef000c3c0 tx=0x7feef000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:43.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0017070 con 0x7fef040b5c70 2024-09-17T03:38:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feef00053c0 con 0x7fef040b5c70 2024-09-17T03:38:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef04154ee0 con 0x7fef040b5c70 2024-09-17T03:38:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0003860 con 0x7fef040b5c70 2024-09-17T03:38:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041554a0 con 0x7fef040b5c70 2024-09-17T03:38:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.467+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef0406b540 con 0x7fef040b5c70 2024-09-17T03:38:43.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7feef001a070 con 0x7fef040b5c70 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee4075a80 0x7feee4077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7feef0092fb0 con 0x7fef040b5c70 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7fef01e00640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee4075a80 0x7feee4077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7feee407b990 0x7feee407de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 --> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feef0010e50 con 0x7feee407b990 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7feef0093360 con 0x7fef040b5c70 2024-09-17T03:38:43.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7fef02800640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7feee407b990 0x7feee407de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7fef01e00640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee4075a80 0x7feee4077f90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7feef40af000 tx=0x7feef40a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:43.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7fef02800640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7feee407b990 0x7feee407de00 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:43.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.471+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== osd.0 v2:172.21.15.46:6810/1962470687 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feef0010e50 con 0x7feee407b990 2024-09-17T03:38:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 --> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fef04154990 con 0x7feee407b990 2024-09-17T03:38:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7feefa000640 1 -- 172.21.15.46:0/2542588951 <== osd.0 v2:172.21.15.46:6810/1962470687 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fef04154990 con 0x7feee407b990 2024-09-17T03:38:43.490 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:43.490 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:43.490 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:43.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7feee407b990 msgr2=0x7feee407de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6810/1962470687,v1:172.21.15.46:6811/1962470687] conn(0x7feee407b990 0x7feee407de00 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee4075a80 msgr2=0x7feee4077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7feee4075a80 0x7feee4077f90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7feef40af000 tx=0x7feef40a7040 comp rx=0 tx=0).stop 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 msgr2=0x7fef041f8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 --2- 172.21.15.46:0/2542588951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef040b5c70 0x7fef041f8200 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7feef000c3c0 tx=0x7feef000c880 comp rx=0 tx=0).stop 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef02800640 1 -- 172.21.15.46:0/2542588951 reap_dead start 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 shutdown_connections 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 >> 172.21.15.46:0/2542588951 conn(0x7fef040b2040 msgr2=0x7fef040cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 shutdown_connections 2024-09-17T03:38:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.487+0000 7fef08a00640 1 -- 172.21.15.46:0/2542588951 wait complete. 2024-09-17T03:38:43.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 -- 172.21.15.46:0/2470896932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 msgr2=0x7f6854004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 --2- 172.21.15.46:0/2470896932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6854004c20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f6858009b10 tx=0x7f685801d230 comp rx=0 tx=0).stop 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 -- 172.21.15.46:0/2470896932 shutdown_connections 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 --2- 172.21.15.46:0/2470896932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6854004c20 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 -- 172.21.15.46:0/2470896932 >> 172.21.15.46:0/2470896932 conn(0x7f68700b4560 msgr2=0x7f68700d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 -- 172.21.15.46:0/2470896932 shutdown_connections 2024-09-17T03:38:43.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 -- 172.21.15.46:0/2470896932 wait complete. 2024-09-17T03:38:43.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 Processor -- start 2024-09-17T03:38:43.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.671+0000 7f6874c00640 1 -- start start 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6870151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6870152100 0x7f68701f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701f70d0 0x7f68700ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68700745d0 con 0x7f6854004810 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6870074450 con 0x7f68701f70d0 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6870074750 con 0x7f6870152100 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6870151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6870151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45296/0 (socket says 172.21.15.46:45296) 2024-09-17T03:38:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 -- 172.21.15.46:0/2416653230 learned_addr learned my addr 172.21.15.46:0/2416653230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6867e00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6870152100 0x7f68701f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686fe00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701f70d0 0x7f68700ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 -- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6870152100 msgr2=0x7f68701f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6870152100 0x7f68701f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 -- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701f70d0 msgr2=0x7f68700ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701f70d0 0x7f68700ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 -- 172.21.15.46:0/2416653230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68700c7df0 con 0x7f6854004810 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f686f400640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6870151aa0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f6860006350 tx=0x7f68600039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860015070 con 0x7f6854004810 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6860011840 con 0x7f6854004810 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860011c90 con 0x7f6854004810 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68700c80d0 con 0x7f6854004810 2024-09-17T03:38:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68700acf40 con 0x7f6854004810 2024-09-17T03:38:43.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.675+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68700b5260 con 0x7f6854004810 2024-09-17T03:38:43.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.679+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f6860018070 con 0x7f6854004810 2024-09-17T03:38:43.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.679+0000 7f6867400640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f684c075ab0 0x7f684c077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:43.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.679+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f68600940f0 con 0x7f6854004810 2024-09-17T03:38:43.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.679+0000 7f6867e00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f684c075ab0 0x7f684c077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:43.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.679+0000 7f6867e00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f684c075ab0 0x7f684c077fc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f68580ad000 tx=0x7f6858019fd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:43.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.683+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6860060520 con 0x7f6854004810 2024-09-17T03:38:43.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f687006b540 con 0x7f684c075ab0 2024-09-17T03:38:43.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6867400640 1 -- 172.21.15.46:0/2416653230 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6858005320 con 0x7f684c075ab0 2024-09-17T03:38:43.861 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f684c075ab0 msgr2=0x7f684c077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f684c075ab0 0x7f684c077fc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f68580ad000 tx=0x7f6858019fd0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 msgr2=0x7f6870151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6870151aa0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f6860006350 tx=0x7f68600039d0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 shutdown_connections 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f684c075ab0 0x7f684c077fc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701f70d0 0x7f68700ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6870152100 0x7f68701f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 --2- 172.21.15.46:0/2416653230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6854004810 0x7f6870151aa0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.859+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 >> 172.21.15.46:0/2416653230 conn(0x7f68700b4560 msgr2=0x7f687015e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.863+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 shutdown_connections 2024-09-17T03:38:43.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:43.863+0000 7f6874c00640 1 -- 172.21.15.46:0/2416653230 wait complete. 2024-09-17T03:38:43.884 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T03:38:43.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:44.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 -- 172.21.15.46:0/2174485883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 msgr2=0x7fd0b40b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 --2- 172.21.15.46:0/2174485883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b40b1dc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009900 tx=0x7fd0a401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 -- 172.21.15.46:0/2174485883 shutdown_connections 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 --2- 172.21.15.46:0/2174485883 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0b40b2300 0x7fd0b40d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 --2- 172.21.15.46:0/2174485883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b40b1dc0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 -- 172.21.15.46:0/2174485883 >> 172.21.15.46:0/2174485883 conn(0x7fd0b406cdf0 msgr2=0x7fd0b406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 -- 172.21.15.46:0/2174485883 shutdown_connections 2024-09-17T03:38:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.043+0000 7fd0ba200640 1 -- 172.21.15.46:0/2174485883 wait complete. 2024-09-17T03:38:44.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 Processor -- start 2024-09-17T03:38:44.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 -- start start 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b4140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b40b2300 0x7fd0b4141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0b40d05d0 0x7fd0b4141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b40cf610 con 0x7fd0b4074d30 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b40cf490 con 0x7fd0b40b2300 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0ba200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b4075300 con 0x7fd0b40d05d0 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0b8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b4140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0b8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b4140410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45304/0 (socket says 172.21.15.46:45304) 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0b8e00640 1 -- 172.21.15.46:0/1473523526 learned_addr learned my addr 172.21.15.46:0/1473523526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:44.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0b9800640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0b40d05d0 0x7fd0b4141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.047+0000 7fd0abe00640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b40b2300 0x7fd0b4141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b8e00640 1 -- 172.21.15.46:0/1473523526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0b40d05d0 msgr2=0x7fd0b4141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b8e00640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fd0b40d05d0 0x7fd0b4141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b8e00640 1 -- 172.21.15.46:0/1473523526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b40b2300 msgr2=0x7fd0b4141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b8e00640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0b40b2300 0x7fd0b4141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b8e00640 1 -- 172.21.15.46:0/1473523526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b4155ac0 con 0x7fd0b4074d30 2024-09-17T03:38:44.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b8e00640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b4140410 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009850 tx=0x7fd0a40a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40b2040 con 0x7fd0b4074d30 2024-09-17T03:38:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0a4003970 con 0x7fd0b4074d30 2024-09-17T03:38:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b41fc0c0 con 0x7fd0b4074d30 2024-09-17T03:38:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4003d80 con 0x7fd0b4074d30 2024-09-17T03:38:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41fc680 con 0x7fd0b4074d30 2024-09-17T03:38:44.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0b406b540 con 0x7fd0b4074d30 2024-09-17T03:38:44.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.051+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fd0a40b8470 con 0x7fd0b4074d30 2024-09-17T03:38:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b1600640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd090075a80 0x7fd090077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fd0a412dc00 con 0x7fd0b4074d30 2024-09-17T03:38:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0abe00640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd090075a80 0x7fd090077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b1600640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd09007b990 0x7fd09007de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 --> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0a40bee50 con 0x7fd09007b990 2024-09-17T03:38:44.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7fd0a412e060 con 0x7fd0b4074d30 2024-09-17T03:38:44.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b9800640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd09007b990 0x7fd09007de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0abe00640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd090075a80 0x7fd090077f90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40b1b90 tx=0x7fd09c00a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b9800640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd09007b990 0x7fd09007de00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.055+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== osd.1 v2:172.21.15.46:6802/2558692648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0a40bee50 con 0x7fd09007b990 2024-09-17T03:38:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.067+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 --> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd0b40b1dc0 con 0x7fd09007b990 2024-09-17T03:38:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0b1600640 1 -- 172.21.15.46:0/1473523526 <== osd.1 v2:172.21.15.46:6802/2558692648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd0b40b1dc0 con 0x7fd09007b990 2024-09-17T03:38:44.072 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:44.072 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:44.072 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd09007b990 msgr2=0x7fd09007de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6802/2558692648,v1:172.21.15.46:6803/2558692648] conn(0x7fd09007b990 0x7fd09007de00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd090075a80 msgr2=0x7fd090077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fd090075a80 0x7fd090077f90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40b1b90 tx=0x7fd09c00a250 comp rx=0 tx=0).stop 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 msgr2=0x7fd0b4140410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 --2- 172.21.15.46:0/1473523526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b4074d30 0x7fd0b4140410 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009850 tx=0x7fd0a40a5810 comp rx=0 tx=0).stop 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0b9800640 1 -- 172.21.15.46:0/1473523526 reap_dead start 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 shutdown_connections 2024-09-17T03:38:44.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 >> 172.21.15.46:0/1473523526 conn(0x7fd0b406cdf0 msgr2=0x7fd0b4074360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 shutdown_connections 2024-09-17T03:38:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.071+0000 7fd0ba200640 1 -- 172.21.15.46:0/1473523526 wait complete. 2024-09-17T03:38:44.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 -- 172.21.15.46:0/287153583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418157030 msgr2=0x7ff4180b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 --2- 172.21.15.46:0/287153583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418157030 0x7ff4180b5a80 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff418160830 tx=0x7ff40801c8c0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 -- 172.21.15.46:0/287153583 shutdown_connections 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 --2- 172.21.15.46:0/287153583 >> v2:172.21.15.99:3300/0 conn(0x7ff418161160 0x7ff418074950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 --2- 172.21.15.46:0/287153583 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff418160d50 0x7ff418074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 --2- 172.21.15.46:0/287153583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418157030 0x7ff4180b5a80 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 -- 172.21.15.46:0/287153583 >> 172.21.15.46:0/287153583 conn(0x7ff4180708c0 msgr2=0x7ff418070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 -- 172.21.15.46:0/287153583 shutdown_connections 2024-09-17T03:38:44.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.247+0000 7ff41e200640 1 -- 172.21.15.46:0/287153583 wait complete. 2024-09-17T03:38:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 Processor -- start 2024-09-17T03:38:44.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 -- start start 2024-09-17T03:38:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff418157030 0x7ff418158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff418160d50 0x7ff418159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 0x7ff4180b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff418072290 con 0x7ff418161160 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff418072110 con 0x7ff418157030 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff41e200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7ff418072410 con 0x7ff418160d50 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 0x7ff4180b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 0x7ff4180b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45316/0 (socket says 172.21.15.46:45316) 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 -- 172.21.15.46:0/3865993847 learned_addr learned my addr 172.21.15.46:0/3865993847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417400640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff418157030 0x7ff418158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff40fe00640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff418160d50 0x7ff418159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 -- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff418160d50 msgr2=0x7ff418159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff418160d50 0x7ff418159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 -- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff418157030 msgr2=0x7ff418158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff418157030 0x7ff418158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.251+0000 7ff417e00640 1 -- 172.21.15.46:0/3865993847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41814ee00 con 0x7ff418161160 2024-09-17T03:38:44.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff417e00640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 0x7ff4180b35c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff41000e9a0 tx=0x7ff41000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410004d30 con 0x7ff418161160 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff41000cb20 con 0x7ff418161160 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41814f0e0 con 0x7ff418161160 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410005020 con 0x7ff418161160 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41814f6a0 con 0x7ff418161160 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff418072900 con 0x7ff418161160 2024-09-17T03:38:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7ff4100051c0 con 0x7ff418161160 2024-09-17T03:38:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.255+0000 7ff40f400640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3f4075a80 0x7ff3f4077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.259+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7ff410093ed0 con 0x7ff418161160 2024-09-17T03:38:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.259+0000 7ff417400640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3f4075a80 0x7ff3f4077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.259+0000 7ff417400640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3f4075a80 0x7ff3f4077f90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff4080af000 tx=0x7ff4080a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.263+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff410016030 con 0x7ff418161160 2024-09-17T03:38:44.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.435+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff41806b540 con 0x7ff3f4075a80 2024-09-17T03:38:44.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.435+0000 7ff40f400640 1 -- 172.21.15.46:0/3865993847 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff4080b3040 con 0x7ff3f4075a80 2024-09-17T03:38:44.438 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3f4075a80 msgr2=0x7ff3f4077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3f4075a80 0x7ff3f4077f90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff4080af000 tx=0x7ff4080a7040 comp rx=0 tx=0).stop 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 msgr2=0x7ff4180b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 0x7ff4180b35c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff41000e9a0 tx=0x7ff41000ee60 comp rx=0 tx=0).stop 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 shutdown_connections 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7ff3f4075a80 0x7ff3f4077f90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff418161160 0x7ff4180b35c0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7ff418160d50 0x7ff418159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 --2- 172.21.15.46:0/3865993847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff418157030 0x7ff418158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 >> 172.21.15.46:0/3865993847 conn(0x7ff4180708c0 msgr2=0x7ff4180d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 shutdown_connections 2024-09-17T03:38:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.439+0000 7ff41e200640 1 -- 172.21.15.46:0/3865993847 wait complete. 2024-09-17T03:38:44.461 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T03:38:44.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 -- 172.21.15.46:0/4098519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 msgr2=0x7f6b7c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 --2- 172.21.15.46:0/4098519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c074e30 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c072e30 tx=0x7f6b6401c800 comp rx=0 tx=0).stop 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 -- 172.21.15.46:0/4098519684 shutdown_connections 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 --2- 172.21.15.46:0/4098519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c074e30 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 --2- 172.21.15.46:0/4098519684 >> v2:172.21.15.99:3300/0 conn(0x7f6b7c0b4de0 0x7f6b7c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 --2- 172.21.15.46:0/4098519684 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6b7c0b5c70 0x7f6b7c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 -- 172.21.15.46:0/4098519684 >> 172.21.15.46:0/4098519684 conn(0x7f6b7c0b2040 msgr2=0x7f6b7c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 -- 172.21.15.46:0/4098519684 shutdown_connections 2024-09-17T03:38:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.623+0000 7f6b82000640 1 -- 172.21.15.46:0/4098519684 wait complete. 2024-09-17T03:38:44.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 Processor -- start 2024-09-17T03:38:44.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- start start 2024-09-17T03:38:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b7c0b4de0 0x7f6b7c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6b7c0b5c70 0x7f6b7c144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c15f700 con 0x7f6b7c1527b0 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c15f580 con 0x7f6b7c0b4de0 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c15f880 con 0x7f6b7c0b5c70 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b7c0b4de0 0x7f6b7c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45330/0 (socket says 172.21.15.46:45330) 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 -- 172.21.15.46:0/2991461533 learned_addr learned my addr 172.21.15.46:0/2991461533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b72a00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6b7c0b5c70 0x7f6b7c144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 -- 172.21.15.46:0/2991461533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6b7c0b5c70 msgr2=0x7f6b7c144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f6b7c0b5c70 0x7f6b7c144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 -- 172.21.15.46:0/2991461533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b7c0b4de0 msgr2=0x7f6b7c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b7c0b4de0 0x7f6b7c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 -- 172.21.15.46:0/2991461533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c0cd370 con 0x7f6b7c1527b0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7b400640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b7c0b4de0 0x7f6b7c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b7be00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c0ced30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400ea00 tx=0x7f6b7400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400cde0 con 0x7f6b7c1527b0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b74004c20 con 0x7f6b7c1527b0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c155430 con 0x7f6b7c1527b0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74005020 con 0x7f6b7c1527b0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c145060 con 0x7f6b7c1527b0 2024-09-17T03:38:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.627+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b7c06b540 con 0x7f6b7c1527b0 2024-09-17T03:38:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f6b740051c0 con 0x7f6b7c1527b0 2024-09-17T03:38:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b73e00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b58075a80 0x7f6b58077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f6b74094ae0 con 0x7f6b7c1527b0 2024-09-17T03:38:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b7b400640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b58075a80 0x7f6b58077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b73e00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7f6b5807b990 0x7f6b5807de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 --> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b74022e50 con 0x7f6b5807b990 2024-09-17T03:38:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f6b74094e90 con 0x7f6b7c1527b0 2024-09-17T03:38:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b72a00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7f6b5807b990 0x7f6b5807de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b7b400640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b58075a80 0x7f6b58077f90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009f30 tx=0x7f6b6c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.631+0000 7f6b72a00640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7f6b5807b990 0x7f6b5807de00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.635+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== osd.2 v2:172.21.15.46:6826/1373384937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b74022e50 con 0x7f6b5807b990 2024-09-17T03:38:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 --> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b7c15fda0 con 0x7f6b5807b990 2024-09-17T03:38:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b73e00640 1 -- 172.21.15.46:0/2991461533 <== osd.2 v2:172.21.15.46:6826/1373384937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6b7c15fda0 con 0x7f6b5807b990 2024-09-17T03:38:44.650 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:44.650 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:44.650 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7f6b5807b990 msgr2=0x7f6b5807de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6826/1373384937,v1:172.21.15.46:6827/1373384937] conn(0x7f6b5807b990 0x7f6b5807de00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b58075a80 msgr2=0x7f6b58077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f6b58075a80 0x7f6b58077f90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009f30 tx=0x7f6b6c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 msgr2=0x7f6b7c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 --2- 172.21.15.46:0/2991461533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b7c1527b0 0x7f6b7c0ced30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400ea00 tx=0x7f6b7400eec0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b7be00640 1 -- 172.21.15.46:0/2991461533 reap_dead start 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 shutdown_connections 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 >> 172.21.15.46:0/2991461533 conn(0x7f6b7c0b2040 msgr2=0x7f6b7c06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 shutdown_connections 2024-09-17T03:38:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.647+0000 7f6b82000640 1 -- 172.21.15.46:0/2991461533 wait complete. 2024-09-17T03:38:44.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T03:38:44.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 -- 172.21.15.46:0/2391894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c1527b0 msgr2=0x7f884c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 --2- 172.21.15.46:0/2391894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c1527b0 0x7f884c15bb60 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f884c0cca90 tx=0x7f883401c800 comp rx=0 tx=0).stop 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 -- 172.21.15.46:0/2391894 shutdown_connections 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 --2- 172.21.15.46:0/2391894 >> v2:172.21.15.99:3300/0 conn(0x7f884c0b4de0 0x7f884c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 --2- 172.21.15.46:0/2391894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c1527b0 0x7f884c15bb60 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 -- 172.21.15.46:0/2391894 >> 172.21.15.46:0/2391894 conn(0x7f884c0b2040 msgr2=0x7f884c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 -- 172.21.15.46:0/2391894 shutdown_connections 2024-09-17T03:38:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 -- 172.21.15.46:0/2391894 wait complete. 2024-09-17T03:38:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.827+0000 7f8852c00640 1 Processor -- start 2024-09-17T03:38:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- start start 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 0x7f884c1fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f884c0b5c70 0x7f884c1f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f884c1527b0 0x7f884c153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f884c168920 con 0x7f884c0b4de0 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f884c1687a0 con 0x7f884c0b5c70 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f884c168aa0 con 0x7f884c1527b0 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 0x7f884c1fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 0x7f884c1fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45340/0 (socket says 172.21.15.46:45340) 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 -- 172.21.15.46:0/2176812220 learned_addr learned my addr 172.21.15.46:0/2176812220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8843e00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f884c0b5c70 0x7f884c1f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8850c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f884c1527b0 0x7f884c153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 -- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f884c1527b0 msgr2=0x7f884c153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f884c1527b0 0x7f884c153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 -- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f884c0b5c70 msgr2=0x7f884c1f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f884c0b5c70 0x7f884c1f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 -- 172.21.15.46:0/2176812220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c1fc770 con 0x7f884c0b4de0 2024-09-17T03:38:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f884be00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 0x7f884c1fbf20 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c3c0 tx=0x7f883c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c017070 con 0x7f884c0b4de0 2024-09-17T03:38:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f883c0053c0 con 0x7f884c0b4de0 2024-09-17T03:38:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c003860 con 0x7f884c0b4de0 2024-09-17T03:38:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c1fca50 con 0x7f884c0b4de0 2024-09-17T03:38:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c1f8a40 con 0x7f884c0b4de0 2024-09-17T03:38:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.831+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c0b51f0 con 0x7f884c0b4de0 2024-09-17T03:38:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.835+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f883c01a070 con 0x7f884c0b4de0 2024-09-17T03:38:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.835+0000 7f8848c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8828075a80 0x7f8828077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.835+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f883c092830 con 0x7f884c0b4de0 2024-09-17T03:38:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.835+0000 7f8843e00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8828075a80 0x7f8828077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.835+0000 7f8843e00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8828075a80 0x7f8828077f90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f88340af000 tx=0x7f88340a7040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:44.839+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f883c09d020 con 0x7f884c0b4de0 2024-09-17T03:38:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.011+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f884c06b540 con 0x7f8828075a80 2024-09-17T03:38:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8848c00640 1 -- 172.21.15.46:0/2176812220 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f88340b3040 con 0x7f8828075a80 2024-09-17T03:38:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T03:38:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8828075a80 msgr2=0x7f8828077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8828075a80 0x7f8828077f90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f88340af000 tx=0x7f88340a7040 comp rx=0 tx=0).stop 2024-09-17T03:38:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 msgr2=0x7f884c1fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 0x7f884c1fbf20 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c3c0 tx=0x7f883c00c880 comp rx=0 tx=0).stop 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 shutdown_connections 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f8828075a80 0x7f8828077f90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f884c1527b0 0x7f884c153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f884c0b5c70 0x7f884c1f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 --2- 172.21.15.46:0/2176812220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f884c0b4de0 0x7f884c1fbf20 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 >> 172.21.15.46:0/2176812220 conn(0x7f884c0b2040 msgr2=0x7f884c15a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 shutdown_connections 2024-09-17T03:38:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.015+0000 7f8852c00640 1 -- 172.21.15.46:0/2176812220 wait complete. 2024-09-17T03:38:45.038 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T03:38:45.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f0171000640 1 -- 172.21.15.46:0/2215736286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0150003a10 msgr2=0x7f0150003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f0171000640 1 --2- 172.21.15.46:0/2215736286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0150003a10 0x7f0150003e80 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c310 tx=0x7f015c01cf90 comp rx=0 tx=0).stop 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f016a800640 1 -- 172.21.15.46:0/2215736286 reap_dead start 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f0171000640 1 -- 172.21.15.46:0/2215736286 shutdown_connections 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f0171000640 1 -- 172.21.15.46:0/2215736286 >> 172.21.15.46:0/2215736286 conn(0x7f016c0b2c40 msgr2=0x7f016c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f0171000640 1 -- 172.21.15.46:0/2215736286 shutdown_connections 2024-09-17T03:38:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.199+0000 7f0171000640 1 -- 172.21.15.46:0/2215736286 wait complete. 2024-09-17T03:38:45.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 Processor -- start 2024-09-17T03:38:45.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 -- start start 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0150003a10 0x7f016c1fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f016c1fc390 0x7f016c14c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f016c14ce40 0x7f016c163510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f016c073040 con 0x7f016c1fc390 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f016c072ec0 con 0x7f0150003a10 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0171000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f016c0731c0 con 0x7f016c14ce40 2024-09-17T03:38:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f016c1fc390 0x7f016c14c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f016c1fc390 0x7f016c14c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45364/0 (socket says 172.21.15.46:45364) 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 -- 172.21.15.46:0/2180641504 learned_addr learned my addr 172.21.15.46:0/2180641504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 -- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f016c14ce40 msgr2=0x7f016c163510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f016a800640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f016c14ce40 0x7f016c163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f016c14ce40 0x7f016c163510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169e00640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0150003a10 0x7f016c1fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 -- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0150003a10 msgr2=0x7f016c1fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0150003a10 0x7f016c1fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 -- 172.21.15.46:0/2180641504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c1f83c0 con 0x7f016c1fc390 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f016a800640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f016c14ce40 0x7f016c163510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169e00640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0150003a10 0x7f016c1fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:45.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.203+0000 7f0169400640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f016c1fc390 0x7f016c14c900 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f0158011b10 tx=0x7f0158009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015801e070 con 0x7f016c1fc390 2024-09-17T03:38:45.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0158004460 con 0x7f016c1fc390 2024-09-17T03:38:45.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c1f86a0 con 0x7f016c1fc390 2024-09-17T03:38:45.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015800fc30 con 0x7f016c1fc390 2024-09-17T03:38:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c14d410 con 0x7f016c1fc390 2024-09-17T03:38:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f016c06b540 con 0x7f016c1fc390 2024-09-17T03:38:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f0158021070 con 0x7f016c1fc390 2024-09-17T03:38:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f014c075ab0 0x7f014c077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f0158094b20 con 0x7f016c1fc390 2024-09-17T03:38:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0169e00640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f014c075ab0 0x7f014c077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f014c07b9c0 0x7f014c07de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 --> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f014c07e500 con 0x7f014c07b9c0 2024-09-17T03:38:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f0158094ed0 con 0x7f016c1fc390 2024-09-17T03:38:45.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.207+0000 7f016a800640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f014c07b9c0 0x7f014c07de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.211+0000 7f0169e00640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f014c075ab0 0x7f014c077fc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c310 tx=0x7f015c019dd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.211+0000 7f016a800640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f014c07b9c0 0x7f014c07de30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.211+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== osd.3 v2:172.21.15.46:6818/2947041972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f014c07e500 con 0x7f014c07b9c0 2024-09-17T03:38:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 --> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f016c06ea60 con 0x7f014c07b9c0 2024-09-17T03:38:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0162000640 1 -- 172.21.15.46:0/2180641504 <== osd.3 v2:172.21.15.46:6818/2947041972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f016c06ea60 con 0x7f014c07b9c0 2024-09-17T03:38:45.226 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:45.227 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:45.227 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f014c07b9c0 msgr2=0x7f014c07de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6818/2947041972,v1:172.21.15.46:6819/2947041972] conn(0x7f014c07b9c0 0x7f014c07de30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f014c075ab0 msgr2=0x7f014c077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f014c075ab0 0x7f014c077fc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c310 tx=0x7f015c019dd0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f016c1fc390 msgr2=0x7f016c14c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 --2- 172.21.15.46:0/2180641504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f016c1fc390 0x7f016c14c900 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f0158011b10 tx=0x7f0158009e10 comp rx=0 tx=0).stop 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f016a800640 1 -- 172.21.15.46:0/2180641504 reap_dead start 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 shutdown_connections 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 >> 172.21.15.46:0/2180641504 conn(0x7f016c0b2c40 msgr2=0x7f016c15e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 shutdown_connections 2024-09-17T03:38:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.223+0000 7f0171000640 1 -- 172.21.15.46:0/2180641504 wait complete. 2024-09-17T03:38:45.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T03:38:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 -- 172.21.15.46:0/1105617034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3c005330 msgr2=0x7f1c3c00b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 --2- 172.21.15.46:0/1105617034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3c005330 0x7f1c3c00b9f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009b30 tx=0x7f1c5001d270 comp rx=0 tx=0).stop 2024-09-17T03:38:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 -- 172.21.15.46:0/1105617034 shutdown_connections 2024-09-17T03:38:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 --2- 172.21.15.46:0/1105617034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3c005330 0x7f1c3c00b9f0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 -- 172.21.15.46:0/1105617034 >> 172.21.15.46:0/1105617034 conn(0x7f1c580b2040 msgr2=0x7f1c580b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 -- 172.21.15.46:0/1105617034 shutdown_connections 2024-09-17T03:38:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.403+0000 7f1c5d400640 1 -- 172.21.15.46:0/1105617034 wait complete. 2024-09-17T03:38:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 Processor -- start 2024-09-17T03:38:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 -- start start 2024-09-17T03:38:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c005330 0x7f1c58152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 0x7f1c5814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c580ae8e0 0x7f1c580aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58074fb0 con 0x7f1c58154970 2024-09-17T03:38:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58074e30 con 0x7f1c3c005330 2024-09-17T03:38:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.407+0000 7f1c5d400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1c58075130 con 0x7f1c580ae8e0 2024-09-17T03:38:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 0x7f1c5814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 0x7f1c5814cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45376/0 (socket says 172.21.15.46:45376) 2024-09-17T03:38:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 -- 172.21.15.46:0/3481956835 learned_addr learned my addr 172.21.15.46:0/3481956835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c57200640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c580ae8e0 0x7f1c580aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c56800640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c005330 0x7f1c58152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 -- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c580ae8e0 msgr2=0x7f1c580aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c580ae8e0 0x7f1c580aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 -- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c005330 msgr2=0x7f1c58152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c005330 0x7f1c58152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 -- 172.21.15.46:0/3481956835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c580af230 con 0x7f1c58154970 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4de00640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 0x7f1c5814cb10 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005ea0 tx=0x7f1c40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40005320 con 0x7f1c58154970 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c40018020 con 0x7f1c58154970 2024-09-17T03:38:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40003860 con 0x7f1c58154970 2024-09-17T03:38:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c58153500 con 0x7f1c58154970 2024-09-17T03:38:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c58153ac0 con 0x7f1c58154970 2024-09-17T03:38:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c580b4f50 con 0x7f1c58154970 2024-09-17T03:38:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.411+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f1c40003a00 con 0x7f1c58154970 2024-09-17T03:38:45.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.415+0000 7f1c4f400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c34075ab0 0x7f1c34077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.415+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f1c40096d50 con 0x7f1c58154970 2024-09-17T03:38:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.415+0000 7f1c56800640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c34075ab0 0x7f1c34077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.415+0000 7f1c56800640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c34075ab0 0x7f1c34077fc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1c48001810 tx=0x7f1c480017a0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.419+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1c40022020 con 0x7f1c58154970 2024-09-17T03:38:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1c5806b540 con 0x7f1c34075ab0 2024-09-17T03:38:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c4f400640 1 -- 172.21.15.46:0/3481956835 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1c48006e10 con 0x7f1c34075ab0 2024-09-17T03:38:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c34075ab0 msgr2=0x7f1c34077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c34075ab0 0x7f1c34077fc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1c48001810 tx=0x7f1c480017a0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 msgr2=0x7f1c5814cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 0x7f1c5814cb10 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005ea0 tx=0x7f1c40005f90 comp rx=0 tx=0).stop 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 shutdown_connections 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1c34075ab0 0x7f1c34077fc0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f1c580ae8e0 0x7f1c580aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c58154970 0x7f1c5814cb10 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 --2- 172.21.15.46:0/3481956835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c3c005330 0x7f1c58152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 >> 172.21.15.46:0/3481956835 conn(0x7f1c580b2040 msgr2=0x7f1c580b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 shutdown_connections 2024-09-17T03:38:45.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.595+0000 7f1c5d400640 1 -- 172.21.15.46:0/3481956835 wait complete. 2024-09-17T03:38:45.619 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T03:38:45.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 -- 172.21.15.46:0/3689648462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e8005360 msgr2=0x7f40e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 --2- 172.21.15.46:0/3689648462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e8005360 0x7f40e800b9d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009ec0 tx=0x7f40fc01d610 comp rx=0 tx=0).stop 2024-09-17T03:38:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 -- 172.21.15.46:0/3689648462 shutdown_connections 2024-09-17T03:38:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 --2- 172.21.15.46:0/3689648462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40e8005360 0x7f40e800b9d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 -- 172.21.15.46:0/3689648462 >> 172.21.15.46:0/3689648462 conn(0x7f41040708c0 msgr2=0x7f4104070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 -- 172.21.15.46:0/3689648462 shutdown_connections 2024-09-17T03:38:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 -- 172.21.15.46:0/3689648462 wait complete. 2024-09-17T03:38:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 Processor -- start 2024-09-17T03:38:45.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.783+0000 7f410a600640 1 -- start start 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40e8005360 0x7f41040c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41041e97a0 0x7f41041471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41041f8e10 0x7f41041f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4104160ed0 con 0x7f41041e97a0 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4104160d50 con 0x7f40e8005360 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4104161050 con 0x7f41041f8e10 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41041e97a0 0x7f41041471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41041e97a0 0x7f41041471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45402/0 (socket says 172.21.15.46:45402) 2024-09-17T03:38:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 -- 172.21.15.46:0/1345788395 learned_addr learned my addr 172.21.15.46:0/1345788395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f4103400640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40e8005360 0x7f41040c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f4103e00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41041f8e10 0x7f41041f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 -- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41041f8e10 msgr2=0x7f41041f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41041f8e10 0x7f41041f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 -- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40e8005360 msgr2=0x7f41040c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40e8005360 0x7f41040c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 -- 172.21.15.46:0/1345788395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41041f9760 con 0x7f41041e97a0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f4103e00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f41041f8e10 0x7f41041f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fbe00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41041e97a0 0x7f41041471e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f40ec003f30 tx=0x7f40ec001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011640 con 0x7f41041e97a0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40ec016020 con 0x7f41041e97a0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec011a40 con 0x7f41041e97a0 2024-09-17T03:38:45.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4104147720 con 0x7f41041e97a0 2024-09-17T03:38:45.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041e7850 con 0x7f41041e97a0 2024-09-17T03:38:45.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.787+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f410406b540 con 0x7f41041e97a0 2024-09-17T03:38:45.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f40ec011be0 con 0x7f41041e97a0 2024-09-17T03:38:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f40fb400640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40e0075a80 0x7f40e0077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f40ec094920 con 0x7f41041e97a0 2024-09-17T03:38:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f4103400640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40e0075a80 0x7f40e0077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f40fb400640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7f40e007b990 0x7f40e007de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 --> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40ec013ec0 con 0x7f40e007b990 2024-09-17T03:38:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f40ec094cd0 con 0x7f41041e97a0 2024-09-17T03:38:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f4103e00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7f40e007b990 0x7f40e007de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f4103400640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40e0075a80 0x7f40e0077f90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f40f4000fc0 tx=0x7f40f40067b0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.791+0000 7f4103e00640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7f40e007b990 0x7f40e007de00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.795+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== osd.4 v2:172.21.15.99:6808/2242927697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40ec013ec0 con 0x7f40e007b990 2024-09-17T03:38:45.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 --> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41040c7990 con 0x7f40e007b990 2024-09-17T03:38:45.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f40fb400640 1 -- 172.21.15.46:0/1345788395 <== osd.4 v2:172.21.15.99:6808/2242927697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f41040c7990 con 0x7f40e007b990 2024-09-17T03:38:45.810 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:45.810 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:45.810 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7f40e007b990 msgr2=0x7f40e007de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.99:6808/2242927697,v1:172.21.15.99:6809/2242927697] conn(0x7f40e007b990 0x7f40e007de00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40e0075a80 msgr2=0x7f40e0077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f40e0075a80 0x7f40e0077f90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f40f4000fc0 tx=0x7f40f40067b0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41041e97a0 msgr2=0x7f41041471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 --2- 172.21.15.46:0/1345788395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41041e97a0 0x7f41041471e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f40ec003f30 tx=0x7f40ec001700 comp rx=0 tx=0).stop 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f4103e00640 1 -- 172.21.15.46:0/1345788395 reap_dead start 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 shutdown_connections 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 >> 172.21.15.46:0/1345788395 conn(0x7f41040708c0 msgr2=0x7f41040aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 shutdown_connections 2024-09-17T03:38:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.807+0000 7f410a600640 1 -- 172.21.15.46:0/1345788395 wait complete. 2024-09-17T03:38:45.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 -- 172.21.15.46:0/602164671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288003c40 msgr2=0x7fb288004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 --2- 172.21.15.46:0/602164671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288003c40 0x7fb288004090 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb294006f80 tx=0x7fb29401ce90 comp rx=0 tx=0).stop 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 -- 172.21.15.46:0/602164671 shutdown_connections 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 --2- 172.21.15.46:0/602164671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288003c40 0x7fb288004090 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 -- 172.21.15.46:0/602164671 >> 172.21.15.46:0/602164671 conn(0x7fb2a40b2040 msgr2=0x7fb2a40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 -- 172.21.15.46:0/602164671 shutdown_connections 2024-09-17T03:38:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.987+0000 7fb2ab200640 1 -- 172.21.15.46:0/602164671 wait complete. 2024-09-17T03:38:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 Processor -- start 2024-09-17T03:38:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 -- start start 2024-09-17T03:38:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb288003c40 0x7fb2a414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2a4154120 0x7fb2a4154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 0x7fb2a40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a415d180 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a415d000 con 0x7fb288003c40 2024-09-17T03:38:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2ab200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a415d300 con 0x7fb2a4154120 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 0x7fb2a40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 0x7fb2a40aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45422/0 (socket says 172.21.15.46:45422) 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 -- 172.21.15.46:0/1186470920 learned_addr learned my addr 172.21.15.46:0/1186470920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a3e00640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb288003c40 0x7fb2a414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a3400640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2a4154120 0x7fb2a4154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 -- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2a4154120 msgr2=0x7fb2a4154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2a4154120 0x7fb2a4154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 -- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb288003c40 msgr2=0x7fb2a414add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb288003c40 0x7fb2a414add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 -- 172.21.15.46:0/1186470920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a40af230 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a3400640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2a4154120 0x7fb2a4154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:38:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.991+0000 7fb2a9200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 0x7fb2a40aecf0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb2880057f0 tx=0x7fb29c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00eea0 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb29c015070 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c004030 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41fbf20 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a41fc4e0 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a415f180 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fb29c0041d0 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb29be00640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb284075a80 0x7fb284077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fb29c0941e0 con 0x7fb2a40ae8e0 2024-09-17T03:38:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.995+0000 7fb2a3e00640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb284075a80 0x7fb284077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.999+0000 7fb2a3e00640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb284075a80 0x7fb284077f90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb2940ab000 tx=0x7fb294019cd0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:45.999+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb2a40af230 con 0x7fb2a40ae8e0 2024-09-17T03:38:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.175+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fb2a415b4f0 con 0x7fb284075a80 2024-09-17T03:38:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.175+0000 7fb29be00640 1 -- 172.21.15.46:0/1186470920 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb2a415b4f0 con 0x7fb284075a80 2024-09-17T03:38:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb284075a80 msgr2=0x7fb284077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb284075a80 0x7fb284077f90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb2940ab000 tx=0x7fb294019cd0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 msgr2=0x7fb2a40aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 0x7fb2a40aecf0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb2880057f0 tx=0x7fb29c00e720 comp rx=0 tx=0).stop 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 shutdown_connections 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fb284075a80 0x7fb284077f90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a40ae8e0 0x7fb2a40aecf0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fb2a4154120 0x7fb2a4154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 --2- 172.21.15.46:0/1186470920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb288003c40 0x7fb2a414add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 >> 172.21.15.46:0/1186470920 conn(0x7fb2a40b2040 msgr2=0x7fb2a415ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 shutdown_connections 2024-09-17T03:38:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.179+0000 7fb2ab200640 1 -- 172.21.15.46:0/1186470920 wait complete. 2024-09-17T03:38:46.201 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T03:38:46.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 -- 172.21.15.46:0/3815718442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc1527b0 msgr2=0x7f04bc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 --2- 172.21.15.46:0/3815718442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc1527b0 0x7f04bc15bb60 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009900 tx=0x7f04a401c8b0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 -- 172.21.15.46:0/3815718442 shutdown_connections 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 --2- 172.21.15.46:0/3815718442 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04bc0b4de0 0x7f04bc0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 --2- 172.21.15.46:0/3815718442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc1527b0 0x7f04bc15bb60 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 -- 172.21.15.46:0/3815718442 >> 172.21.15.46:0/3815718442 conn(0x7f04bc0b2040 msgr2=0x7f04bc0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 -- 172.21.15.46:0/3815718442 shutdown_connections 2024-09-17T03:38:46.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.363+0000 7f04c2c00640 1 -- 172.21.15.46:0/3815718442 wait complete. 2024-09-17T03:38:46.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 Processor -- start 2024-09-17T03:38:46.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 -- start start 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04bc0b4de0 0x7f04bc1fca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc0b5c70 0x7f04bc158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04bc1527b0 0x7f04bc1f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc168950 con 0x7f04bc0b5c70 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1687d0 con 0x7f04bc1527b0 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc168ad0 con 0x7f04bc0b4de0 2024-09-17T03:38:46.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc0b5c70 0x7f04bc158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc0b5c70 0x7f04bc158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45432/0 (socket says 172.21.15.46:45432) 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 -- 172.21.15.46:0/3138078478 learned_addr learned my addr 172.21.15.46:0/3138078478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c0c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04bc1527b0 0x7f04bc1f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04bbe00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04bc0b4de0 0x7f04bc1fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 -- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04bc0b4de0 msgr2=0x7f04bc1fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f04bc0b4de0 0x7f04bc1fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 -- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04bc1527b0 msgr2=0x7f04bc1f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04bc1527b0 0x7f04bc1f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 -- 172.21.15.46:0/3138078478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc141c40 con 0x7f04bc0b5c70 2024-09-17T03:38:46.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b3400640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc0b5c70 0x7f04bc158420 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009850 tx=0x7f04a4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a40bc070 con 0x7f04bc0b5c70 2024-09-17T03:38:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04a4005310 con 0x7f04bc0b5c70 2024-09-17T03:38:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a40b2040 con 0x7f04bc0b5c70 2024-09-17T03:38:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc141f20 con 0x7f04bc0b5c70 2024-09-17T03:38:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.367+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1424e0 con 0x7f04bc0b5c70 2024-09-17T03:38:46.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04bc06b540 con 0x7f04bc0b5c70 2024-09-17T03:38:46.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f04a40046d0 con 0x7f04bc0b5c70 2024-09-17T03:38:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04b8c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0498075a80 0x7f0498077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f04a412dee0 con 0x7f04bc0b5c70 2024-09-17T03:38:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04bbe00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0498075a80 0x7f0498077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04b8c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f049807b990 0x7f049807de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 --> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04a40a7ea0 con 0x7f049807b990 2024-09-17T03:38:46.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f04a412e660 con 0x7f04bc0b5c70 2024-09-17T03:38:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04c0c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f049807b990 0x7f049807de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04bbe00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0498075a80 0x7f0498077f90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001650 tx=0x7f04ac009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.371+0000 7f04c0c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f049807b990 0x7f049807de00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.375+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== osd.5 v2:172.21.15.99:6800/4253831876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04a40a7ea0 con 0x7f049807b990 2024-09-17T03:38:46.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 --> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f04bc1419b0 con 0x7f049807b990 2024-09-17T03:38:46.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04b8c00640 1 -- 172.21.15.46:0/3138078478 <== osd.5 v2:172.21.15.99:6800/4253831876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f04bc1419b0 con 0x7f049807b990 2024-09-17T03:38:46.391 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:46.391 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:46.392 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f049807b990 msgr2=0x7f049807de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.99:6800/4253831876,v1:172.21.15.99:6801/4253831876] conn(0x7f049807b990 0x7f049807de00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0498075a80 msgr2=0x7f0498077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f0498075a80 0x7f0498077f90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001650 tx=0x7f04ac009040 comp rx=0 tx=0).stop 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc0b5c70 msgr2=0x7f04bc158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 --2- 172.21.15.46:0/3138078478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04bc0b5c70 0x7f04bc158420 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009850 tx=0x7f04a4005e00 comp rx=0 tx=0).stop 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c0c00640 1 -- 172.21.15.46:0/3138078478 reap_dead start 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 shutdown_connections 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 >> 172.21.15.46:0/3138078478 conn(0x7f04bc0b2040 msgr2=0x7f04bc0b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.387+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 shutdown_connections 2024-09-17T03:38:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.391+0000 7f04c2c00640 1 -- 172.21.15.46:0/3138078478 wait complete. 2024-09-17T03:38:46.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 -- 172.21.15.46:0/3202856970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80b4de0 msgr2=0x7f7db80b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 --2- 172.21.15.46:0/3202856970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80b4de0 0x7f7db80b51f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7db8071cb0 tx=0x7f7db001cae0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 -- 172.21.15.46:0/3202856970 shutdown_connections 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 --2- 172.21.15.46:0/3202856970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db80b4de0 0x7f7db80b51f0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 --2- 172.21.15.46:0/3202856970 >> v2:172.21.15.99:3300/0 conn(0x7f7db80b5c70 0x7f7db815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 -- 172.21.15.46:0/3202856970 >> 172.21.15.46:0/3202856970 conn(0x7f7db80b2040 msgr2=0x7f7db80b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 -- 172.21.15.46:0/3202856970 shutdown_connections 2024-09-17T03:38:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.567+0000 7f7dc0600640 1 -- 172.21.15.46:0/3202856970 wait complete. 2024-09-17T03:38:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 Processor -- start 2024-09-17T03:38:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 -- start start 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7db80b4de0 0x7f7db81f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7db80b5c70 0x7f7db8158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 0x7f7db814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8168950 con 0x7f7db81527b0 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7db81687d0 con 0x7f7db80b5c70 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dc0600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f7db8168ad0 con 0x7f7db80b4de0 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 0x7f7db814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 0x7f7db814bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45450/0 (socket says 172.21.15.46:45450) 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 -- 172.21.15.46:0/965936700 learned_addr learned my addr 172.21.15.46:0/965936700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbdc00640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7db80b4de0 0x7f7db81f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbd200640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7db80b5c70 0x7f7db8158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 -- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7db80b4de0 msgr2=0x7f7db81f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7db80b4de0 0x7f7db81f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 -- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7db80b5c70 msgr2=0x7f7db8158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7db80b5c70 0x7f7db8158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 -- 172.21.15.46:0/965936700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db8157cc0 con 0x7f7db81527b0 2024-09-17T03:38:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbd200640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7db80b5c70 0x7f7db8158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.571+0000 7f7dbe600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 0x7f7db814bfb0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009bd0 tx=0x7f7db0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00bc070 con 0x7f7db81527b0 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db00042f0 con 0x7f7db81527b0 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00b2040 con 0x7f7db81527b0 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db8157fa0 con 0x7f7db81527b0 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db8158560 con 0x7f7db81527b0 2024-09-17T03:38:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db80b51f0 con 0x7f7db81527b0 2024-09-17T03:38:46.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f7db0004720 con 0x7f7db81527b0 2024-09-17T03:38:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7db5600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d98075a80 0x7f7d98077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.575+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f7db012d720 con 0x7f7db81527b0 2024-09-17T03:38:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.579+0000 7f7dbdc00640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d98075a80 0x7f7d98077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.579+0000 7f7dbdc00640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d98075a80 0x7f7d98077f90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00ab40 tx=0x7f7dac00a250 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.579+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7db00c5080 con 0x7f7db81527b0 2024-09-17T03:38:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.755+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f7db806b540 con 0x7f7d98075a80 2024-09-17T03:38:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.755+0000 7f7db5600640 1 -- 172.21.15.46:0/965936700 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7dac00fb60 con 0x7f7d98075a80 2024-09-17T03:38:46.759 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T03:38:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d98075a80 msgr2=0x7f7d98077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d98075a80 0x7f7d98077f90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00ab40 tx=0x7f7dac00a250 comp rx=0 tx=0).stop 2024-09-17T03:38:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 msgr2=0x7f7db814bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 0x7f7db814bfb0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009bd0 tx=0x7f7db0003f80 comp rx=0 tx=0).stop 2024-09-17T03:38:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 shutdown_connections 2024-09-17T03:38:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f7d98075a80 0x7f7d98077f90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db81527b0 0x7f7db814bfb0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7db80b5c70 0x7f7db8158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 --2- 172.21.15.46:0/965936700 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f7db80b4de0 0x7f7db81f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 >> 172.21.15.46:0/965936700 conn(0x7f7db80b2040 msgr2=0x7f7db815a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 shutdown_connections 2024-09-17T03:38:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.759+0000 7f7dc0600640 1 -- 172.21.15.46:0/965936700 wait complete. 2024-09-17T03:38:46.781 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T03:38:46.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 -- 172.21.15.46:0/3122360411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11501527b0 msgr2=0x7f115015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 --2- 172.21.15.46:0/3122360411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11501527b0 0x7f115015bb60 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f11500cca90 tx=0x7f113801c800 comp rx=0 tx=0).stop 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 -- 172.21.15.46:0/3122360411 shutdown_connections 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 --2- 172.21.15.46:0/3122360411 >> v2:172.21.15.99:3300/0 conn(0x7f11500b4de0 0x7f11500b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 --2- 172.21.15.46:0/3122360411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11501527b0 0x7f115015bb60 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 -- 172.21.15.46:0/3122360411 >> 172.21.15.46:0/3122360411 conn(0x7f11500b2040 msgr2=0x7f11500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 -- 172.21.15.46:0/3122360411 shutdown_connections 2024-09-17T03:38:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 -- 172.21.15.46:0/3122360411 wait complete. 2024-09-17T03:38:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 Processor -- start 2024-09-17T03:38:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.943+0000 7f1155400640 1 -- start start 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11500b4de0 0x7f11501fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11500b5c70 0x7f11501f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11501527b0 0x7f1150153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1150168920 con 0x7f11500b5c70 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f11501687a0 con 0x7f11500b4de0 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1150168aa0 con 0x7f11501527b0 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11500b5c70 0x7f11501f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11500b5c70 0x7f11501f9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45464/0 (socket says 172.21.15.46:45464) 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 -- 172.21.15.46:0/2257032866 learned_addr learned my addr 172.21.15.46:0/2257032866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f114e800640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11500b4de0 0x7f11501fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f114f200640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11501527b0 0x7f1150153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 -- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11501527b0 msgr2=0x7f1150153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f11501527b0 0x7f1150153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 -- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11500b4de0 msgr2=0x7f11501fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11500b4de0 0x7f11501fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 -- 172.21.15.46:0/2257032866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11501fc770 con 0x7f11500b5c70 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1147e00640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11500b5c70 0x7f11501f9240 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f11380b0040 tx=0x7f11380a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11380b2040 con 0x7f11500b5c70 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11380044b0 con 0x7f11500b5c70 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138005280 con 0x7f11500b5c70 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11501fca50 con 0x7f11500b5c70 2024-09-17T03:38:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11501f8a40 con 0x7f11500b5c70 2024-09-17T03:38:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.947+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f115006b540 con 0x7f11500b5c70 2024-09-17T03:38:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f1138005420 con 0x7f11500b5c70 2024-09-17T03:38:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f1146a00640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f112c075a80 0x7f112c077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f113812d390 con 0x7f11500b5c70 2024-09-17T03:38:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f114e800640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f112c075a80 0x7f112c077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f1146a00640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f112c07b990 0x7f112c07de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 --> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1138004e70 con 0x7f112c07b990 2024-09-17T03:38:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f113812db10 con 0x7f11500b5c70 2024-09-17T03:38:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f114f200640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f112c07b990 0x7f112c07de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f114e800640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f112c075a80 0x7f112c077f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1140001650 tx=0x7f1140009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.951+0000 7f114f200640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f112c07b990 0x7f112c07de00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.955+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== osd.6 v2:172.21.15.99:6816/788789999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1138004e70 con 0x7f112c07b990 2024-09-17T03:38:46.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 --> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f11501fc520 con 0x7f112c07b990 2024-09-17T03:38:46.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1146a00640 1 -- 172.21.15.46:0/2257032866 <== osd.6 v2:172.21.15.99:6816/788789999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f11501fc520 con 0x7f112c07b990 2024-09-17T03:38:46.970 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:46.970 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:46.970 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f112c07b990 msgr2=0x7f112c07de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.99:6816/788789999,v1:172.21.15.99:6817/788789999] conn(0x7f112c07b990 0x7f112c07de00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f112c075a80 msgr2=0x7f112c077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f112c075a80 0x7f112c077f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1140001650 tx=0x7f1140009040 comp rx=0 tx=0).stop 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11500b5c70 msgr2=0x7f11501f9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 --2- 172.21.15.46:0/2257032866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11500b5c70 0x7f11501f9240 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f11380b0040 tx=0x7f11380a5820 comp rx=0 tx=0).stop 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f114f200640 1 -- 172.21.15.46:0/2257032866 reap_dead start 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 shutdown_connections 2024-09-17T03:38:46.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 >> 172.21.15.46:0/2257032866 conn(0x7f11500b2040 msgr2=0x7f115015a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 shutdown_connections 2024-09-17T03:38:46.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:46.967+0000 7f1155400640 1 -- 172.21.15.46:0/2257032866 wait complete. 2024-09-17T03:38:46.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T03:38:47.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 -- 172.21.15.46:0/2708794630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c0b4de0 msgr2=0x7fc56c0b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 --2- 172.21.15.46:0/2708794630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c0b4de0 0x7fc56c0b51f0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc56c070710 tx=0x7fc56401ccc0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 -- 172.21.15.46:0/2708794630 shutdown_connections 2024-09-17T03:38:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 --2- 172.21.15.46:0/2708794630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c0b4de0 0x7fc56c0b51f0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 --2- 172.21.15.46:0/2708794630 >> v2:172.21.15.99:3300/0 conn(0x7fc56c1527b0 0x7fc56c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 -- 172.21.15.46:0/2708794630 >> 172.21.15.46:0/2708794630 conn(0x7fc56c0b2040 msgr2=0x7fc56c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 -- 172.21.15.46:0/2708794630 shutdown_connections 2024-09-17T03:38:47.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.147+0000 7fc572c00640 1 -- 172.21.15.46:0/2708794630 wait complete. 2024-09-17T03:38:47.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 Processor -- start 2024-09-17T03:38:47.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 -- start start 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c0b4de0 0x7fc56c155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc56c0b5c70 0x7fc56c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 0x7fc56c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c168920 con 0x7fc56c1527b0 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1687a0 con 0x7fc56c0b4de0 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7fc56c168aa0 con 0x7fc56c0b5c70 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 0x7fc56c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 0x7fc56c158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45480/0 (socket says 172.21.15.46:45480) 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 -- 172.21.15.46:0/1347962364 learned_addr learned my addr 172.21.15.46:0/1347962364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc563e00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc56c0b5c70 0x7fc56c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc56be00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c0b4de0 0x7fc56c155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 -- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc56c0b5c70 msgr2=0x7fc56c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc56c0b5c70 0x7fc56c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 -- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c0b4de0 msgr2=0x7fc56c155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c0b4de0 0x7fc56c155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 -- 172.21.15.46:0/1347962364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c142480 con 0x7fc56c1527b0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc563e00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc56c0b5c70 0x7fc56c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc570c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 0x7fc56c158710 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc564009bd0 tx=0x7fc564005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640bb070 con 0x7fc56c1527b0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc564004d40 con 0x7fc56c1527b0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640b2040 con 0x7fc56c1527b0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56c1416c0 con 0x7fc56c1527b0 2024-09-17T03:38:47.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.151+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56c141c80 con 0x7fc56c1527b0 2024-09-17T03:38:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.155+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56c0b51f0 con 0x7fc56c1527b0 2024-09-17T03:38:47.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.155+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7fc5640be070 con 0x7fc56c1527b0 2024-09-17T03:38:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.155+0000 7fc568c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc548075ab0 0x7fc548077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.155+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fc56412da80 con 0x7fc56c1527b0 2024-09-17T03:38:47.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.155+0000 7fc56be00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc548075ab0 0x7fc548077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.155+0000 7fc56be00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc548075ab0 0x7fc548077fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc55c001650 tx=0x7fc55c009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.159+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5640c5020 con 0x7fc56c1527b0 2024-09-17T03:38:47.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fc56c06b540 con 0x7fc548075ab0 2024-09-17T03:38:47.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc568c00640 1 -- 172.21.15.46:0/1347962364 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc55c0050d0 con 0x7fc548075ab0 2024-09-17T03:38:47.340 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc548075ab0 msgr2=0x7fc548077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc548075ab0 0x7fc548077fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc55c001650 tx=0x7fc55c009040 comp rx=0 tx=0).stop 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 msgr2=0x7fc56c158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 0x7fc56c158710 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc564009bd0 tx=0x7fc564005b00 comp rx=0 tx=0).stop 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 shutdown_connections 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7fc548075ab0 0x7fc548077fc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc56c1527b0 0x7fc56c158710 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7fc56c0b5c70 0x7fc56c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 --2- 172.21.15.46:0/1347962364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc56c0b4de0 0x7fc56c155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 >> 172.21.15.46:0/1347962364 conn(0x7fc56c0b2040 msgr2=0x7fc56c15a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:47.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 shutdown_connections 2024-09-17T03:38:47.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.339+0000 7fc572c00640 1 -- 172.21.15.46:0/1347962364 wait complete. 2024-09-17T03:38:47.363 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T03:38:47.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 -- 172.21.15.46:0/4077296961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b4de0 msgr2=0x7f57380b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 --2- 172.21.15.46:0/4077296961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b4de0 0x7f57380b51f0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f5730009c80 tx=0x7f573001cce0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 -- 172.21.15.46:0/4077296961 shutdown_connections 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 --2- 172.21.15.46:0/4077296961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b4de0 0x7f57380b51f0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 --2- 172.21.15.46:0/4077296961 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57380b5c70 0x7f573815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 -- 172.21.15.46:0/4077296961 >> 172.21.15.46:0/4077296961 conn(0x7f57380b2040 msgr2=0x7f57380b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 -- 172.21.15.46:0/4077296961 shutdown_connections 2024-09-17T03:38:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.523+0000 7f5740600640 1 -- 172.21.15.46:0/4077296961 wait complete. 2024-09-17T03:38:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 Processor -- start 2024-09-17T03:38:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 -- start start 2024-09-17T03:38:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57380b4de0 0x7f5738151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b5c70 0x7f57381fc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57381527b0 0x7f57381f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5738168ab0 con 0x7f57380b5c70 2024-09-17T03:38:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5738168930 con 0x7f57380b4de0 2024-09-17T03:38:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f5740600640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f5738168c30 con 0x7f57381527b0 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b5c70 0x7f57381fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b5c70 0x7f57381fc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45486/0 (socket says 172.21.15.46:45486) 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 -- 172.21.15.46:0/2032368652 learned_addr learned my addr 172.21.15.46:0/2032368652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573dc00640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57380b4de0 0x7f5738151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573e600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57381527b0 0x7f57381f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 -- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57381527b0 msgr2=0x7f57381f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f57381527b0 0x7f57381f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 -- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57380b4de0 msgr2=0x7f5738151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57380b4de0 0x7f5738151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.527+0000 7f573d200640 1 -- 172.21.15.46:0/2032368652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5738161dc0 con 0x7f57380b5c70 2024-09-17T03:38:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f573d200640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b5c70 0x7f57381fc810 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f572400c390 tx=0x7f572400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724017070 con 0x7f57380b5c70 2024-09-17T03:38:47.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57240053c0 con 0x7f57380b5c70 2024-09-17T03:38:47.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57381620a0 con 0x7f57380b5c70 2024-09-17T03:38:47.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724003840 con 0x7f57380b5c70 2024-09-17T03:38:47.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5738160c50 con 0x7f57380b5c70 2024-09-17T03:38:47.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57381f9760 con 0x7f57380b5c70 2024-09-17T03:38:47.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f572401a070 con 0x7f57380b5c70 2024-09-17T03:38:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.531+0000 7f5735600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5718075ab0 0x7f5718077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f5724092cf0 con 0x7f57380b5c70 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f573dc00640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5718075ab0 0x7f5718077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f5735600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f571807b9c0 0x7f571807de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 --> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f571807e500 con 0x7f571807b9c0 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=149) ==== 24+0+0 (secure 0 0 0) 0x7f57240930a0 con 0x7f57380b5c70 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f573e600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f571807b9c0 0x7f571807de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f573dc00640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5718075ab0 0x7f5718077fc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f572c004630 tx=0x7f572c00b2e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f573e600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f571807b9c0 0x7f571807de30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.535+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== osd.7 v2:172.21.15.99:6824/3170570057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f571807e500 con 0x7f571807b9c0 2024-09-17T03:38:47.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 --> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5738161b70 con 0x7f571807b9c0 2024-09-17T03:38:47.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5735600640 1 -- 172.21.15.46:0/2032368652 <== osd.7 v2:172.21.15.99:6824/3170570057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5738161b70 con 0x7f571807b9c0 2024-09-17T03:38:47.552 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-17T03:38:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:38:47.553 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-17T03:38:47.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f571807b9c0 msgr2=0x7f571807de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.99:6824/3170570057,v1:172.21.15.99:6825/3170570057] conn(0x7f571807b9c0 0x7f571807de30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5718075ab0 msgr2=0x7f5718077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f5718075ab0 0x7f5718077fc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f572c004630 tx=0x7f572c00b2e0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b5c70 msgr2=0x7f57381fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 --2- 172.21.15.46:0/2032368652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57380b5c70 0x7f57381fc810 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f572400c390 tx=0x7f572400c850 comp rx=0 tx=0).stop 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f573e600640 1 -- 172.21.15.46:0/2032368652 reap_dead start 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 shutdown_connections 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 >> 172.21.15.46:0/2032368652 conn(0x7f57380b2040 msgr2=0x7f57380b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 shutdown_connections 2024-09-17T03:38:47.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.551+0000 7f5740600640 1 -- 172.21.15.46:0/2032368652 wait complete. 2024-09-17T03:38:47.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T03:38:47.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 -- 172.21.15.46:0/3199109432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 msgr2=0x7f2bb415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/3199109432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb415bb60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4074b20 tx=0x7f2b9c01c820 comp rx=0 tx=0).stop 2024-09-17T03:38:47.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 -- 172.21.15.46:0/3199109432 shutdown_connections 2024-09-17T03:38:47.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/3199109432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb415bb60 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 -- 172.21.15.46:0/3199109432 >> 172.21.15.46:0/3199109432 conn(0x7f2bb40b2040 msgr2=0x7f2bb40b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:47.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 -- 172.21.15.46:0/3199109432 shutdown_connections 2024-09-17T03:38:47.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 -- 172.21.15.46:0/3199109432 wait complete. 2024-09-17T03:38:47.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 Processor -- start 2024-09-17T03:38:47.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.731+0000 7f2bb8e00640 1 -- start start 2024-09-17T03:38:47.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bb40b4de0 0x7f2bb41444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb40b5c70 0x7f2bb41fa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb4158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4169bd0 con 0x7f2bb41527b0 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4169a50 con 0x7f2bb40b5c70 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb4169d50 con 0x7f2bb40b4de0 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb4158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb4158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45506/0 (socket says 172.21.15.46:45506) 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 -- 172.21.15.46:0/1915657008 learned_addr learned my addr 172.21.15.46:0/1915657008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2babe00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb40b5c70 0x7f2bb41fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb1e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bb40b4de0 0x7f2bb41444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 -- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bb40b4de0 msgr2=0x7f2bb41444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bb40b4de0 0x7f2bb41444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 -- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb40b5c70 msgr2=0x7f2bb41fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb40b5c70 0x7f2bb41fa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 -- 172.21.15.46:0/1915657008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb40cf8b0 con 0x7f2bb41527b0 2024-09-17T03:38:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb2800640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb4158c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c760 tx=0x7f2bac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac003b80 con 0x7f2bb41527b0 2024-09-17T03:38:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bac003d20 con 0x7f2bb41527b0 2024-09-17T03:38:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb41523b0 con 0x7f2bb41527b0 2024-09-17T03:38:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00aab0 con 0x7f2bb41527b0 2024-09-17T03:38:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb40cd690 con 0x7f2bb41527b0 2024-09-17T03:38:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.735+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb40cd120 con 0x7f2bb41527b0 2024-09-17T03:38:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.739+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f2bac00ac50 con 0x7f2bb41527b0 2024-09-17T03:38:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.739+0000 7f2baa000640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2b90075ab0 0x7f2b90077fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.739+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f2bac093b40 con 0x7f2bb41527b0 2024-09-17T03:38:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.739+0000 7f2bb1e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2b90075ab0 0x7f2b90077fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.739+0000 7f2bb1e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2b90075ab0 0x7f2b90077fc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007fd0 tx=0x7f2ba4009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:47.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.739+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2bac060180 con 0x7f2bb41527b0 2024-09-17T03:38:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.919+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f2bb406b540 con 0x7f2b90075ab0 2024-09-17T03:38:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.919+0000 7f2baa000640 1 -- 172.21.15.46:0/1915657008 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2ba4001490 con 0x7f2b90075ab0 2024-09-17T03:38:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2b90075ab0 msgr2=0x7f2b90077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2b90075ab0 0x7f2b90077fc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007fd0 tx=0x7f2ba4009040 comp rx=0 tx=0).stop 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 msgr2=0x7f2bb4158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb4158c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c760 tx=0x7f2bac00cc20 comp rx=0 tx=0).stop 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 shutdown_connections 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f2b90075ab0 0x7f2b90077fc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb41527b0 0x7f2bb4158c80 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb40b5c70 0x7f2bb41fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 --2- 172.21.15.46:0/1915657008 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f2bb40b4de0 0x7f2bb41444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 >> 172.21.15.46:0/1915657008 conn(0x7f2bb40b2040 msgr2=0x7f2bb4154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 shutdown_connections 2024-09-17T03:38:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:47.923+0000 7f2bb8e00640 1 -- 172.21.15.46:0/1915657008 wait complete. 2024-09-17T03:38:47.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:38:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 -- 172.21.15.46:0/3735214035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 msgr2=0x7f4e5c070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 --2- 172.21.15.46:0/3735214035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c070e80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c070e00 tx=0x7f4e5001c8c0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 -- 172.21.15.46:0/3735214035 shutdown_connections 2024-09-17T03:38:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 --2- 172.21.15.46:0/3735214035 >> v2:172.21.15.99:3300/0 conn(0x7f4e5c0cf1b0 0x7f4e5c0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 --2- 172.21.15.46:0/3735214035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c070e80 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 -- 172.21.15.46:0/3735214035 >> 172.21.15.46:0/3735214035 conn(0x7f4e5c0b2c40 msgr2=0x7f4e5c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 -- 172.21.15.46:0/3735214035 shutdown_connections 2024-09-17T03:38:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 -- 172.21.15.46:0/3735214035 wait complete. 2024-09-17T03:38:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 Processor -- start 2024-09-17T03:38:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.103+0000 7f4e63a00640 1 -- start start 2024-09-17T03:38:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e5c0cf1b0 0x7f4e5c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e5c1602a0 0x7f4e5c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c169e30 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c169cb0 con 0x7f4e5c1602a0 2024-09-17T03:38:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5c169fb0 con 0x7f4e5c0cf1b0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c13f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45524/0 (socket says 172.21.15.46:45524) 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 -- 172.21.15.46:0/2874927727 learned_addr learned my addr 172.21.15.46:0/2874927727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e5be00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e5c0cf1b0 0x7f4e5c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e5c1602a0 0x7f4e5c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 -- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e5c0cf1b0 msgr2=0x7f4e5c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e5c0cf1b0 0x7f4e5c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 -- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e5c1602a0 msgr2=0x7f4e5c1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e5c1602a0 0x7f4e5c1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 -- 172.21.15.46:0/2874927727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5c158b30 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e61000640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c13f1c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4e500194d0 tx=0x7f4e500a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e500b2040 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e500042c0 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50005290 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5c158e10 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5c13f700 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.107+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5c070e80 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.111+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f4e50005430 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.111+0000 7f4e58c00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4e3c075a80 0x7f4e3c077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:38:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.111+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f4e5012d590 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.111+0000 7f4e5be00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4e3c075a80 0x7f4e3c077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:38:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.111+0000 7f4e5be00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4e3c075a80 0x7f4e3c077f90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c1f9d90 tx=0x7f4e48009040 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:38:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.115+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4e500c4060 con 0x7f4e5c0cd2e0 2024-09-17T03:38:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e5c06b540 con 0x7f4e3c075a80 2024-09-17T03:38:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e58c00640 1 -- 172.21.15.46:0/2874927727 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13384 (secure 0 0 0) 0x7f4e480050d0 con 0x7f4e3c075a80 2024-09-17T03:38:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:38:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:38:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4e3c075a80 msgr2=0x7f4e3c077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4e3c075a80 0x7f4e3c077f90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c1f9d90 tx=0x7f4e48009040 comp rx=0 tx=0).stop 2024-09-17T03:38:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 msgr2=0x7f4e5c13f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:38:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c13f1c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4e500194d0 tx=0x7f4e500a56f0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 shutdown_connections 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4e3c075a80 0x7f4e3c077f90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e5c1602a0 0x7f4e5c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4e5c0cf1b0 0x7f4e5c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 --2- 172.21.15.46:0/2874927727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e5c0cd2e0 0x7f4e5c13f1c0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 >> 172.21.15.46:0/2874927727 conn(0x7f4e5c0b2c40 msgr2=0x7f4e5c15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 shutdown_connections 2024-09-17T03:38:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:38:48.287+0000 7f4e63a00640 1 -- 172.21.15.46:0/2874927727 wait complete. 2024-09-17T03:38:48.311 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":678,"stamp":"2024-09-17T03:38:47.637944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1207276,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":1191347,"kb_avail":753767444,"statfs":{"total":773094113280,"available":771857862656,"internally_reserved":0,"allocated":16097280,"data_stored":7729320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1219939888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.134934"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":347,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T03:38:37.577068+0000","last_change":"2024-09-17T03:20:57.214498+0000","last_active":"2024-09-17T03:38:37.577068+0000","last_peered":"2024-09-17T03:38:37.577068+0000","last_clean":"2024-09-17T03:38:37.577068+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:38:37.577068+0000","last_undegraded":"2024-09-17T03:38:37.577068+0000","last_fullsized":"2024-09-17T03:38:37.577068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:20:55.951112+0000","last_clean_scrub_stamp":"2024-09-17T03:20:55.951112+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:01.342163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509657,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150136,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221704,"statfs":{"total":96636764160,"available":96483024896,"internally_reserved":0,"allocated":2457600,"data_stored":1408941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":151255498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509657,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159220,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157374,"kb_avail":94212620,"statfs":{"total":96636764160,"available":96473722880,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161151430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509657,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155128,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216712,"statfs":{"total":96636764160,"available":96477913088,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137076,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234764,"statfs":{"total":96636764160,"available":96496398336,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509657,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159284,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212556,"statfs":{"total":96636764160,"available":96473657344,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509657,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141816,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230024,"statfs":{"total":96636764160,"available":96491544576,"internally_reserved":0,"allocated":2457600,"data_stored":1408941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542361,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123188,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248652,"statfs":{"total":96636764160,"available":96510619648,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542361,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181428,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":179582,"kb_avail":94190412,"statfs":{"total":96636764160,"available":96450981888,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183892421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:38:48.314 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T03:20:55.951112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=3, tm_min=20, tm_sec=55, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=3, tm_min=38, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T03:38:48.315 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T03:39:01.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:01.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:08.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f4556200640 1 -- 172.21.15.46:0/3552157456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4534003a10 msgr2=0x7f4534003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f4556200640 1 --2- 172.21.15.46:0/3552157456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4534003a10 0x7f4534003e80 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f4548009ec0 tx=0x7f454801d5f0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f454fe00640 1 -- 172.21.15.46:0/3552157456 reap_dead start 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f4556200640 1 -- 172.21.15.46:0/3552157456 shutdown_connections 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f4556200640 1 -- 172.21.15.46:0/3552157456 >> 172.21.15.46:0/3552157456 conn(0x7f45500b2040 msgr2=0x7f45500b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f4556200640 1 -- 172.21.15.46:0/3552157456 shutdown_connections 2024-09-17T03:39:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.487+0000 7f4556200640 1 -- 172.21.15.46:0/3552157456 wait complete. 2024-09-17T03:39:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 Processor -- start 2024-09-17T03:39:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 -- start start 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4534003a10 0x7f45501f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455013db60 0x7f4550143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 0x7f4550143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4550140710 con 0x7f4550143740 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4550140590 con 0x7f455013db60 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f4556200640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f4550140890 con 0x7f4534003a10 2024-09-17T03:39:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 0x7f4550143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 0x7f4550143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39934/0 (socket says 172.21.15.46:39934) 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 -- 172.21.15.46:0/1635423910 learned_addr learned my addr 172.21.15.46:0/1635423910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454ea00640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455013db60 0x7f4550143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454f400640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4534003a10 0x7f45501f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 -- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4534003a10 msgr2=0x7f45501f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4534003a10 0x7f45501f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 -- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455013db60 msgr2=0x7f4550143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455013db60 0x7f4550143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 -- 172.21.15.46:0/1635423910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4550142270 con 0x7f4550143740 2024-09-17T03:39:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.491+0000 7f454fe00640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 0x7f4550143b50 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f45500cd150 tx=0x7f453c00d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:39:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c00dd50 con 0x7f4550143740 2024-09-17T03:39:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f453c004030 con 0x7f4550143740 2024-09-17T03:39:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c004480 con 0x7f4550143740 2024-09-17T03:39:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4550142550 con 0x7f4550143740 2024-09-17T03:39:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4550142b10 con 0x7f4550143740 2024-09-17T03:39:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4550140e20 con 0x7f4550143740 2024-09-17T03:39:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f453c003600 con 0x7f4550143740 2024-09-17T03:39:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4547400640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4530079ec0 0x7f453007c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f453c093ac0 con 0x7f4550143740 2024-09-17T03:39:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.495+0000 7f454f400640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4530079ec0 0x7f453007c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.499+0000 7f454f400640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4530079ec0 0x7f453007c3d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f45480ae000 tx=0x7f454801a390 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:39:08.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.503+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f453c05fef0 con 0x7f4550143740 2024-09-17T03:39:08.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 --> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f455006b540 con 0x7f4530079ec0 2024-09-17T03:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4547400640 1 -- 172.21.15.46:0/1635423910 <== mgr.4168 v2:172.21.15.46:6800/537437096 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7f4548004d50 con 0x7f4530079ec0 2024-09-17T03:39:08.677 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:39:08.677 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4530079ec0 msgr2=0x7f453007c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4530079ec0 0x7f453007c3d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f45480ae000 tx=0x7f454801a390 comp rx=0 tx=0).stop 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 msgr2=0x7f4550143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 0x7f4550143b50 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f45500cd150 tx=0x7f453c00d3e0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 shutdown_connections 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f4530079ec0 0x7f453007c3d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4550143740 0x7f4550143b50 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f455013db60 0x7f4550143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 --2- 172.21.15.46:0/1635423910 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f4534003a10 0x7f45501f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 >> 172.21.15.46:0/1635423910 conn(0x7f45500b2040 msgr2=0x7f455015ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:39:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 shutdown_connections 2024-09-17T03:39:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.675+0000 7f4556200640 1 -- 172.21.15.46:0/1635423910 wait complete. 2024-09-17T03:39:08.699 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":688,"stamp":"2024-09-17T03:39:07.642834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1207260,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":1191347,"kb_avail":753767460,"statfs":{"total":773094113280,"available":771857879040,"internally_reserved":0,"allocated":16097280,"data_stored":7729320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1219939888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"1.0","version":"149'74","reported_seq":358,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T03:38:48.839167+0000","last_change":"2024-09-17T03:38:48.839096+0000","last_active":"2024-09-17T03:38:48.839167+0000","last_peered":"2024-09-17T03:38:48.839167+0000","last_clean":"2024-09-17T03:38:48.839167+0000","last_became_active":"2024-09-17T03:20:57.213441+0000","last_became_peered":"2024-09-17T03:20:57.213441+0000","last_unstale":"2024-09-17T03:38:48.839167+0000","last_undegraded":"2024-09-17T03:38:48.839167+0000","last_fullsized":"2024-09-17T03:38:48.839167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'74","last_scrub_stamp":"2024-09-17T03:38:48.838958+0000","last_deep_scrub":"149'74","last_deep_scrub_stamp":"2024-09-17T03:38:48.838958+0000","last_clean_scrub_stamp":"2024-09-17T03:38:48.838958+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:41.950858+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509661,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150128,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221712,"statfs":{"total":96636764160,"available":96483033088,"internally_reserved":0,"allocated":2457600,"data_stored":1408941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":151255498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509661,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159220,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157374,"kb_avail":94212620,"statfs":{"total":96636764160,"available":96473722880,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161151430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509661,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":155128,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216712,"statfs":{"total":96636764160,"available":96477913088,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":156957125},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509662,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":137076,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234764,"statfs":{"total":96636764160,"available":96496398336,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138475975},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509661,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159284,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212556,"statfs":{"total":96636764160,"available":96473657344,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509661,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141808,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230032,"statfs":{"total":96636764160,"available":96491552768,"internally_reserved":0,"allocated":2457600,"data_stored":1408941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542365,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123188,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248652,"statfs":{"total":96636764160,"available":96510619648,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542365,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":181428,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":179582,"kb_avail":94190412,"statfs":{"total":96636764160,"available":96450981888,"internally_reserved":0,"allocated":1863680,"data_stored":818573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183892421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:39:08.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f1050000640 1 -- 172.21.15.46:0/2111943308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030005160 msgr2=0x7f103000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f1050000640 1 --2- 172.21.15.46:0/2111943308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030005160 0x7f103000b9d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f10480734d0 tx=0x7f104001d4b0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f104e000640 1 -- 172.21.15.46:0/2111943308 reap_dead start 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f1050000640 1 -- 172.21.15.46:0/2111943308 shutdown_connections 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f1050000640 1 -- 172.21.15.46:0/2111943308 >> 172.21.15.46:0/2111943308 conn(0x7f10480b2040 msgr2=0x7f10480b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f1050000640 1 -- 172.21.15.46:0/2111943308 shutdown_connections 2024-09-17T03:39:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.855+0000 7f1050000640 1 -- 172.21.15.46:0/2111943308 wait complete. 2024-09-17T03:39:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 Processor -- start 2024-09-17T03:39:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 -- start start 2024-09-17T03:39:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1030005160 0x7f10481f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 0x7f1048157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10481452e0 0x7f10481456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1048074500 con 0x7f1048157340 2024-09-17T03:39:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1048074380 con 0x7f1030005160 2024-09-17T03:39:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f1050000640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f1048074680 con 0x7f10481452e0 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 0x7f1048157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 0x7f1048157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39946/0 (socket says 172.21.15.46:39946) 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 -- 172.21.15.46:0/928074424 learned_addr learned my addr 172.21.15.46:0/928074424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104d600640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1030005160 0x7f10481f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104e000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10481452e0 0x7f10481456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 -- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10481452e0 msgr2=0x7f10481456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10481452e0 0x7f10481456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 -- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1030005160 msgr2=0x7f10481f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1030005160 0x7f10481f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 -- 172.21.15.46:0/928074424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10481483c0 con 0x7f1048157340 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104d600640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1030005160 0x7f10481f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.859+0000 7f104cc00640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 0x7f1048157750 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f103400e6d0 tx=0x7f103400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:39:08.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034015070 con 0x7f1048157340 2024-09-17T03:39:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f103400cb30 con 0x7f1048157340 2024-09-17T03:39:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10481486a0 con 0x7f1048157340 2024-09-17T03:39:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034011460 con 0x7f1048157340 2024-09-17T03:39:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1048148c60 con 0x7f1048157340 2024-09-17T03:39:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1048074c10 con 0x7f1048157340 2024-09-17T03:39:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f1034018070 con 0x7f1048157340 2024-09-17T03:39:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1045600640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1028075a80 0x7f1028077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.863+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f1034093b60 con 0x7f1048157340 2024-09-17T03:39:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.867+0000 7f104d600640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1028075a80 0x7f1028077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:08.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.867+0000 7f104d600640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1028075a80 0x7f1028077f90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f103c009ad0 tx=0x7f103c0091e0 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:39:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:08.871+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f10340605e0 con 0x7f1048157340 2024-09-17T03:39:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.047+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f10480b3740 con 0x7f1048157340 2024-09-17T03:39:09.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v62)=0 v62) ==== 123+0+0 (secure 0 0 0) 0x7f103405fee0 con 0x7f1048157340 2024-09-17T03:39:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1045600640 1 -- 172.21.15.46:0/928074424 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f103405f9f0 con 0x7f1048157340 2024-09-17T03:39:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1028075a80 msgr2=0x7f1028077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1028075a80 0x7f1028077f90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f103c009ad0 tx=0x7f103c0091e0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 msgr2=0x7f1048157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 0x7f1048157750 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f103400e6d0 tx=0x7f103400eb90 comp rx=0 tx=0).stop 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 shutdown_connections 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f1028075a80 0x7f1028077f90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f10481452e0 0x7f10481456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1048157340 0x7f1048157750 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 --2- 172.21.15.46:0/928074424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1030005160 0x7f10481f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 >> 172.21.15.46:0/928074424 conn(0x7f10480b2040 msgr2=0x7f10481589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 shutdown_connections 2024-09-17T03:39:09.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.051+0000 7f1050000640 1 -- 172.21.15.46:0/928074424 wait complete. 2024-09-17T03:39:09.077 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:39:09.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.231+0000 7f92a7800640 1 -- 172.21.15.46:0/2960575219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 msgr2=0x7f928800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.231+0000 7f92a7800640 1 --2- 172.21.15.46:0/2960575219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f928800b9d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f929800c610 tx=0x7f929801d2a0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.231+0000 7f92a7800640 1 -- 172.21.15.46:0/2960575219 shutdown_connections 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.231+0000 7f92a7800640 1 --2- 172.21.15.46:0/2960575219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f928800b9d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.231+0000 7f92a7800640 1 -- 172.21.15.46:0/2960575219 >> 172.21.15.46:0/2960575219 conn(0x7f92a00b2040 msgr2=0x7f92a00b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 -- 172.21.15.46:0/2960575219 shutdown_connections 2024-09-17T03:39:09.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 -- 172.21.15.46:0/2960575219 wait complete. 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 Processor -- start 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 -- start start 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f92a01456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 --2- >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92a0141a80 0x7f92a00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0134970 0x7f92a01369c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92a00cf3b0 con 0x7f9288005260 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92a00752e0 con 0x7f92a0134970 2024-09-17T03:39:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a7800640 1 -- --> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] -- mon_getmap magic: 0 -- 0x7f92a00cf530 con 0x7f92a0141a80 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.235+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f92a01456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f92a01456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39964/0 (socket says 172.21.15.46:39964) 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 -- 172.21.15.46:0/2543374489 learned_addr learned my addr 172.21.15.46:0/2543374489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f929fe00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92a0141a80 0x7f92a00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a5800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0134970 0x7f92a01369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 -- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92a0141a80 msgr2=0x7f92a00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92a0141a80 0x7f92a00cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 -- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0134970 msgr2=0x7f92a01369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0134970 0x7f92a01369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 -- 172.21.15.46:0/2543374489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0145c20 con 0x7f9288005260 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a4e00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f92a01456e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f9294011860 tx=0x7f9294011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929400f9a0 con 0x7f9288005260 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f929400fc50 con 0x7f9288005260 2024-09-17T03:39:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294003600 con 0x7f9288005260 2024-09-17T03:39:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a0144640 con 0x7f9288005260 2024-09-17T03:39:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a0144c00 con 0x7f9288005260 2024-09-17T03:39:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a00b3b70 con 0x7f9288005260 2024-09-17T03:39:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.239+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) ==== 100906+0+0 (secure 0 0 0) 0x7f92940037a0 con 0x7f9288005260 2024-09-17T03:39:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.243+0000 7f929cc00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9280075a80 0x7f9280077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:39:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.243+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f9294094740 con 0x7f9288005260 2024-09-17T03:39:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.243+0000 7f929fe00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9280075a80 0x7f9280077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:39:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.243+0000 7f929fe00640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9280075a80 0x7f9280077f90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f928c006fd0 tx=0x7f928c006b10 comp rx=0 tx=0).ready entity=mgr.4168 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:39:09.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.247+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9294061250 con 0x7f9288005260 2024-09-17T03:39:09.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.451+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92a01452c0 con 0x7f9288005260 2024-09-17T03:39:09.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f929cc00640 1 -- 172.21.15.46:0/2543374489 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+3919 (secure 0 0 0) 0x7f9294060b50 con 0x7f9288005260 2024-09-17T03:39:09.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-17T03:39:09.456 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"btime":"2024-09-17T03:38:35:635961+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":247745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6834/1619006645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1619006645},{"type":"v1","addr":"172.21.15.99:6834","nonce":1619006645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":247751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2388261204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2388261204},{"type":"v1","addr":"172.21.15.46:6835","nonce":2388261204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":248752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3356683498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6833","nonce":3356683498},{"type":"v1","addr":"172.21.15.99:6835","nonce":3356683498}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":249762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1386590881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1386590881},{"type":"v1","addr":"172.21.15.46:6837","nonce":1386590881}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[]} 2024-09-17T03:39:09.456 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 97 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9280075a80 msgr2=0x7f9280077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9280075a80 0x7f9280077f90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f928c006fd0 tx=0x7f928c006b10 comp rx=0 tx=0).stop 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 msgr2=0x7f92a01456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f92a01456e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f9294011860 tx=0x7f9294011d20 comp rx=0 tx=0).stop 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 shutdown_connections 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:6800/537437096,v1:172.21.15.46:6801/537437096] conn(0x7f9280075a80 0x7f9280077f90 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0134970 0x7f92a01369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.99:3301/0,v1:172.21.15.99:6790/0] conn(0x7f92a0141a80 0x7f92a00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 --2- 172.21.15.46:0/2543374489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9288005260 0x7f92a01456e0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:39:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 >> 172.21.15.46:0/2543374489 conn(0x7f92a00b2040 msgr2=0x7f92a00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:39:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 shutdown_connections 2024-09-17T03:39:09.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-17T03:39:09.455+0000 7f92a7800640 1 -- 172.21.15.46:0/2543374489 wait complete. 2024-09-17T03:39:09.478 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T03:39:09.479 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T03:39:09.479 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:39:09.479 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:09.500 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:39:09.500 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:39:09.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:09.560 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:39:09.560 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:39:09.560 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:09.575 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:39:09.575 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:39:09.575 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:09.631 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:39:09.631 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T03:39:09.631 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T03:39:09.631 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:12.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-17T03:39:15.733 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T03:39:15.733 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T03:39:15.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:15.830 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T03:39:15.830 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T03:39:15.830 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:18.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-17T03:39:18.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-17T03:39:21.933 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T03:39:21.933 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T03:39:21.933 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:22.022 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T03:39:22.022 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T03:39:22.022 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:24.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-17T03:39:24.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-17T03:39:28.124 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T03:39:28.124 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T03:39:28.124 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:28.214 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T03:39:28.215 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T03:39:28.215 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:30.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-17T03:39:30.057 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-17T03:39:31.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:31.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:34.317 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T03:39:34.317 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T03:39:34.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:34.412 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T03:39:34.413 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T03:39:34.413 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T03:39:34.413 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:35.464 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-09-17T03:39:35.464 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-17T03:39:40.515 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T03:39:40.515 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T03:39:40.515 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:40.573 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T03:39:40.573 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T03:39:40.573 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T03:39:40.573 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:40.597 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T03:39:40.597 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T03:39:40.597 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:40.615 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T03:39:40.616 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T03:39:40.616 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:39:40.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-17T03:39:40.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-17T03:39:40.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-09-17T03:39:40.674 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T03:39:40.674 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T03:39:40.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-17T03:39:40.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:39:40.718 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T03:39:40.990 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:39:40.991 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T03:39:41.306 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:39:41.306 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T03:39:41.494 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:39:41.494 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T03:39:41.668 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:39:41.668 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T03:39:41.867 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:39:41.867 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T03:39:42.050 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:39:42.050 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T03:39:42.302 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:39:42.302 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T03:39:42.526 INFO:tasks.ceph:Archiving mon data... 2024-09-17T03:39:42.527 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/data/mon.a.tgz 2024-09-17T03:39:42.527 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-17T03:39:42.537 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.gHuejaqUGW 2024-09-17T03:39:42.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.gHuejaqUGW 2024-09-17T03:39:42.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.gHuejaqUGW 2024-09-17T03:39:42.976 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.gHuejaqUGW is 2MB 2024-09-17T03:39:43.059 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.gHuejaqUGW 2024-09-17T03:39:43.068 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/data/mon.b.tgz 2024-09-17T03:39:43.069 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-17T03:39:43.077 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.NsRP3tWSOi 2024-09-17T03:39:43.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.NsRP3tWSOi 2024-09-17T03:39:43.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.NsRP3tWSOi 2024-09-17T03:39:43.512 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.NsRP3tWSOi is 2MB 2024-09-17T03:39:43.604 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.NsRP3tWSOi 2024-09-17T03:39:43.612 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/data/mon.c.tgz 2024-09-17T03:39:43.613 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-17T03:39:43.664 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.2XK62NOEbz 2024-09-17T03:39:43.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2XK62NOEbz 2024-09-17T03:39:44.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.2XK62NOEbz 2024-09-17T03:39:44.109 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.2XK62NOEbz is 2MB 2024-09-17T03:39:44.194 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.2XK62NOEbz 2024-09-17T03:39:44.203 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T03:39:44.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T03:39:44.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T03:39:44.277 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T03:39:44.312 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T03:39:44.312 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T03:39:44.314 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/remote/smithi046/crash 2024-09-17T03:39:44.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T03:39:44.344 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/remote/smithi099/crash 2024-09-17T03:39:44.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T03:39:44.373 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T03:39:44.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:44.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:44.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:44.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T03:39:44.518 INFO:tasks.ceph:Compressing logs... 2024-09-17T03:39:44.519 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T03:39:44.526 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T03:39:44.561 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45243.log 2024-09-17T03:39:44.563 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2024-09-17T03:39:44.564 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43947.log 2024-09-17T03:39:44.564 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45243.log.gz 2024-09-17T03:39:44.564 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-09-17T03:39:44.564 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2024-09-17T03:39:44.565 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49921.log 2024-09-17T03:39:44.565 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43947.log.gz 2024-09-17T03:39:44.566 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40103.log 2024-09-17T03:39:44.566 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-09-17T03:39:44.567 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-09-17T03:39:44.567 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49921.log.gz 2024-09-17T03:39:44.568 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2024-09-17T03:39:44.568 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40103.log.gz 2024-09-17T03:39:44.568 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46342.log 2024-09-17T03:39:44.569 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-09-17T03:39:44.570 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53660.log 2024-09-17T03:39:44.570 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2024-09-17T03:39:44.570 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-09-17T03:39:44.570 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46342.log.gz 2024-09-17T03:39:44.571 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2024-09-17T03:39:44.572 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53660.log.gz 2024-09-17T03:39:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2024-09-17T03:39:44.572 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-09-17T03:39:44.573 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2024-09-17T03:39:44.573 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2024-09-17T03:39:44.574 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2024-09-17T03:39:44.574 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2024-09-17T03:39:44.575 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48383.log 2024-09-17T03:39:44.575 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2024-09-17T03:39:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55784.log 2024-09-17T03:39:44.576 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2024-09-17T03:39:44.577 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38764.log 2024-09-17T03:39:44.577 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48383.log.gz 2024-09-17T03:39:44.578 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48556.log 2024-09-17T03:39:44.578 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55784.log.gz 2024-09-17T03:39:44.579 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-09-17T03:39:44.579 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38764.log.gz 2024-09-17T03:39:44.579 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52263.log 2024-09-17T03:39:44.580 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48556.log.gz 2024-09-17T03:39:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-09-17T03:39:44.581 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-09-17T03:39:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-09-17T03:39:44.582 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52263.log.gz 2024-09-17T03:39:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52655.log 2024-09-17T03:39:44.582 INFO:teuthology.orchestra.run.smithi046.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-09-17T03:39:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47702.log 2024-09-17T03:39:44.583 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-09-17T03:39:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33965.log 2024-09-17T03:39:44.584 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52655.log.gz 2024-09-17T03:39:44.585 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34205.log 2024-09-17T03:39:44.585 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29092.log 2024-09-17T03:39:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50165.log 2024-09-17T03:39:44.585 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47702.log.gz 2024-09-17T03:39:44.586 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46368.log 2024-09-17T03:39:44.588 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33965.log.gz 2024-09-17T03:39:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38391.log 2024-09-17T03:39:44.588 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50165.log.gz 2024-09-17T03:39:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2024-09-17T03:39:44.588 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46368.log.gz 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.3.34205.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-osd.7.log 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.3.34205.log.gz 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29171.log 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29092.log.gz 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34103.log 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29171.log.gz 2024-09-17T03:39:44.589 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T03:39:44.592 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.1.34103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34103.log.gz 2024-09-17T03:39:44.592 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-17T03:39:44.592 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29123.log 2024-09-17T03:39:44.592 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29020.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-17T03:39:44.592 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29068.log 2024-09-17T03:39:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-09-17T03:39:44.593 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38391.log.gz 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43479.log 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56226.log 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43479.log.gz 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34148.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56226.log.gz 2024-09-17T03:39:44.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38920.log 2024-09-17T03:39:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37386.log 2024-09-17T03:39:44.598 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz 2024-09-17T03:39:44.598 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41770.log 2024-09-17T03:39:44.598 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38920.log.gz 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48215.log 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37386.log.gz 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36061.log 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41770.log.gz 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48215.log.gz 2024-09-17T03:39:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29147.log 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29123.log.gz 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29068.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29068.log.gz 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29147.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29147.log.gz 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29044.log 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.29044.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29044.log.gz 2024-09-17T03:39:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T03:39:44.603 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34367.log 2024-09-17T03:39:44.603 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36061.log.gz 2024-09-17T03:39:44.603 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39312.log 2024-09-17T03:39:44.603 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43137.log 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34367.log.gz 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28991.log 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39312.log.gz 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39800.log 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43137.log.gz 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28991.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28991.log.gz 2024-09-17T03:39:44.604 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55888.log 2024-09-17T03:39:44.605 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T03:39:44.605 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T03:39:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44240.log 2024-09-17T03:39:44.606 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39800.log.gz 2024-09-17T03:39:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2024-09-17T03:39:44.606 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55888.log.gz 2024-09-17T03:39:44.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33441.log 2024-09-17T03:39:44.606 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44240.log.gz 2024-09-17T03:39:44.607 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44940.log 2024-09-17T03:39:44.607 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-09-17T03:39:44.608 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-09-17T03:39:44.608 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33441.log.gz 2024-09-17T03:39:44.609 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35080.log 2024-09-17T03:39:44.609 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44940.log.gz 2024-09-17T03:39:44.609 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mds.b.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T03:39:44.609 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T03:39:44.610 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29195.log 2024-09-17T03:39:44.610 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36420.log 2024-09-17T03:39:44.611 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-09-17T03:39:44.611 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2024-09-17T03:39:44.611 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35080.log.gz 2024-09-17T03:39:44.612 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.36420.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.36420.log.gz 2024-09-17T03:39:44.612 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44776.log 2024-09-17T03:39:44.612 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.29195.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T03:39:44.612 INFO:teuthology.orchestra.run.smithi099.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29195.log.gz 2024-09-17T03:39:44.613 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T03:39:44.613 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53924.log 2024-09-17T03:39:44.614 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2024-09-17T03:39:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53052.log 2024-09-17T03:39:44.614 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44776.log.gz 2024-09-17T03:39:44.614 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43383.log 2024-09-17T03:39:44.614 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53924.log.gz 2024-09-17T03:39:44.615 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-09-17T03:39:44.615 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53052.log.gz 2024-09-17T03:39:44.616 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54390.log 2024-09-17T03:39:44.616 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz 2024-09-17T03:39:44.617 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55635.log 2024-09-17T03:39:44.617 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-09-17T03:39:44.618 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45955.log 2024-09-17T03:39:44.618 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54390.log.gz 2024-09-17T03:39:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43784.log 2024-09-17T03:39:44.619 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55635.log.gz 2024-09-17T03:39:44.619 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2024-09-17T03:39:44.620 INFO:teuthology.orchestra.run.smithi046.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.45955.log.gz 2024-09-17T03:39:44.620 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50663.log 2024-09-17T03:39:44.621 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43784.log.gz 2024-09-17T03:39:44.621 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56382.log 2024-09-17T03:39:44.621 INFO:teuthology.orchestra.run.smithi046.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gz 2024-09-17T03:39:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40672.log 2024-09-17T03:39:44.622 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50663.log.gz 2024-09-17T03:39:44.623 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36948.log 2024-09-17T03:39:44.623 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56382.log.gz 2024-09-17T03:39:44.624 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41174.log 2024-09-17T03:39:44.624 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40672.log.gz 2024-09-17T03:39:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47365.log 2024-09-17T03:39:44.625 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36948.log.gz 2024-09-17T03:39:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35795.log 2024-09-17T03:39:44.626 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41174.log.gz 2024-09-17T03:39:44.627 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48139.log 2024-09-17T03:39:44.627 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47365.log.gz 2024-09-17T03:39:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 2024-09-17T03:39:44.628 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35795.log.gz 2024-09-17T03:39:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48139.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T03:39:44.629 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48139.log.gz 2024-09-17T03:39:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-09-17T03:39:44.630 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gz 2024-09-17T03:39:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45380.log 2024-09-17T03:39:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-09-17T03:39:44.632 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-09-17T03:39:44.632 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2024-09-17T03:39:44.632 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45380.log.gz 2024-09-17T03:39:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35382.log 2024-09-17T03:39:44.634 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gz 2024-09-17T03:39:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49591.log 2024-09-17T03:39:44.635 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2024-09-17T03:39:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37725.log 2024-09-17T03:39:44.636 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35382.log.gz 2024-09-17T03:39:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49182.log 2024-09-17T03:39:44.637 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49591.log.gz 2024-09-17T03:39:44.637 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.37725.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36035.log 2024-09-17T03:39:44.638 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37725.log.gz 2024-09-17T03:39:44.639 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52890.log 2024-09-17T03:39:44.639 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49182.log.gz 2024-09-17T03:39:44.639 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36035.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36035.log.gz 2024-09-17T03:39:44.640 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34173.log 2024-09-17T03:39:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-09-17T03:39:44.641 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52890.log.gz 2024-09-17T03:39:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36879.log 2024-09-17T03:39:44.642 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34173.log.gz 2024-09-17T03:39:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37230.log 2024-09-17T03:39:44.642 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2024-09-17T03:39:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35132.log 2024-09-17T03:39:44.643 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36879.log.gz 2024-09-17T03:39:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47085.log 2024-09-17T03:39:44.644 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37230.log.gz 2024-09-17T03:39:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-09-17T03:39:44.645 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35132.log.gz 2024-09-17T03:39:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41200.log 2024-09-17T03:39:44.646 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47085.log.gz 2024-09-17T03:39:44.647 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46090.log 2024-09-17T03:39:44.647 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-09-17T03:39:44.648 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2024-09-17T03:39:44.648 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41200.log.gz 2024-09-17T03:39:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48459.log 2024-09-17T03:39:44.649 INFO:teuthology.orchestra.run.smithi046.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.46090.log.gz 2024-09-17T03:39:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39024.log 2024-09-17T03:39:44.650 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2024-09-17T03:39:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40963.log 2024-09-17T03:39:44.650 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48459.log.gz 2024-09-17T03:39:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50096.log 2024-09-17T03:39:44.651 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39024.log.gz 2024-09-17T03:39:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43531.log 2024-09-17T03:39:44.652 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40963.log.gz 2024-09-17T03:39:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.50096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50096.log.gz 2024-09-17T03:39:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41226.log 2024-09-17T03:39:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43505.log 2024-09-17T03:39:44.654 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43531.log.gz 2024-09-17T03:39:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55609.log 2024-09-17T03:39:44.655 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41226.log.gz 2024-09-17T03:39:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43505.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43505.log.gz 2024-09-17T03:39:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53332.log 2024-09-17T03:39:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44322.log 2024-09-17T03:39:44.657 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55609.log.gz 2024-09-17T03:39:44.658 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47734.log 2024-09-17T03:39:44.658 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53332.log.gz 2024-09-17T03:39:44.659 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40313.log 2024-09-17T03:39:44.659 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44322.log.gz 2024-09-17T03:39:44.659 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36489.log 2024-09-17T03:39:44.660 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47734.log.gz 2024-09-17T03:39:44.660 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41535.log 2024-09-17T03:39:44.661 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40313.log.gz 2024-09-17T03:39:44.661 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36489.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36489.log.gz 2024-09-17T03:39:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28845.log 2024-09-17T03:39:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36515.log 2024-09-17T03:39:44.662 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41535.log.gz 2024-09-17T03:39:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28845.log: gzip -5 --verbose 74.8% -- -- replaced with /var/log/ceph/ceph-client.admin.28845.log.gz /var/log/ceph/ceph-client.admin.54861.log 2024-09-17T03:39:44.663 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-17T03:39:44.664 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36515.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.36515.log.gz --verbose 2024-09-17T03:39:44.664 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.37256.log 2024-09-17T03:39:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38699.log 2024-09-17T03:39:44.665 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.54861.log.gz 2024-09-17T03:39:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51272.log 2024-09-17T03:39:44.666 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37256.log.gz 2024-09-17T03:39:44.667 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45981.log 2024-09-17T03:39:44.667 INFO:teuthology.orchestra.run.smithi046.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.38699.log.gz 2024-09-17T03:39:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-09-17T03:39:44.668 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51272.log.gz 2024-09-17T03:39:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46913.log 2024-09-17T03:39:44.669 INFO:teuthology.orchestra.run.smithi046.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.45981.log.gz 2024-09-17T03:39:44.670 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.4.log: 94.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T03:39:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2024-09-17T03:39:44.670 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-09-17T03:39:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43201.log 2024-09-17T03:39:44.671 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46913.log.gz 2024-09-17T03:39:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54002.log 2024-09-17T03:39:44.671 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2024-09-17T03:39:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55992.log 2024-09-17T03:39:44.672 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43201.log.gz 2024-09-17T03:39:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37595.log 2024-09-17T03:39:44.676 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54002.log.gz 2024-09-17T03:39:44.678 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2024-09-17T03:39:44.678 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55992.log.gz 2024-09-17T03:39:44.678 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2024-09-17T03:39:44.678 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37595.log.gz 2024-09-17T03:39:44.678 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47586.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2024-09-17T03:39:44.678 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46006.log 2024-09-17T03:39:44.679 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45683.log 2024-09-17T03:39:44.679 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2024-09-17T03:39:44.679 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-09-17T03:39:44.679 INFO:teuthology.orchestra.run.smithi046.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.46006.log.gz 2024-09-17T03:39:44.680 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-09-17T03:39:44.680 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45683.log.gz 2024-09-17T03:39:44.680 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35672.log 2024-09-17T03:39:44.680 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-09-17T03:39:44.681 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39169.log 2024-09-17T03:39:44.681 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-09-17T03:39:44.681 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28924.log 2024-09-17T03:39:44.681 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35672.log.gz 2024-09-17T03:39:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47849.log 2024-09-17T03:39:44.682 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz 2024-09-17T03:39:44.683 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-09-17T03:39:44.683 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28924.log.gz 2024-09-17T03:39:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47849.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47849.log.gz 2024-09-17T03:39:44.684 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2024-09-17T03:39:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46574.log 2024-09-17T03:39:44.685 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-09-17T03:39:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53487.log 2024-09-17T03:39:44.686 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2024-09-17T03:39:44.687 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39114.log 2024-09-17T03:39:44.687 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46574.log.gz 2024-09-17T03:39:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53487.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53487.log.gz 2024-09-17T03:39:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37060.log 2024-09-17T03:39:44.688 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51691.log 2024-09-17T03:39:44.689 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39114.log.gz 2024-09-17T03:39:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33134.log 2024-09-17T03:39:44.689 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37060.log.gz 2024-09-17T03:39:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54614.log 2024-09-17T03:39:44.690 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51691.log.gz 2024-09-17T03:39:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35953.log 2024-09-17T03:39:44.691 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33134.log.gz 2024-09-17T03:39:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44530.log 2024-09-17T03:39:44.692 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz 2024-09-17T03:39:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2024-09-17T03:39:44.693 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35953.log.gz 2024-09-17T03:39:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52019.log 2024-09-17T03:39:44.694 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44530.log.gz 2024-09-17T03:39:44.695 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34171.log 2024-09-17T03:39:44.695 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2024-09-17T03:39:44.696 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-09-17T03:39:44.696 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52019.log.gz 2024-09-17T03:39:44.697 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28821.log 2024-09-17T03:39:44.697 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34171.log.gz 2024-09-17T03:39:44.697 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51836.log 2024-09-17T03:39:44.697 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-09-17T03:39:44.698 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28821.log.gz 2024-09-17T03:39:44.698 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55097.log 2024-09-17T03:39:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39591.log 2024-09-17T03:39:44.699 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51836.log.gz 2024-09-17T03:39:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55097.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55097.log.gz 2024-09-17T03:39:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45648.log 2024-09-17T03:39:44.701 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2024-09-17T03:39:44.701 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39591.log.gz 2024-09-17T03:39:44.702 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35516.log 2024-09-17T03:39:44.702 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45648.log.gz 2024-09-17T03:39:44.703 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-09-17T03:39:44.703 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-09-17T03:39:44.705 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56148.log 2024-09-17T03:39:44.705 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35516.log.gz 2024-09-17T03:39:44.705 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33549.log 2024-09-17T03:39:44.705 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-09-17T03:39:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33653.log 2024-09-17T03:39:44.706 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56148.log.gz 2024-09-17T03:39:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2024-09-17T03:39:44.706 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33549.log.gz 2024-09-17T03:39:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54499.log 2024-09-17T03:39:44.707 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33653.log.gz 2024-09-17T03:39:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-09-17T03:39:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42837.log 2024-09-17T03:39:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-09-17T03:39:44.709 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54499.log.gz 2024-09-17T03:39:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35412.log 2024-09-17T03:39:44.710 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42837.log.gz 2024-09-17T03:39:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37034.log 2024-09-17T03:39:44.711 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-09-17T03:39:44.712 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39539.log 2024-09-17T03:39:44.712 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35412.log.gz 2024-09-17T03:39:44.713 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2024-09-17T03:39:44.713 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37034.log.gz 2024-09-17T03:39:44.714 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50821.log 2024-09-17T03:39:44.714 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39539.log.gz 2024-09-17T03:39:44.714 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49078.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz 2024-09-17T03:39:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34589.log 2024-09-17T03:39:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-09-17T03:39:44.715 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50821.log.gz 2024-09-17T03:39:44.716 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38838.log 2024-09-17T03:39:44.716 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34589.log.gz 2024-09-17T03:39:44.717 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-09-17T03:39:44.717 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-09-17T03:39:44.718 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2024-09-17T03:39:44.718 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38838.log.gz 2024-09-17T03:39:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54052.log 2024-09-17T03:39:44.719 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-09-17T03:39:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2024-09-17T03:39:44.720 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2024-09-17T03:39:44.720 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54052.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54052.log.gz 2024-09-17T03:39:44.720 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-09-17T03:39:44.721 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51914.log 2024-09-17T03:39:44.721 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2024-09-17T03:39:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2024-09-17T03:39:44.722 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-09-17T03:39:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40795.log 2024-09-17T03:39:44.723 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51914.log.gz 2024-09-17T03:39:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46316.log 2024-09-17T03:39:44.724 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2024-09-17T03:39:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2024-09-17T03:39:44.725 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40795.log.gz 2024-09-17T03:39:44.726 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51428.log 2024-09-17T03:39:44.726 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46316.log.gz 2024-09-17T03:39:44.727 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47879.log 2024-09-17T03:39:44.727 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2024-09-17T03:39:44.727 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2024-09-17T03:39:44.728 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51428.log.gz 2024-09-17T03:39:44.728 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47879.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T03:39:44.729 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47879.log.gz 2024-09-17T03:39:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49331.log 2024-09-17T03:39:44.729 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2024-09-17T03:39:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41970.log 2024-09-17T03:39:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49331.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49331.log.gz 2024-09-17T03:39:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37491.log 2024-09-17T03:39:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37008.log 2024-09-17T03:39:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41970.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41970.log.gz 2024-09-17T03:39:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41509.log 2024-09-17T03:39:44.733 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37491.log.gz 2024-09-17T03:39:44.734 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51888.log 2024-09-17T03:39:44.735 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37008.log.gz 2024-09-17T03:39:44.735 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28953.log 2024-09-17T03:39:44.736 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41509.log.gz 2024-09-17T03:39:44.737 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-09-17T03:39:44.737 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51888.log.gz 2024-09-17T03:39:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28953.log.gz 2024-09-17T03:39:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46232.log 2024-09-17T03:39:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.46174.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45592.log 2024-09-17T03:39:44.739 INFO:teuthology.orchestra.run.smithi046.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-09-17T03:39:44.740 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2024-09-17T03:39:44.740 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46232.log.gz 2024-09-17T03:39:44.741 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34355.log 2024-09-17T03:39:44.741 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45592.log.gz 2024-09-17T03:39:44.742 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56252.log 2024-09-17T03:39:44.742 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz 2024-09-17T03:39:44.743 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55123.log 2024-09-17T03:39:44.743 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34355.log.gz 2024-09-17T03:39:44.744 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43414.log 2024-09-17T03:39:44.744 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56252.log.gz 2024-09-17T03:39:44.745 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-09-17T03:39:44.745 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55123.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55123.log.gz 2024-09-17T03:39:44.746 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50255.log 2024-09-17T03:39:44.746 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43414.log.gz 2024-09-17T03:39:44.747 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47650.log 2024-09-17T03:39:44.747 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-09-17T03:39:44.748 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44814.log 2024-09-17T03:39:44.749 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50255.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50255.log.gz 2024-09-17T03:39:44.749 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-09-17T03:39:44.749 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47650.log.gz 2024-09-17T03:39:44.750 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43635.log 2024-09-17T03:39:44.750 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44814.log.gz 2024-09-17T03:39:44.751 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.50911.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43914.log 2024-09-17T03:39:44.752 INFO:teuthology.orchestra.run.smithi046.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-09-17T03:39:44.752 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42473.log 2024-09-17T03:39:44.752 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43635.log.gz 2024-09-17T03:39:44.753 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43914.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.33627.log -- replaced with /var/log/ceph/ceph-client.admin.43914.log.gz 2024-09-17T03:39:44.753 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-17T03:39:44.754 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T03:39:44.755 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42473.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42473.log.gz 2024-09-17T03:39:44.755 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54640.log 2024-09-17T03:39:44.756 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33627.log.gz 2024-09-17T03:39:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44426.log 2024-09-17T03:39:44.758 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55557.log 2024-09-17T03:39:44.758 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54640.log.gz 2024-09-17T03:39:44.759 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46991.log 2024-09-17T03:39:44.759 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44426.log.gz 2024-09-17T03:39:44.760 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-09-17T03:39:44.760 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55557.log.gz 2024-09-17T03:39:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37953.log 2024-09-17T03:39:44.761 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46991.log.gz 2024-09-17T03:39:44.762 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36798.log 2024-09-17T03:39:44.762 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-09-17T03:39:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54284.log 2024-09-17T03:39:44.764 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37953.log.gz 2024-09-17T03:39:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41109.log 2024-09-17T03:39:44.765 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36798.log.gz 2024-09-17T03:39:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51201.log 2024-09-17T03:39:44.766 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54284.log.gz 2024-09-17T03:39:44.767 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38232.log 2024-09-17T03:39:44.767 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41109.log.gz 2024-09-17T03:39:44.768 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43661.log 2024-09-17T03:39:44.768 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51201.log.gz 2024-09-17T03:39:44.769 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.38232.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.38232.log.gz --verbose 2024-09-17T03:39:44.769 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.53175.log 2024-09-17T03:39:44.771 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39487.log 2024-09-17T03:39:44.771 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43661.log.gz 2024-09-17T03:39:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-09-17T03:39:44.772 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53175.log.gz 2024-09-17T03:39:44.772 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56200.log 2024-09-17T03:39:44.773 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39487.log.gz 2024-09-17T03:39:44.774 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34615.log 2024-09-17T03:39:44.774 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-09-17T03:39:44.775 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33497.log 2024-09-17T03:39:44.775 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56200.log.gz 2024-09-17T03:39:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54694.log 2024-09-17T03:39:44.776 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34615.log.gz 2024-09-17T03:39:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49895.log 2024-09-17T03:39:44.777 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33497.log.gz 2024-09-17T03:39:44.778 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-09-17T03:39:44.778 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54694.log.gz 2024-09-17T03:39:44.779 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42048.log 2024-09-17T03:39:44.779 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49895.log.gz 2024-09-17T03:39:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52787.log 2024-09-17T03:39:44.781 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-09-17T03:39:44.781 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41744.log 2024-09-17T03:39:44.782 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42048.log.gz 2024-09-17T03:39:44.782 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48961.log 2024-09-17T03:39:44.783 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52787.log.gz 2024-09-17T03:39:44.784 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2024-09-17T03:39:44.784 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41744.log.gz 2024-09-17T03:39:44.785 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53716.log 2024-09-17T03:39:44.785 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48961.log.gz 2024-09-17T03:39:44.786 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-09-17T03:39:44.786 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz 2024-09-17T03:39:44.787 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47443.log 2024-09-17T03:39:44.787 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53716.log.gz 2024-09-17T03:39:44.788 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51784.log 2024-09-17T03:39:44.789 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-09-17T03:39:44.789 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43059.log 2024-09-17T03:39:44.789 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47443.log.gz 2024-09-17T03:39:44.791 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43583.log 2024-09-17T03:39:44.791 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51784.log.gz 2024-09-17T03:39:44.792 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48498.log 2024-09-17T03:39:44.792 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43059.log.gz 2024-09-17T03:39:44.793 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55225.log 2024-09-17T03:39:44.793 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43583.log.gz 2024-09-17T03:39:44.794 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54537.log 2024-09-17T03:39:44.794 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48498.log.gz 2024-09-17T03:39:44.795 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46206.log 2024-09-17T03:39:44.796 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55225.log.gz 2024-09-17T03:39:44.796 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42510.log 2024-09-17T03:39:44.797 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54537.log.gz 2024-09-17T03:39:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-09-17T03:39:44.798 INFO:teuthology.orchestra.run.smithi046.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46206.log.gz 2024-09-17T03:39:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2024-09-17T03:39:44.799 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42510.log.gz 2024-09-17T03:39:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40261.log 2024-09-17T03:39:44.800 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-09-17T03:39:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42203.log 2024-09-17T03:39:44.801 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2024-09-17T03:39:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48920.log 2024-09-17T03:39:44.802 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40261.log.gz 2024-09-17T03:39:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37927.log 2024-09-17T03:39:44.803 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42203.log.gz 2024-09-17T03:39:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-09-17T03:39:44.804 INFO:teuthology.orchestra.run.smithi046.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.48920.log.gz 2024-09-17T03:39:44.805 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41065.log 2024-09-17T03:39:44.805 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37927.log.gz 2024-09-17T03:39:44.806 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40129.log 2024-09-17T03:39:44.807 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-09-17T03:39:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36758.log 2024-09-17T03:39:44.808 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41065.log.gz 2024-09-17T03:39:44.809 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38660.log 2024-09-17T03:39:44.809 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40129.log.gz 2024-09-17T03:39:44.810 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41278.log 2024-09-17T03:39:44.810 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36758.log.gz 2024-09-17T03:39:44.811 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2024-09-17T03:39:44.811 INFO:teuthology.orchestra.run.smithi046.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.38660.log.gz 2024-09-17T03:39:44.812 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56435.log 2024-09-17T03:39:44.812 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41278.log.gz 2024-09-17T03:39:44.813 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2024-09-17T03:39:44.813 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2024-09-17T03:39:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56435.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23941.log 2024-09-17T03:39:44.815 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56435.log.gz 2024-09-17T03:39:44.816 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28804.log 2024-09-17T03:39:44.816 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2024-09-17T03:39:44.817 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.tmp-client.admin.23941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28977.log 2024-09-17T03:39:44.817 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23941.log.gz 2024-09-17T03:39:44.818 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34382.log 2024-09-17T03:39:44.818 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28804.log.gz 2024-09-17T03:39:44.819 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54666.log 2024-09-17T03:39:44.819 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28977.log.gz 2024-09-17T03:39:44.820 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50769.log 2024-09-17T03:39:44.820 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34382.log.gz 2024-09-17T03:39:44.821 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45621.log 2024-09-17T03:39:44.821 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54666.log.gz 2024-09-17T03:39:44.822 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53000.log 2024-09-17T03:39:44.822 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50769.log.gz 2024-09-17T03:39:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42435.log 2024-09-17T03:39:44.824 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45621.log.gz 2024-09-17T03:39:44.824 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43028.log 2024-09-17T03:39:44.825 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53000.log.gz 2024-09-17T03:39:44.826 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56330.log 2024-09-17T03:39:44.826 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42435.log.gz 2024-09-17T03:39:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52761.log 2024-09-17T03:39:44.827 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43028.log.gz 2024-09-17T03:39:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38998.log 2024-09-17T03:39:44.828 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56330.log.gz 2024-09-17T03:39:44.829 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37178.log 2024-09-17T03:39:44.829 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52761.log.gz 2024-09-17T03:39:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40363.log 2024-09-17T03:39:44.830 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38998.log.gz 2024-09-17T03:39:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46287.log 2024-09-17T03:39:44.831 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37178.log.gz 2024-09-17T03:39:44.832 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41587.log 2024-09-17T03:39:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40363.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40363.log.gz 2024-09-17T03:39:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2024-09-17T03:39:44.833 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46287.log.gz 2024-09-17T03:39:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35262.log 2024-09-17T03:39:44.835 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41587.log.gz 2024-09-17T03:39:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-09-17T03:39:44.836 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2024-09-17T03:39:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54809.log 2024-09-17T03:39:44.837 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35262.log.gz 2024-09-17T03:39:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45843.log 2024-09-17T03:39:44.838 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-09-17T03:39:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51513.log 2024-09-17T03:39:44.839 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54809.log.gz 2024-09-17T03:39:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47339.log 2024-09-17T03:39:44.840 INFO:teuthology.orchestra.run.smithi046.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.45843.log.gz 2024-09-17T03:39:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44698.log 2024-09-17T03:39:44.841 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51513.log.gz 2024-09-17T03:39:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37569.log 2024-09-17T03:39:44.843 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47339.log.gz 2024-09-17T03:39:44.843 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38353.log 2024-09-17T03:39:44.844 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44698.log.gz 2024-09-17T03:39:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35542.log 2024-09-17T03:39:44.845 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37569.log.gz 2024-09-17T03:39:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35184.log 2024-09-17T03:39:44.846 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38353.log.gz 2024-09-17T03:39:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51992.log 2024-09-17T03:39:44.847 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35542.log.gz 2024-09-17T03:39:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55251.log 2024-09-17T03:39:44.848 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35184.log.gz 2024-09-17T03:39:44.849 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2024-09-17T03:39:44.849 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-09-17T03:39:44.850 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40989.log 2024-09-17T03:39:44.850 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55251.log.gz 2024-09-17T03:39:44.851 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-09-17T03:39:44.852 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz 2024-09-17T03:39:44.853 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33389.log 2024-09-17T03:39:44.853 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40989.log.gz 2024-09-17T03:39:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47417.log 2024-09-17T03:39:44.854 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-09-17T03:39:44.855 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37777.log 2024-09-17T03:39:44.855 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33389.log.gz 2024-09-17T03:39:44.856 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48724.log 2024-09-17T03:39:44.856 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47417.log.gz 2024-09-17T03:39:44.857 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36567.log 2024-09-17T03:39:44.857 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37777.log.gz 2024-09-17T03:39:44.858 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2024-09-17T03:39:44.858 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48724.log.gz 2024-09-17T03:39:44.859 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52173.log 2024-09-17T03:39:44.860 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36567.log.gz 2024-09-17T03:39:44.860 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2024-09-17T03:39:44.861 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2024-09-17T03:39:44.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43557.log 2024-09-17T03:39:44.865 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52173.log.gz 2024-09-17T03:39:44.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44606.log 2024-09-17T03:39:44.865 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2024-09-17T03:39:44.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-17T03:39:44.865 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43557.log.gz 2024-09-17T03:39:44.866 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40607.log 2024-09-17T03:39:44.866 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44606.log.gz 2024-09-17T03:39:44.866 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28766.log 2024-09-17T03:39:44.866 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-17T03:39:44.867 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39702.log 2024-09-17T03:39:44.867 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40607.log.gz 2024-09-17T03:39:44.868 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36437.log 2024-09-17T03:39:44.868 INFO:teuthology.orchestra.run.smithi046.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.28766.log.gz 2024-09-17T03:39:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44556.log 2024-09-17T03:39:44.870 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39702.log.gz 2024-09-17T03:39:44.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51376.log 2024-09-17T03:39:44.871 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36437.log.gz 2024-09-17T03:39:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41824.log 2024-09-17T03:39:44.872 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44556.log.gz 2024-09-17T03:39:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log 2024-09-17T03:39:44.873 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51376.log.gz 2024-09-17T03:39:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51545.log 2024-09-17T03:39:44.874 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41824.log.gz 2024-09-17T03:39:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40911.log 2024-09-17T03:39:44.875 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2024-09-17T03:39:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-09-17T03:39:44.876 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51545.log.gz 2024-09-17T03:39:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43836.log 2024-09-17T03:39:44.877 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40911.log.gz 2024-09-17T03:39:44.879 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2024-09-17T03:39:44.879 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz 2024-09-17T03:39:44.880 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37517.log 2024-09-17T03:39:44.880 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43836.log.gz 2024-09-17T03:39:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46522.log 2024-09-17T03:39:44.881 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2024-09-17T03:39:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41853.log 2024-09-17T03:39:44.882 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37517.log.gz 2024-09-17T03:39:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55199.log 2024-09-17T03:39:44.883 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46522.log.gz 2024-09-17T03:39:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38327.log 2024-09-17T03:39:44.885 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41853.log.gz 2024-09-17T03:39:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47762.log 2024-09-17T03:39:44.886 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55199.log.gz 2024-09-17T03:39:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38571.log 2024-09-17T03:39:44.887 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38327.log.gz 2024-09-17T03:39:44.888 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53026.log 2024-09-17T03:39:44.888 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47762.log.gz 2024-09-17T03:39:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35769.log 2024-09-17T03:39:44.889 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38571.log.gz 2024-09-17T03:39:44.890 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51661.log 2024-09-17T03:39:44.890 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53026.log.gz 2024-09-17T03:39:44.891 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35769.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T03:39:44.891 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35769.log.gz 2024-09-17T03:39:44.892 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41483.log 2024-09-17T03:39:44.893 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51661.log.gz 2024-09-17T03:39:44.894 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40427.log 2024-09-17T03:39:44.895 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-09-17T03:39:44.895 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41483.log.gz 2024-09-17T03:39:44.896 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40427.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40427.log.gz 2024-09-17T03:39:44.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33469.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-09-17T03:39:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2024-09-17T03:39:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2024-09-17T03:39:44.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34719.log 2024-09-17T03:39:44.901 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2024-09-17T03:39:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53976.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2024-09-17T03:39:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34719.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34719.log.gz 2024-09-17T03:39:44.903 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-09-17T03:39:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44878.log 2024-09-17T03:39:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52419.log 2024-09-17T03:39:44.905 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-09-17T03:39:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44878.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44878.log.gz 2024-09-17T03:39:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52419.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52419.log.gz 2024-09-17T03:39:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29001.log 2024-09-17T03:39:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56018.log 2024-09-17T03:39:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29001.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29001.log.gz -5 2024-09-17T03:39:44.910 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41796.log 2024-09-17T03:39:44.911 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50978.log 2024-09-17T03:39:44.912 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56018.log.gz 2024-09-17T03:39:44.913 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41796.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41796.log.gz 2024-09-17T03:39:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50978.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50978.log.gz 2024-09-17T03:39:44.914 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37979.log 2024-09-17T03:39:44.916 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39286.log 2024-09-17T03:39:44.917 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-09-17T03:39:44.917 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37979.log.gz 2024-09-17T03:39:44.918 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39286.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39286.log.gz 2024-09-17T03:39:44.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45323.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-09-17T03:39:44.919 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41252.log 2024-09-17T03:39:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45780.log 2024-09-17T03:39:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 2024-09-17T03:39:44.922 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41252.log.gz 2024-09-17T03:39:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45780.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.45780.log.gz 2024-09-17T03:39:44.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56122.log 2024-09-17T03:39:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35848.log 2024-09-17T03:39:44.925 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2024-09-17T03:39:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44069.log 2024-09-17T03:39:44.927 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56122.log.gz 2024-09-17T03:39:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35848.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35848.log.gz 2024-09-17T03:39:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44069.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2024-09-17T03:39:44.929 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40505.log 2024-09-17T03:39:44.930 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51011.log 2024-09-17T03:39:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36087.log 2024-09-17T03:39:44.932 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40505.log.gz 2024-09-17T03:39:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51011.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51011.log.gz 2024-09-17T03:39:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36087.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36087.log.gz 2024-09-17T03:39:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38249.log 2024-09-17T03:39:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53597.log 2024-09-17T03:39:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38249.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38249.log.gz 2024-09-17T03:39:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53597.log.gz 2024-09-17T03:39:44.938 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-09-17T03:39:44.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36463.log 2024-09-17T03:39:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34163.log 2024-09-17T03:39:44.941 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-09-17T03:39:44.942 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36463.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36463.log.gz 2024-09-17T03:39:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34163.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34163.log.gz 2024-09-17T03:39:44.943 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42409.log 2024-09-17T03:39:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-09-17T03:39:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52506.log 2024-09-17T03:39:44.946 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42409.log.gz 2024-09-17T03:39:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42074.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-09-17T03:39:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2024-09-17T03:39:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55661.log 2024-09-17T03:39:44.949 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52506.log.gz 2024-09-17T03:39:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38809.log 2024-09-17T03:39:44.951 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2024-09-17T03:39:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55661.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55661.log.gz 2024-09-17T03:39:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38809.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38809.log.gz 2024-09-17T03:39:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35490.log 2024-09-17T03:39:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39995.log 2024-09-17T03:39:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42759.log 2024-09-17T03:39:44.955 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35490.log.gz 2024-09-17T03:39:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39995.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39995.log.gz 2024-09-17T03:39:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42759.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.42759.log.gz 2024-09-17T03:39:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37871.log 2024-09-17T03:39:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41304.log 2024-09-17T03:39:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37871.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37871.log.gz 2024-09-17T03:39:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44188.log 2024-09-17T03:39:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35464.log 2024-09-17T03:39:44.963 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41304.log.gz 2024-09-17T03:39:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44188.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44188.log.gz 2024-09-17T03:39:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35464.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35464.log.gz 2024-09-17T03:39:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34641.log 2024-09-17T03:39:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33955.log 2024-09-17T03:39:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51719.log 2024-09-17T03:39:44.968 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34641.log.gz 2024-09-17T03:39:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33955.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33955.log.gz 2024-09-17T03:39:44.970 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51719.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51719.log.gz 2024-09-17T03:39:44.971 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49383.log 2024-09-17T03:39:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46652.log 2024-09-17T03:39:44.973 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44840.log 2024-09-17T03:39:44.973 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49383.log.gz 2024-09-17T03:39:44.974 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46652.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46652.log.gz 2024-09-17T03:39:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44840.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44840.log.gz 2024-09-17T03:39:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37543.log 2024-09-17T03:39:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29381.log 2024-09-17T03:39:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38501.log 2024-09-17T03:39:44.979 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37543.log.gz 2024-09-17T03:39:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29381.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29381.log.gz 2024-09-17T03:39:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38501.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38501.log.gz 2024-09-17T03:39:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44914.log 2024-09-17T03:39:44.982 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52735.log 2024-09-17T03:39:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40181.log 2024-09-17T03:39:44.984 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-09-17T03:39:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52735.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52735.log.gz 2024-09-17T03:39:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40181.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40181.log.gz 2024-09-17T03:39:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37751.log 2024-09-17T03:39:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41457.log 2024-09-17T03:39:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38539.log 2024-09-17T03:39:44.989 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37751.log.gz 2024-09-17T03:39:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41457.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41457.log.gz 2024-09-17T03:39:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38539.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38539.log.gz 2024-09-17T03:39:44.991 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-09-17T03:39:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2024-09-17T03:39:44.994 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33913.log 2024-09-17T03:39:44.994 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-09-17T03:39:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44153.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2024-09-17T03:39:44.996 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33913.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33913.log.gz 2024-09-17T03:39:44.997 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49700.log 2024-09-17T03:39:44.998 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T03:39:44.999 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34170.log 2024-09-17T03:39:44.999 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49700.log.gz 2024-09-17T03:39:45.001 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.34170.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34170.log.gz 2024-09-17T03:39:45.002 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53280.log 2024-09-17T03:39:45.003 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50139.log 2024-09-17T03:39:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53280.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53280.log.gz 2024-09-17T03:39:45.005 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50139.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50139.log.gz 2024-09-17T03:39:45.006 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-09-17T03:39:45.007 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46939.log 2024-09-17T03:39:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51454.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2024-09-17T03:39:45.009 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46939.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46939.log.gz 2024-09-17T03:39:45.010 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40453.log 2024-09-17T03:39:45.011 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49869.log 2024-09-17T03:39:45.013 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40453.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40453.log.gz 2024-09-17T03:39:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49869.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49869.log.gz 2024-09-17T03:39:45.014 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56411.log 2024-09-17T03:39:45.015 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53569.log 2024-09-17T03:39:45.017 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56411.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56411.log.gz 2024-09-17T03:39:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53569.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53569.log.gz 2024-09-17T03:39:45.018 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-09-17T03:39:45.019 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44095.log 2024-09-17T03:39:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47163.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-09-17T03:39:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44095.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44095.log.gz 2024-09-17T03:39:45.022 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-09-17T03:39:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37901.log 2024-09-17T03:39:45.025 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43706.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-09-17T03:39:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37901.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37901.log.gz 2024-09-17T03:39:45.026 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2024-09-17T03:39:45.028 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39050.log 2024-09-17T03:39:45.029 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38005.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-09-17T03:39:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39050.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39050.log.gz 2024-09-17T03:39:45.030 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38185.log 2024-09-17T03:39:45.032 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55687.log 2024-09-17T03:39:45.033 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38185.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38185.log.gz 2024-09-17T03:39:45.034 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55687.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55687.log.gz 2024-09-17T03:39:45.035 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46116.log 2024-09-17T03:39:45.036 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48262.log 2024-09-17T03:39:45.038 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46116.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46116.log.gz 2024-09-17T03:39:45.038 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.48262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48262.log.gz 2024-09-17T03:39:45.038 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54783.log 2024-09-17T03:39:45.040 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50715.log 2024-09-17T03:39:45.041 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54783.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54783.log.gz 2024-09-17T03:39:45.042 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50715.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50715.log.gz 2024-09-17T03:39:45.042 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-09-17T03:39:45.044 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34805.log 2024-09-17T03:39:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45870.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-09-17T03:39:45.046 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34805.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34805.log.gz 2024-09-17T03:39:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39338.log 2024-09-17T03:39:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46031.log 2024-09-17T03:39:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39338.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39338.log.gz 2024-09-17T03:39:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46031.log: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.46031.log.gz 2024-09-17T03:39:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44974.log 2024-09-17T03:39:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2024-09-17T03:39:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44974.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44974.log.gz 2024-09-17T03:39:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53846.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2024-09-17T03:39:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51350.log 2024-09-17T03:39:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34375.log 2024-09-17T03:39:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51350.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2024-09-17T03:39:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34375.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34375.log.gz 2024-09-17T03:39:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48645.log 2024-09-17T03:39:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54258.log 2024-09-17T03:39:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48645.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48645.log.gz 2024-09-17T03:39:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54258.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54258.log.gz 2024-09-17T03:39:45.063 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42914.log 2024-09-17T03:39:45.064 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45101.log 2024-09-17T03:39:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42914.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.42914.log.gz 2024-09-17T03:39:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45101.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2024-09-17T03:39:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49999.log 2024-09-17T03:39:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51071.log 2024-09-17T03:39:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49999.log: /var/log/ceph/ceph-client.admin.51071.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49999.log.gz 2024-09-17T03:39:45.071 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51071.log.gz 2024-09-17T03:39:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52123.log 2024-09-17T03:39:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35704.log 2024-09-17T03:39:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52123.log: /var/log/ceph/ceph-client.admin.35704.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52123.log.gz 2024-09-17T03:39:45.076 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35704.log.gz 2024-09-17T03:39:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39908.log 2024-09-17T03:39:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54991.log 2024-09-17T03:39:45.077 INFO:teuthology.orchestra.run.smithi099.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T03:39:45.078 INFO:teuthology.orchestra.run.smithi099.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T03:39:45.078 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39908.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39908.log.gz 2024-09-17T03:39:45.079 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54991.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54991.log.gz 2024-09-17T03:39:45.079 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55043.log 2024-09-17T03:39:45.081 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40025.log 2024-09-17T03:39:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55043.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55043.log.gz 2024-09-17T03:39:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40025.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40025.log.gz 2024-09-17T03:39:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45139.log 2024-09-17T03:39:45.085 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49973.log 2024-09-17T03:39:45.086 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45139.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.45139.log.gz 2024-09-17T03:39:45.087 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49973.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49973.log.gz 2024-09-17T03:39:45.087 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33961.log 2024-09-17T03:39:45.089 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46600.log 2024-09-17T03:39:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33961.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33961.log.gz 2024-09-17T03:39:45.091 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46600.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46600.log.gz 2024-09-17T03:39:45.092 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49643.log 2024-09-17T03:39:45.093 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36139.log 2024-09-17T03:39:45.095 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49643.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49643.log.gz 2024-09-17T03:39:45.095 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36139.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36139.log.gz 2024-09-17T03:39:45.096 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39734.log 2024-09-17T03:39:45.097 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47242.log 2024-09-17T03:39:45.099 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39734.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39734.log.gz 2024-09-17T03:39:45.099 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47242.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47242.log.gz 2024-09-17T03:39:45.100 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-09-17T03:39:45.101 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39565.log 2024-09-17T03:39:45.103 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41142.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-09-17T03:39:45.104 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39565.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39565.log.gz 2024-09-17T03:39:45.104 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35158.log 2024-09-17T03:39:45.105 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54220.log 2024-09-17T03:39:45.107 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35158.log.gz 2024-09-17T03:39:45.108 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55407.log 2024-09-17T03:39:45.108 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54220.log.gz 2024-09-17T03:39:45.109 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52480.log 2024-09-17T03:39:45.111 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55407.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55407.log.gz 2024-09-17T03:39:45.111 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52480.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52480.log.gz 2024-09-17T03:39:45.112 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55862.log 2024-09-17T03:39:45.113 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50949.log 2024-09-17T03:39:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55862.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55862.log.gz 2024-09-17T03:39:45.115 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50949.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50949.log.gz 2024-09-17T03:39:45.116 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35054.log 2024-09-17T03:39:45.117 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53423.log 2024-09-17T03:39:45.118 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.2.35054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35054.log.gz 2024-09-17T03:39:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53423.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53423.log.gz 2024-09-17T03:39:45.120 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52237.log 2024-09-17T03:39:45.121 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54142.log 2024-09-17T03:39:45.123 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52237.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52237.log.gz 2024-09-17T03:39:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54142.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54142.log.gz 2024-09-17T03:39:45.124 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54913.log 2024-09-17T03:39:45.125 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-09-17T03:39:45.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54913.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54913.log.gz 2024-09-17T03:39:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36243.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-09-17T03:39:45.128 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2024-09-17T03:39:45.130 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47676.log 2024-09-17T03:39:45.131 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-09-17T03:39:45.132 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49026.log 2024-09-17T03:39:45.133 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2024-09-17T03:39:45.133 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50466.log 2024-09-17T03:39:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49026.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49026.log.gz 2024-09-17T03:39:45.136 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50466.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50466.log.gz 2024-09-17T03:39:45.136 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40569.log 2024-09-17T03:39:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52825.log 2024-09-17T03:39:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40569.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40569.log.gz 2024-09-17T03:39:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52825.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52825.log.gz 2024-09-17T03:39:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55436.log 2024-09-17T03:39:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34370.log 2024-09-17T03:39:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55436.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55436.log.gz 2024-09-17T03:39:45.144 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35979.log 2024-09-17T03:39:45.147 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34370.log.gz 2024-09-17T03:39:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35322.log 2024-09-17T03:39:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35979.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35979.log.gz 2024-09-17T03:39:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35322.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35322.log.gz 2024-09-17T03:39:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42293.log 2024-09-17T03:39:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54168.log 2024-09-17T03:39:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42293.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-09-17T03:39:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54168.log.gz 2024-09-17T03:39:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39416.log 2024-09-17T03:39:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53104.log 2024-09-17T03:39:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39416.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39416.log.gz 2024-09-17T03:39:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53104.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53104.log.gz 2024-09-17T03:39:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41330.log 2024-09-17T03:39:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54576.log 2024-09-17T03:39:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41330.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41330.log.gz 2024-09-17T03:39:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54576.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54576.log.gz 2024-09-17T03:39:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T03:39:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42939.log 2024-09-17T03:39:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.42939.log: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.42939.log.gz 2024-09-17T03:39:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28943.log 2024-09-17T03:39:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28943.log.gz 2024-09-17T03:39:45.167 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T03:39:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29522.log 2024-09-17T03:39:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44296.log 2024-09-17T03:39:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29522.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29522.log.gz 2024-09-17T03:39:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44296.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44296.log.gz 2024-09-17T03:39:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-17T03:39:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-09-17T03:39:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48802.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-17T03:39:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34383.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-09-17T03:39:45.175 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-09-17T03:39:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52220.log 2024-09-17T03:39:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41888.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-09-17T03:39:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.52220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52220.log.gz 2024-09-17T03:39:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-09-17T03:39:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36373.log 2024-09-17T03:39:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39669.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-09-17T03:39:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36373.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36373.log.gz 2024-09-17T03:39:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55069.log 2024-09-17T03:39:45.184 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42811.log 2024-09-17T03:39:45.186 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55069.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55069.log.gz 2024-09-17T03:39:45.186 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42811.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42811.log.gz 2024-09-17T03:39:45.187 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47554.log 2024-09-17T03:39:45.188 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2024-09-17T03:39:45.190 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47554.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47554.log.gz 2024-09-17T03:39:45.190 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51324.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2024-09-17T03:39:45.191 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2024-09-17T03:39:45.192 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49305.log 2024-09-17T03:39:45.194 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38301.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2024-09-17T03:39:45.194 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49305.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49305.log.gz 2024-09-17T03:39:45.195 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45504.log 2024-09-17T03:39:45.196 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-09-17T03:39:45.198 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45504.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45504.log.gz 2024-09-17T03:39:45.198 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41918.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-09-17T03:39:45.199 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51940.log 2024-09-17T03:39:45.200 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41015.log 2024-09-17T03:39:45.202 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51940.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51940.log.gz 2024-09-17T03:39:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41015.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41015.log.gz 2024-09-17T03:39:45.203 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38429.log 2024-09-17T03:39:45.204 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55505.log 2024-09-17T03:39:45.206 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38429.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38429.log.gz 2024-09-17T03:39:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55505.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55505.log.gz 2024-09-17T03:39:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55355.log 2024-09-17T03:39:45.209 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-09-17T03:39:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55355.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55355.log.gz 2024-09-17T03:39:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48331.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-09-17T03:39:45.212 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53228.log 2024-09-17T03:39:45.213 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-09-17T03:39:45.215 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53228.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53228.log.gz 2024-09-17T03:39:45.215 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38868.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-09-17T03:39:45.216 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-09-17T03:39:45.217 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42127.log 2024-09-17T03:39:45.219 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38946.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-09-17T03:39:45.219 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42127.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42127.log.gz 2024-09-17T03:39:45.219 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-09-17T03:39:45.221 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54348.log 2024-09-17T03:39:45.222 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44348.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-09-17T03:39:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54348.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54348.log.gz 2024-09-17T03:39:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52709.log 2024-09-17T03:39:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39852.log 2024-09-17T03:39:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52709.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52709.log.gz 2024-09-17T03:39:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39852.log.gz 2024-09-17T03:39:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49526.log 2024-09-17T03:39:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52683.log 2024-09-17T03:39:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49526.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49526.log.gz 2024-09-17T03:39:45.231 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52683.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52683.log.gz 2024-09-17T03:39:45.232 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33960.log 2024-09-17T03:39:45.233 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-09-17T03:39:45.234 INFO:teuthology.orchestra.run.smithi099.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T03:39:45.235 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33960.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33960.log.gz 2024-09-17T03:39:45.235 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44724.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-09-17T03:39:45.236 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40410.log 2024-09-17T03:39:45.237 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35106.log 2024-09-17T03:39:45.238 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.40410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40410.log.gz 2024-09-17T03:39:45.239 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35106.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35106.log.gz 2024-09-17T03:39:45.240 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48776.log 2024-09-17T03:39:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33731.log 2024-09-17T03:39:45.243 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48776.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48776.log.gz 2024-09-17T03:39:45.243 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33731.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33731.log.gz 2024-09-17T03:39:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37439.log 2024-09-17T03:39:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44670.log 2024-09-17T03:39:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37439.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37439.log.gz 2024-09-17T03:39:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44670.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44670.log.gz 2024-09-17T03:39:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40744.log 2024-09-17T03:39:45.249 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40077.log 2024-09-17T03:39:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40744.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.40744.log.gz 2024-09-17T03:39:45.252 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40077.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40077.log.gz 2024-09-17T03:39:45.252 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48856.log 2024-09-17T03:39:45.253 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41646.log 2024-09-17T03:39:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48856.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48856.log.gz 2024-09-17T03:39:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45025.log 2024-09-17T03:39:45.257 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41646.log.gz 2024-09-17T03:39:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34158.log 2024-09-17T03:39:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45025.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45025.log.gz 2024-09-17T03:39:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34158.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34158.log.gz 2024-09-17T03:39:45.260 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53634.log 2024-09-17T03:39:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-09-17T03:39:45.263 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53634.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53634.log.gz 2024-09-17T03:39:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46678.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-09-17T03:39:45.264 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-09-17T03:39:45.265 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34966.log 2024-09-17T03:39:45.267 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54757.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-09-17T03:39:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34966.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34966.log.gz 2024-09-17T03:39:45.268 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37308.log 2024-09-17T03:39:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28877.log 2024-09-17T03:39:45.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37308.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37308.log.gz 2024-09-17T03:39:45.272 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56461.log 2024-09-17T03:39:45.272 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28877.log.gz 2024-09-17T03:39:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49843.log 2024-09-17T03:39:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56461.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56461.log.gz 2024-09-17T03:39:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49843.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49843.log.gz 2024-09-17T03:39:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35438.log 2024-09-17T03:39:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36217.log 2024-09-17T03:39:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35438.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35438.log.gz 2024-09-17T03:39:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36217.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36217.log.gz 2024-09-17T03:39:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-09-17T03:39:45.281 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52857.log 2024-09-17T03:39:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-09-17T03:39:45.285 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52857.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52857.log.gz 2024-09-17T03:39:45.285 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48607.log 2024-09-17T03:39:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2024-09-17T03:39:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48607.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48607.log.gz 2024-09-17T03:39:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52948.log 2024-09-17T03:39:45.291 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz 2024-09-17T03:39:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52922.log 2024-09-17T03:39:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52948.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52948.log.gz 2024-09-17T03:39:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52922.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52922.log.gz 2024-09-17T03:39:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53742.log 2024-09-17T03:39:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49357.log 2024-09-17T03:39:45.294 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53742.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53742.log.gz 2024-09-17T03:39:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49357.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49357.log.gz 2024-09-17T03:39:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42267.log 2024-09-17T03:39:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50357.log 2024-09-17T03:39:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42267.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42267.log.gz 2024-09-17T03:39:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50357.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50357.log.gz 2024-09-17T03:39:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39076.log 2024-09-17T03:39:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55329.log 2024-09-17T03:39:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39076.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39076.log.gz 2024-09-17T03:39:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55329.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55329.log.gz 2024-09-17T03:39:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45295.log 2024-09-17T03:39:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54099.log 2024-09-17T03:39:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45295.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45295.log.gz 2024-09-17T03:39:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.54099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54099.log.gz 2024-09-17T03:39:45.308 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42371.log 2024-09-17T03:39:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2024-09-17T03:39:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42371.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42371.log.gz 2024-09-17T03:39:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44750.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2024-09-17T03:39:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42617.log 2024-09-17T03:39:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2024-09-17T03:39:45.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42617.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42617.log.gz 2024-09-17T03:39:45.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43331.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2024-09-17T03:39:45.316 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55732.log 2024-09-17T03:39:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34374.log 2024-09-17T03:39:45.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55732.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55732.log.gz 2024-09-17T03:39:45.320 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log 2024-09-17T03:39:45.320 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34374.log.gz 2024-09-17T03:39:45.321 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52315.log 2024-09-17T03:39:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49156.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2024-09-17T03:39:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52315.log.gz 2024-09-17T03:39:45.324 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39969.log 2024-09-17T03:39:45.325 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-09-17T03:39:45.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39969.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39969.log.gz 2024-09-17T03:39:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35594.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-09-17T03:39:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55583.log 2024-09-17T03:39:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34667.log 2024-09-17T03:39:45.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55583.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55583.log.gz 2024-09-17T03:39:45.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34667.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34667.log.gz 2024-09-17T03:39:45.332 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44043.log 2024-09-17T03:39:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37660.log 2024-09-17T03:39:45.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44043.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44043.log.gz 2024-09-17T03:39:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-09-17T03:39:45.336 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37660.log.gz 2024-09-17T03:39:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35236.log 2024-09-17T03:39:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42889.log: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-09-17T03:39:45.339 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35236.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35236.log.gz 2024-09-17T03:39:45.340 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45269.log 2024-09-17T03:39:45.341 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41996.log 2024-09-17T03:39:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45269.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45269.log.gz 2024-09-17T03:39:45.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-09-17T03:39:45.344 INFO:teuthology.orchestra.run.smithi046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41996.log.gz 2024-09-17T03:39:45.345 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35900.log 2024-09-17T03:39:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38083.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-09-17T03:39:45.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35900.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35900.log.gz 2024-09-17T03:39:45.348 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-17T03:39:45.349 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40531.log 2024-09-17T03:39:45.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-17T03:39:45.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40531.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40531.log.gz 2024-09-17T03:39:45.352 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43085.log 2024-09-17T03:39:45.353 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-09-17T03:39:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43085.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43085.log.gz 2024-09-17T03:39:45.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50293.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-09-17T03:39:45.356 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-09-17T03:39:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34175.log 2024-09-17T03:39:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41944.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 2024-09-17T03:39:45.359 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34175.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34175.log.gz 2024-09-17T03:39:45.360 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37113.log 2024-09-17T03:39:45.361 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33575.log 2024-09-17T03:39:45.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37113.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37113.log.gz 2024-09-17T03:39:45.364 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-09-17T03:39:45.364 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33575.log.gz 2024-09-17T03:39:45.365 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53202.log 2024-09-17T03:39:45.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43253.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-09-17T03:39:45.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53202.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53202.log.gz 2024-09-17T03:39:45.368 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2024-09-17T03:39:45.369 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51635.log 2024-09-17T03:39:45.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54835.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz 2024-09-17T03:39:45.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51635.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51635.log.gz 2024-09-17T03:39:45.372 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42319.log 2024-09-17T03:39:45.373 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39513.log 2024-09-17T03:39:45.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42319.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42319.log.gz 2024-09-17T03:39:45.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39513.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39513.log.gz 2024-09-17T03:39:45.376 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49672.log 2024-09-17T03:39:45.377 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39202.log 2024-09-17T03:39:45.379 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49672.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49672.log.gz 2024-09-17T03:39:45.379 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39202.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39202.log.gz 2024-09-17T03:39:45.380 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51175.log 2024-09-17T03:39:45.381 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47137.log 2024-09-17T03:39:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51175.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51175.log.gz 2024-09-17T03:39:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47137.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47137.log.gz 2024-09-17T03:39:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-09-17T03:39:45.385 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42655.log 2024-09-17T03:39:45.387 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50547.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-09-17T03:39:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42655.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.42655.log.gz 2024-09-17T03:39:45.388 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40479.log 2024-09-17T03:39:45.390 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52974.log 2024-09-17T03:39:45.391 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40479.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40479.log.gz 2024-09-17T03:39:45.392 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-09-17T03:39:45.393 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52974.log.gz 2024-09-17T03:39:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44214.log 2024-09-17T03:39:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39617.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-09-17T03:39:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44214.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44214.log.gz 2024-09-17T03:39:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-09-17T03:39:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37412.log 2024-09-17T03:39:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49761.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-09-17T03:39:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37412.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37412.log.gz 2024-09-17T03:39:45.400 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54416.log 2024-09-17T03:39:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T03:39:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54416.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54416.log.gz 2024-09-17T03:39:45.403 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T03:39:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38109.log 2024-09-17T03:39:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.38109.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38109.log.gz 2024-09-17T03:39:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-09-17T03:39:45.411 INFO:teuthology.orchestra.run.smithi046.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T03:39:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46808.log 2024-09-17T03:39:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45412.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-09-17T03:39:45.415 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38057.log 2024-09-17T03:39:45.416 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46808.log.gz 2024-09-17T03:39:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42345.log 2024-09-17T03:39:45.420 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38057.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38057.log.gz 2024-09-17T03:39:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T03:39:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42345.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42345.log.gz 2024-09-17T03:39:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36165.log 2024-09-17T03:39:45.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.36165.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36165.log.gz 2024-09-17T03:39:45.432 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35288.log 2024-09-17T03:39:45.433 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41561.log 2024-09-17T03:39:45.435 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35288.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35288.log.gz 2024-09-17T03:39:45.436 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28833.log 2024-09-17T03:39:45.437 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48279.log 2024-09-17T03:39:45.438 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41561.log.gz 2024-09-17T03:39:45.439 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28833.log.gz 2024-09-17T03:39:45.442 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48279.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48279.log.gz 2024-09-17T03:39:45.442 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43979.log 2024-09-17T03:39:45.443 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46261.log 2024-09-17T03:39:45.445 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43979.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43979.log.gz 2024-09-17T03:39:45.449 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39826.log 2024-09-17T03:39:45.450 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46261.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46261.log.gz 2024-09-17T03:39:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39826.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39826.log.gz 2024-09-17T03:39:45.452 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38894.log 2024-09-17T03:39:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56356.log 2024-09-17T03:39:45.458 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38894.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38894.log.gz 2024-09-17T03:39:45.462 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56356.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56356.log.gz 2024-09-17T03:39:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53254.log 2024-09-17T03:39:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41401.log 2024-09-17T03:39:45.470 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53254.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53254.log.gz 2024-09-17T03:39:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36269.log 2024-09-17T03:39:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41401.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41401.log.gz 2024-09-17T03:39:45.478 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36269.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36269.log.gz 2024-09-17T03:39:45.479 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36979.log 2024-09-17T03:39:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-09-17T03:39:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36979.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36979.log.gz 2024-09-17T03:39:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38275.log 2024-09-17T03:39:45.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48828.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-09-17T03:39:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42785.log 2024-09-17T03:39:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38275.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38275.log.gz 2024-09-17T03:39:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53820.log 2024-09-17T03:39:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42785.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.42785.log.gz 2024-09-17T03:39:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28901.log 2024-09-17T03:39:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50383.log 2024-09-17T03:39:45.500 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53820.log.gz 2024-09-17T03:39:45.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28901.log.gz 2024-09-17T03:39:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50795.log 2024-09-17T03:39:45.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50383.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50383.log.gz 2024-09-17T03:39:45.508 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35353.log 2024-09-17T03:39:45.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50795.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50795.log.gz 2024-09-17T03:39:45.512 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45473.log 2024-09-17T03:39:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35353.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35353.log.gz 2024-09-17T03:39:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52540.log 2024-09-17T03:39:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45473.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45473.log.gz 2024-09-17T03:39:45.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52540.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52540.log.gz 2024-09-17T03:39:45.524 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28809.log 2024-09-17T03:39:45.528 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36007.log 2024-09-17T03:39:45.529 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28809.log: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.28809.log.gz 2024-09-17T03:39:45.532 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33323.log 2024-09-17T03:39:45.534 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36007.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36007.log.gz 2024-09-17T03:39:45.535 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33809.log 2024-09-17T03:39:45.536 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33323.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33323.log.gz 2024-09-17T03:39:45.537 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36295.log 2024-09-17T03:39:45.538 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33809.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33809.log.gz 2024-09-17T03:39:45.540 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-09-17T03:39:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36295.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36295.log.gz 2024-09-17T03:39:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35568.log 2024-09-17T03:39:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33887.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-09-17T03:39:45.544 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46965.log 2024-09-17T03:39:45.552 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35568.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35568.log.gz 2024-09-17T03:39:45.552 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45191.log 2024-09-17T03:39:45.552 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46965.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46965.log.gz 2024-09-17T03:39:45.554 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45191.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45191.log.gz 2024-09-17T03:39:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-09-17T03:39:45.560 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55471.log 2024-09-17T03:39:45.562 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44374.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-09-17T03:39:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49409.log 2024-09-17T03:39:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55471.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55471.log.gz 2024-09-17T03:39:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43227.log 2024-09-17T03:39:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42731.log 2024-09-17T03:39:45.568 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49409.log.gz 2024-09-17T03:39:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43227.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43227.log.gz 2024-09-17T03:39:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-09-17T03:39:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42731.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42731.log.gz 2024-09-17T03:39:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56174.log 2024-09-17T03:39:45.576 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43175.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-09-17T03:39:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42022.log 2024-09-17T03:39:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56174.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56174.log.gz 2024-09-17T03:39:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42022.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42022.log.gz 2024-09-17T03:39:45.588 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37204.log 2024-09-17T03:39:45.592 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28864.log 2024-09-17T03:39:45.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37204.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37204.log.gz 2024-09-17T03:39:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52045.log 2024-09-17T03:39:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28864.log.gz 2024-09-17T03:39:45.598 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49279.log 2024-09-17T03:39:45.599 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39234.log 2024-09-17T03:39:45.600 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52045.log.gz 2024-09-17T03:39:45.601 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49279.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49279.log.gz 2024-09-17T03:39:45.604 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-09-17T03:39:45.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39234.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39234.log.gz 2024-09-17T03:39:45.607 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51609.log 2024-09-17T03:39:45.608 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2024-09-17T03:39:45.608 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-09-17T03:39:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51609.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51609.log.gz 2024-09-17T03:39:45.614 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51298.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2024-09-17T03:39:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40710.log 2024-09-17T03:39:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55277.log 2024-09-17T03:39:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40710.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40710.log.gz 2024-09-17T03:39:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-09-17T03:39:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55277.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55277.log.gz 2024-09-17T03:39:45.629 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53794.log 2024-09-17T03:39:45.629 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37334.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-09-17T03:39:45.634 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53794.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53794.log.gz 2024-09-17T03:39:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41613.log 2024-09-17T03:39:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53950.log 2024-09-17T03:39:45.670 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41613.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41613.log.gz 2024-09-17T03:39:45.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-17T03:39:45.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-17T03:39:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-09-17T03:39:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53950.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53950.log.gz 2024-09-17T03:39:45.686 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49208.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-09-17T03:39:45.686 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33523.log 2024-09-17T03:39:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36191.log 2024-09-17T03:39:45.693 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40885.log 2024-09-17T03:39:45.694 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36191.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36191.log.gz 2024-09-17T03:39:45.697 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33523.log.gz 2024-09-17T03:39:45.697 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36635.log 2024-09-17T03:39:45.698 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40885.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.40885.log.gz 2024-09-17T03:39:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46756.log 2024-09-17T03:39:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36635.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36635.log.gz 2024-09-17T03:39:45.734 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46756.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46756.log.gz 2024-09-17T03:39:45.734 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34693.log 2024-09-17T03:39:45.737 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43002.log 2024-09-17T03:39:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34693.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34693.log.gz 2024-09-17T03:39:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39364.log/var/log/ceph/ceph-client.admin.43002.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43002.log.gz 2024-09-17T03:39:45.743 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-17T03:39:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-09-17T03:39:45.749 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39364.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39364.log.gz 2024-09-17T03:39:45.751 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44504.log 2024-09-17T03:39:45.752 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45558.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-09-17T03:39:45.756 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36113.log 2024-09-17T03:39:45.758 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44504.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44504.log.gz 2024-09-17T03:39:45.762 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36113.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36113.log.gz 2024-09-17T03:39:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48750.log 2024-09-17T03:39:45.768 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50743.log 2024-09-17T03:39:45.770 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48750.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48750.log.gz 2024-09-17T03:39:45.772 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56096.log 2024-09-17T03:39:45.774 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50743.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50743.log.gz 2024-09-17T03:39:45.776 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-09-17T03:39:45.778 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56096.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56096.log.gz 2024-09-17T03:39:45.780 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2024-09-17T03:39:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43758.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-09-17T03:39:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33967.log 2024-09-17T03:39:45.786 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51123.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2024-09-17T03:39:45.788 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50113.log 2024-09-17T03:39:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33967.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33967.log.gz 2024-09-17T03:39:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50113.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50113.log.gz 2024-09-17T03:39:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48061.log 2024-09-17T03:39:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-09-17T03:39:45.802 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48061.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48061.log.gz 2024-09-17T03:39:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54448.log 2024-09-17T03:39:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43810.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-09-17T03:39:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35821.log 2024-09-17T03:39:45.808 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54448.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54448.log.gz 2024-09-17T03:39:45.812 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55810.log 2024-09-17T03:39:45.814 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35821.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35821.log.gz 2024-09-17T03:39:45.816 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50415.log 2024-09-17T03:39:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55810.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55810.log.gz 2024-09-17T03:39:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49461.log 2024-09-17T03:39:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50415.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50415.log.gz 2024-09-17T03:39:45.827 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.49461.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49461.log.gz 2024-09-17T03:39:45.827 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34368.log 2024-09-17T03:39:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55303.log 2024-09-17T03:39:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34368.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34368.log.gz 2024-09-17T03:39:45.832 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56278.log 2024-09-17T03:39:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55303.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55303.log.gz 2024-09-17T03:39:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-09-17T03:39:45.836 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-09-17T03:39:45.836 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56278.log.gz 2024-09-17T03:39:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43279.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-09-17T03:39:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51862.log 2024-09-17T03:39:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46056.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-09-17T03:39:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39461.log 2024-09-17T03:39:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51862.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51862.log.gz 2024-09-17T03:39:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39461.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39461.log.gz 2024-09-17T03:39:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-09-17T03:39:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50847.log 2024-09-17T03:39:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45350.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-09-17T03:39:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53391.log 2024-09-17T03:39:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50847.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50847.log.gz 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41427.log 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40287.log 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41427.log: /var/log/ceph/ceph-client.admin.53391.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41427.log.gz 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53391.log.gz 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51480.log 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40287.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40287.log.gz 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52379.log 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51480.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51480.log.gz 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54322.log 2024-09-17T03:39:45.889 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52379.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52379.log.gz 2024-09-17T03:39:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50885.log 2024-09-17T03:39:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54322.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54322.log.gz 2024-09-17T03:39:45.892 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54732.log 2024-09-17T03:39:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50885.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50885.log.gz 2024-09-17T03:39:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33705.log 2024-09-17T03:39:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54732.log: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.54732.log.gz 2024-09-17T03:39:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33705.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33705.log.gz 2024-09-17T03:39:45.903 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42250.log 2024-09-17T03:39:45.908 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-09-17T03:39:45.910 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42250.log.gz 2024-09-17T03:39:45.912 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47521.log 2024-09-17T03:39:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36323.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-09-17T03:39:45.916 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29357.log 2024-09-17T03:39:45.918 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47521.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47521.log.gz 2024-09-17T03:39:45.920 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47797.log 2024-09-17T03:39:45.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29357.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29357.log.gz 2024-09-17T03:39:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47215.log 2024-09-17T03:39:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47797.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47797.log.gz 2024-09-17T03:39:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55940.log 2024-09-17T03:39:45.930 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47215.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47215.log.gz 2024-09-17T03:39:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53306.log 2024-09-17T03:39:45.934 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55940.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55940.log.gz 2024-09-17T03:39:45.937 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53306.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53306.log.gz 2024-09-17T03:39:45.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46887.log 2024-09-17T03:39:45.944 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-09-17T03:39:45.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46887.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46887.log.gz 2024-09-17T03:39:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53078.log 2024-09-17T03:39:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33835.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-09-17T03:39:45.951 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40937.log 2024-09-17T03:39:45.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53078.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53078.log.gz 2024-09-17T03:39:45.957 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40051.log 2024-09-17T03:39:45.958 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40937.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40937.log.gz 2024-09-17T03:39:45.960 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47313.log 2024-09-17T03:39:45.962 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40051.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40051.log.gz 2024-09-17T03:39:45.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47313.log: -- /var/log/ceph/ceph-client.admin.34166.log 2024-09-17T03:39:45.964 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47313.log.gz 2024-09-17T03:39:45.968 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44478.log 2024-09-17T03:39:45.970 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34166.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34166.log.gz 2024-09-17T03:39:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44478.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44478.log.gz 2024-09-17T03:39:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36541.log 2024-09-17T03:39:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39260.log 2024-09-17T03:39:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36541.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36541.log.gz 2024-09-17T03:39:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54887.log 2024-09-17T03:39:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39260.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39260.log.gz 2024-09-17T03:39:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29437.log 2024-09-17T03:39:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54887.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54887.log.gz 2024-09-17T03:39:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45165.log 2024-09-17T03:39:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29437.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29437.log.gz 2024-09-17T03:39:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-09-17T03:39:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45165.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45165.log.gz 2024-09-17T03:39:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47268.log 2024-09-17T03:39:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47823.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-09-17T03:39:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37282.log 2024-09-17T03:39:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47268.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47268.log.gz 2024-09-17T03:39:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37282.log: -- /var/log/ceph/ceph-client.admin.50506.log 2024-09-17T03:39:46.008 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37282.log.gz 2024-09-17T03:39:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50506.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50506.log.gz 2024-09-17T03:39:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49052.log 2024-09-17T03:39:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48305.log 2024-09-17T03:39:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49052.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49052.log.gz 2024-09-17T03:39:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33601.log 2024-09-17T03:39:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48305.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48305.log.gz 2024-09-17T03:39:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49735.log 2024-09-17T03:39:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33601.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33601.log.gz 2024-09-17T03:39:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38738.log 2024-09-17T03:39:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49735.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49735.log.gz 2024-09-17T03:39:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39880.log 2024-09-17T03:39:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38738.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38738.log.gz 2024-09-17T03:39:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33757.log 2024-09-17T03:39:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39880.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39880.log.gz 2024-09-17T03:39:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28889.log 2024-09-17T03:39:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33757.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33757.log.gz 2024-09-17T03:39:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28889.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28889.log.gz 2024-09-17T03:39:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51227.log 2024-09-17T03:39:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38031.log 2024-09-17T03:39:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51227.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51227.log.gz 2024-09-17T03:39:46.060 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34924.log 2024-09-17T03:39:46.062 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38031.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38031.log.gz 2024-09-17T03:39:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44452.log 2024-09-17T03:39:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.34924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34924.log.gz 2024-09-17T03:39:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33861.log 2024-09-17T03:39:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44452.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44452.log.gz 2024-09-17T03:39:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-09-17T03:39:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33861.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33861.log.gz 2024-09-17T03:39:46.076 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T03:39:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-09-17T03:39:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37628.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-09-17T03:39:46.079 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54965.log 2024-09-17T03:39:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54965.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54965.log.gz 2024-09-17T03:39:46.083 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33679.log 2024-09-17T03:39:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33679.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.33679.log.gz 2024-09-17T03:39:46.086 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-09-17T03:39:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40207.log 2024-09-17T03:39:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40207.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40207.log.gz 2024-09-17T03:39:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33947.log 2024-09-17T03:39:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33947.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33947.log.gz 2024-09-17T03:39:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45713.log 2024-09-17T03:39:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45713.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45713.log.gz 2024-09-17T03:39:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44653.log 2024-09-17T03:39:46.101 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.44653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44653.log.gz 2024-09-17T03:39:46.101 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38622.log 2024-09-17T03:39:46.104 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38622.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38622.log.gz 2024-09-17T03:39:46.104 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56304.log 2024-09-17T03:39:46.105 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56304.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56304.log.gz 2024-09-17T03:39:46.105 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52629.log 2024-09-17T03:39:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52629.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52629.log.gz 2024-09-17T03:39:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2024-09-17T03:39:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49817.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2024-09-17T03:39:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53513.log 2024-09-17T03:39:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-09-17T03:39:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55381.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-09-17T03:39:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53513.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53513.log.gz 2024-09-17T03:39:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-09-17T03:39:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53872.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-09-17T03:39:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41356.log 2024-09-17T03:39:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41356.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41356.log.gz 2024-09-17T03:39:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40155.log 2024-09-17T03:39:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40155.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40155.log.gz 2024-09-17T03:39:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52603.log 2024-09-17T03:39:46.128 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40859.log 2024-09-17T03:39:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52603.log: /var/log/ceph/ceph-client.admin.40859.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40859.log.gz 2024-09-17T03:39:46.132 INFO:teuthology.orchestra.run.smithi046.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52603.log.gz 2024-09-17T03:39:46.132 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-09-17T03:39:46.135 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50191.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-09-17T03:39:46.135 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39390.log 2024-09-17T03:39:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-09-17T03:39:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39390.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39390.log.gz 2024-09-17T03:39:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45929.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-09-17T03:39:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55017.log 2024-09-17T03:39:46.143 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55531.log 2024-09-17T03:39:46.144 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55017.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39943.log 2024-09-17T03:39:46.144 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-17T03:39:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55531.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55531.log.gz 2024-09-17T03:39:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-09-17T03:39:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46470.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-09-17T03:39:46.151 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46394.log 2024-09-17T03:39:46.151 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39943.log.gz 2024-09-17T03:39:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35874.log 2024-09-17T03:39:46.154 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46394.log.gz 2024-09-17T03:39:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35874.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35874.log.gz 2024-09-17T03:39:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55149.log 2024-09-17T03:39:46.160 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55149.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55149.log.gz 2024-09-17T03:39:46.160 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-09-17T03:39:46.163 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39643.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-09-17T03:39:46.163 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49494.log 2024-09-17T03:39:46.164 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-17T03:39:46.166 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49494.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49494.log.gz 2024-09-17T03:39:46.167 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29012.log 2024-09-17T03:39:46.171 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42153.log: /var/log/ceph/ceph-client.admin.29012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29012.log.gz 2024-09-17T03:39:46.171 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-09-17T03:39:46.171 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47053.log 2024-09-17T03:39:46.173 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53686.log 2024-09-17T03:39:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47053.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47053.log.gz 2024-09-17T03:39:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2024-09-17T03:39:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53686.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53686.log.gz 2024-09-17T03:39:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54116.log 2024-09-17T03:39:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36828.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2024-09-17T03:39:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2024-09-17T03:39:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54116.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54116.log.gz 2024-09-17T03:39:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50049.log 2024-09-17T03:39:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45063.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2024-09-17T03:39:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-09-17T03:39:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50049.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50049.log.gz 2024-09-17T03:39:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42100.log 2024-09-17T03:39:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52097.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-09-17T03:39:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42100.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42100.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40235.log 2024-09-17T03:39:46.192 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-17T03:39:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46148.log 2024-09-17T03:39:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35646.log 2024-09-17T03:39:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46148.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46148.log.gz 2024-09-17T03:39:46.200 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40235.log.gz 2024-09-17T03:39:46.200 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35646.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35646.log.gz 2024-09-17T03:39:46.200 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2024-09-17T03:39:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47469.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2024-09-17T03:39:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29024.log 2024-09-17T03:39:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29024.log.gz 2024-09-17T03:39:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42566.log 2024-09-17T03:39:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42566.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42566.log.gz 2024-09-17T03:39:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33952.log 2024-09-17T03:39:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55914.log 2024-09-17T03:39:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33952.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33952.log.gz 2024-09-17T03:39:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.55914.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55914.log.gz 2024-09-17T03:39:46.215 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28912.log 2024-09-17T03:39:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28912.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28912.log.gz 2024-09-17T03:39:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33783.log 2024-09-17T03:39:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33963.log 2024-09-17T03:39:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33783.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33783.log.gz 2024-09-17T03:39:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33963.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33963.log.gz 2024-09-17T03:39:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45441.log 2024-09-17T03:39:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45441.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45441.log.gz 2024-09-17T03:39:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53768.log 2024-09-17T03:39:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53768.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53768.log.gz 2024-09-17T03:39:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42964.log 2024-09-17T03:39:46.232 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42964.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42964.log.gz 2024-09-17T03:39:46.232 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-09-17T03:39:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43111.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-09-17T03:39:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52071.log 2024-09-17T03:39:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52071.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52071.log.gz 2024-09-17T03:39:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40833.log 2024-09-17T03:39:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-09-17T03:39:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40833.log: /var/log/ceph/ceph-client.admin.41718.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40833.log.gz 2024-09-17T03:39:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52341.log 2024-09-17T03:39:46.241 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-09-17T03:39:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52341.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52341.log.gz 2024-09-17T03:39:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33415.log 2024-09-17T03:39:46.247 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33415.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33415.log.gz 2024-09-17T03:39:46.247 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2024-09-17T03:39:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49253.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2024-09-17T03:39:46.251 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56044.log 2024-09-17T03:39:46.252 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.56044.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56044.log.gz 2024-09-17T03:39:46.253 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53358.log 2024-09-17T03:39:46.255 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2024-09-17T03:39:46.256 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53358.log.gz 2024-09-17T03:39:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-09-17T03:39:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45217.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2024-09-17T03:39:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51810.log 2024-09-17T03:39:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48672.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-09-17T03:39:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53149.log 2024-09-17T03:39:46.264 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51810.log.gz 2024-09-17T03:39:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53149.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53149.log.gz 2024-09-17T03:39:46.268 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35926.log 2024-09-17T03:39:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35926.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35926.log.gz 2024-09-17T03:39:46.271 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2024-09-17T03:39:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38135.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2024-09-17T03:39:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37806.log 2024-09-17T03:39:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37806.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37806.log.gz 2024-09-17T03:39:46.277 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54939.log 2024-09-17T03:39:46.280 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.54939.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54939.log.gz 2024-09-17T03:39:46.280 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-09-17T03:39:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34745.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-09-17T03:39:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-09-17T03:39:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35620.log 2024-09-17T03:39:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51149.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-09-17T03:39:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35620.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35620.log.gz 2024-09-17T03:39:48.330 INFO:teuthology.orchestra.run.smithi099.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T03:39:48.945 INFO:teuthology.orchestra.run.smithi099.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T03:39:49.117 INFO:teuthology.orchestra.run.smithi099.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T03:39:49.242 INFO:teuthology.orchestra.run.smithi046.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T03:39:49.286 INFO:teuthology.orchestra.run.smithi099.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T03:39:49.287 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-17T03:39:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m4.722s 2024-09-17T03:39:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m17.468s 2024-09-17T03:39:49.288 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m1.568s 2024-09-17T03:39:50.407 INFO:teuthology.orchestra.run.smithi046.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T03:39:50.442 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T03:39:50.792 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T03:39:50.803 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T03:39:51.448 INFO:teuthology.orchestra.run.smithi046.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T03:39:52.062 INFO:teuthology.orchestra.run.smithi046.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T03:39:52.065 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-17T03:39:52.065 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m7.535s 2024-09-17T03:39:52.065 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m39.572s 2024-09-17T03:39:52.065 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m3.953s 2024-09-17T03:39:52.066 INFO:tasks.ceph:Archiving logs... 2024-09-17T03:39:52.066 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/remote/smithi046/log 2024-09-17T03:39:52.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T03:39:56.227 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/remote/smithi099/log 2024-09-17T03:39:56.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T03:39:58.319 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T03:39:58.329 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T03:39:58.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T03:39:58.332 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T03:39:58.411 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T03:39:58.421 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T03:39:58.421 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:39:58.424 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:39:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T03:39:58.437 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-09-17T03:39:58.437 INFO:teuthology.orchestra.run.smithi046.stdout:*hv01.front.sepi 67.205.162.81 3 u 58 64 377 0.108 -2.655 0.483 2024-09-17T03:39:58.437 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 63.231.80.2 3 u 58 64 377 0.087 -0.620 0.300 2024-09-17T03:39:58.437 INFO:teuthology.orchestra.run.smithi046.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 64 377 0.094 +1.377 0.525 2024-09-17T03:39:58.437 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T03:39:58.438 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T03:39:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2024-09-17T03:39:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:*hv01.front.sepi 67.205.162.81 3 u 48 64 377 0.081 -11.061 2.116 2024-09-17T03:39:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:+hv02.front.sepi 63.231.80.2 3 u 55 64 377 0.074 -8.786 1.978 2024-09-17T03:39:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.093 -5.934 1.741 2024-09-17T03:39:58.438 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T03:39:58.440 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T03:39:58.470 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T03:39:58.470 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T03:39:58.479 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T03:39:58.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T03:39:58.496 INFO:teuthology.task.internal:Duration was 1875.369818 seconds 2024-09-17T03:39:58.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T03:39:58.505 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T03:39:58.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T03:39:58.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T03:39:58.593 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T03:39:58.593 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-17T03:39:58.593 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T03:39:58.614 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-17T03:39:58.614 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T03:39:58.633 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T03:39:58.634 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T03:39:58.662 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T03:39:58.717 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T03:39:58.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T03:39:58.754 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T03:39:58.845 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T03:39:58.853 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T03:39:58.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T03:39:58.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T03:39:58.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T03:39:58.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:39:58.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:39:58.937 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-17T03:39:58.952 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-17T03:39:58.970 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:39:59.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:39:59.007 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:39:59.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:39:59.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T03:39:59.028 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T03:39:59.029 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/remote/smithi046 2024-09-17T03:39:59.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T03:39:59.074 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908415/remote/smithi099 2024-09-17T03:39:59.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T03:39:59.100 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T03:39:59.100 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T03:39:59.118 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T03:39:59.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T03:39:59.155 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T03:39:59.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T03:39:59.164 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T03:39:59.164 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T03:39:59.174 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T03:39:59.179 INFO:teuthology.orchestra.run.smithi046.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 03:39 /home/ubuntu/cephtest 2024-09-17T03:39:59.191 INFO:teuthology.orchestra.run.smithi099.stdout: 655468 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 03:39 /home/ubuntu/cephtest 2024-09-17T03:39:59.193 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T03:39:59.203 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T03:39:59.245 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1875.3698179721832 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-17T03:39:59.246 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T03:39:59.351 INFO:teuthology.run:pass